I don't think you are going to get any kind of reliable answer. There are too many variables which depend on your area, the type of custom house, and the general housing market.

If you are trying to determine a fair percentage for your area, be sure to compare 'apples to apples' in terms of what services are actually provided, and for what type of housing. The "custom" house which is one of several models repeated in variation (a common use of the word) with the customer selecting/customizing colors, finishes, flooring, and/or appliances is a very different category of apple from the "custom" house which was designed by an architect and has all the details worked out (well, hopefully so) before the builder oversees the construction. And even the designer/builder combo probably has a thousand or so variations depending on who does what!

I have cousins who were "custom builders" (recently retired now)specializing in designing and building family residences for the decidedly well-off stratum; most of their SF houses are valued at over $2 million each, while a section of townhouses [they did for"fun"] originally sold for $700K and up. Their 'profit' ranged from about 50% of the selling value to an occasional loss taken when the house built didn't even come close to selling for the cost of building. That loss didn't happen very often but it's a real fact of life for custom builders. Over the 30+ years they were in business they each made a very comfortable living, yet they certainly had losses due to collapsing housing markets and/or defaulting customers.

I'm going to guess somewhere between 10% and 15% of what they're charging you. Or at least what they're charging me. But that doesn't include any relationships they may have with their subs, etc. I know that some of the subs are relatives of my builder, but they also seem to be competent, and in the end, it's all a matter of the builder taking responsibility anyway. When we got 3 bids, 2 of them were very, very close. I think they do just eyeball it somewhat, and throw in 15-20% on top of that, and figure they'll ultimately get close to 15% or so, averaged out.

In my market a cost plus is typically 15% for a nice home under $1M. It certainly was 18% and there are people doing it for 10% now.

To be fair, that is not all profit. There are certainly expenses. My fixed cost (about 15%) was broken down to about 1/2 overhead and 1/2 profit. The builder had 1 superviser and I take the overhead to be mostly his salary. But he used to have an office and that would have been overhead. Not to mention any advertising. You could call his truck overhead but it was his personal vehicle.

"To be fair, that is not all profit. There are certainly expenses. My fixed cost (about 15%) was broken down to about 1/2 overhead and 1/2 profit. The builder had 1 superviser and I take the overhead to be mostly his salary. But he used to have an office and that would have been overhead. Not to mention any advertising. You could call his truck overhead but it was his personal vehicle."

There are a myriad of costs that come out of the builder percentage.

Phones, accounting costs,office equipment, vehicle costs
continuing education on everything from permitting requirements to code requirements, insurance, tools.