Personally I'd agree. If you want to add a third color I'd do it on a smaller scale somewhere other than window trim... Just my $.02

If you make the other changes as suggested, I don't think you'll feel the need to paint the brick. There's no sense creating a maintenance issue where there is none now. The house has the potential to be very handsome. Some pretty pots with flowers on either side of your door will add some color as well. Don't forget that nothing looks pretty this time of year when trees are bare and lawns are brown.

The rest of the house has wood trim. Also, we are doing wood look porcelain tile throughout, including the kitchen.

Is the whole house going to be brick? All sides? Are you essentially asking if it should be greyer like brick or black like windows?