The brick is much nicer than the siding. I'd keep it. That said, I don't know of any siding that can't be painted with the right paint. Frankly, since you don't have a lot of siding, I'd vote for replacing the siding before painting the brick.

I'm not sure maroon goes with the brick. I'd recomment painting the shutters.

I know a lot of people like the brick. But I would paint.

I vote yes, for Celery's image. Agree, also, no shutters. I would like to suggest my ideas for the paint colors, based on the color of your roof.

For the siding: B.M. Atrium White and for the front door B.M. Racoon Fur

I honestly hate the look of any painted brick and why introduce maintenance to a basically maintenance free product. And a definite no on shutters as they would not look like they belong.

I do sometimes wonder if I’m in a keep- the - brick rut but I just can’t see painting brick, especially a good monochrome type. Maybe some of those awful moth- eaten vomit brick styles with multiple colors, are bad. But just because it’s orangey- that’s a brick tone. You can soften that with your siding color change and landscaping. Play around with some color imaging for trims and door.
SO many people are painting brick and I think a few years in, it won’t show well. Even fresh, yes, it’s a uniform color ( almost always white or light) but it looks more fakey to me, loses the appeal of the strength of regular brick. Plus several white painted brick homes I pass by are showing dirt and mildew , now need pressure- washing or something.

I also like Celery’s example. Definitely no shutters on double and triple windows. The roof color looks challenging based on your photos. Check out Judianna’s paint color suggestions.

Looking at the brick, it is a dated style and the front is so broken up...particularly with the garage and siding. I painted our ugly red brick house yrs ago and LOVED it as did many neighbors who made a point to tell us. It's not high maintenance if prepped correctly. I love pretty brick but mine wasn't & your's isn't great either. Why do you say you can't paint the siding? One can paint anything...I've painted siding and vinyl garage doors several times. Prime it well, let it cure and choose a good paint. SW Duration is expensive but worth every penny. It's has a polymer mixed in, a LONG warranty. We used it 2x and it is unbelievable; stood up to a severe 1 hr hailstorm only days after painting. NOT a bit came off; we couldn't believe it.

Speaking of the garage, I'd paint the door and the gutters/downspouts the body color. You want them to fade away, not become the focal points. :) I'd select a medium taupe tone, paint the entire house that and then just an accent color or black for the door. No shutters.

When choosing paint, be sure to splotch and choose a paint darker than you think you want because it's going to appear a lot lighter when painted; this is always true but esp on exteriors. One easy way to choose paint? Drive around, find a house you like and then ring their bell and ask. People are flattered.! I actually had 2 people do that with me...didn't mind at all telling them and I gave one woman a few ozs of paint to play with. It gives you a good idea what it'll look like when you look at chips too. Actually I think the siding color is not bad...at least on my monitor. Maybe steal a tiny piece or take a close photo and have it color matched. They're pretty good by eye too.

IMO painting the brick is the fastest way to devalue the house . Just because you see it on TV does notmake it a good idea.Take the shutters off and maybe do a bit of more interesting landscaping. I love this for inspiration if you want to go the MCM direction Notice the front door and garage door and you could do the dark gray for your siding.

Thank you for all the comments and suggestions. I’ve been sitting on making this decision for quite awhile. I’m now looking for a low budget way to make the exterior look more unified. My current thinking is white shutters, white front door and white garage door and keeping the brick and siding as-is to keep costs down.

Painted brick will devalue your house because it creates a forever maintenance problem/cost.

I think that looks good OP but be sure to choose a white that is flattering of earthy brick tones - i.e. something very slightly creamy that leans more into browns/reds. Dont worry it will still read as white! A white that is leans more blue or is too stark can make the brick look dingy in comparison. A lighter/whiter/brighter version of your siding color might be a good place to start looking at whites.