Depends on what your idea of breaking the bank is. You can try painting, I would suggest hiring a professional, not a DIY, as the stone is already such a focal point. Some color change might make this more desirable for you.

Paint or stain (Romabio) the brick and stone the same colour. I suggest dark gray, but it really depends on the rest of your room.

A more expensive option would be to have the upper portion (both brick and stone) removed, add a mantle, and drywall above the mantle. Or possibly, you could add a full-width mantle then drywall over the brick and stone above it. Depends on how far the stone sticks out. Then with either scenario, paint/stain the brick below the mantle if the colour doesn't suit your decor.

Do you have any pictures before the 70's update? If the stone is just an add on, on top of the brick maybe it could be removed. Do you have an idea board of what you would like?

Break out the stones and reface the entire fireplace:

Concrete Facing

Wide Format Tile Facing

Unfortunately I don't know if it ever looked different but the whole room was added in the seventies. My neighbor thought they had a greenhouse originally and made it into a family room.

I haven't made an idea board yet as I just started thinking about this but am currently working on some other projects. I would definitely like a more modern look but have some flexibility. Thanks so much for answering!

How about painting the smaller bricks white and leaving the bigger bricks? Country cottage / ranch house chic.

leave the brick remove the stone. add a functional mantel. art above. to keep midcentury look maybe a starburst

If being limited by funds, I would hold off. Given the brick AND stone and what seems to be a crack in the hearth, as well as the overall heaviness of the whole thing....I’d collect ideas for now, have a few people in for ideas and quotes, and save up to do it right. In the long run, it will make a huge difference in enjoyment and value.

Paint brick and stone the same color with Romabio.

I wouldn't do more than that, unless you know for sure that the room addition was added with city approval and is up to code.