Remove all the shelves. Sort your hanging clothes by length - long dresses, tops, slacks and then figure out how much space you need for each length. Reinstall the shelves. Put seldom used, seasonal or extra stuff on the high shelf, and have a stepstool tucked into the closet so I can get to the high up stuff easily.

Would you have room to turn clothes rods 90°, hanging in the deep part of the closet? I would put double rods in one closet, and a longer one for dresses in the other. Opposite the hanging clothes you can build 12" shelves (I would not use wire shelving). Essentially you would have 2 shallow walk-in closets.

I'd also consider taking the doors off and replacing with curtains, especially if you rarely close the doors.

Remove the doors and try it for a month. If it's good, then install a single door on each closet. If it is not good, call a contractor for an estimate on reframing the closets with larger (wider and taller- floor to ceiling) openings. They can also investigate the possible HVAC in the middle. A full opening with sliding doors would be much better than the current structure.