If that were my tree I'd be cutting everything way back. All I see is bare wood. But if you want all that wood and height just cut it off below the break. I new shoot will take the place of that broken off.

should I be cutting the limbs off near the trunk or should I be leaving them cut them down to the first set of leaves?

I'd cut them back to the lowest leaves on each branch. On the central leader I'd leave a few of those lower sprouts. This will give you a much better tree to shape in future years.

acce,

You didn't inquire about this, but your tree looks to me to be in a questionable location.

From the shadows, floor litter, and trees in the background, it looks to be in a shady area.

If that's the case you're fighting an uphill battle. Not only will the shade make things more difficult, but the competition from roots of other forest trees will add to the stress on your tree.

Peach trees aren't like Oak or Maple trees that can be planted pretty much anywhere and grow without care. Instead for peach trees to thrive, they must be planted in a sunny location with good drainage, and little root competition. At least that's my experience.

If it were mine, I'd take the leader off all the way down to the top branch and train it to an open center. It looks like it has a good start in that direction already.

Chop it back. Peaches grow super fast (give it some N) and they fruit on year old wood.

Watch the shade. So hard to tell from this pic or without being there.