I'm trying to limit to just a handful of hours a day instead of all-day marathons, but there's always something to do so that doesn't always go as planned -- but I pay for it the next day if I over-do so that is the primary reason I try to pace myself. I'm usually too tired after work to really do anything except maybe water the pots, I really don't even usually go out there to pick any vegetables on workdays, even though that shouldn't take too long. I'd say maybe 6-8 hours a week during the season is needed? Maybe?
I really let things get away from me this summer due to family circumstances, I'm starting to feel overwhelmed again. This may not be such a bad thing because when I get like this (or fed up with stuff) I go on a culling rampage -- which really does need to be done, I over-planted purposely when I made the new beds, can't stand an un-filled-in look, so now I need to thin a lot of it to cut down on the upkeep over time and keep room for the access paths (I knew this going in). The vegetable garden is a mess, though. I made a list of what to cut down on for next year, I'm just over-doing it year after year. Because I like to eat good food LOL! But here too I need to be more selective and just quit trying to grow stuff that doesn't do well for me.
Maybe next year I'll do better at spending 30-60 minutes after work deadheading or other small chores so I'll have more time to myself on the weekends. Maybe? Who knows.