Water celery can get a bit invasive, I haven't had nearly the issues with parrot's feather and I'm zone 4/5 also. Pulling out the loose celery doesn't do a whole bunch unless you get to the roots. One tip we often do is a 'burrito' for the parrots feather. A little bit of underlay wrapped around the soil will create a manageable 'burrito' that will limit spreading in our zones.
As far as the string algae we try and stay as natural as possible also. We go with a combo of barley extract and pellets and it inhibits the growth, not an algaecide, but does a solid job. Bales are okay but the must break down to be effective and have to be in early to make any difference. Otherwise some string is basically unavoidable without treating you pond like a pool or having massive plants to starve the algae.

If I am going to get in my pond one more time this year, I could pull out most of the water celery. I do like the look of it and I am not afraid of pulling it all out-way too big of a job. Last year I let it freeze and it did look a bit ugly. Is there any benefit to get most of it off of the rocks.

Thanks for the burrito tip.