This is very normal. It's abnormal when it doesn't happen.

Simple forum type answer is to add a UV filter.

Long answer

It's not specifically related to the goldfish. The fish do produce ammonia which algae uses to grow, but and this is a very important but, do not jump to the conclusion like virtually everyone does that the algae wouldn't grow if the fish are removed. It would probably grow less fast, but you'd still have the exact same problem. It's the same with sunlight. Algae grows slower in shade but still grows. To most people it doesn't really matter if there is 10,20% less algae, it's still too much.

The other important thing to understand is in ponds we break algae into 2 general classes.

1. Green Water Algae, aka Micro Algae
Micro just means small. This type turns water green. Meaning if you put pond water into clear water glass and hold it up to the sun the water has a green tint. Generally pond owners will say something like "I can see the bottom" or "I can't see the bottom very well" or "I can't see my hand when it's 6" deep" to describe how green the water is. These green water algae is a single cell algae that is so small it stays suspended in the water. Although these cells will clump together into larger balls and then sink eventually, but new cells will replace those.

Also called suspended algae.

2. String Algae, aka Macro Algae
Macro just means big. String algae is an unfortunate term because it's only one type of macro algae, but string algae is the term most often used in forums. There are many species of these kinds of algae. These are more like plants, you can touch them, pull it out, etc.

Your description, which is common, covers both kinds. It's important to know which kind because the remedies are very different. The string algae type stuff generally grows on the pond walls, rocks, pipes. Some people seeing this say my pond is green even though the water is clear because they see the green stuff growing on the pond walls.

You said It happens both in the pond itself, hosing and the waterfall. That describes macro algae which grows on things.

But you also say can no longer see the fish which sounds like green water algae (micro algae).

Micro algae does clump and sink and so the pond bottom, hoses etc, can get covered in a kind of green dust. To most people that could appear to be algae growing on those objects.

You also said green scum. This is a term we use to describe a green film floating on the water surface. It can be very dense, almost like green paint floating on the water. This is a less common algae but could lead you to say you can no longer see the fish.

So it's kind of important to narrow your description. Many people don't like algae and so like to use terms like "scum" just as a way to insult the algae. But this type of thing will only generate bad advice and make your life much more complex. Besides, algae don't read forums so they won't be insulted at all. But if your algae actually is scum then that's an important clue.

The time frame you're describing also sounds like micro algae. New water, couple of weeks (2-4) and the water turns green.

The most effective fix for green water (micro) algae is a UV filter. These are pretty cheap to buy and run and are 100% effective in removing green water (micro) algae in 3-7 days. It is so effective that the test for knowing if you have it sized and installed correctly is the color of the water. If there is any green tint at all it isn't sized or installed correctly. They're not hard to install, but unfortunately many people don't seem to like any kind of detail. They plug them in and if it doesn't work give up and start trying the many "fixes" that don't work. Such is the world.

After the water is cleared you will very likely get a few kinds of macro algae. Interesting when this happens you can most likely turn off the UV and you may never need it again except may in the spring depending on your climate. Macro algae produces a chemical that kills micro algae. But micro algae also produce a chemical to kill macro algae. A war. The UV filter kills the micro algae and so their chemical goes away allowing macro algae to grow.

Macro algae can be controlled by chemicals you buy, but I'm not a fan of those. Lots of issues, but is possible. Much easier to remove macro algae by hand. Like weeding the garden or mowing the lawn.

Emptying the water and refilling is the most difficult way to deal with algae. You will often read in forums people saying "you're starting over" because it's widely believed ponds clear on their own given time, but most people don't know why. It's because of the chemicals produced by macro algae.

Another effective when to deal with green water algae is to have a constant water drip into the pond. This is tricky in small ponds (100-200 gals) and almost impossible in most pre-form ponds because an overflow is needed. Here's a video that describes the system.

Thank you so much for your answer! I told my wife that you must know your stuff - no one could make up such a complete answer!

So...to diagnose the type of algae - my answer is I'm not sure. Is it possible to have both types? If not, I would probably use your definition of macro algae. I say this because when I drained and cleaned the pond, I had to scrub the algae off the sides (pretty aggressively). The waterfall and tubing were especially caked, the actual pond less so. However, because I had to pull the fish out too, I do recall the water being a little cloudy as well. So if it's possible to have both, I'd say I had both. If not, I'm going to guess it was macro. When I referenced not being able to see the fish, the pool went from a black background with clear water (and I could see the fish) to a cloudy mess. However, that could have been because the algae caked the inside of the pond, thereby causing the fish to 'disappear.'

That said, my pond is pretty small - I doubt it's 50 gallons total. I watched part of the video - would you recommend a drip? That's going to be a challenge, I'm guessing, because it's so small. The only way I can think of to accomplish this would be to drill a hole in the side to allow water to escape at or above this line, but then I'm not sure how to create constant water into the pond. I have a water source nearby but wouldn't want to keep a hose constantly running into it.

As for the reference to 'scum,' I was referring to the green build-up on the sides and bottom of the pool, waterfall and tubing. It is not a situation where it's floating or present elsewhere as a floating form.

I hope that makes sense in addressing some of your questions. Please let me know your thoughts. THANK YOU!

Sorry, I still don't know what kind of algae you have. Here's how to narrow it down.

You can pick up macro algae.

Micro algae is like green tea, you can't hold it in your hand.

Also, as micro algae dies it settles like dust on everything. When you stir that dust up it clouds the water and it will stay cloudy for an hour, maybe a day, and then settle again.

This settled stuff can kind of be held in you hand, but its like more like a slime. Macro algae has a texture.

It is possible to have both micro and macro algae at the same time. There are too many species to know for sure how all act. The war they wage is not only with chemicals. The green water algae (micro) reduces sunlight to the bottom where macro algae wants to grow. But macro algae can get a foot hold in waterfalls, streams, and on the edge of the pond near the surface. But generally one is, or will become, dominant. Them there are some really strange species that defy the normal.

Scrubbing algae off the sides sounds like macro algae. I wouldn't do that any more. It's too much work as you've seen and it's what will keep water clear. keeping the sides spotless requires almost daily scrubbing. Most people consider algae an enhancement to the look of a pond, within reason. Which is why people pull it off as it gets too long. Mowing the lawn.

If you really want spotless you can use an oxidizer. Basically it's hydrogen peroxide like you buy in the drug store. But that isn't very strong so better to buy a granular type. I don't have a specific brand, I don't use it often. And don't really recommend using it unless having algae would mean the pond would be removed. Like most chemicals you have to really follow directions and accept the risk to fish. The other reason I don't recommend it is that it isn't magic by any means. It only kills the algae, which means instead of green algae you now have brown algae. Not much of an improvement in my book. So the dead algae has to be removed.

At least that's how most people use these products. They wait until there's a problem, toss in the chemicals and have a bigger mess. The way these are actually suppose to be used is when there's no visible algae or very little. The idea is to kill the algae when there's not much and it's tiny. A pond can handle disposing of that little bit. So the idea is to have a routine of adding this stuff every few days, once a week, what ever it takes.

There are copper based chemicals too but these are even more risky to fish. But these do last longer.

In all cases using chemicals would also require a lot of manual scrubbing. That just what it takes for a spotless pond if that's what you want.

On the drip system.. The set up is described in the video in detail. You definitively don't want to use a hose if you have chlorinated water because the drip would not be consistent. 50 gal pond imo is too small for this type of setup because a 0.25 GPH emitters would be the max I would use and I don't think they make them that small. Maybe you next pond ;-)