White aluminum and vinyl trim can leave white residue after rainstorms. We have it as well and there is no aluminum on our house - only vinyl trim. Pretty hard to get off without using something special.

I have trained my eyes to ignore it, but yours is much more prominent. Use the advice above.