just wanted to add that the little bugs I mentioned I dont think are fungus gnats. They are too tiny I think and dont look anything like the photos Ive seen.

A pic would help. Are they anything like the woolly aphids in the pix at the link below?

If you think it's a bug, try one of these sprays:

CITRUS SPRAY FOR APHIDS:
Boil 2 cups water in a saucepan. Remove from heat and add the peel of 1 lemon or grapefruit. Cover, leave overnight. Mix half water and half citrus liquid, and use as a spray for aphids and other soft-bodied pests. The spray must come into contact with the pests to be effective.

FUNGICIDE/POWDERY MILDEW SPRAY:
4 litres water
3 tablespoons bicarbonate of soda
1 tablespoon household bleach
1 teaspoon dishwashing detergent

Snip and remove leaves that are worst affected. Mix ingredients with water. Spray remaining leaves top and undersides. Apply a heavier dose on leaves that have signs of infection and only lightly on unaffected leaves as bleach can actually harm and discolor the leaves.

BASIL SPRAY:
Pour 1 litre of boiling water over 2 firmly packed cupfuls of basil leaves. Leave to get cold, strain. Use within a few days. Basil spray will deter and sometimes kill aphids, woolly aphids, flies, fruit flies, leaf-miners, mosquitoes, scale, spider mites, thrips, whitefly, bugs, caterpillars, earwigs, millipedes, beetles, slaters, spiders and weevils.

CANOLA OIL SPRAY:
Mix 1 part canola oil into 50 parts water (approximately 1 tablespoon oil to 1 litre water). Add a few drops of diswashing detergent and shake well. Spray tops and bottoms of leaves and stems generously to get rid of aphids, mealy bugs, spider mites, soft scale or earwigs.

CHIVE SPRAY:
An infusion of 1 litre of water and 1 cup of chopped leaves is a mild insect repellent, a fungicide and an antiseptic. It may assist in the treatment of downy or powdery mildew if applied regularly. Use the spray within a few hours.

You can also try a Garlic Spray, a Coriander Spray (using the leaves made into a strong tea), a Eucalyptus Spray (using the leaves made into a tea, or some diluted essential oil), Nasturtium Spray (if it's aphid) , Mint or Lavender or Lemongrass (as a tea, if it's aphid), Chamomile (if it's a fungus). Or Rhubarb (leaf tea) or Tomato (leaf tea) (both for assorted bugs). Pyrethrum is a good all-rounder.

http://gardening.about.com/od/gardenproblems/qt/PowderyMildew.htm

All these will wash off easily to make the rosemary safe to eat, and won't affect the flavour.

Here is a link that might be useful: woolly aphid

I've had good luck getting rid of powdery mildew with a baking soda and water solution (look around online for the proportions), a diluted 1:10 milk and water solution, and with an infusion of chamomile (brew it like tea and spray it on). All of these are fine for eating...and more palatable than the powdery mildew!

Have you tried just hosing it off? Many times this is all it takes.

Thanks, Daisy, for all those recipes.

Interesting. Thanks all for the great ideas. I will be trying the baking soda and milk/chamomile stuff first. Although I did see one on another place in the forum with horticultural oil and baking soda as I said in the first post. I think I will try those 3 first (not at the same time of course!) and see what happens if washing the leaves off doesnt work. As for the little buggies, theyre not little wooly aphids or anything.