Without knowing your location, I'll go out on a limb with Pine Sawfly larvae (Diprion similis)?

It does sound like one of the mud dobbers or potters wasps. Mud (dirt) daubers are a species of wasp. The adult wasp manufactures her nest, finds a grub, caterpillar, spider, etc. to paralyze (but not kill), places it in the nest and lays her egg. The wasp larvae is able to feed on the live (but immobile) prey until it is ready to emerge as an adult wasp.

Here's where proper id can be important. Proper id can give you the precise life cycle, revealing when the insect is the most susceptible. Timing can be incredibly important to the success of any control mechanisms. So here's a few general things to consider. How severe is this problem? If of limited scope, you can simply dispose of the problem by pucking them off, pruning off affected branches, or similar mechanical control. Once enveloped in the protective webbing, such pests can be hard to control. Contact insecticides (sevin, orthene) need to contact the body of the insect. In that nearly impervious pouch, not a good control mechanism. Ingested insectides (Bt) can be effective if the insect in question will still go through a metamorphosis, ie, the "worm" pupates into a moth. Spinosad can be quite effective as well...with either of the last two approaches, timing can be incredibly important.

Now I'm reading about them, and I am sorry for the few I put in the bluebird feeder! They were probably eating the assorted bugs who destroyed my corn. Yey soldier beetles!