I had green caterpillars eating holes in the leaves of my broccoli and brussell sprouts. Since they are all the same species, perhaps that is what you have? They blended in really well and were tough to spot when they were small, but became more obvious as they grew. I usually found them on the inner leaves. I just picked them off manually and have seen no new caterpillars or leaf damage for weeks.

I've had cauliflowers set out as transplants in early/mid May (around Mother's Day or just after) go all through the summer vegetative, get enormous, and then make absolutely fabulous heads in October. I set out 75 plants of Snow Crown in 2008, and none of them did anything until fall, when they were great and much larger than normal for the variety. Variety may make a difference, too. Graffiti and Cheddar planted the very same day in 2008 made cauliflowers in July. The Snow Crown waited until October. If they're "blind" I think they kind of just stop growing, with no new leaves coming from the center. If they're still growing new leaves and getting taller, I'd just leave them if you can spare the space, keep them well watered and fertilized over the summer heat, and I think that you will probably get nice fall cauliflowers as well.

It's a cabbage looper. Use Bacillus thuringiensis against them if hand picking isn't practical. Bacillus thuringiensis (BT) is simply a bacteria that likes eating caterpillars. It is sold as Dipel or Thuricide. Here is a link on cabbage loopers

Likely caterpillars of one kind or another. Don't know which specific ones are busy at your place, but the key is to look closely at top & underside of leaves. At my place, the current bad guys on the cabbage family crops are these: 1. imported white butterfly (cats are up to 1 inch long, velvety green color helps them bled into the leaf. 2. diamondback moth -- very small cats, about 1/4 inch, also small pupal cases webbed to the leaf. These cats make "windows" (eat cells on surface, only) also holes.