It sounds like horn worms aka the "tomato worm". Don't be fooled by the name; they love peppers and eggplants too. More info can be found here. If that is what it is, I feel your pain. A trio of them ruined my best Jalapeno plant last week, consuming 1/2 of the total foliage on the plant over 24 hours and leaving most of the peppers exposed to direct sunlight burn. They are ravenously hungry beasts. If you get lucky enough to see one feeding, you will be amazed how fast the gulp down the leaves, going from one to the next to the next. They also eat the fruit - actually, they burrow into the fruit enough to ruin it for your own eating, and then leave behind a ruined pod carcass with a hole in the middle. Best strategy seems to be to just go out in the morning and at night and pick them off. They are kind of hard to find, but can be found. They like to hang out on leaf stems, undersides of leaves, and occasionally on a main stem. If you search on horn worm, you'll get some other suggestions too, including corn starch, insecticidal soap, etc. But I was able to get them under control old-school style by just grabbing them and pulling them off the plants. They are destroyers of gardens, but few in number. (One or two can eat an entire plant, but once you get rid of them, they're gone). They're not like aphids - swarming in vast numbers. They're large and few in number. As soon as you see those droppings, that's your alarm bell to examine every inch of the plant to get rid of the worms before they gorge.

If I had to guess, I'd say slugs. They did a number on my peppers last year when we had an unusually wet summer (I found some IN peppers that I had picked. I didn't always notice the small holes. Even hot peppers). I've seen them on my rhubarb too. Beer traps work to an extent. I don't know for certain, but I believe that sluggo is approved for organic gardening. Diatomaceous earth is organic as well. Hope that helps -- good luck!

Ken, I agree, it looks like snails or slugs. I had lots of chilis and Scotch bonnets in the ground last year and had constant problems with the snails. Also, since the plants were in the ground too near other plants, powdery mildew became a problem. So this year I am growing the peppers in individual pots and so far...no problems. I don't know how many plants you have but if you only have a few, keeping them in pots may be a solution. It is easier to control conditions with them in pots.

Pee on your plants, that will keep the mini-deer at bay. It works for me every time them little bitty bastards hit my peppers. Bruce