Inspect the plant all over, inside the canopy, under the leaves, you might find something that is munching on your plant. I found in my tropical hibiscus flowers and buds lots of small ants and they were after the sticky honeydew from scale. I had to spray it with neem several times to get rid of both.

Many pests that damage leaves do the damage at night and hide during the day. Continue trying to catch them in the act or in places where they hide (under rocks, mulch, debris, etc)

Many thanks for your replies. I had gone over the plant pretty carefully in initially trying to find an obvious suspect, so I took your advise and did so again after dusk and relying as much on hand feel as sight. Lo and behold, in palpating bits of the plant (including a blossom that had been nearly entirely decimated in just a few hours), I felt something fall into my palm: the same sort of green caterpillar-y worm I'd evicted in spring -- which I hadn't seen even though I'd been looking directly at its eating spot as I felt around! Heartened by the find, I kept at it, got nothing then went back again at night with a flashlight and managed to get another dropping into my hand that I couldn't see while looking (tricky, these guys -- I'm not *that* old to have Mister Magoo syndrome!)

The double eviction has really helped, though I'm fairly sure there's a third lurking (and chewing) somewhere. I'll continue taking your advice and finger my way around the plant some more.

Thanks once again for your pointers!