My guess is deer. Slugs don't eat that way. Deer will eat flowers and do not touch leaves especially if he is not that hungry and the leaves are not so tender.

The type of damage and the droppings point to an insect larva and not a slug or a snail (snails tend to graze across a leaf or bud, leaving holes surrounded by undamaged leaf). Since you haven't found any caterpillars it's probably a sawfly larva or something similar. These look like caterpillars, but hide in the soil near the plant during the day. If you go out with a flashlight just after it gets dark you should be able to spot them. If there aren't too many you can handpick them & dispose of them, otherwise I'd suggest a systemic insecticide (and perhaps a preventative spray early next season before the damage gets done).

buy some live traps from your local hardware store, visit farmers market or your friends and ask where is the best place to deliver your furry friends after you trap them. you need to go at least 20 miles from your home or they will make it back. to get rid of our ground-hog infestation under the porch we resorted to some drastic measures. put some lye around and down their little holes to chase them out. have not seen any since

They avoid aromatics--some of which are known for beautiful flowers as well (lavender, and rosemary where hardy), you can always plant the hardy cactus which are covered in beautiful yellow lowers in June. Other flowers that are same are bleeding heart, columbines, day lilies, daffodils, iris, lungwort and rhododendros. There are lots actually. But sometimes I will find things damaged that they are not supposed to touch (but may not always just be deer!) Flowers to protect from deer--azaleas, hibiscus, camellia, roses, hydrangea, HOSTA. Mine seem to love Euonymus japonica best of all.--I give p on that one. Would move them into the house but the deer would only be breaking into the house to eat them!