Pretty sure you have delphiniums there - not foxgloves :-) And as long as it is only the lowest leaves with a few holes and drying or browning, you don't have anything to worry about. Could be any one of several dozen small plants pests lurking in the soil having a munch but nothing that will cause any serious damage.

Just so you are aware, delphiniums prefer a pretty rich organic soil and consistent soil moisture. When they are grown in less than ideal conditions, they become stressed, which can attract insect predation. They tend not to be very perennial under those circumstances also.

Wow, good eye, I guess I have a Delphinium and a Foxglove right next to each other, these are their flowers, maybe the only one getting eaten is the Delphinium, I'll have to check when I get home. Yes, I think it might be the soil, it's clay-like and I definitely need to put something better in there. As soon as I put them in the soil they got sad.