no you'll be fine. The only problem is if you or someone else mistakes a poisonous plant for an herb, which is unlikely. One plant producing poison doesn't affect the toxicity of a plant nearby.

The only way I could really see a problem is if you like have castor bean as an ornamental, which apparently also has nasty allergenic foliage besides its deadly-poisonous seeds, the foliage might rub against that of the herbs and transfer some irritative stuff, and this could happen with ouchy-leaved things too, I guess. But very unlikely.

It's a matter of common-sense, really. I wouldn't plant your stinging nettles next to your veges, or harvesting could be a painful experience. Many people grow basil and other herbs near their tomato plants - and tomato leaves are very poisonous.

I wouldn't grow something deadly poisonous like belladonna near the vege patch. With some plants, just touching the leaves is a health hazard.

As a general rule, you can easily grow edible plants with non-edible plants. It's rare that one will contaminate another.

Castor bean is a good example of a poisonous plant. For tomatoes, the big green hornworm loves the leaves as he absorbs that toxin, making him quite poisonous too. Tomatoes and peppers are in the nightshade family.

I would probably be more concerned with any chemical pesticides, weedkillers, etc. used around edibles. If you must use a pesticide or fertilizer or weed-killer near to your edibles, look for an organic vegetable gardening product.

FataMorgana

My concern was somewhat like FataMorgana's in that you can't spray the other plants with insecticides, etc. and then expect that the poisons didn't get on the herbs. I had a friend who planted all her herbs under her heavily sprayed Rose bushes...... not wise.