Anagallis arvensis, scarlet pimpernel

Or, possibly Cheirianthus allionii, Siberian Wallflower

Thanks to both of you. So good to know. I quite like them. I am not too troubled by them showing up everywhere in my garden but I would not want to think that they are invasive and it would be a bit challenging to thoroughly remove the spent flowers though I could try. Across the road is a forested area and a path with tall wild plants like goldenrod and aster and beyond that a marsh. I don't want to do anything irresponsible. I understand they are short lived. Should I be concerned about them spreading to this natural area across the road?

They don't appear to be beneficial insects, so even though you don't know what they are, it is safe to squish them, hose them off or spray them with soapy water. You can also use an insecticidal soap such as Safer. You may not have to resort to strong chemicals which can also kill beneficial insects.