You will find that next year it will take off. In my opinion, I wouldn't change where you have it. I've grown them from seed and once they establish their roots, that's when they amazed you. Be patient. I haven't watered mine and we are in what is considered to be the driest early summer in years. The soil has retained it's moisture, but they haven't had any water in about two weeks and are showing no signs of needing it. I'm just north of you. Hope you made room for them. :O)

Joe Pye weed is a great plant in the right place, beautifully suited to swamps and wetlands. It is a very large plant with a robust presence. It has an appealing scent. I don't think it would be well-suited to a small English garden - it's a space eater. I don't care if people don't like the plants I grow or the way I garden. Because our gardens are public, we see the gamut of visitors. Most are kind. Occasionally, there is a gardening snob wandering through, but I have learned to ignore them. But everybody's garden is different and their experience of any given plant is bound to reflect that difference. I didn't invent any of the plants I grow, so I cannot take it personally if anyone dislikes a particular plant. Also, my own tastes change. Plants I loved 15 years ago may now be banned from the garden, or they just might be working their way back into my own top 10 list.

I love information about what will probably happen from collected seed - any seed -- snapdragons, petunias, anything! That's interesting about the Choc Joe. I also have a (purchased) plant and can put this information on the seed envies. The thing is, I'm not sure my purchased plant is as deep dark chocolate as it should be. I ought to take a picture of it to ask. FWIW, Another plant along those lines is Heliopsis 'Loraine Sunshine' which is variegated-leaf Heliopsis. I would say about 20% are nicely variegated and the rest should be tossed. Lime

While the leafminers make the leaves they inhabit appear not very attractive they do not do much harm to those leaves. Although some sources suggest pulling the affected leaves off the plant unless the larva is still in the leaf there is no really good reason to do that. Control is pretty much limited to the adults that lay the eggs, and systemic poisons do little to control the larva, although some sources will state that if systemic poisons are used they need to be applied about two weeks before the larva start to feed on the interior of the leaf. Since these larva are the hosts for a number of parasitic insects I'd not do anything about them since topical poisons to control the adults will also kill those parasitic insects.