The leaves on that tree look like they have "ears" at the base; if so its probably Frazer magnolia, Magnolia frazeri. The little herb might be spreading chervil, Chaerophyllum procumbens, try looking that one up.

As well as closeups, location would also be useful, especially for the wild flower. I can't make out the form of the flowers enough to even pin down a family. eta ok, I see you are in Georgia. Sorry I missed that in the text. p.s. Have a look at images of Polygala polygama. Is that your plant?

Well, I've never seen one with little spines, and don't think that stock images count for a high degree of accuracy. But I refuse to be someone who thinks that just because I haven't experienced something that it cannot be true. There are enough of those in the forums, lol. The P. caroliniana is certainly quite common in its range, that's for sure. Most consider it a trash tree, though I have seen numerous specimens that have been groomed into amazingly beautiful large shrubs or small trees.

Small woody invasive species I pull out by the roots with a large set of pliers. There are also tools designed for pulling woody plants typically called something like a weed wrench that can be used for things a bit larger. If you cannot pull or dig it out, cut it to the ground after which you have a couple choices. For Japanese knotweed and larger woodies, I cut down and paint stumps with a brush strength herbicide. I am not an indiscriminate user of lawn and garden herbicides, but in this instance I think it is the lesser of two evils, and since I have more than 100 acres with lots of invasives, this is my only hope of removal. I have researched to find what won’t stay in the soil and wear protective equipment. Painting on cut stems uses less herbicide and in a very targeted fashion. Alternatively, cut them to ground level and then religiously check for sprouts every week or so. Remove the sprouts. Continue this for a season or two, or for Japanese knotweed, it might be longer. This will use up the plant’s stored resources without allowing it to replenish. But it only works if you are very, very consistent. IMO removing invasives from public land that adjoins your property is totally reasonable. But as I said above, don’t remove until you have a positive ID.