I grow both varieties(Southern large tree growing evergreen M. virginiana var. australis and Northern small tree growing mostly deciduous M. virginiana var. virginiana) and cultivars of both varieties, as well as some intermediate between the 2 described varieties, of Sweetbay Magnolia here in Eastern Kansas. I've never had any die-back or problems with any of them, except one. The one exception is a wild grown seedling from far South Florida, which some people think is distinct and should be a third named variety and which also seems to be almost a sub-tropical tree. So that's another story. I've seen Sweetbay Magnolia planted in open exposed windy sites here in Kansas and yes they do get some of their early leaves beat up a bit. But usually it's not really noticeable at a distance. The evergreen ones can have their leaves beat up and ragged looking in late winter and early spring before new leaves emerge. So, I'm of the opinion that there's something wrong with the soil in which yours are planted. Near a driveway there's no telling what buried debris around. Limestone gravel, large chunks of concrete, other construction debris, trash, etc. Using soil acidifiers(Aluminum Sulfate, Sulfur, etc.), hollytone, ironite, etc. all could help if it's an alkalinity problem. But you can add too much of some of these things at once, so space out any additional treatments. Also, making sure the trees are well mulched and/or adding compost before mulching could both help out as well. snasxs, I suspect there's something else wrong with your Sweetbay Magnolia too. I'd be shocked if any Sweetbay had any winter die-back in zone 8, especially if you have hot summers. If you were in a cool summer zone 8 area, then it'd be possible, but still not likely. Even the far South Florida origin ones would only have winter problems in a hot summer zone 8 in cold years.

Leaf miners can be addressed with other than systemic insecticides, but your timing has to be spot on if you want to use sprays. They don't spend their entire life-cycle inside the leaf. And you will have to spray in that interval when they are external to the plant. I used to raise bearded iris for sale, and that meant the State inspector had to monitor my mother stock. I had a beginning infestation of iris borer on one variety, and she explained to me how they enter the leaves and then migrate into the rhizome and how to monitor for the earliest signs of them before they damaged the root stock. Same way with leaf miners. Due to your zone, I don't know if what you have are magnolia leaf miners or not, but they do produce damage like leaf scorch, in addition to the mines. Leaf miners can be moths, weevils, etc. it's a generic name for any insect who does this type of damage. So you have to identify the causative pest. Magnolia leaf miners do lay their eggs inside the leaf, the insect pupates in there after their larval stage, and emerges to the outside in summer.

Give it another year. Terrific species btw, love the aroma. People are here for the short term; trees live on a different time scale. If you have the room, you might wanna consider another species with a longer bloom period. This Little Gem blooms for me from May till Early November, though not quite as aromatic as the Sweet Bay.

Several houses ago we had a sweetbay magnolia that as you say is a very desirable tree for several reason. Like you I could never understand why it only had branches at the top. After puzzling over this for several years, one day in the spring I go my answer. Apparently not only humans like the smell of the tree, but squirrel also appreciate it. That morning I looked out of the window and noticed a squirrel busily working in our yard. I took a few minutes of observation to see what she was doing. She would climb the Magnolia and cut several of the lower branches off. Come down the tree and get them in the shape she wanted, and then carry them up another to where she was building a nest, . I realized that other that shooting the squirrel, since we live in a urban residential area not a option. I should learn to live with a bean pole sweetbay magnolia tree. I also suspect that since there had been many new babies in the yard and the sweet bay magnolia never had lower branches, if I got rid of the older squirrel, he babies who were brought up in a sweetbay magnolia lined nest would continue the tradition. So again it seem a futile effort to trap the older female.