dig them out ... remove everything.. but a bare 4 foot trunk .... you will need that for leverage ... and then start digging about 3 foot out ..... it wont be easy ... the undisturbed soil will probably be very hard/compacted and then just start cutting roots ... it wont be easy ..a sawzall helps.. and use the trunk to rock it back and forth to find roots as you continue to remove soil ... ... there will be one maddening large one.. going straight down ... yes.. its very hard labor... but cheaper than paying for it ... the lesson you should learn here is twofold .. one those cute little babes get very big over the decades ... so on replanting.. plan accordingly ... and second.. foundation planting are planted to hide the foundation.. NOT ON THE FOUNDATION ... so when you go to replant.. think about a 6 foot wide bed .. and plant about 3 to 4 feet out.. leaving at least a 2 foot walkway behind for house maintenance .. window washing.. xmas lights.. painting.. washing siding.. etc ... good luck ken ps: yes you could cut them to the ground.. and apply roundup.. but you will still have to do the digging.. once you try to replant there ... so may as well do it right in the first place ...

You could look into a Black Lace Zambucus. Those can die back each year but will still get quite large. Mine took about three years for it to hit its stride, and now it is beautiful every year. It can be pruned to a tree form. I think it was known as the Japanese maple of the north. Or you can also try to find a standard physocarpus which appear to be more readily available these days. If you want to stick with the dark or reddish/purple leaf colors.

Ostrich, the stems are rather flexible and you should easily be able to lay the plant down to the ground for winter. The problem you'll notice is it needs a long hot growing season in order to ripen and harden off, otherwise much of the new growth will freeze right back. The first few years, I had covered it with about 3 inches of peat moss and then let this freeze hard (to keep mice from burrowing into it) and then put plastic over to keep moisture out and later piled snow on top. This plant requires work if it's to obtain any real size in this climate and it's likely going to be more frustration than satisfaction.

Is it the vine type or the shrubby type of invasive honeysuckle that you want to replace? There is a native honeysuckle vine called Trumpet Honeysuckle (Lonicera sempervirens) which attracts hummingbirds like crazy! It's easy to grow. For a replacement of the shrubby variety, you could plant Viburnum or Serviceberry for berries. The Viburnum berries stay on the plant longer than the Serviceberry but you need to plant at least two (of different cultivars of the same species) to get good berries. Serviceberry (Amelanchier) is a great shrub with white flowers in early spring. The birds LOVE the berries but they tend to strip them bare within days :-) Chokeberry (Aronia) is also a great berry plant that the birds love. Also a native.