Aquilegia, Columbine. Nice plant.

Usually peonies are moved and planted in the fall, though if they aren't bare root, they can be moved early spring. If you need to move it this spring, take lots of soil with it and don't damage the one leaf stalk. Here's a page from the U of RI, so relatively close to you, about care of peonies. Here is a link that might be useful: URI peony care fact sheet

I'll repeat what others have said about waiting a year before doing anything. You don't know what hasn't come up & what you like/don't like right now may change once you get settled in. You should go to your settings & include your zone & at least what state you are in to give others an idea. What works for me living in south Arkansas may not work for someone in Georgia. I personally do not like trees planted near enough to a house that in a few years the roots cause problems with the foundation & septic system (been there, done that). I do like shrubs like azaleas that are green all year & then pop with color in the summer. Do your big stuff first like trees & shrubs, then work the small plants like flowers around that. A local mom & pop nursery would appreciate the business & could give you a landscape planning overview, maybe give you a discount on your order if you buy from them. Before contacting anyone like that, go to your local library & check out the books they have on what grows in your area. Also visit nurseries in the area. You'll get an idea of what grows in your area. Go to your local county or parish (if you live in Louisiana) extension office. They'll have tons of information on everything you can think of for vegetable, flowers, ornamental grasses, etc. And everything is free. Write down everything you want, even if the odds are you won't get it. Make a trade list here on GW & on any other gardening sites or Facebook gardening groups you are a member of. You never know who will have something for trade or postage. I really like that pergola!!!! That looks like a great place for a barbeque grill & furniture for outside gatherings. The stepping stones - remember that if they're sunk into the ground you should probably be able to mow without bothering them. This may be your first home, it may be the only home you ever have so you'll want to make sure the regular maintenance you have to do isn't any harder than it has to be. As for a herb & vegetable garden, start small with the things you guys like to eat & just keep building as you can.

The first is pulmonaria and the second might be a coreopsis, both of which would ordinarily be planted intentionally. As garden flowers. The third does look like an artemisia - which also may have gotten there as an intentional planting - and the last is lesser periwinkle, as far as I can tell using the two not very close views of it. The periwinkle is sold and planted for decorative use all of the time but can make large persisting patches in places where this is not wanted.