Yellow Jackets do not re-use the previous year's underground cavity. There may be one or more over-wintering queens in the cavity, they will fly away in the spring.

I will leave to the lawn experts any advice regarding filling in cavities.

You should know EXACTLY what made the holes before doing anything. More harm is done to the earth attempting to eradicate unidentified , misidentified and harmless creatures than anything else people do in their yards. I can tell you it is not yellow jackets making holes in soil. I doubt if filling holes will make a difference so if it makes you feel better use topsoil or compost from local garden center. Above all else I'd like you to realize everything that creeps and crawls is not harmful or detrimental to mankind. Gardening is supposed to be relaxing so enjoy it while you can or you will regret it when you are old and crippled.

Maybe I should have mentioned i live in southern PA, near MD

If i fill the underground cavity on a cold day, and one or more queens are inside, do i need to be ready for them to come after me in anger? or are they more or less hibernating? should i put soap or vinegar or anything in that cavety first?

I cannot be sure they are yellowjackets but they have yellow and black bodies and resemble a yellowjacket but im sure the amount and size and sequence of the body stripes matter and i didnt get a close enough look. thankfully they didnt get me when i mowed over them this summer. but that was once as i then stopped when i located them. and no harm came to the kids as its not in the center of the yard but it did significantly limit the yard where kids play.

a bit off topic, we also had cicada killers in a different part of the yard. they also caused no harm to any of us and i researched how beneficial they are but id rather them locate elsewhere next year. any recommendations? (i can start a new post too if thats better)

Mr. Ro wants to fill the underground lawn void regardless of the insect species.

Ground-dwelling Eastern Yellowjacket has the same life-cycle as many other yellowjacket species.

Any over-wintering yellowjackets will be sluggish, especially on a cold day.

But it may be warmer underground; if you pry up some sod to access the cavity, they may briefly be able to fly. Have a can of wasp killer handy. The queens may have already abandoned the underground nest and are over-wintering in another protected location.

I have had ”ground hornets” - not sure what they are called - in the garden beds as well as in the lawn. For the garden beds I bought some wasp and hornet killer which is some kind of foamy stuff that you squirt into their hole at night. It worked! When they are in my lawn I mark their spot with a flag and never get around to spraying them. They have always disappeared once it starts getting cold out and they have not come back to their previous hole the next year. I just put dirt in their hole, after I know they’re gone, and stomp it down.

it sounds like this problem should solve itself and they are unlikely to return.

what type of dirt should i put into the hole? i assume i need something that isnt too loose or something else might access the cavity later this summer

I just use the cheapest bag of soil I can find to fill the hole. In your case I would peel back the grass layer on top of the hollow sounding part to see how big the hole is, so I could get a better idea of the number of bags of soil needed to fill it in. I also put some grass seed down where needed. This past summer the skunks demolished the nest, which was also another reason I didn’t spray. I hate spraying

Here’s the stuff I use, when I need to.

I'd just use soil from elsewhere in the garden.

I heard once that putting a clear glass bowl over their entrance confuses them and somehow results in their demise. The only thing I remember is that they don't try to build a new exit, because they can't figure out the problem with the glass domed one.

(For dealing with bees: those thick, white Mr. Clean gloves do a really good job protecting hands against honeybee stings.)

That’s very interesting annpat - I’m going to try that next time

ID them if you can. Not everything in the yellowish/blackish category of winged bug is a stinging bad guy. Yes, there are some ground nesting stingers. But a lot of ground dwellers are fairly harmless, many don't sting. And have a good spot in the ecology of the garden.