Maybe get to know your neighbor a bit better. Or say nothing and put up some stakes with flagging tape - nothing more passive aggressive than that.

Id be inclined to buy some inexpensive, hardy plants and dig them in along the boundary. If they're halfway decent people, they won't want to trample them.

Maybe your yard looks bad enough that they think you don’t care. Get a sprinkler and place it where it hits the area they play in during the time they play. Then you take care of two birds: the yellow grass and them. If you can’t afford landscaping plants plant a garden. Seeds cost very little.

Do you actually care if they play in your yard? I always let the neighbor kids run through my yard, etc. I just told them areas they had to stay out of or if it wasn't a good time for them to cut through because I had company or something. If it was a particularly good strawberry or raspberry year, I'd specifically invite them over to pick berries for their families. If they're not damaging anything or doing something particularly risky (like using your swimming pool without permission), then I'd let it go. Mind you, I'm pretty easy going about these things - and always pretty popular with the neighbor kids even if I had to tell them to quit squabbling or to pick up a candy wrapper once in a while.

I would resent the presumption as well as the invasion of privacy. Perhaps
mow your yellow lawn as low as possible, throw out some inexpensive
grass seed and some little signs: "Newly seeded grass, please stay off."

There are also small, inexpensive temporary fences available at big box stores or online that would define the property line, which, along with the signs will make it clear.

Aw geez, lighten up already. People playing in your yard? Really?

If they had a big dog who dumped in your yard and they didn't clean up, you'd have a problem.

I can very easily imagine in a "family friendly" neighborhood, people treating their backyards, especially if they are not particularly cultivated looking, as semi-common property. A large percentage of families care much more about their kids than their yards. I grew up in such a neighborhood. Kids cut through the yard and sometimes would play on our swingset (which was home-fabbed and superior) without asking. Nobody seemed to care. But these were the days before everyone sued everyone else. Since those days I have become considerably more concerned and defensive over any yard I've cared for, but also quit living in family friendly type neighborhoods.

I would have second thoughts about planting any low-cost shrubs even on a temporary basis. Those are usually the ones that grow super fast and tend toward being weedy or have other undesirable traits. A row of corn, castor beans, or something that will die at summer end, would do the same thing and be a lot easier to get rid of.

It would of been nice if he asked your permission. We have a neighbor who assumes it's ok without permission, but his yard is "no touch ". If it bothers you I would say something to them.

My $.02 - you moved into a family friendly neighborhood, kids are always playing outside. I doubt this will last forever. If you want to make enemies, say something, if not, let it go, chances are they’ll stop once we get into winter, or the kid gets into something else, etc.. By the time you get that fence up they probably won’t even be out there anymore. Nothing worse than having an awkward situation with people you have to live next door to. You might need them someday. Pick your battles. ;)

We had a woman move into the house next door. She complained about the noise the kids made at the basketball hoop. I reminded her the hoop was there when she bought the house and Florida is full, full, full of 55 and over communities where children would not have been an issue. I would hesitate to make a big deal, but, I would say something about potential damage to the siding the next time the ball hits your house. A simple, “Oh, my goodness, it startled me awake from my much needed nap!” Or, “ As your cute little boy gets stronger, my siding could get dented and I know you’re raising him to be thoughtful. So, Dad, maybe you can show him the boundaries?”.

How big are the lots in question?

I bought a second/vacation home in a beach community, so the lots are much smaller because the land is so valuable. We looked at so many homes that butted up so close to each other that the backyards all run together.

If you're in a place with small yards and they're all open....maybe they're playing partially in yours because they need the extra space for the game or the balls?

I agree it's rather presumptuous of them and I would not personally ever do it - but everyone is so different. I see kids in my beach neighborhood playing ball out behind houses and none of them are paying attention to where the lot lines are.

Without talking to them it's hard to know what their motives are. I wouldn't assume it's anything malicious. They certainly shouldn't be letting their balls hit your actual house - but again, if these are small lots occasionally mistakes and errant throws are going to happen.

I would plant some plants ( 2 or 3 scrubs) in property line, in the middle.They would understand, they are not welcome to play. And it is without arguing.