I would be more concerned about the apple trees than your juniper. The gall is harmless to your tree but it can be very serious for apple trees nearby. If you don't like it, you can prune out the infected branch, but I personally would just leave it. It's actually kind of cool looking (in a weird sort of way).

Just part of nature. Found one yesterday on a juniper that looks just like the one by M. Kuo.

Yeah, not a big deal. I enjoy finding those things. They're so weird looking.

By the way, I have read that Juniperus virginiana is one of the highest-rated trees for wildlife value. The berries are important food for birds, and hummingbirds favor the fibrous bark for nesting. Loved by many kinds of birds for shelter and food.

I find the trees have lots of character, gnarled and twisty like Italian cypresses. With all that room in your yard, you could put in a nice group of three somewhere. The color of the foliage in wild populations is variable. If you go to the nursery, you can find named varieties selected for foliage that is greener than typical, if that appeals to you.

We have quite a few red cedars around here - They're one of the first things to pop up in abandoned fields. There's a fairly old one in front of our house, and it has a couple galls on it every year. We've also got tons of wild crabapples and a self-seeded orchard apple in the hedgerow (The old thing actually produces pretty good fruit - similar to a jonathan. Might try grafting a few just for kicks.), but they never seem to get the rust. The only thing I don't like is trying to mow around young red cedars - Very prickly needles.

Unless your J.virginiana is the only redcedar within miles, removing it probably will not have any significant diminishing effect on cedar-apple rust infections on apples(or pears, serviceberries, quince, hawthorn)in the immediate area.