The fruit pictured in the 4th image is from the rootstock, (tri-foliate orange or one of its many offspring), that the budwood was grafted onto. If you want to have only the "desirable" citrus, you should prune the rootstock sucker off. These are the branches that have the tri-foliate leaves as pictured in the 4th image. If you are unsure, contact one of the master gardeners in your area for assistance in correctly pruning this tree. The tree may look lopsided for a while after this pruning, but it will even itself out after a few years.

agree with the trifolate leaves on the right side. the previous owner prob didnt know to prune off the suckers below the graft.

i would cut off the right side. and pluck off any new suckers that start to grow below that split area.

Okay, thanks for the info. Did some more reading, I wonder if the tree was bud grafted. As the right side looks completely straight. The normal fruit left side actually looks like the off shoot.

So I guess the Hardy Oranges, or Trifoliate Oranges, don't really taste all that great. Kind of sour by what I read. I think I'd enjoy a more enjoyable fruit that can eaten right off the tree, instead of using them for marmalade or candying.

Do the last 2 images look like any certain fruit, or is it too early to tell? I thought maybe the tree was a lemon tree, but if the rootstock is a trifoliate orange, then maybe they're oranges?

I think one of my other trees may be an improved Meyer Lemon tree. As the remaining fruit when I moved in looked round and the color was between lemon yellow and orange, but had a hint of both lemon and orange flavor.

It is too early to tell what kind of fruit it is, but oranges, even at that stage of development tend to be rounder, so I suspect it is either a lemon, (most likely) or a lime. The meyer lemon is usually rounder than a true lemon, but should retain the lemon yellow color of a true lemon. The fruit you described may be one of the many varieties of mandarins, or a citrus hybrid; but without actually seeing a mature fruit, I wouldn't even try to guess.

But I've read that Meyer Lemons turn a little orange color when they're ripe. The other tree had some fairly round fruits. I also have a regular lemon tree that had some fruit on it. My suspect Meyer Lemons were right in between the color of my lemons and oranges. They weren't yellow, and they weren't orange.

Would it be common to use a trifoliate orange rootstock with a lemon tree?

Here's a picture of the whole tree in the images above. Once I chop the trifoliate orange, there won't be much else left. Maybe I'll rip the whole tree out and plant an apple or pear tree. :)

Is it normal for this time of year for some of these trees to be getting yellow leaves and having them drop? See if this picture, some are all yellow and they just fall off. Some of my trees have curled leaves too. I wonder if they're starved of nutrients.
{{gwi:588864}}

It's not abnormal for them to drop leaves this time of year. You could replace the tree with something else, or prune off the tri-foliate branch adn wait a year to see what happens. It does look like a lemon tree, and yes it is common to use tri-foliate orange for rootstock for many different varieties of citrus as it induces early ripening, resistance to some diseases, adaptability to certain soil types, etc.