My best guess is osage orange. Are there are thorns on the twigs or giant green fruit-balls underneath?

The tree looks similar to the invasive Chinese Tallow Tree (Triadica sebifera). Take a look at this link for further identification. If it is a tallow, you might want to consider getting rid of it before you and your neighbors develop a Tallow-tree jungle!

Here is a link that might be useful: Invasive Species Tallow-tree

Habit and leaves more like Maclura.

Yes, I'm pretty sure this was part of a huge grove of Maclura pomifera (Osage Orange)!

Here is a link that might be useful: Osage Orange