I don't know where you live but a multi grafted espalier apple tree would fit in nicely there. Check out your local nurseries.Some do carry them allready growing well. But before you do that if you don't know much about apple growing. Rip out that tree and start now reading past posts on growing and caring for apple trees. This is a great place to read plenty of old posts about growing apple trees. Then by next spring you will be prepared to grow again. My espalier I bought at a local nursery has four different apples on it and fits snugly in a small area of my veg garden.

Thanks everyone for the replies. @ken_adrian that is the thing that had me alarmed this particular apple tree usually does not shed its leaves until around mid-September. So I knew there was something wrong with it. After some consideration I think the brown spots was my doing :/ I tried to spray the tree with some "natural" bug killer since there was an unusual amount of wasps circling it, and I think I might have sprayed a little too much. @i_like_pi it does look like it has Cedar Apple Rust. The neighbors have a crab apple tree in the back, and it looks like that tree has the same growths. So I think that tree might have passed the cedar rust on to mine. I plucked off all of the leaves with the growths and so far I have not seen any more growths pop up. So I think I caught it in time. Once again thanks for the help.

Paint the trunk with some old latex (white or yellow will do) paint it looks like sun burn to me, which can happen in the winter as well, when the sun is low in the sky. Sorry but I would pull that gravel away from the base of the tree out to the drip line. Rock holds the heat near the roots and reflects even more sun on to the trunk. The tree will thank you. Camp Verde

Sorry, but the two issues are not at all the same!! Rick, your apple blossoms are just showing some weather damage......nothing to worry about and nothing to treat. Fwiw, a great many early blooming trees and shrubs will produce flowers that show this sort of damage....typically just the result of a heavy rainfall. We see this all the time in our wet PNW springs......and we grow a heck of a lot of apples here in WA state!!