might be gumi or elaeagnus multiflora.

Here is a link that might be useful: gumi

Also, Autumn olive(Eleaganus umbellata) has red fruit usually and can be alright tasting. Here is pic of couple colors of the fruit
{{gwi:445587}}

I stand corrected. It is a gumi, Elaeagnus multiflora. Locally it's called Russian olive which I see is incorrect.
The link states that the're hard to get started from seeds but here locally the birds have spread the seeds extensivly.
I have ocassionally eaten the fruit from other bushes but this one is unique.

I went back to show my son the gumi bush and we tried some fruit. It was more astringent on Tuesday that on Sunday. We had a couple of nights of rain. I don't know if this affected it or what happened but it's not a great a find as I had thought.

you probably were hungrier the other day

That could have been. I had just finished a project when I happened upon the bush and the berries were sweet and juicy.

I was eating some of the berries of Autumn Olive (Elaeagnus umbellata) the other day, as I was cutting them down (to clear out a border and remove invasive plants). They are tasty when fully ripe, a mix of tart and sweet, and are supposed to be rich in Vit c and lycopene. Perhaps you stumbled across a plant that was genetically sweeter than most?

It's a very invasive plant in Mass. so I remove it from the property, but it's too bad since it's a pretty shrub (unlike Buckthorn and Burning Bush) has nice fragrant flowers, and the berries are edible.

Gumi/goumi fruit is ripe in midsummer, not September.

I just ate some of the red & orange berries last week in Butler Pa. Was with an older Polish guy who called them Russian Olives. The orange ones were generally sweeter. Tasted like Sour cherries with a slight alum bite. So r they Russian Olives or Gumi. I bought a bush from Gurneys several years ago that produces loads of berries in the spring. Twice the size of the fall berries from last week.
Looked exactly the same, bright red with gold flecks. It was advertised as some kind of Russian berry but didn't write it down

I have 5 Russian Olive trees and would not eat the fruit until it is processed. Too puckery.

I believe the berries I tried were gumi berries.
I had a pinch of chewing tobacco under my gums when I tried them the time they were so juicy and sweet. I tried them with out the tobacco later and they were the same as the others, juicy and sweet but with a mild astringentcy.
At first I thought I had discovered something new but it was not so.
Although from posting this I have learned a lot about them.

Check out buffalo berries. The fruit really looks like the autumn olive trees above, add a nice touch of blue/green/silver to the landscape, and produce wonderful small red berries like those pictured above. When ripe, they are sour like a pie cherry or grapefruit and have a hint of grapefruit flavor and a small seed you can swallow or spit out.

Their main drawback is that they are very thorny and picking the berries results in numerous pricked fingers. Best to wait until just after a good frost, then lay a tarp under the tree and shake it good to knock the berries off. It is a native, not invasive, and very hardy with few pests.

Like alexander has brought up, gumi fruit bears in the summer, not the fall. Autumn olive is the most likely fruit that was ate.

I believe your right it is autum olive. I googled autum olive and believe this is the plant and fruit.
Locally everyone calls it Russian Olive which I'm sure is incorrect.
I want to thank everyone for their input. I can now speak intelligently about this invasive species.

Yea ! if you can stand seeds the best way to eat them is to get a hand full of them when it takes just a little bit of efort to knock them off the branches. (when ripe) then chew the seeds and all, its almost like the breaking of the seeds mellows out the bit factor of the berry witch i kinda like . If you think that jams and stuff are good and your a wine drinker AutumOlive wine is the best ever ! i made 5 gal last year and after it sat about 8 months it was unreal how good it was. not to mention about 12% alc to. if interested ill send my wine instructions to you.