Many spring-flowering bulbs obviously fit the bill, so I'll skip those. AS far as perennials, for early spring foliage and bloom I must recommend Pulmonaria (lungwort). There are various species, of course, but I love those with variegated foliage. And the early flowers are a treat (visually for me and food-wise for early bees)as they turn from blue to pink (or is it the other way round?--no matter--there are always some of each happening at once!).
As a sucker for blue-flowering plants, for fall one of my favorites is Plumbago (leadwort), which in my garden began blooming a few weeks ago & is going strong. It would go wonderfully with my yellow fall-blooming crocus located elsewhere in the yard, so I must try to get more & plant them to come up in & among the plumbago.

And of course hellebores are a must! I have them blooming sometimes even in February, sometimes under the snow. The foliage is, of course, evergreen, and as it is very sturdy provides nice winter interest. My perennial foxgloves are good too; stay greem a long time and bloom pretty early.

Then there is "mystery [or "surprise"] lily"--aka "Naked Ladies" -- Lycoris squamigera. The hardy bulbs put out a lot of robust foliage in the spring (like daffodils on steroids - LOL) and then fade away but the real kicker is the way the fat stems (sans foliage!) come shooting up suddenly in mid- to late August and burst into slow pink fireworks for about a week. I love the silly things; they are ubiquitous here in central Ohio and many of the old farm yards (or older small-town homes)have them.

Christmas fern is evergreen, though flattened by snow; ostrich fern sends up edible fiddleheads fairly early and is quite dominant in the shade bed till late (it's still going strong). I have a very large white bleeding heart here when we moved in 3 years ago; I noticed that it keeps self-sowing so have transplanted a number of the young-uns around and these sometimes bloom the first year. It lasts a lot longer in the garden than the wild native in the woods but soon yellows and then disappears when summer arrives (the ostrich fern happily stretches its fronds to fill in the empty space).

Sweet woodruff is a wonderful ground cover with a wonderful fragrance & delightful little white flowers in May. I much prefer it to various other alien ground covers like Pachysandra terminalis, periwinkle (vinca vine), English ivy, etc. I do have a bit of our native Pachysandra procumbens (Allegheny spurge), but it spreads much more slowly than the others.

That's enough--I'll let others jump in! CK

Pansies and Bergenia are 2 things that come to mind when I think about "the off season" when I lived in OH. Primrose I considered "winter annual" I could never get to live through summer. Trilliums might interest you. Anemones. Never got around to actually getting any but remember wanting some heaths/heathers for this reason.

Thanks to both of you, there are definitely some plants I will look up.

Sweet Woodruff - when I looked that up, I found a plant I had seen growing in gardens around Denver but hadn't been able to identify! Thanks for that tip.

I just planted some pulmonaria and also some hellebores!

I wasn't aware Bergenia was evergreen. Recently I declined to buy some because it looked like it wouldn't be drought-tolerant, but if it grows year-round I think it could really take advantage of the early spring and fall to get established. I'll have to give it a second look.

I had Bergenia cordifolia in central OH (5b) and although it's supposed to be evergreen, it wasn't for me, I called it ever-red. The leaves did stay on and they turned a pretty burgundy color. When everything's covered with snow, this plant will probably be under it, but until then it should provide some interest for you.

I left that house during that plant's 3rd spring, so I never got to see how it grew long-term. It may have remained more green in subsequent years after getting more settled in, or that may be it's normal MO in the colder climates. I also think it would have had bigger leaves after a few more years. When I bought it, it was pretty root-bound and had leaves about 5" tall. It did absolutely nothing the rest of that year. The next spring, it grew 3 flowers stalks and a lot of new but smaller leaves. When new leaves came out in the 2nd spring, along with the flower stalks, the older leaves died off. When I left the house in March and that plant was still wearing last years' leaves. I'd say this plant "takes a lot of patience" and hope someone who has had some for an extended period can tell you more about how it grows long-term. No doubt the flowers and their bright reddish stalks are very pretty.

You may also want to look at witch hazel and forsythia. Maybe pussy willow. Generally, you need a woody plant - a tree or shrub which is tall enough to be up above the likely snow cover, to get much of anything interesting going on in the winter. If you include the term ephemeral in your searching, you will probably run onto other candidates that break that general rule.

There are ornamental grasses to consider. If you wait until spring to trim/clean, any leaves or seed stalks that aren't knocked down by snow (or encased in ice) will move in the slightest breeze. Any movement gives a sensation of alive-ness when you look out the window or walk around in your yard.

The Arum italicum have emerged for the Winter, here in Zone 7a. In this acid sand they have never been invasive. In freezing weather the leaves look like cooked spinach but perfect when they thaw.'Marmoratum' is not the only clone, and I was fortunate to find one nursery that offers a form with different variagation than 'marmoratum', for variety sake.
A great favorite is the hardy Cyclamen hederacea, in various clones.
Both plants disappear entirely by late Spring.
If you climate is 8 or warmer, there are beautiful Lycoris whose foliage disappears in Summer. Magnificent flowers.

I have a toothwort, Cardamine diphylla, that comes up in late fall and is green all winter. I grow it under rhododendrons. It blooms in spring and goes dormant for the summer. They grow by rhizomes.

I got my plants several years ago from Plant Delights nursery but I don't know if they offer them any more.

Sandy