I'm in zone 4/5 and have a Crimson Pygmy Barberry (Berberis thun. atropur). I had it since 1986; it's completely winter hardy and is bright red in the summer and slowly turns burgandy in the fall. Mine is flanked by two miniature golden cypress...great color combination!
I believe the crimson is also available in a standard size.

Thanks Schenley but Barberries are Deciduous. I'm looking for something that doesn't lose its foliage in the winter.

Well... if you're willing to give it some protection, there are different pierises that have new, bright red growth that will slowly mature to light green and finally dark green. Some cultivars are slower to turn than others. "Mountain Fire" is often noted as one of the redder ones.

I am trying a little leucothoe fontanesiana "Girard's Rainbow" this year that also has reddish new growth that matures to a varigated mix of dark green, white, and pink. It is supposed to be hardy to Zone 5.

That sounds like the 'Vanilla Swirl' leucothoe I got from Wayside a few months ago. Are you happy with it? I'm not sure if I'm terribly pleased with mine, but I think it's just a matter of getting used to how incredibly different the new growth looks from the old. DH says it looks like two different plants grafted together (sigh!) I'm hoping once it gets bigger he will have gotten past that. I think the problem is that I'd like something like a red-leaved Rhododendron or Spruce or the like, which don't exist (yet?)

Well... with all the rain and cool overcast that we've had this spring, I haven't had to water my leucothoe, so it has been sortof maintenance free. And it has actually thrived. I just think it's a neat little shrub (with a common name "dog hobble" LOL).

I don't think that there are any red-leafed rhodies, although there are kurume azaleas that have reddish leaf coloration through winter. I'm not sure of conifers explictly reddish, although there are golden ones that are pretty dramatic.

Other than that, the pieris I recommended, either one like 'Mountain Fire' or other "-fires", have that intense red new growth and they'll vary with respect to how long the color hangs around before it matures to dark green. I'm sortof a pieris junky. LOL

Some thoughts...

The two leucothoes, 'Gerard's Rainbow' and 'Vanilla Swirl' are different plants. 'Girard's Rainbow' is L. fontanesiana and 'Vanilla Swirl' is L. axillaris. I have several for the former and they are good, sturdy plants although really gangly and open to about 6 feet tall. The difference in the foliage color between new and old foliage on 'Girard's Rainbow' is not as dramatic as it sounds like it is on 'Vanilla Swirl.' All the leaves are mottled with olive green and cream and then the new growth looks like a coppery red wash over the top of those colors.

You might consider Leucothoe axillaris 'Scarletta' or 'Red Lips' which are two dwarf, red leaved varieties. Said to be hardy to zone 4. I have 'Scarletta', and, unfortunately, it doesn't color-up well in the shaded location where I've planted it--but it should do better with more sun. It is a compact, nicely shaped plant about 18'X18'. Has red new growth which gets more burgandy in the winter and then the leaves fade to dark green so it looks kind of speckled with red leaves.

I like Jenny's idea about the Rhodys or Azaleas. You might try a rhody with really red petioles.

Also, check out the following link with many Euphorbias that have red/reddish foliage (at least at some point in the season). Some are even hardy to zone 5!

Here is a link that might be useful: Euphorbia Link

You have presented some highly interesting options - I'm even rethinking my original opposition to the Barberry (kind of overdone in my area); perhaps a Rosy Glow used in an unexpected spot.... I am also now contemplating how I might work the Mountain Fire pieris, leucothoe fontanesiana and kurume azalea into the scheme of my garden (original plan called for a single red 'evergreen' LOL)! Thank you ALL so much! Feel free to suggest more if there are any that have been missed - I'm on the very edge of the Lake Erie-induced Zone 6 so that might give me more options than the local nursery growers are comfortable selling to the general public.

How about a heather- calluna vulgaris or heath- erica carnea. I'm sure there are many varieties that would do, but a nearby catalog describes three: 'Winter Red', 'Wickwar Flame' and 'Firefly' as having red foliage in the winter.

There's a leucothoe variety called 'Red Robin' with red stems and new growth.

Heavenly Bamboo or nandina:'Firepower', 'Harbor Dwarf' and 'Woods Dwarf' are supposed to become different shades of red in the winter.

I suspect there would be suitable varieties of either Heuchera or Tiarella to bring that color into winter.

Happy searching!

I also thought of sempervivums- hens and chicks, there are many different color selections, some quite bright.
I think bergenia gets reddish during the winter,too.

Have you considered red twigged shrubs as a source of winter color? Particulary, one of the brilliant red-twig dogwoods or willows?

Good design usually repeats itself- so you may need a few splashes of winter red here and there to pull it all together!