Dwarf Ornamental Crab Apple
Mugo Pine
Siberian Iris
Japanese Iris (mulch good for the winter)
Hostas would look good too.

Sharon

Peonies, too, i think.

Varigated ornamental grass would be nice.

The hardy sambucus racemosa 'sutherland gold' elder would work too, though depending on the size of your bed it might grow too large (or need annual trimming). Maybe coppertina ninebark? I find the elders like 'black lace' basically die back to near the ground here, but that makes them great smaller size shrubs for me.

The falsecypress might do okay for you (depending on your location, they are borderline hardy in zone 3). I have one called sungold and one called curly tops. Maybe euonymus fortunei varieties like 'emerald gaiety' too.

Japanese blood grass would be striking, but treat it like an annual in zone 3 (and it's a bonus if it survives winter.) One hardy grass that I like is bulbous oat grass. Then again blue fescue is great too.

The bonus is that, if you are into Japanese-type ornaments for your garden, the stores really seem to have a lot of variety this year.

Good luck with your project, It's fun to go with a theme and try to achieve it.

Glen

Itoh (aka: Intersectional Peonies) that are a cross between herbaceous and tree peonies. Expensive but they would look awesome in a Japanese theme.

Another shrub suggestion that will give that Japanese Maple look is Sorbaria sorbifolia 'Sem'. They just got them in at our local Canadian Tire today, so I bought one. ;)

I have the regular species, but this one has pink/red/bronze shades. It's just gorgeous and should be very hardy for zone 2.

Here is a link that might be useful: Sorbaria sorbifolia 'Sem'

Thanks everyone for your suggestions!!!!!! I am excited to get going on this new bed. DH bought me a weeping crab, so now I'm anxious to get digging :-)

How about "Blue Moon" or "Lawrence" wisteria? "Blue Moon" is good in zone 3. "Lawrence" in zone 4. Have you tried tree peonies? They are definitely oriental looking. I have 9 tree peonies that have survived for years in zone 4. You might incorporate Japanese maples, clump bamboo and bonsai some trees that you can keep indoors in an unheated basement corner during the winter if you have such as place. Gambate kudasai!