Thornless blackberry plants love sandy soil. Fuyu Japanese Persimmons are a small size, very delicious fruit.

I'm in the process of gradually doing the same thing in my Northern VA front yard. Your ideas sound like an excellent start.

There is a very good Edible Landscaping nursery in Afton, VA near Charlottesville - not too close to you, but likely a reasonably day trip. They are at http://www.ediblelandscapin.com - they do mail order, but if you go yourself you can hand select your plants, save on shipping, and draw on the extensive knowledge of the owner (at least during the week).

For fruit trees it depends on what you like to eat the most. Most varieties are available on dwarf root stock, which you should be able to space 8-10' apart.

I planted Lingonberry and Wintergreen between my blueberry bushes - they grow slowly, but make pretty groundcover year round, happy in some shade as an understory plant, and are edible, too! I'm also interested in trying some wild ginger, which is native, though I don't have it yet.

You can get bush cherries, and bush varieties of filberts (non-native), both grow into fairly large shrubs.

Artichokes die back in the winter, but are very attractive in the warmer months, and are perennials you can grow from seed.

There are also hardy kiwis which grow on thick, sturdy vines (similar to wisteria, but they don't grab on as aggressively). They take a fair bit of space, heavy pruning, and at least 1 male plant to pollinate up to 5-6 females. I prefer their small, smooth-skinned fruits over the hairy ones in the grocery stores. They fruit prolifically, and yield kiwis the size of large grapes - you can just pop them in your mouth without peeling!

So far in my yard I have Sunshine Blue blueberries (tolerate less acidic soil than most blueberries), Hosui Asian pear, a non-astringent Asian persimmon (Shenko?), a contorted filbert, and a bush filbert (don't recall the variety) in my front yard. I will add another small tree (Asian pear or sour cherry, I think), and a pair of hardy kiwi. I'll also start new Artichokes - I let the ones I planted in 2007 get overgrown with grass, and they didn't survive last winter.

Sorry for the lengthy post - I hope it helps a bit! Enjoy your project.
-Leah

Remember, one of the keys to this sort of thing is keeping it attractive as well as productive, especially if you're in the sort of area where neighbors, homeowners associatons, or local government is likely to complain if things aren't Stepford Wives cookie-cutter same old-same old yews and hostas. So, keep in mind that what you plant should be colorful, interesting, not too messy, and look attractive throughout the year.

There were some excellent suggestions above. Persimmons, pears, apricots, quinces are all very attractive looking trees. Starks and other nurseries sell columnar-shaped apple trees which work well in landscapes. Many small fruits like blueberries, currants are quite attractive. Consider adding in some of the more attractive and colorful veggies -- pepper varieties which bear colorful fruits, especially the smaller-fruited types, eggplants, kales, leeks, lettuces, etc. Keep the more messy and more sprawling things for the backyard.

speaking of edible landscaping in Afton; they have this miniature peach tree that does not require spraying-though I didn't believe it (I had one, but gave it to my neighbor, which I now rue to some extent seeing how well it is doing). Might get another one, if I could only figure out where to put it (getting down to lingonberries & herbs, at this point).
Might you want to consider edible weeds (e.g.-purslane)?

Dennis already touched on HOA problems. One of my friends has been able to grow zucchini in the front yard for several years, and he has a rabid HOA.

Besides zucchini, veggies that can reasonably pass for decorative plants

- flowering cabbage (the most delicate of edible cabbages)
- chard, all types
- runner beans
- cardoon

in compositions, cabbages, carrots and lettuces will manage to look good in a bed.

You have far more choice with fruits (and I am thinking only no-spray stuff). Blueberries are an excellent choice. Aronia is even better, but it is too warm where you live (berries will be bitter). Hardy kiwis are a wonderful, no spray plant that will make a great arbor. Persimmons and pawpaws are also small, handsome trees. Chestnuts are large, magnificent trees. Mulberries are a great shade tree.

Another thumbs up for edible landscaping. get their catalog and browse and daydream; it's a lot of fun.

I am too cold for a pomegranate, but if I lived in your zone, I'd get one of the hardier ones and try it (like kazaki).

I have an amalanchier and it's very beautiful (aka serviceberry, shadblow, juneberry). I would plant more of those if I had more space.

I think nectarine trees have lovely spring blossoms (pink). could easily pass for an ornamental.

If I had the space I would definitely get a paw paw.

Also, if you get everbearing raspberries, you can opt for one late crop and cut them down to the ground every year, so no messy canes to look at all winter. Otoh, some of the crimson-purply frosted-looking canes can be quite lovely. Perhaps theres a thread somewhere about which varieties of raspberry have the best winter beauty. Raspberries are just so easy; I would never be without them. I have had great luck getting them (and triple crown thornless blackberries) thru mail order from nourse farms in MA.

I definitely second the suggestions for paw paw and persimmon. They are attractive trees and low maintenance. Asian persimmons will produce fruit without a second tree for pollination. Paw paws require two trees for pollination, with the exception of the variety 'Sunflower', which is self fertile.

Alex