I would suggest finding information on what is native to your location. What is "Native" varies widely depending upon where you live. I did a google search of "Wildflowers Ontario Canada" and came up with a number of sites. I did a similar search for "Native Plants Ontario Canada" and came up with some hits including this excellent list of resources:

Native Plant Crossroads

That said, I'm guessing that we have many natives that are similar since I live just south of you in Western NY State. I have found that the areas where the native goldenrod, asters, and milkweed are well established and have made colonies, the garlic mustard doesn't get a chance to grow. I think these plants grow too tall and thickly for the 1st year garlic mustards (which tend to be basal rosettes) to get the light they need. Or probably any other resources.

As far as removing stuff, I wouldn't at this point. But if you want to, I find that right around your house is a tough location for natives and many of the ornamentals do much better there.

FataMorgana

Ajuga and Japanese Pachysandra can get a little invasive so watch those. There is a north american pachysandra that I happen to think looks better than the standard landscaping (Japanese) pachysandra but is, no doubt, not as hardy. Keeping out garlic mustard will require some labor on your part for at least a few years. Some aggressive natives like Canada Goldenrod should help to compete with the garlic mustard but you will still have to weed some out by hand for a while.

From some gardens I visited, Cornus alternifolia does well in Ontario so I would add more plus more amelenchier. Unless you're a purist I see nothing wrong in using cultivars. Acer pensylvanicum I would recommend as well as Cornus, viburnum and Ilex shrubs plus other sumacs.

Garlic mustard will sprout in quite dense woods, so It will take a few years of vigilance to get it eradicated. Pachysandra procumbens is hardy, but will not be evergreen in a cold winter and it is a nicer plant than the Japanese.

If you have a deer problem anything in the Lily familyis on the top of their list, e.g. Trilliums, Solomon Seal, Hostas.
There are a lot of wildflowers native to Ontario including orchids.

I don't think there is anything wrong with using a few cultivars or non-native plants, as long as you choose plants that won't spread to the wild. To be safe I'd use only those non-natives that have been in cultivation on this continent a long time. For example, lilacs don't spread to the wild as far as I know, and they've had a hundred years or more to try. On the other hand, new introductions like Russian Sage - Perovskia spp. - are new to this continent and I think will be recognized as invasive of natural areas within a few years.

It's your yard, and you should plant what you like as long as it won't negatively impact the wild areas near you (that is, as long as it is not invasive.) Nevertheless, I think you may find, like many of us have, that as you get more into native gardening you prefer natives and sometimes wish you hadn't planted non-natives, especially those that live a long time and take up space where you could have one more nice native plant if the exotic one wasn't there.

Garlic Mustard will sprout and set seed in shade, under shrubs, and even within blackberry thickets. To get rid of it you'll have to diligently stop it from flowering and seeding for a number of years in a row. After the first couple of years you'll see fewer and fewer new plants, but you'll have to be on guard for a long time. You can pull, spray, or smother the plants, but it will take some work.

Raspberry, on the other hand, is native but grows a bit too rampantly to make it welcome in many yards. It will not grow well in the shade, so once you have trees and shrubs established you won't have nearly as much raspberry. Also, as others have said, establishing thick stands of other plants like native grasses and wildflowers will help slow down the raspberry and the Garlic Mustard. Raspberry grows from a system of underground roots or stems, and when you pull or cut a patch a lot of the underground material remains and will quickly resprout. After a few cuttings or, especially, pulling of canes, the underground part will diminish and you'll have a lot less resprouting.

Thanks all for some good advise. I must admit I have been struggling with how to proceed this spring. I think I am more in the middle like ladyslppr in that I will try and use a lot of native material but there are just some cultivars and exotics that I like so much and want to use. I will look into the Pachysandra procumbens suggested. Evergreen groundcover is not an important issue since (hopefully) the ground is mostly snow covered back there most of the winter. I also didn't know that there was a native holly and have looked that up too and will likely add some of that. I think the approach I will try and take is stick with native deeper into the wooded area as this is closer to the nature brush area anyway and limit the ornamental to the "yard" area. Hopefully this will satisfy my need for variety and yet also give me a challenge of trying more native material.

As for the garlic mustard and raspberry canes (and grapevine) yes I know I have to keep at it. I'll never get rid of it entirely as it fills the areas not on my property so will seed regularly. Hence I was trying to choose some 'dense' groundcovers like ajuga for certain areas where it creeps in quite heavily. Almost like creating a fire wall. Perhaps not the best approach based on your suggestions so I'll rethink that as well.

Thanks again

Sharon