You should leave them outside, as Henry says, unless the temperature should fall to -3 C, or possibly -2 C with a clear sky. Zero is no problem. Exposure to cold and bright sun is useful to help the plants, which were forced in a greenhouse, to adjust. Don't plant outside until they are safe from hard freezes.

Just a word of advice about locating the tree peonies. It sounds great that the plants will get early morning sun but it causes problems in late winter. What happens is the sun hits the stems and warms the plant up. But when the air is freezing only the side the sun hits is thawed. The other side remains frozen. The area in between is the problem in that the cells are trying to thaw and freeze at the same time. In this area plant cells can burst and cause scar tissue damage the layer that provides nutrician to the plant weakening the plant. If you can provide shade especially on the lower parts of the tp during the winter it is better for the plant. The shade can be as little as a cloth banner or latticework placed in front of them. Don't give up on the scent as sometimes it is so overpowering that it is nasty when the flowers first open. Of course to my nose almost all of the herbaceous corals smell like re-hydrating dried fish. Very beautiful to look ate but please don't ask me to stick my nose anywhere near them.

Are your peonies herbaceous ones or a tree peony? I think your soil will be great provided you also add in the blood & bone & well rotted cow manure. This is what I did for my herbaceous ones. They are heavy feeders and I tried some in pots to start with while I was preparing the beds. They like to be raised up for good drainage. I firstly dug down and added handfuls of blood and bone & lime then covered it with good soil/mulch or potting mix. Then I planted the potted peony on top, so it was still raised above the soil by about 12 inches. Then I covered the sides with good mulch. I planted in Autumn as they are least disturbed at this stage. The cow manure was a bagged variety so I assumed it was fairly old and sprinkled this on the top and watered well. Last year I did have a couple of flowers from the ones I had planted. They like a really cold winter for successful flowering. Someone on the gardenweb tried ice-cubes for a few weeks but as I recall their efforts produced no flowers. They like full sun. I think you have to be patient. The autumn colour of the leaves is also very attractive. I'm still keen to give them a go. Good luck.

No problem! Especially since you had buds already---if it's still growing cut it off as soon as you can. Once your weather is such that you won't see new growth again this fall you might want to spread a bit of bulb fertilizer around the immediate area and scratch it in, do the same thing when you first see growth in the spring. I don't know how much benefit the fall application will provide but several forum members that I trust do it and seem to have amazing results. I put it on my peonies last spring and several of my established peonies bloomed bigger than they ever have so I'm a convert--spread my fertilizer for the fall already. The one recommended to me was an organic one called BulbTone. I used it and am really happy with the results, would certainly recommend it. There are several types of BulbTone, the one for bulbs is the one you'd want, or any other (preferably organic since it should help improve the soil, at least a tiny bit) bulb fertilizer.