Many early buds will survive in a typical NY winter and others will not. Buds that remain tight and near ground level typically are less affected, but ones that grow too quickly upward will often suffer later, often at the expense of seed set etc (cold can damage the sepals visually and also the sex organs etc). It won't harm the plant if it happens. Much of it is genetic and simply bad timing by certain plants. It is most common with a few species like H. bocconei and also white hybrids from H. orientalis etc. Some plants (for example those of the same species from lower elevations and/or near coastal areas) bloom early in the wild and to some extent these plants start earlier in the garden regardless of weather conditions. On the other hand, plants are very early this year for me because of this spring-like winter.

agreeing with robeb here. Let nature be nature. It knows more than any human about what it's ready to do and not ready to do. It'll drop buds and flowers until it decides it's time to make a pepper, and then it will.

Instead of the bottom, drill around the circumference about 2" off the bottom every 4-5 inches. Holes in the bottom allow water and fertilizer the ability to drain too fast. I have some jalapeño plants that are mild and others that are hotter than any other jalapeño I've tasted. A golden cayenne feels as hot as my habanero peppers. I think it is the professional growers know/understand the care aspect better. I water every 5/6 days normally, but super dry here so every 3 days I flood them and protect them from too much sun. The super hots tend to be more sensitive in my experience. The standard cayenne and pepperoncini seem to just take care of themselves as long as I stay out of their way. My bhut and scorpions are very very hot but smaller in size than many people have in better climates and longer growing season. By the way, 5 gallon buckets from HD are much bigger than 5 gallon landscape containers. Since you're in HD anyway, they sell earplugs which are a great asset when the complaining starts ;)

Limiting the volume of soil limits the size of the plant. Limiting the size of the plant limits the amount of peppers it can sustain. Fish fertilizer is very high in N relative P and K and this triggered the plant for foliar growth, but too much N can prevent the plant from switching to setting fruit when it normally would. People who choose to overwinter generally do so by yanking the plants out of their pots, cutting both the root ball and the top growth way back, sticking that into a smaller pot for the winter, then repotting into a bigger pot for the growing season. Your chosen pot might do for a slower growing C. chinense like a habanero or a reaper, but cayenne varieties grow fast and big and need a lot of root for roots. Worm castings contain very limited amounts of nutrients. Both they and bone meal will break down in potting soil, but it will take a long, long time because both depend on microbial action to do so. Microbial activity is either limited or missing entirely from containers due to the highly fluctuating environment. The container gardening forum here has lots of discussion about good choices of container fertilizers, both organic and synthetic. The general consensus is that the purely organic route is possible but takes more effort and money. I have similar problems with tap water. If you fill the watering can, bucket, whatever and let it sit out, uncovered, for a day or so a lot of the bad stuff will offgas. Cheaper than buying bottled water, which is way less regulated than your tap water and may contain who knows what... Good call on the garden soil, though. The stuff tends to compact in containers and will starve the roots of oxygen. Epsom salts (magnesium sulfate) are one of the myths that the Internet (and the seller) loves to propagate as a miracle fix-all cure, but in reality they may only solve one problem (magnesium deficiency) that is really not all that commonly occurring to begin with. Foliar application does no good because the plants do not absorb magnesium through their leaves. There's a good discussion over at the Growing Tomatoes forum right now about the kind of damage ill-informed application of Epsom salts can do to tomato plants. Pepper plants are in the same family as tomatoes. If you do a bit of research you can find any one of many good fertilizers that contain more than enough magnesium. Not trying to harsh your mellow, bring you down, etc., just speaking from experience so that you can avoid making the kinds of mistakes I, and many others, have already made.