The entire plant will benefit from being covered, but if they're nice large heads, why risk losing them to frost damage when they could be harvested?

How should I cover it? I have frost cloth, 3.5mil plastic, and cardboard boxes. Well, they're not quite ready to be harvested. This is my first time growing broccoli and the heads are probably only 2-3 inches maybe (some smaller), and they are still growing because the middle of them are the yellow-green color while outer parts are dark green and they are still tight (for each head). They are just big to me because my other three plants only produced like 1-2 inch heads lol :) I want to see if I can get them bigger, and these are the only 2 nights that are supposed to get this cold for the next 2 weeks.

Okay, good to have more detail. 6 two-inch heads is roughly equivalent to one large head. The plant is capable of supporting a limited amount of that growth, but it will survive the forecast. I would harvest half of the heads prior to the predicted low temperatures, and enjoy that for dinner. I would water after harvest with compost tea or some other hormone-rich low N supplement to help the plant recover. Cover the plant with a cardboard box if you are able to uncover it when the daytime temperature rises above freezing, otherwise use fabric, which will let some light through during the day. Plastic provides less insulation and magnifies the heating effects of sunlight, which you do not want. You could put a bottle of warm water under the cover as well, this will further stabilize the temperature. In a few weeks the remaining florets that you did not harvest will likely have reached a size that is worth a second harvest. Normal broccoli culture aims for one large head, but multiple small heads can equal the same yield. There is a point with most crops when the return won't justify continued effort, and you are better off starting fresh. You could leave one head on the plant to see if it will grow larger as the days get longer, you could cut off all of the florets and see if the plant produces more. or you could pull the plant and replace it with something that will give a more immediate gratification, like some salad greens.

Covering the plants to protect the heads if the temperatures get down into the mid 20's or lower is a good idea--you won't necessarily get damage if you don't, but you easily could.

I've never understood the conventional method of growing broccoli--grow for that one central massive head, cut it, then cut off the florets to get 1-2" pieces, then throw away the woody stalk and more than half the weight, pull the plant and start over with something else. It certainly is satisfying to see the huge head, but I'm glad once I've picked it and can get to all the secondary growth, which is what you have now, getting gobs of heads from 1-4" that you can just cut, wash and cook or eat with no waste. Total edible weight from the secondary cuttings each picking seems higher than from the big head when you discount that central woody stalk that gets thrown out, and it continues to produce the rest of the summer and into fall until hard freezes take the plants out. At least that's been my experience. I don't get the first heads until mid July though (transplant in early May), and it seems crazy to spend the effort and space to make multiple plantings when 4 plants will keep us in fresh broccoli a couple times a week from mid summer until late October, plus fill the freezer with enough until the next year's harvest, almost all from the secondary heads.

I agree, beeone. High side shoot production is one thing I look for in a broccoli.