1/4" around the windows is no issue. You just have to get low expanding can foam. Look at the labeling on the can prior to buying.
Will you have hvac equipment in the attic? (besides the ducts you mentioned). Is it possible to get the ducts in the floor instead? this will get them out of the the hot attic and supply the air conditioning from the floor rather the ceiling where the heat already is.
One issue about the foam in the attic is 5" does not meet code. Sure, they can throw out the performance method of code compliance because, after all, the foam will prevent more infiltration then the cellulose will by far. However if air tight drywall approach is used on the ceiling and proper air sealing done (around can lights, electrical boxes, plumbing stacks, etc) then the difference in performance will show cellulose as a winner vs the 5" of foam, especially in the price category. Spray foams rarely prove to have a payoff. If one of your heat pumps is in the attic, that is a different story. It MAY make sense to foam the rafters. However to be code compliant, you need between 9-10 inches. Any foam installer will deny this saying it is over kill. That is just their effort to get around code compliance by waving around performance method printoffs vs prescriptive methods.

I bought some Great Stuff Pro Window and Door foam and the gun for it. Just worried about getting it on the window frame since it is such a small space. Not sure if the tip of the gun will even fit.

No HVAC equipment in the attic, only the ducts. Would be difficult to put ducts in floor trusses.

Can you explain the airtight drywall to me? I know I can get in the attic before they insulate to seal around the can lights and the plumbing stacks but is there anything else?

Thank you so much!

Do you know when the Extreme Makeover will air?

that is the correct foam to use. Assume expansion of roughly 1:10, so you do not need very much at all. However if you can not fit it in there, then for your climate I might recommend simply installing a foam backer rod and caulking the joint instead.
Air tight drywall is just that...creating an interior air barrier to prevent conditioned air movement from inside or outside of the gyp layer. Basically concepts use rubber gaskets or a layer of caulk around the edges of each sheet prior to being screwed on. Gyp is caulk too and around electrical boxes/lights. For can lights, I actually prefer building a plywood box in the attic around the can light fixture and simply taping the edges of the box and caulking it to the gyp. This creates a tight seal around the fixture since it is hard to seal up the fixture to the gyp alone.
Basically any and all penetrations through your ceiling needs to be sealed. Think of it just as that..."air tight".
Google "air tight drywall" for more information.
Assuming you use r5-10 insulated duct and blow the cellulose completely over them, they should be ok in your climate. Remember you MUST seal the duct work to prevent air leakage as well. This includes sealing the ductwork to the registers and the registers box surrounds to the sheetrock.
Caulking and air sealing is the single most important part of efficient and tight construction. It happens to be also the cheapest way to drastically change the performance of the house. It is more important than u values, etc.
On that topic, it is also important for the rest of your house as well. The more caulking/air sealing you can do the better.
On the EMHE, I assume you are asking about my house. It airs first of January I think.

I will add...another option is to shoot an inch or so of spray foam in your attic on the top of your gyp to avoid installing via air tight methods. This will have a similar affect and seal everything up. Then you can blow cellulose on top of that.

wait a minute now..op is in florida.
hot humid climate.
in these areas we can't bury ducts in insulation
as it will cause condensation. even ducts lying on
top of insulation will condensate where duct touches insulation.

open cell insulation's R-value is R-4 per inch
so meeting the R-30 requirement by code (in La.)
means 7-7 1/2". (some open cell foams have slightly
higher R-values..R-4.2)

foaming the roofline does extend the payback, but
houses withstand hurricanes better as uplift is
reduced. putting ductwork in conditioned attic is
also a plus. on a 2500 sq ft house with foamed roofline only I get a 8 year payback..foam all the walls
and that payback jumps to 25 years..not a good deal.

better to put 1" foam boards to exterior of walls, tape
all seams and insulate conventionally.

even with an unvented attic, mastic sealing of ductwork
returns and supply boxes should be done.
as for recessed lights, simply buy ICAT cans at $15 more
per box than IC. ICAT is insulation contact air tight
no holes in housing. IC is insulation contact..holes in housing.

op, you should look at the builder's guide to hot humid climates @ www.buildingscience.com

best of luck.

For can lights, I actually prefer building a plywood box in the attic around the can light fixture and simply taping the edges of the box and caulking it to the gyp.

How 1983! BTW, the box should be lined with gypsum board.

I wouldn't be worried about using the window/door foam even with that small a gap. It's right about the size of the tubes that come with the guns/cans. If it overfills, you can just cut it back afterward. Any way you slice it, it'll be less work than caulk and backer.

I can see doing styrofoam or other boxes on lights in a retrofit, but why do them for a new build? I agree with energyrater- just get the ICAT cans to start with and not worry about building a bunch of boxes.

the problem with sealed cans on super tight houses (I am completely assuming the house is tight at this point, so this is where I am most likely wrong in my recommendation)is the can may be sealed, but to create the air barrier plane on the interior of the building requires sealing and detailing of the drywall (and vapor barrier if there is one) around the cans, which very few people do. Boxes are old school, but its cheap and easy insurance if you are going for super tight construction. Then again, I rarely recommend cans anyway for this very reason. Infrared testing and blower door testing I have done in many houses shows leaks around can lights (yes, even ICAT) are one of the biggest sources of air in/exfiltration. But, these projects I work on also have goals of sub <.1 pac which is far better then of the houses people build on this site i know. with that being said go for sealed cans and do their best to caulk them gyp ceiling.>We also have hot (100+) and high humidity (80%+) summers and never have issues with condensing duct work (ductwork is insulated) in the unconditioned attic.

"in these areas we can't bury ducts in insulation "

If you have condensation it means the the ducts need MORE insulation.

Wile burying them in loose fill is probably not a good option, adding more sheet insulation is likely to be very productive.

If the insulation is thick enough its outer surface will not be at the dew point.