Yes, it really does make a difference.

It makes a difference. Specially if the ground was hardpan before. If it has had loads of organic matter put into the soil previous, then it won't be so noticeable. I am in the minority that thinks that some American gardeners are a little fertilizer happy, but still, it's like eating right and taking yer vitamins... putting compost in the soil is getting the plants to eat right, giving them fertilizer is like giving them a vitamin on top of eating right.
THEN, to make it more complicated, there are fertilizers for growth, that if applied to heavily to fruit bearing plants, like tomatoes or squash, means that you get large vines and teenie fruits. So, you have to give them a well rounded 'vitamin' with potassium and phosphorus, for root growth and fruit/flowering.

start with a soil test to see what/if your soil needs anything then you will know what to do in the fall

During the peak growing season it's generally a safe bet that most crops can use some quick-acting nitrogen. That would have to be quite some soil to be so rich as to make any such additional nitro be too much. So save your urine and mix it about 10-1 with water and apply, about once a week at this time of year.

It's organic and fast-acting.

Add compost. safe to do any time of the year, makes everything a little better.

mmmm.... compost

2nd the compost...oh yea
keepin in mind that tooo much nitro at the wrong time can cause other probs
but hey yea compost is balanced

It is soil health that makes the difference. Fertilizers may contribute to soil health if the right fertilizer in the right amount is used, but since most all fertilizers are fairly water soluble most of those applied simply flow with water through the soil and pollute the ground water. One indication of soil health is the level of organic matter in your soil.

3rd the compost. You'll be amazed.
Sally

Hi fenway-gardener...you must have a water faucet close enough to hook up a hose to right? Well...ya get yourself some combo seaweed/fish fertilizer...liquid...can buy in garden centers...and a liquid fertilizer dispenser that ya attach to your hose end. Mix up some fert. and spread with the hose end applicator. Use applicator and fert. as per instructions. Also get yourself a do it yourself PH testing kit and check your soil PH...it will get you close enough to add lime if ya need to. Your plants will not take up fert. if the PH is out of wack...slightly on the acid side would be good. Happy gardenin'...Franklin

4th the compost.

But if I were in your place, I would look for the best quality amendments available locally, whether compost or a fertilizer. (Sometimes the compost that is offered for sale does not look wholesome to me.) Also, I was noticing that the summer squash I planted in March was smaller than the ones the next plot over that my neighbor planted in May, and I know the neighbor uses manure, so I asked on the Soil Forum, "Is Compost Enough?" The consensus seemed to be that some people believe in compost and some believe in fertilizer, almost as a matter of faith.

Anyway, here is one version of a make-it-yourself fert recipe (from http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2006/06/03/HOGC7J5DA21.DTL). I have not used it, but if I were to use fertilizer, this is the kind of thing I would use.

Mix uniformly, in parts by volume:
4 parts seed meal (soy, flax, sunflower, cottonseed, canola, etc.)
1/4 part agricultural lime, best finely ground
1/4 part gypsum (or double the agricultural lime)
1/2 part dolomitic lime
Plus, for best results:
1 part rock phosphate
1/2 to 1 part kelp meal (or 1 part basalt dust)