Yes, cheap seed is filled with millet, loved by grackles, House Sparrows, pigeons, etc. You can get rid of many vermin birds and attract Goldfinches by switching to Black Oil sunflower seed (BOSS) and using tube feeders, or feeding the more expensive Nyjer, or thistle seed.

Suet feeders will attract woodpeckers and wrens (mine especially love the 99 cent peanutbutter flavored suet packs from WalMart), orange halves will attract orioles, hummeringbird feeders will attract hummingbirds, but to attract Bluebirds you might have to serve live Mealworms.

You can buy most feeders and feed inexpensively at a big box store like WalMart, as well as inexpensive backyard bird field guide booklets. Or you can order a more expensive field guide book, like The Sibley Guide to Birds, used, on Amazon.com

Rachel's advice is excellent. Probably the first thing you should do is to buy a good book and learn about what food attracts the birds you want to see in your yard. Obviously if you want to attract hummingbirds, you plant nectar producing flowers or put out a sugar water feeder. The same holds true for other birds. If you would like to see Bluebirds but are squeamish about insect larvae, you can plant trees or shrubs that produce berries that they eat, and keep your lawn cut and free of insecticides so they can hunt insects. If you don't mind handling insect larvae, you can feed them mealworms in a feeder designed to keep the bigger birds (like Blue Jays) out.