Assuming we're talking about the "ordinary" type of programmable thermostat, not the new fancy ones like Nest, the folks who are going to experience the greatest savings are those who have fairly regular schedule and were lousy at "managing" their manual thermostat. If you regularly adjust your thermostat before leaving for work or going to bed, then any benefit of a programmable thermostat is going to be minimal.

Remember too that any calculations post conversion need to take into account degree days and changes in utility rates.

I did not notice a change in cost when we put in a programmable thermostat, but I love the convenience of it. We can warm the house up before we get out of bed. When I work around the house and get hot, I can change the temp manually and if I forget to change it back it will do it automatically to the preset setting. Our elec company will put one in for you free, but they will control it and during peak seasons it will be turned down for 15 minutes and back up ever so often. I think that means it does save you money or they wouldn't do that. We bought our own.

We did not save much money, we saved time.
We no longer have to do it manually.

There's no need to over-complicate or overly quantify the assessment, it's not going to be a life-changing or bank-breaking decision. Putting one or two in isn't that expensive. Any money savings comes from not having to remember to turn the system off before the normal hours when the house is vacated or to adjust for sleeping temperatures. Or, not having to remember to turn the setting down (or up, if for A/C) if you tend to overcompensate for a quick warm up or cool down when it first comes on.

An even bigger benefit to some, certainly to me, is the convenience factor. I've had setback thermostats for (it seems) 20 years or more, I wouldn't be without them.

My programmable thermostat probably costs me money as opposed to saving it.
When I used to turn the temperature up and down manually, I'd do it religiously not only when I go to bed and leave for work, but also if I left the house on the weekend for extended periods of time.
Since it's now programmed to my usual routine, I don't remember/bother to adjust it at other times. So if I leave for the weekend, it usually remains on it's schedule.
But I love the convenience and wouldn't want to go back.

It only saves money if you didn't used to adjust it manually.

Our Ecobee3 is programmable and WiFi connected. We enjoy the convenience of setting it and forgetting it. Does it save? Of course it does.
We sleep at 60 degrees, get ready for work at 70 degrees, and the house temp is 60 degrees while we are away. Less heat/less blower time = gas and electric savings.

Optional room sensors are available if you wish to use them. The sensors do a good job of keeping our ranch an even temperature.

The WiFi advantage allows us to makes changes while we're away. If we're heading home early, we simply pull up the app and touch HOME.

When you go away or for any reason you can set the temp and press the 'hold' button and it will not change on it's own until you push the program button.

Yep, I'm aware of that functionality and sometimes use it. But since I'm not obsessive about changing the thermostat anymore, it usually slips my mind.

If you have a heat pump I understand you are to avoid these stats...just set it and forget it. The folks who installed our unit said that while dropping the house temp from say 70 to 62 while away at work for 8 hours obviously saves some energy, the amount of energy used to bring the temp back up trumps that since 'backup heat" (heat strips) kick in. These are the strips that assist the heat pump when it so cold out that the nature of the heat pump is not enough to heat or maintain the heat in the house. We can always tell when these strips are enaged...the air coming out of the vents is HOT instead of luke warm...I also know the $$$ are spinning when this occurs.

andrelaphlume2 - this is exactly what I think too. For heat pumps, the recovery period energy use far exceeds the savings obtained from turning the heat back for 8-10hrs... in short, it is more expensive to allow the space(and furniture in the space) to cool down while away just to have the energy hog (auxiliary heat) kick on to get the space warmed back up. This winter I have taken the approach to set-it-and-forget-it (at 67 F) and so far I am ahead compared to last year when I turned the unit up/down(between 60-68 daily). This year has also been colder to-date compared to last year. (at least according to the information printed on my electric bill)

There are "smart" thermostats that monitor system performance over time in relation to how fast the temp drops during a setback period and how fast it rises during recovery. They adjust the start-up time for recovery and ramp the temp up slowly to avoid triggering the auxiliary. Most systems also have an option to add an outdoor temp sensor and lock-out the auxiliary from running at all unless the outdoor temp is below a programmable level (it still runs during defrost cycles).

When I was working two jobs, I routinely did 10°F setbacks for 12+ hours per day. Sometimes the indoor temp dropped by only 6 to 8 degrees so the system never ran during the setback periods. Took some unusually cold weather for the "smart recovery" function to really come into play. In any case, it wasn't unusual for the recovery to fall 1°F to 2°F short of the target temp at the target time, which is what triggered the auxiliary to quickly make up the difference ... which is what prompted me to have the outdoor temp sensor added for the lock-out. Not a problem for me if the room temp was 68°F instead of 70°F at the target time. The system was blowing warm air so it was perfectly comfortable in the house. I'd rather it run longer without the auxiliary than trigger the auxiliary unnecessarily to quickly make up the difference.

I don't have a heat pump, just a gas furnace; the programmable stat was installed with the new furnace. So, it is hard to say how much effect the stat alone had vs. replacing the 30 yr old furnace with a hi efficiency one. I have definitely lowered my bills though.