Pulling off dead blossoms and removing dead, diseased, damaged, or rubbing branches certainly won't hurt your tree. As for how light pruning would effect blooming next year, it depends on how much pruning is done. I guess any pruning could potentially effect bloom performance to some degree, but multiple other considerations seem more significant to me, and light, proper pruning is unlikely to result in a noticeable difference.

Judy,
Smoke bushes don't like light pruning at all.
Have you ever heard people complaining that they look weird and start to grow "whips" on them?
Leave it alone.
If it ever gets to the point where you really want to prune it down, you have to COPPICE it.
Go online and do a search on Coppicing of smoke bush.
There are some great instructions on coppicing of smoke bush from professional growers.
By the way, if you ever did coppice the bush, you must do this before it starts to leaf out (sometime in March), and the leaves are gorgeous on it that year, it puts all it's energy into the foliage.

"Smoke bushes don't like light pruning at all. Have you ever heard people complaining that they look weird and start to grow "whips" on them?"

The growth spurts that cause these whips are due to more major pruning, not what I would call light pruning. It is the plant's way of responding to a significant loss of foliage potential.

Coppicing tends to change the natural shape of the plant, and may or may not be desirable. Coppicing will definitely effect the blooming potential next year, but could result in a smaller, fuller shrub-like plant. Since Judiblueyes' plant is up against her house, this smaller version might be more appropriate.

Thanks for the feedback brandon and butterfly....I will just do a minimal 'cleanup' of dead blossoms and dead branches, that's all. I'm new to this site, and can see I will so enjoy the benefits of sharing!

Smokebushes bloom on prior seasons growth so when you prune in the fall you are removing next year's bloom. However, the best foliage color is on new growth. This is especially important for cultivars with purple, burgundy or yellow-green leaves. Always a trade off with these shrubs.

Thanks for your feedback, all. To get a little specific, the branches with the blossoms on them are like starbursts at the ends....with thin, dead, dry bits that had blooms on spanning out from the end (last 1/4 of the branch.) Are you saying that even if I leave most of the bulk of the old branch intact but trim off those ends (about 12 inches or so) I'll be risking less blooms? Will next years blooms still just come on the exact section of the branch they were on this year?

It sounds like you are talking about heading back the branches. If so, that's NOT the way to go about pruning trees and shrubs. But, I'm a little uncertain about exactly what you are proposing. It IS ok to remove dead limbs or make proper reduction cuts on limbs with dead ends. If you still have questions, pictures might help.

Judi,
Just prune off DEAD wood in the fall, now.
If in doubt, do nothing.
The only way a smoke bush or tree should be pruned is COPPICING.
So, if you don't want to cut it down to the frame, just look real close and make sure what you are cutting off is absolutely brown, dead.
Next spring you will have your bloom then.
Just cut dead. Not live.
If you look at the branch closely, you will see little "bumps" or spots where the branches for blooms come from. (just for your info, to answer your questions on where it blooms from).
WHen it has no leaves on it, and it is bare, stand back and look at it from about 12 feet. You will see the bush's frame. WHen you go to coppice it, you cut off every single branch down to the main trunk of each arm. That is what coppicing is.
Some people cut it all the way to the ground nd leave about 3 inches of main stems, which you can also do, but you will not get any blooms that year cutting it to the ground.
When you just cut all the stems back to the trunk, you will get some blooms, and beautiful foliage too.
It all depends on how big you want the bush to be and if you want blooms or not.
Good luck to you, you have a wonderful bush.
I'm looking for a Velvet Cloak for next spring.

"The only way a smoke bush or tree should be pruned is COPPICING."

That statement is FALSE.

"So, if you don't want to cut it down to the frame..."

The word "frame" is not a term typically used by horticulturists/arborists. I think I have a general idea from the context in your post, but I can't be 100% sure I completely understand its use here.

"WHen (sic) you go to coppice it, you cut off every single branch down to the main trunk of each arm. That is what coppicing is."

As was explained by someone else in another recent thread, in which you posted a similar statement, that is NOT what coppicing means.