If they pulled out fairly easily and had their own roots, then they are seedlings from dropped berries. Especially since you said they were tiny, it sounds like seedlings.

If they pulled out without roots, it's likely they are suckers.

Do you remember how they looked enough to make the distinction?

I carefully worked around each one and there were no roots so I guess it must be suckers but there were so many I am wondering if this plant it going to be very invasive. This is its first year and there were no flowers so its not seedlings. The Black Lace Sambucus doesn't show any signs of suckers and it is much larger. I read on the forum that this type doesn't sucker but I guess that was wrong.

I sympathize with you. I planted some Clethra (I think it is the 'Hummingbird' cultivar) last year and it has shot out in all directions! I'm going to try and root prune some of the suckers so I can pot them up to share with others, but I wonder if I may have way more than I can deal with. My species clethra shows no sign of suckering.

I also have a Black Lace sambucus and no sign of suckering. Both the sambucus and clethra were installed at the same time.

Shrubs that get to be 5' to 10' wide are going to put up growth from the rootball to get them to be the size that they need to be. Unless you bought shrubs already at full size?