It is most commonly a soil moisture issue/hot temperature issue. The first thing to go when the plant is water stressed are flower buds and flowers. Otherwise, you would see the blooms fade (after a while) and start a color change progression that ends in brown blooms.

I already went thru that early browning when my temperatures zoomed to the 100s. Heat dries out the sepals and makes them brown out if the plant cannot provide extra water. The heat can be in the environment or bounced off walls or in rocks that people put as organic mulch.

Try giving it more shade (I aim for shade after 11am) so cover it with an umbrella or chair. Also supply more water and see if these things helps. In my also clayish but amended soil, I provide plants with 1.5 gallons of water per bush per watering. Try watering the soil (not the leaves) early in the morning and water from the root ball outwards. Mulch -about 2-3" up to the drip line also helps.

Unusual causes: gray mold is a fungal infection of the blooms. The blooms brown out, feel papery and are covered with a gray-ish mold. But it is usually more common in the Spring or Fall.