What you describe is generally a result of significant wet weather patches when orchard trees are blooming.

Pollinators in rainy weather

The honeybee is not a good traveler in wet weather, so pollination success drops way off. There are also some issues with honeybees and a mite that have depleted the honeybee population. Many beekeepers lost their colonies over this winter/early spring.

Mason bees can fill the gap since they are native and willing to fly between showers. They also don't fly as far, so a gardener that has supported them will generally have better pollination results.

Frost on fruit tree blooms

One additional factor is the series of frosts during bloom where it is cloudy/cool during the day and then cleared up at night with temperatures below freezing. That effect was very site specific with one garden affected and one less than a quarter mile away, not. Wind flow and cold air sinking and pooling were quite evident in some locations.