Dryland ecosystems cover large regions of the Earth and have important impacts on global biogeochemistry and the carbon cycle. The plant species that occupy dryland environments have traits that enable them to withstand harsh environmental conditions, and some researchers have hypothesized that dryland vegetation may be comparatively resilient to changing climate, while others have pointed out that dryland vegetation often operates close to the physiological limits of many species, implying a possible vulnerability to warming. Here we use the Landsat archive to analyze vegetation dynamics for part of the Sonoran Desert and adjacent mountains in southern California. We show that large decreases in vegetation cover occurred over the last 34 years (1984–2017), especially across the xeric portions of our study region, where we observed a normalized difference vegetation index (NDVI) decline of 1.1 ± 0.3% yr−1. Changes in precipitation explain most of the year-to-year variation but are unable to fully explain the observed long-term decline in NDVI. Statistical models that combined summer temperature and mean annual precipitation explained more of the spatial and temporal structure of NDVI trends and implicate climate warming as an important driver of declining vegetation cover. The impact of warming contributed to a change in the precipitation-vegetation relationship through time for this desert region, indicating a structural change in ecosystem function during the study period. These results suggest that recent climate change has already had significant impact on these drylands and highlight the potential for future warming to increase risks for dryland ecosystems in other regions.