Preheat the oven to 350 degrees F (175 degrees C).

Combine rosemary, lemon zest, salt, and pepper in a bowl. Rub over chicken thighs and place into a baking dish. Spray with cooking spray.

Bake in the preheated oven until no longer pink in the center and the juices run clear, 35 to 40 minutes. An instant-read thermometer inserted into the center should read at least 165 degrees F (74 degrees C).