Preheat oven to 350°F (177°C).

Combine butter and brown sugar together in a small saucepan over medium heat. Whisk occasionally until butter has melted. Cook, whisking constantly, for 1 minute as mixture thickens. Remove from heat and whisk in the cinnamon and vanilla extract. Pour into an ungreased 9×2 inch pie dish or round cake pan. (Make sure the pan is 2 inches deep. I recommend this pie dish, which is 1.8 inches deep but I never have an overflow issue.) Arrange the apple slices neatly on top, overlapping where necessary. See photo above. Place pan in the refrigerator for a few minutes as you prepare the cake batter. This helps solidify or “set” the topping’s arrangement.

Whisk the flour, baking powder, cinnamon, nutmeg, and salt together. Set aside.

Using a handheld or stand mixer fitted with a paddle or whisk attachment, beat the butter on high speed until smooth and creamy, about 1 minute. Add both sugars and beat on high speed until creamed together, about 1 minute. Scrape down the sides and up the bottom of the bowl with a rubber spatula as needed. On high speed, beat in the eggs and vanilla extract until combined. Scrape down the sides and up the bottom of the bowl as needed. Pour the dry ingredients into the wet ingredients. Turn the mixer onto low speed and as the mixer runs, slowly pour in the milk. Beat on low speed just until all of the ingredients are combined. Do not over-mix. You may need to whisk it all by hand to make sure there are no lumps at the bottom of the bowl. The batter will be slightly thick.

Remove topping from the refrigerator. Pour and spread cake batter evenly over topping.

Bake for 40–46 minutes, tenting foil on top of the cake halfway through bake time to prevent the top from over-browning before the center has a chance to fully cook. The cake is done when a toothpick inserted into the center of the cake comes out mostly clean—a couple moist crumbs are OK. Don’t be alarmed if your cake takes longer or if the cake rises up and sticks to the foil. (We serve the cake upside down anyway!)

Remove cake from the oven and cool on a wire rack for just 15 minutes. Invert the slightly cooled cake onto a cake stand or serving plate. Some of the juices from the topping will seep over the sides—that’s ok. You can slice and serve the cake warm, but the slices will be messy. I find it’s best to cool the cake completely at room temperature before slicing and serving. Do not refrigerate the cake to speed up the cooling process because it could end up tasting overly dense.

Cover leftover slices and store for up to 3 days in the refrigerator or 3 months in the freezer. Thaw at room temperature. I don’t recommend freezing the cake as a whole because the topping arrangement doesn’t thaw very nicely. See make ahead instructions below.