In a large bowl using a handheld or stand mixer fitted with a paddle attachment, beat the butter, granulated sugar, and brown sugar together on medium high speed until fluffy and light in color, about 2-3 minutes. Add the egg and vanilla extract, and then beat on high speed until combined. Scrape down the sides and bottom of the bowl as needed.

In a separate bowl, whisk the flour, cocoa powder, baking soda and salt together until combined. With the mixer running on low speed, slowly pour into the wet ingredients. Beat on low until combined. The cookie dough will be quite thick. Switch to high speed and beat in the milk. The cookie dough will be sticky and tacky. Cover dough tightly and chill in the refrigerator for at least 3 hours and up to 3 days. Chilling is mandatory for this sticky cookie dough.

Remove cookie dough from the refrigerator and allow to sit at room temperature for 10 minutes. If the cookie dough chilled longer than 3 hours, let it sit at room temperature for about 20 minutes. This makes the chilled cookie dough easier to scoop and roll.

Preheat oven to 350°F (177°C). Line large baking sheets with parchment paper or silicone baking mats. (Always recommended for cookies.) Set aside.

Scoop and roll dough, a heaping 1.5 Tablespoons of dough each, into balls (I use this medium cookie scoop). Roll each into graham cracker crumbs. Arrange 2-3 inches apart on the baking sheets. The cookie dough is certainly sticky, so wipe your hands clean after every few balls of dough you shape.

Bake for 10 minutes. Remove from the oven and press 3-4 pieces of marshmallow on top of each cookie. Return to the oven for another 1-2 minutes. Remove from the oven and allow to cool on the cookie sheet for at least 5 minutes before transferring to a wire rack to cool completely. The cookies will slightly deflate as they cool.

Cover leftover cookies tightly and store at room temperature for up to 1 week.