Preheat oven to 350 degrees F (175 degrees C).

Fill a pot with lightly-salted water and bring to a boil. Stir the macaroni into the water and return to a boil. Cook and occasionally stir until the pasta has cooked through, but is still firm to the bite, about 8 minutes; drain.

Melt the butter in a large pot over medium heat. Add the flour to the melted butter and whisk continually for 1 to 2 minutes to make a roux. Whisk 2 cups of the milk into the roux until smooth and then add the remaining milk. Raise the heat to medium-high and continue cooking and stirring until thickened, but not boiling. Stir the Gouda, Cheddar, Swiss, and 1 cup Parmesan cheese into the mixture; once the cheeses have melted completely, fold the drained macaroni into the mixture to coat. Pour the macaroni mixture into a 9x13-inch baking dish.

Stir 1/2 cup Parmesan cheese, bread crumbs, and cayenne pepper together in a small bowl; sprinkle over the macaroni. Drizzle the olive oil over the bread crumbs.

Bake in the preheated oven until golden and bubbly, about 30 minutes. Allow to rest 10 minutes before cutting.