Heat 2 cups milk in a heavy saucepan over low heat until simmering, about 5 minutes.

Beat remaining 1/2 cup milk and cornstarch together in a bowl using a whisk until dissolved; add sugar and egg yolks and beat until smooth. Slowly pour milk mixture into simmering milk while gently whisking the simmering milk; simmer, stirring frequently, until thickened, about 3 minutes. Remove saucepan from heat and stir vanilla extract into sauce.

Pour the sauce through a wire mesh sieve into a bowl to remove any cornstarch lumps. Place a piece of plastic wrap on top of the sauce, ensuring the plastic wrap is touching the sauce to prevent a skin from forming; cool to room temperature, about 20 minutes.

Move sauce to refrigerator and chill completely cold, at least 30 minutes.