Whisk together 2 cups whipping cream, milk, lemon pudding mixes, and lemon juice in a large bowl until creamy and smooth. Cover and refrigerate pudding until set, about 1 hour.

Beat remaining 2 cups whipping cream in another bowl until foamy. Gradually add confectioners' sugar and vanilla, beating until stiff peaks form. Lift the beater or whisk straight up: whipped cream should form sharp peaks.

Layer 1/2 of the angel food cake in a trifle bowl. Top with 1/2 of the pudding, 1/2 of the strawberries, and 1/2 of the blueberries. Spread 1/2 of the whipped cream over berries. Repeat layers. Cover and refrigerate until ready to serve.