Preheat the oven to 375 degrees F (190 degrees C).

To make crust: Combine 3 cups flour, 3/4 cups white sugar, baking powder, and 1/8 teaspoon salt in a large bowl; mix thoroughly. Cut in 2/3 cup butter with a pastry blender until pieces are the size of small peas. Stir in eggs, vanilla extract, milk, and lemon zest. Mix just until all ingredients are combined. Allow dough to rest in refrigerator while you make the topping.

To make streusel topping: Mix 1/2 cup flour, brown sugar, 1/2 teaspoon cinnamon, 1/4 teaspoon salt, chopped nuts, and grated lemon zest in a medium bowl. Rub in butter until all ingredients are well combined. Set aside.

To make fruit filling: Place pitted and sliced plums in a large bowl. Mix remaining sugar, flour, cinnamon, and nutmeg in a small bowl until thoroughly combined. Pour over plums and stir gently until fruit is evenly coated.

Roll out pie crust and arrange in a 9-inch pie pan; trim and flute edges. Pour in plum filling and evenly cover with streusel topping.

Bake pie in the preheated oven until filling is bubbly and streusel is golden brown on top, 45 to 55 minutes. Serve warm, or at room temperature.