Whisk milk, cream, sugar, eggs, lemon zest, salt, cinnamon, and vanilla bean together in a bowl; fold in baguette slices, raisins, and cranberries. Set aside to soak for 40 minutes.

Preheat the oven to 350 degrees F (175 degrees C). Butter a 12-inch deep-dish pie dish.

Transfer bread mixture to prepared pie dish; remove and discard vanilla bean. Cover dish with aluminum foil.

Bake in the preheated oven for 45 minutes. Remove foil and bake until pudding is set and lightly browned, about 15 more minutes.