Preheat oven to 350 degrees F (175 degrees C). Grease a 9x13-inch baking dish.

Bring a large pot of lightly salted water to a boil; cook the pasta in the boiling water, stirring occasionally, until tender but not mushy, about 10 minutes. Drain.

Thaw the broccoli in a microwave oven until you can break it apart into small pieces.

Heat the olive oil in a large skillet over medium heat; cook and stir the diced ham in the hot oil until the edges start to brown, about 10 minutes. Stir in the broccoli and cook and stir until any excess water has cooked away and the ham and broccoli are hot. Pour in the jar of Alfredo sauce and the milk; stir to blend, sprinkle with pepper, and add the cooked pasta. Stir everything together to coat the pasta with sauce and bring to a simmer.

Spread the bubbling pasta mixture into the prepared baking dish, top with shredded Colby-Monterey jack cheese, and bake in the preheated oven until the casserole is hot and the cheese is melted and starting to brown, about 30 minutes.