Combine potatoes and carrot in a large pot. Add enough water to cover and bring to a boil over high heat. Cook until vegetables are tender, 10 to 15 minutes.

Meanwhile, melt butter in a large skillet over medium heat. Add celery and onion; cook, stirring occasionally, until softened, about 10 minutes.

Drain the potatoes and carrots, reserving about 2 cups cooking liquid. Return vegetables and reserved liquid to the pot. Stir in 4 cups milk, then season with salt and pepper. Reduce heat to low.

Transfer the onion and celery mixture to the pot and stir in the chicken soup base. Heat slowly, then add the cheese. Allow the cheese to melt, stirring all together well.

Whisk cornstarch and remaining 1/2 cup milk together in a small bowl until smooth. Stir into the soup and cook until slightly thickened, 2 to 3 minutes. Serve hot.