Place sweet potatoes in a medium saucepan, cover with water, and bring to a boil. Reduce heat to medium; simmer until tender, about 20 minutes. Drain. Transfer to a large bowl; set aside.

Melt butter in a small saucepan over low heat. Stir in sugar, cinnamon, and allspice. Add apple slices, cover, and simmer until apples are tender, about 5 minutes.

Stir apple mixture and milk into sweet potatoes; beat using an electric mixer or a fork until potatoes are mashed.