Peel and wash the potatoes and sweet potatoes. Depending on size, cut sweet potatoes into quarters. Cover the potatoes and sweet potatoes and cook in boiling salted water for about 20 minutes. Drain. Mash them with a potato masher and mix in pieces of butter.
Stir in the crème fraiche. Season to taste with salt and pepper.