Heat the olive oil or butter in a large pot over medium heat and sauté the onion, garlic, carrot, and celery for about five minutes until softened. Add the cubed sweet potatoes and broth, bring to a boil, then reduce heat and let it simmer for about twenty minutes until the sweet potatoes are tender. Blend the soup with an immersion blender until smooth or use a regular blender in batches. Stir in the coconut milk or heavy cream, grated ginger if using, and a pinch of nutmeg. Adjust seasoning with salt and pepper and simmer for another five minutes. Serve warm, garnished with fresh herbs, toasted seeds, or a drizzle of cream.