Add your cubed potatoes to a large pot of salted water and bring it to a boil. Let them cook for about 15 minutes, until a fork slides in easily. Drain well and return the potatoes to the warm pot right away.
Pour the warmed milk and butter into the pot with the drained potatoes. Mash everything together until smooth and creamy, adjusting the texture with a splash more milk if needed. Season with salt to taste and set the pot aside, covered, to stay warm.
In a mixing bowl, toss the shrimp with olive oil, paprika, garlic powder, salt, and pepper until every piece is coated.
Heat a skillet over medium-high heat and add the shrimp in a single layer. Cook for 2 to 3 minutes per side, just until they turn pink and opaque. Transfer the shrimp to a plate and set aside, since they'll finish warming through in the sauce later.
In that same skillet, melt the butter over medium heat and add the minced garlic. Sauté for about a minute, stirring often so it turns fragrant but never browns. This is where a lot of the dish's flavor comes from, so don't rush it.
Pour in the heavy cream and let it come to a gentle simmer, scraping up any browned bits from the shrimp. Stir in the Parmesan and Dijon mustard if using, then season with salt and pepper. Let it bubble gently for 3 to 5 minutes until it thickens slightly.
Return the shrimp to the skillet, spooning the sauce over each piece so they warm through and soak up the flavor. Let everything simmer together for another minute or two. This final step ties the shrimp and sauce into one cohesive, saucy dish.