Start by rolling your chilled pie dough or puff pastry out on a lightly floured surface until it's about an eighth of an inch thick. Keeping the dough cold is important here, since warm dough gets sticky and hard to shape. If it starts to soften while you're working, pop it back in the fridge for a few minutes before continuing.
Using a pizza cutter or sharp knife, slice the dough into even rectangles, roughly three by four inches each. Make sure you cut an even number, since you'll need a top and bottom piece for every pop tart. I like keeping a ruler nearby so all my pieces match up neatly when I seal them later.
Spoon about a tablespoon of Nutella onto half of your rectangles, leaving a small border around the edges. Don't overfill them, or the filling will ooze out during baking. This is also where homemade pop tart filling ideas come in handy if you want to switch things up, like swirling in a little fruit jam alongside the Nutella.
Place a second rectangle of dough on top of each filled piece, then press the edges together firmly. Knowing how to seal homemade pop tarts properly makes all the difference, so crimp the border with the tines of a fork all the way around. This locks the filling in and gives you that classic pop tart edge.
Transfer the assembled pop tarts to your parchment-lined baking sheet and pop them in the fridge for about ten minutes. This little chill helps the dough hold its shape in the oven instead of puffing unevenly. I usually preheat my oven to 375 degrees Fahrenheit during this step.
Brush the tops with your egg wash and sprinkle a little sugar over each one. Bake for 18 to 22 minutes, until the pastries are deeply golden and flaky. Let them cool on the tray for five minutes before moving them to a cooling rack.
Once the pop tarts have cooled completely, whisk together the powdered sugar, milk, and vanilla until smooth. Drizzle or spread the icing over the top of each pastry, then add sprinkles right away before the icing sets.