Preheat the oven to 425°F (220°C). Line a baking sheet with parchment paper.
Cut the puff pastry into 16 equal rectangles. Keep the pastry cold while working so it stays easy to handle.
Add about ½ tablespoon of thick strawberry jam to the center of 8 pastry rectangles. Leave a clear border around the edges.
Brush the pastry edges with egg wash. Place the remaining pastry rectangles on top.
Press the edges together firmly with a fork to seal each pop tart. Make a few small holes in the top with a fork or knife so steam can escape.
Brush the tops with the remaining egg wash. This helps the pastry turn golden and flaky as it bakes.
Bake for 10–12 minutes, or until the pastry is puffed and lightly golden brown.
Let the pop tarts cool for about 10 minutes before adding the glaze. They will be delicate when they first come out of the oven.
Mix powdered sugar, vanilla, and milk or water until you have a thick but spreadable glaze.
Spread the glaze over the cooled pop tarts and add sprinkles if desired. Let the glaze set briefly before serving.