Soak up the pumpkin mixture: Line a medium bowl with 2 paper towels. Place the 1/2 cup pumpkin puree in the bowl. Using another paper towel, press down to blot excess moisture out of the pumpkin. Let it sit with the paper towels while you prepare the dough.
Prep for baking: Heat oven to 350°F and line 2 baking sheets with parchment paper or silicone baking sheets.
Combine the dry ingredients: Whisk together the 4 cups flour, the 2 teaspoons baking soda, the 2 teaspoons cinnamon, 2 teaspoons ginger, and 1 teaspoon allspice, 1 teaspoon cloves, and 1/2 teaspoon salt in a medium bowl.
Make the dough: Combine the 1 cup room temperature butter and the 3/4 cup brown sugar in the bowl of a stand mixer fitted with the paddle attachment. Beat on medium speed until light and fluffy. Add the 1/2 cup pumpkin puree, the 1 egg yolk, 1/2 cup molasses, and 2 tablespoons maple syrup and beat until combined. Turn off mixer and scrape down sides with a rubber spatula. Add flour mixture and beat on low until flour is incorporated. Dough will be sticky.
Bake: Using a large cookie scoop, scoop cookie dough into balls, about 3 tablespoons of dough per cookie, and arrange 1 inch apart on the baking sheets. Press down to flatten. Bake for 9-12 minutes or until lightly browned on the sides. The centers will still be soft. Let sit on baking sheet a couple of minutes, pressing down gently to flatten before removing to a cooling rack to cool completely.
Make the icing: Place powdered sugar in a medium bowl. Add the vanilla extract and 1 tablespoon of milk. Use a small whisk to combined. It will be impossible to fully combine because this isn’t enough liquid. Add only enough extra milk to make a very very thick icing. I only add about 1 more Tablespoon of milk. Lightly dip the tops of the cooled cookies into the icing. Return cookies to the cooling rack and let icing harden before continuing with the recipe.
Make ice cream sandwiches: Scoop 1/3 cup of ice cream onto the bottom of 1 cookie, top with another cookie and press down to evenly distribute ice cream, wrap in plastic wrap and freeze. Repeat with remaining cookies and ice cream. I recommend freezing the sandwiches for at least an hour before serving.
If you would like perfectly round discs of ice cream like the photos, line a small baking sheet with parchment paper. Let the ice cream sit out until it is soft but not melted. Spread the ice cream into an even (1/2-inch thick) layer on the baking sheet and freeze. Once the ice cream is frozen, use a 3-inch biscuit cutter and cut circles out of the ice cream. Sandwich each circle between the two cookies, wrap and freeze.