Preheat the oven to 350 degrees F. Grease a 9x13-inch baking pan with nonstick cooking spray and set aside.
To make the streusel, in a medium bowl, mix together the flour, brown sugar, salt, cinnamon, and melted butter until it resembles coarse crumbs. Use your hands to mix to form larger crumb pieces. You want it to be crumbly. Set aside.
In a medium bowl, whisk together the flour, baking powder, baking soda, and salt. Set aside.
In the bowl of a stand mixer or using a hand mixer, cream the butter and sugar together until creamy and smooth, about 2 minutes. You might need to stop the mixer and scrape down the sides of the bowl with a spatula.
With the mixer on medium speed, add 1 egg at a time and beat until combined. Mix in the sour cream and vanilla extract. Scrape down the sides and the bottom of the bowl with a spatula as needed.
Turn off the mixer and add in all of the dry ingredients. Mix on low until just combined. Do not overmix. It is ok if there are still a few streaks of flour.
Spread half of the batter into the prepared pan. Sprinkle with half of the streusel. Spread the remaining half of the batter evenly on top with a spatula. Sprinkle the remaining streusel over the batter.
Bake for 50 to 60 minutes or until a toothpick can be inserted into the center and comes out clean. Remove from the oven. Let the cake cool completely.
To make the glaze, in a medium bowl, whisk together the confectioners’ sugar, milk, and vanilla until smooth.
Drizzle the cooled coffee cake with the glaze. Cut the cake into squares and serve.