I was mostly tongue-in-cheek with my comment on the 2 degree thing.
Sure, you can go out with enough layers. Can you fit in the car comfortably? A restaurant? Shopping? Removing layers is a pain in the arse. That's what I hated.
I've been to cold games outside at Camp Randall and Soldier Field. They barely give you enough room to sit in shorts and a t-shirt. Add 3-4" to everyone's body and it just downright sucks.