The problem with any "most annoying fanbase" article is that the most annoying fanbase is usually a team that's winning and has a TON of bandwagon fans.
When USC was in their heyday, you had TONS of Trojan fans out and about here in SoCal. Did any of them attend the school? Of course not. It's a small private institution that none of them could afford, much less get into. Then when they started falling off? Yeah, you'd still see some, but it disappeared--especially now that we have NFL teams and one of them won a Super Bowl.
It's surprising here in SoCal how much OSU gear I saw when OSU was dominating, Alabama gear when Alabama was dominating, etc. Do I think these were OSU/Bama alumni? Or even people who had historic fandom of the program? Of course not. Michigan is certainly one of them but less so because while they've gotten to the CFP of late, they haven't actually done anything so Michigan bandwagon fans aren't coming out of the woodwork--that's Georgia (who is the new thing I see constantly here in SoCal 2000+ miles away from Athens).
There is only one fanbase that deserves to ALWAYS be atop every most annoying list. And that's Notre Dame. Any Joe Schmo who didn't go to college and is either Catholic or Irish suddenly thinks they're Notre Dame fans, despite the fact that they've never been to South Bend and couldn't find it on a map. Most probably don't even know what state it's in. And even worse are the folks in Indiana who are the reversible jacket type--Notre Dame football fan and Indiana basketball fan despite having attended neither. Blech. And Notre Dame fans are the most insufferable because they haven't done anything since the 80s. They don't even have recent accomplishments to hang their hat on, unless you count being overrated and getting destroyed every time they face REAL competition in the postseason.
But even then? While I detest most Notre Dame fans, I don't have any problem with anyone who is an alumni or actually has a family connection to the school. Hell, I even know a guy who in his blogging days was known as the Irish Trojan because he attended USC for undergrad and Notre Dame for law school (or vice versa). No reason to hate--he came by that moniker honestly.