I'm generally of the opinion to never attribute to malice what can be attributed to incompetence. I also think people are full of biases, some of which can't be overcome. i.e., you may have it in your head that LSU DBs of yesteryear are amazing and whatever should look like holding to you doesn't look like holding because LSU DBs are amazing and they don't hold to make great plays. A subconscious circular reasoning of sorts. Not out to screw a team, but the result is the same.
That said, in light of credible evidence I'd be open to changing my mind. Some people do shady things, refs are likely no different.