Logically, one would expect a 7-3 team ranked say 15th to move UP if they lose a close road game against #2. I've seen it happen, but very rarely.
Usually, as noted, it starts with number of losses, any team with 4 isn't going to get considered, generally, at this point. Unless they are in the SEC.