Just looked it up, here's what AI had to say about it... 
Yes, the U.S. Copyright Act's Fair Use Doctrine (Section 107) may allow music instructors to use copyrighted music without permission for educational purposes, but it is not a blanket exemption. Fair use is decided on a case-by-case basis, generally allowing small, transformative excerpts for analysis, but rarely covering the copying of entire works.
Key Considerations for Fair Use in Music Education:
- Purpose & Amount: Using small excerpts for commentary, criticism, or analysis is more likely to be fair use than using the entire, original work.
- Educational Setting: Section 110(1) allows for face-to-face performance of copyrighted music in non-profit educational settings, but this does not authorize copying, distributing, or uploading the music to the internet.
- Four Factors: Courts consider the purpose (non-profit/educational), nature of the work, amount used, and the effect on the market value.
- Safety Rule: Never assume usage is fair; when in doubt, especially for for-profit instruction, obtain permission.
Common Misconceptions & Limits:
- Copying for Students: Making multiple copies of a copyrighted song for a class is generally not considered fair use.
- Arrangements: You cannot make a new arrangement (e.g., changing the style) of a copyrighted work without permission, even for your own ensemble, if it changes the fundamental melody.
- Digital Distribution: Posting copyrighted music on a website or learning management system often violates copyright, even for educational use.
Yeah, and where did AI scrape that from?

But seriously, a guitar teacher explaining how to play the song of specific riffs doesn't need to use the actual recorded version of the song. And if using some portion of the song, probably also doesn't have to use the entire thing.
Back when I was a blogger it was well-known that copyright prevented the republishing of other's work. But in blogging and journalism, republishing an excerpt, quote, or small portion of a work for the use of explaining / refuting / using it as an example is absolutely covered by Fair Use.
A YouTube guitarist probably can't upload them covering another artist's work. But "here's how you play this difficult riff" or "here's how you get that specific sound effect used in song X" would almost assuredly be fine.
Fair Use can get murky, sure. But I think the bigger problem is this:
And you only get three strikes, and you're booted. They can close your channel. You can contest strikes, but YouTube is notoriously hard to deal with, both in getting a hold of anyone to explain things, and in their persistent vagueness when issuing defenses of copyright strikes. When YouTubers show how their video did not violate YouTube's terms and seek explanation for why a video is still taken down and a strike still exists on their account, YT may or may not ever answer, and if they do, they're not big on really explaining anything. They just copy and paste their terms, which is what a content creator is disputing in the first place. The only option from there is to fight YouTube itself, which is impossible for most creators, and it also runs the risk of biting the hand that feeds you, if you're a full-time YouTuber.
It's just not worth fighting, because you end up having to justify to YouTube that you're not infringing. As Mike says, they're difficult to deal with. You may be legally in the clear but YouTube doesn't want that risk.
Ultimately it's just easier for YouTube to take stuff down and not have to deal with it.
Which is something that bands like The Eagles and their lawyers can exploit. Even if they're legally in the wrong, they can still get stuff taken down because it's easier to comply with their accusation on infringement--especially at the platform level like YouTube--than it is to prove you (or your user) are right and they're wrong.