I've been through two, back-to-back IRS audits. It's not fun. And their assumption appears to be (I think they told us this explicitly) that you have done something wrong. In one instance we had a very minor adjustment (due to something our accountant had amortized incorrectly), the other one they found no problems. But both times it was invasive, targeting, and felt awful.
They also explicitly told us that it's much easier to go after lower wage earners than the truly wealthy. The rich lawyer up and make things a nightmare--so while the amount at issue with the IRS is much bigger, it's also much more costly for the IRS to pursue them. That makes for an interesting enforcement dynamic.
However, without actual tax enforcement, there would be a lot more tax cheating than there is (and there is already a decent amount). This is one of those situations where improvements are definitely needed, but how we ended up with the enforcement system we have wasn't random or ridiculous, it responsed to the ways taxpayers were avoiding their taxes.
Now, as to the tax code, it too, is the result of a lot of history and process. Our basic view of a progressive income tax isn't especially controversial (sure, there are those who advocate for a flat tax, etc., but the overall support for progressive taxation--even among the wealthy--is pretty strong). What makes our tax code difficult is that we use taxes as a way to encourage all kinds of behavior. At a basic level, the following are some of our fairly popularly known and popular tax decisions: we value being married and having children (reduced taxes--normally--for married, joint returns, and tax rebates--not merely write offs--for having children), we value investments differently than income (our tax code actually shows that we value investing money more than making money for directly performing services, i.e., we tax investment income less than we tax income we make from our own direct services), we value home ownership (we give a huge tax break, smaller following the Trump tax changes, for interest on home mortgages), we value saving for college and retirement (Sections 529 and 401K of the tax code), we value supporting non-profits (we provide a tax break for that), we value reinvesting profits in companies (as a general matter, you don't pay taxes on gross revenue, you pay for net revenue). Taking away these--and many other--tax breaks would cause a huge uproar. So the idea that we need a new tax code briefs well, but generally doesn't stand up to public scrutiny. Most of the complication of our tax code comes from these policy choices, and the big changes in how We the People collect revenue would come from eliminating some of these very popular tax policies.
Regardless, being on the receiving end of an IRS audit SUCKS.
Tangentially, direct services is one of the key jobs for our elected representatives. Helping us navigate a complicated federal government (or leaning on a government agency) is one of the ways--as Badge shows--that our representatives can earn their constituents' loyalty. Even in my Army days, when a constituent (in our case, Joe Snuffy) contacted their congressperson, and we received a letter of inquiry, we hopped right to addressing it (although ordinarily in our case it was explaining to Congressperson X that Joe Snuffy was a dirt bag and was appropriately disciplined as such).