Well, I think it's twofold.
Your house, for most people in this country, is the biggest financial decision you'll ever make that doesn't include marriage or children. Not only is it massive from a financial sense, but it's also emotional. It's about choosing who you are, where you want to be, and what sort of community you want to live in. You put down roots, you have kids (who go to school), you become a part of that community.
- When outside forces like "zoning boards" try to make changes and that means the very nature of your community might change, you can't simply decouple the emotional aspects. It's your life. Do you want to blow up friendships, do you want to force your kids into new schools in a new community, do you want to uproot exactly what you've built? No.
- When those same forces do something that will negatively affect the financial value of your asset, and you can exert political pressure to support the financial value, it's basically a no-brainer to exert that pressure to preserve it. If not only does the actions of a zoning board mean that your community will change and make you want to sell your house and move somewhere else, if the actions of that zoning board means that you're not going to get for your house what it was worth before they made their decision, you're gonna fight like hell.
NIMBY is a lot of things, but irrational isn't one of them.
It's funny because in my econ major brain, I see the strong rationality in No. 2, but scoff at the irrational part of No. 1, even if the mechanism is essentially the same.
As you said, it's emotional, and people will leverage their wealth and power to protect their feeling. But there's such a rich sense of entitlement that as a stakeholder in a community, you deserve a really strong hand in deciding the shape of it like that. The nature of a community is ever-changing. Factors, such as zoning, likely brought it into existence and can tweak it as well. I'm sure wherever someone lives, someone way back wouldn't have been the most keen on someone like the newcomers being there, but in America there's freedom and changes and whatever else.
And the entitlement to be free of that without giving something up strikes me as irrational in its way, even if the mechanism follows it's own type of rationality.
(Some of this comes out of convos with a close friend. He contended his company should pay employees at a certain level, the one he was at and a high-ish one, enough to be able to afford a house in the town where the company is based. And in some ways, this is rational. But it's in an expensive place. And he wanted to be part of a certain "community." Which meant inflexibility when it came to location. And of course the people in the "community" he eventually found a place in are either happy with his spending driving up prices, if their have an appreciating asset, or unhappy becuase rising housing prices mean they'll be pushed out. Only they're the sort of folks without the financial weight to be listened to, which means they'll lose friendships, move kids because of the flows of economics and policy)