Before 1967, Israel got relatively little from the US. Their military gear was mostly French or British based stuff. The Six Day War changed that. Egypt et al. was in the Soviet sphere, and we wanted to counter that, Israel was a proxy.
Over time, Israel became a political focal point. Evangelicals for various reasons decided it was worth supporting, as did most Jews, both groups usually voting for the other party in bloc, so no politician could afford to alienate his "side". Camp David then secured aid to Israel AND Egypt.
I personally see some value in foreign aid, value for Americans. This is a global economy, things that happen over there have an impact on us here. And it really is a small part of overall spending (which I agree should be cut, but won't be, and taxes should go up, but that won't happen either to any real degree). So, focus on foreign aid, to me, is focusing on your third string long snapper when your QB room is a problem.