A "literalist" might argue that since Congress is not making any law, the whole thing is fine. Of course, "originalists" have broadly interpreted such matters as being more general.
I'd note crimes like murder are widely illegal even in countries with no Christian tradition. "Blue laws" I'm guessing are unique to the US. You might defend a low closing bars on Sunday evening as not being religiously based, as it would tend to perhaps enable drinkers to get to work the next day.
The core truth is that SCOTUS Justices do indeed ask "What about ..." for a bunch of very solid reasons. One could claim it's BS, but that sounds to me more like "I'm too lazy to consider the point." I could post some weird stuff about say quantum mechanics and folks might also say "BS", because it does sound like BS.