They often do warn, but it can be so late or so muddled it's not clear what they say. The other day a rider yelled something causing me to pivot around and he slammed on his brakes because my pivot put me into his intended path, it was a normal turn, and he had nowhere to go. I told him he shouldn't be riding on sidewalks. I had to grab his handlebars to stop him.
I'm a pretty large fellow, but I worry about my wife who isn't. I played basketball and it would be pretty easy to stick out an elbow at times, but I don't do it. I try and pay attention to what may be behind us but I don't do it all the time of course. Some bikes have bells which if I hear I know they are coming somewhere.
It is illegal for them to ride on sidewalks, but hardly enforced of course.