1) Pleading the fifth amendment always looks shady.
2) Pleading the fifth amendment is often advised by attorneys, even if the person doesn't think they did anything wrong.
3) Refusing to answer on the basis of the physician patient privilege also confuses this topic a bit more--doctors are not supposed to testify about their patients (except in limited circumstances) without patient approval, and a great deal of medical information is highly personal.
4) If you are answering questions that are intended to illicit embarrassing answers for you or your client, you are going to try to avoid doing that to the full extent afforded by law.
5) As much as I know and respect several cops, they--and government agencies like them--are looking for things that are wrong, not things that are right. Refusing to give the government information that it might then use to prosecute you is generally smart, even if it is embarrassing in the court of public opinion.
6) As noted above, the right to avoid self incrimination does not protect the witness from civil claims. They can refuse to testify, but the finder of fact can use that as a reason to find against them.
So yeah, this looks shady.