Most mirrors have the ability to flip that little tab on the bottom of them that puts it in a different angle and then has somewhat of a tint effect. It does wonders. Do you not have this? If not go get one.
I can and do, it still bothers me
I also always have my windows down and was thinking about tinting the side mirrors too