That's pretty bold of him. Of course, if you were to stay now... he'd know you were his slave. Give him a handshake and wish them the best of luck. Taking the high road is your best option on this one.