Never pull a gun out unless you intend to shoot to kill.
If they had called and reported you, and you got caught, you would be facing felony charges. Even if you won in court, it would always be on your arrest record.