I had the same problem, but with second gear. It turned out to be my shifter tuning forks. Had a warranty on the car, and got a new trans!

Good luck with the fix.