Was this "team dinner" organized by your employer, or was it just a private gathering of people who happen to work together? If it was organized by your employer, consider involving HR. If I were another colleague present with you in this situation, I would go to HR for it. (This may be different by location. Please indicate your country in the tags.)

Do you feel like he was trying to be friendly? Or trying to be cocky and arrogant?

Do you know the colleague? Did you joke with him before? Is he from your culture?

@Theodore - It was in Singapore.

@BernhardDöbler - we both are from different culture (and countries)

Why are you worried about "politely"? He clearly isn't.

Do you want to know what you should have done then or what you should do now, or what you should do if the situation arises again in the future?

To what extent was the "team dinner" work, or a social event? If it was an official work event, with a working agenda, that you were required to attend, then this might be a Workplace matter. If it was a largely social event and the only "official" element was, say, team building, would that not make this about good manners in general, not as applied to the Workplace?

I know you didnt ask this, but "I don't drink" is an acceptable statement.

"he should not make fun of/rag on someone for their beliefs?" to what extend is your drinking style a matter of religion?

As someone who has been on and off with alcohol in my lifetime, people like that are so annoying.

@Theodore I am so sad that your stupid comment is liked by so many people. I really hope people are not going to listen to your bad advice and involve HR.

@enhzflep folk etymology is fun isn't it. english.stackexchange.com/questions/100990

@BЈовић I suppose it strongly depends on culture. In some regions HR can help resolve interpersonal issues in a subtle way. In others, contacting them can be seen as being a kind of 'snitch' ;)