What authentication protocol will be used between linux & windows?

I know we can communicate between these different platforms by config Samba server, but here I need the authentication protocol used to logging in.