What is up with this guy?

this guy and i have known each other for 2 years and in the past month got really close. we hold hands, cuddle and he plays with my hair. we never kissed or anything and when i rub his thigh he moves and shakes his head. well i only did it when my brother was in the room so do you think its just because of my brother or am i moving to fast or what?