I just know if I said something at the start he would probably freak out like 90% of guys, should I have told him when he kissed me?

This reminds me of the time that a coworker told our boss that I was his girlfriend. Eventually he told me that he knew that I would say 'no' so he didn't ask me first.