Actually, if I remember correctly, on her first sight of him, she didn't want some creep following her down a dark alley, and told him to get lost, and she didn't like him at all for quite a few more episodes after that, until she learned she could trust him.
You do indeed remember correctly. However, it was clear from the onset--at least to her friends--that Buffy found Angel attractive despite her concerns over him following her (which, it turned out, he did for legitimate reasons that had nothing to do with stalking), and though she didn't trust him at first, it was clear she had the hots for him from the start, and the sexual tension was always there.
And yup, it's true that they didn't have sex until her 17th birthday, but they kissed for the first time when she was 15 going on 16, and they often made out quite heavily (and graphically) prior to having sex for the first time--an act which cost Angel something far more precious than it cost Buffy, as all fans of the show will recall.