A man (even an old man) can be attracted (even sexually) to a girl (even a very young girl) without any danger to the girl, society, or himself.
He can seek out associations with girls and get close to them and be trusted. He can hug, kiss, and pet girls he gets close to when the girls exhibit their acceptance of his love and attention. It's cute. He loves kids and they love him. It's okay.
Nothing happens that causes concern.
The girl is not sexy, she's just cute. The man is not lascivious, he is just affectionate and kind and kids love him.
It's a social lie. It hurts. Why does everything change if his little secret is revealed? He suddenly becomes a "pedophile", lumped in with rapists, molesters, situational offenders, and sociopathic predators, dangerous, suspect, untrustworthy.
He is still the same person. But now a pedophile. The word is poison.