But you can not say committing the act is not a crime, you can't.

Thats when they become a sexual predator / molester, on top of being a pedophile. Its an important distinction to make because while we can get rid of people breaking the law for violating the rights of others, we simply can't get rid of the entire group of people who could be labeled 'pedophile'. The mindset is similar in nature to those who imagine running people over, shooting their boss, robbing a bank, etc - not illegal to think it, illegal to do it. I know this seems nuanced but it really is important to make this separation because we can't get carried away in our pursuit for justice. And its also important to realize that just because someone thinks it or desires it, does not mean they will act on it.