It means he built something that visits every site it can find, much like a search engine would. And he stored a text only copy of every page each time it found one. It's truly a herculean effort, and if this is an individual effort I would expect a huge gap in frequency for revisiting the same website after hitting it once. If he were to truly index every website daily he would need NSA level computing power.