I have a text-only archive of the Internet since 1992. 52TB raw including edits. 22TB without. Crawlers have indexed 82 search engines since AltaVista. (whatever)

submitted ago by virge

I'm going to start seeding this tomorrow. Encryption key below.

AES-256-CFB1 key: 0193A106375355D07724EFD8F650042F95BE491881CD02FA93CCD39A3CCFA857

Edit: Interesting. The system I made this post on is now experiencing an extremely high level of probing for security vulnerabilities.

You are viewing a single comment's thread.

view the rest of the comments →

KebabAndNoseRemoval ago

Telling the glow in the darks you've got all of their clearweb fuckery over the last 27 years.

Interesting choice for your health friendo.

Morbo ago

This makes Virge look very glow in the dark himself. Why and how would someone have such an archive covering 27 years? This isn't the sort of thing a normal everyday Joe would do for fun. Seems quite suspicious.

Wahaha ago

If it's just text, then it's fairly easy to do. You just had to be there, 27 years ago. Since the Internet was a very different, smaller place, I can see how something like this started as a for fun project which wouldn't need much resources and then escalate together with the growth the Internet experienced, but by then you were already doing it, so you might as well continue. It's only eating harddrives, and those are cheap.

Morbo ago

It's not as easy as you think. You would have to crawl the entire internet several times an hour to catch the changes. Bandwidth back in the early days of the web was low. Crawling was intensive on the machines at the time. Hard drive space is only cheap today. Back in 1995, 1 TB of storage space would be very costly. It would also require some fancy setups since operating systems of the time could not address 1 TB of storage as one volume. I was there before the web and in the somewhat early days of the academic internet. What you speak of is based on technology as you know it right now. It was not a trivial thing to do any of this. Virge is a liar and did not make good on the drop of the archive since it was all bullshit. If you think you can do it so easily, start today and archive the entire web for just one year. You won't be able to do it, but you will learn that on your own. Come back next year and either drop your incomplete archive or admit defeat.

Wahaha ago

Back in 1992 the entire Internet in text format wouldn't even amount to one gigabyte. Wikipedia is only about 5GB, even today. Nobody claimed to have the entire Internet every hour since 27 years ago. Maybe the crawling was only done once a year.

I never cared for the entire Internet, but I did backup some sites I cared for in a way that lets me browse them as if they still existed. If I only saved it as text it would've been easier. And looking back, 1992 only had like ten websites. Ten. I could've saved those manually. There would've been enough time to improve this by writing scripts that do this for me. OP also never claimed a complete copy. Lots of stuff is behind paywalls and logins anyway and then there's the Internet Archives, which are redundant anyway. But a "text-only archive of the Internet since 1992" - definitively possible. Text-only archive of the entire Internet since 1992? Not so much possible.

Morbo ago

But a "text-only archive of the Internet since 1992" - definitively possible. Text-only archive of the entire Internet since 1992? Not so much possible.

Now you're just playing word games. You're saying it is both possible and impossible based on whether or not it is a complete copy. You can think whatever you want about this. The reality is that OP (Virge) did not make anything available and is a known liar. Semantics and word salad mean nothing since she failed to deliver. Many of us knew she was lying from the get-go and it is obvious now to everyone that she had nothing. Now Virge has disappeared thanks to WhiteRonin constantly calling her out on the bullshit. Good riddance to bad rubbish. Nothing of value was lost.

Wahaha ago

Yes, since you can't access everything (paywall, logins) a complete copy is impossible. I thought that was obvious, but apparently not to you, since you put an emphasis on "complete".

Morbo ago

What paywalls existed in 1992, moron? And complete would mean ALL changes on ALL sites, but I guess that was not apparent to you. So either put up an archive or prove yourself a liar like Virge. We'll wait for your archive to drop.

Wahaha ago

"I have a text-only archive of the Internet since 1992. 52TB raw including edits. 22TB without. Crawlers have indexed 82 search engines since AltaVista."

Where does it say "complete"?