I have a text-only archive of the Internet since 1992. 52TB raw including edits. 22TB without. Crawlers have indexed 82 search engines since AltaVista. (whatever)

submitted ago by virge

I'm going to start seeding this tomorrow. Encryption key below.

AES-256-CFB1 key: 0193A106375355D07724EFD8F650042F95BE491881CD02FA93CCD39A3CCFA857

Edit: Interesting. The system I made this post on is now experiencing an extremely high level of probing for security vulnerabilities.

SearchVoatBot ago

This submission was linked from this anonymous v/confessions comment.

Posted automatically (#60017) by the SearchVoat.co Cross-Link Bot. You can suppress these notifications by appending a forward-slash(/) to your Voat link. More information here. (@virge: Click here to suppress your anonymous crosslink notifications)

SearchVoatBot ago

This submission was linked from this v/AskVoat comment by @zxcvzxcv.

Posted automatically (#59996) by the SearchVoat.co Cross-Link Bot. You can suppress these notifications by appending a forward-slash(/) to your Voat link. More information here. (@virge: Click here to suppress your crosslink notifications from @zxcvzxcv)

libman ago

A few retard points:

  • You don't need "encryption key" to distribute public content. You probably mean "checksum", you retard.

  • There's no benefit to compressing everything into one giant file you retard. You can break it up by domain, or at least the top X thousand domains and one big file for the rest. That way people can first download a sites they're interested in (ex. a popular message forum) and figure out what they can do with it.

  • You didn't submit anything submit or comment anything in 13+ days, so fuck you retard.

dfgsdfgsdfg ago

this submission has been HEAVILY voat & comment manipulated welcome @puttitout.... might as well switch it too 0 comments

no wonder everyone is leaving

-dial_indicator

CognitiveDissident5 ago

When are you leaving? I seem to recall you telling @Kevdude you would leave.

Conejo_loco ago

Any updates on this????

ilikeskittles ago

You’re so full of shit. LOL. And these morons that believe you. ROTFLMAO.

Tarrock ago

Does it have my old geocities site?

Gabfag ago

2.2 years 20 ccp... totally not a alt @puttitout

fluhthreeex ago

who cares this place is 95% pentagon sockpuppets like OP eat a dick

harmlessgryphon ago

I have storage space. I don't have much upload speed but I can basically let it run forever.

PlutoInCapricorn ago

Pinging @virge

Goats are eagerly awaiting your upload. This will even take priority above my porn queue.

Dogsoldiertoo ago

Dang! That is very impressive. Well done, sir. Awesome foresight.

winterdreams7 ago

I want to download this, but how would I even use this? Where would I get a drive with 52TB? I would need to first setup a raid to be able to download this.

Corpse_washer ago

I would not give that to anyone. Ever.

phillyjoe ago

What is this, utzoo?

albatrosv15 ago

Edit: Interesting. The system I made this post on is now experiencing an extremely high level of probing for security vulnerabilities.

Everyone is hacking everything. It's just sad it is used for destruction of civilization.

abattoirdaydream ago

This is fucking amazing. Have you cleared it of CP? If not, let those who would access it know.

Einsatzgruppen1939 ago

How?!

abattoirdaydream ago

I love you!

WatchListMe ago

Would love a link to the torrent!

MDEneverdies1488 ago

I'd watch out and make sure not to seed any child porn and the like. Could get into loads of fucking trouble

HillaryClintonsShoe ago

Would not want unrevised history to rear it's ugly head.

TopTierCIAShill ago

@virge is a lying faggot

Gorillion ago

Will this include hidden areas of websites, like the "special menu" section of Comet Ping Pong?

Smells_Like_Tacos ago

If true, you better watch yourself. They goona say, whoah cowboy, what you doing with all that data?, Where you going you better give that to us.

Better have an air gapped copy also.

Shinsha ago

(҂‾ -‾)︻デ═一 (˚▽˚’!)/

progressbin ago

Fuck! I want that! What would I do with that? Spatially organize it using a Kohonen self-organizing map in N-dimensions normalized to an N+1th dimension. (if I had the horsepower). For the connecting weights, I'd use an error-squashed inverse-document-frequency of phrases (not from full on POS tagging, but from adjacent words.) When completed, the archive could be navigated spatially. Entry points into the system could be made from a similarly phrased Bayesian classification system. Google's mistake was in using their blunt un-phrased dictionaries. Their phrased searches are shit, and 50,000 words can't compete with 4,000,000,000 phrases.

heygeorge ago

Air-gapped no longer!

And just in time.

I’ve been searching for a genuine archive of the TimeCube.

Smells_Like_Tacos ago

Ok

Rotteuxx ago

Interesting. The system I made this post on is now experiencing an extremely high level of probing for security vulnerabilities.

So you're accusing @puttitout of giving your IP away or accusing him of probing your system ?

TradMan ago

Please tag me

Smells_Like_Tacos ago

Hay uh, what did you use use to discriminate what to save and trash or never scrape for as a ruler or whatever?

NoisySilence ago

Hello mr elite hacker man. Could you please focus your efforts on 1.Recent Texas Walmart security footage. 2.Frazzled.rip and 3.Anthony Weiner's laptop HDD. Thanks for your effort.

NamelessCrewmember ago

Had that happen once, traced it, complained to the top level tech at the NOC hosting that server hitting me. He was typical nerd then suddenly “sir I can’t talk about this”. Got real scared, said it agin and hung up.

You know whose knocking on the doors to your server, make sure it’s copied elsewhere within hours in a physical way, data can be traced and deleted.

chrimony ago

Can you send it to me on 3.5" floppies?

GasChamber ago

Your post intrigued me, there was this burning question that i had to know the answer to. The answer is almost 40 million floppies, which would occupy more than half a million cubic feet.

toobaditworks ago

jesus christ lol

Jimbonez_Jonez ago

From 1992 to what day?

TheBookWasBetter ago

proly like... today.

blumen4alles ago

I take if you have a copy of that archive on a system NOT connected to the internet. Considering it looks as if someone doesn't want you to share it.

Zoldam ago

I remember AltaVista

goatboy ago

WORST PORN SITE EVER!

Smells_Like_Tacos ago

Found tha jewbag

sAVAgeBeastN ago

Pew . Pew..

sirRantsalot ago

Oh.....shit.

AshesAshes ago

Following for interest.

carlip ago

Maybe it is a bad idea to connect to someone who you don't even know and is shopping around files... just a thought.

Nullisect ago

I'd be interested in checking this out but how would I even store 22TB?

GrandNagus ago

Hard drives very cheap nowadays,10TB is about $300.

Smells_Like_Tacos ago

Your going to want to double/tripple up on this type data for redundancy.

winterdreams7 ago

Impressive!

NarrativeControl ago

FUCK I DON'T HAVE ENOUGH HARD DISKS! D:

endernug ago

D:? You're gonna need E:, F:, G:, and H: for this shit!

Pronebone45 ago

What does this even mean?

Cronenberg ago

It means he built something that visits every site it can find, much like a search engine would. And he stored a text only copy of every page each time it found one. It's truly a herculean effort, and if this is an individual effort I would expect a huge gap in frequency for revisiting the same website after hitting it once. If he were to truly index every website daily he would need NSA level computing power.

Pronebone45 ago

Thank you. I'm quite computarded but it sounded important, yet also "herculean" as you said.

toobaditworks ago

Shit I need to buy a new HD. Can you make the font size smaller and shrink it to like 25TB?

harmlessgryphon ago

Just download more storage space.

deleteme123 ago

make the font size smaller

lol.

AlexanderMorose13 ago

This is incredible. You rock so hard! I can't wait to see what the future holds! I'm working with someone on Blockchain website design. I'm still very new at this, but I might have a few solutions to use later when it comes to navigating through the networks that you're trying to build. I'll keep you in mind; until then, keep up the good work!

insanitea ago

Getting a few seeders before making it entirely public might be a good idea.

pby1000 ago

Why don’t you seed it now?

KebabAndNoseRemoval ago

Telling the glow in the darks you've got all of their clearweb fuckery over the last 27 years.

Interesting choice for your health friendo.

Morbo ago

This makes Virge look very glow in the dark himself. Why and how would someone have such an archive covering 27 years? This isn't the sort of thing a normal everyday Joe would do for fun. Seems quite suspicious.

Wahaha ago

If it's just text, then it's fairly easy to do. You just had to be there, 27 years ago. Since the Internet was a very different, smaller place, I can see how something like this started as a for fun project which wouldn't need much resources and then escalate together with the growth the Internet experienced, but by then you were already doing it, so you might as well continue. It's only eating harddrives, and those are cheap.

Morbo ago

It's not as easy as you think. You would have to crawl the entire internet several times an hour to catch the changes. Bandwidth back in the early days of the web was low. Crawling was intensive on the machines at the time. Hard drive space is only cheap today. Back in 1995, 1 TB of storage space would be very costly. It would also require some fancy setups since operating systems of the time could not address 1 TB of storage as one volume. I was there before the web and in the somewhat early days of the academic internet. What you speak of is based on technology as you know it right now. It was not a trivial thing to do any of this. Virge is a liar and did not make good on the drop of the archive since it was all bullshit. If you think you can do it so easily, start today and archive the entire web for just one year. You won't be able to do it, but you will learn that on your own. Come back next year and either drop your incomplete archive or admit defeat.

Wahaha ago

Back in 1992 the entire Internet in text format wouldn't even amount to one gigabyte. Wikipedia is only about 5GB, even today. Nobody claimed to have the entire Internet every hour since 27 years ago. Maybe the crawling was only done once a year.

I never cared for the entire Internet, but I did backup some sites I cared for in a way that lets me browse them as if they still existed. If I only saved it as text it would've been easier. And looking back, 1992 only had like ten websites. Ten. I could've saved those manually. There would've been enough time to improve this by writing scripts that do this for me. OP also never claimed a complete copy. Lots of stuff is behind paywalls and logins anyway and then there's the Internet Archives, which are redundant anyway. But a "text-only archive of the Internet since 1992" - definitively possible. Text-only archive of the entire Internet since 1992? Not so much possible.

Morbo ago

But a "text-only archive of the Internet since 1992" - definitively possible. Text-only archive of the entire Internet since 1992? Not so much possible.

Now you're just playing word games. You're saying it is both possible and impossible based on whether or not it is a complete copy. You can think whatever you want about this. The reality is that OP (Virge) did not make anything available and is a known liar. Semantics and word salad mean nothing since she failed to deliver. Many of us knew she was lying from the get-go and it is obvious now to everyone that she had nothing. Now Virge has disappeared thanks to WhiteRonin constantly calling her out on the bullshit. Good riddance to bad rubbish. Nothing of value was lost.

Wahaha ago

Yes, since you can't access everything (paywall, logins) a complete copy is impossible. I thought that was obvious, but apparently not to you, since you put an emphasis on "complete".

Morbo ago

What paywalls existed in 1992, moron? And complete would mean ALL changes on ALL sites, but I guess that was not apparent to you. So either put up an archive or prove yourself a liar like Virge. We'll wait for your archive to drop.

Wahaha ago

"I have a text-only archive of the Internet since 1992. 52TB raw including edits. 22TB without. Crawlers have indexed 82 search engines since AltaVista."

Where does it say "complete"?

Anoxim ago

I dunno. I had a friend who used to steal HDD from Walmart so he could fill them with screen caps of 4 chan. Last I remember he had at least 13 500GB HDD full of 4chan caps, mostly from /b/. This was back in the late 00s like 07 - 09.

abattoirdaydream ago

CP on board likely. Trust but verify.

Smells_Like_Tacos ago

A lot of techies like thier own hard copy. A copy they can work with without anybody else watching.

Morbo ago

The vast majority of techies don't have any means to download or store a 52 TB file. This is not some like a movie or TV show sized file. This kind of storage would set you back a lot of money and is not something most techies would even attempt to do. Imagine building an addition to your house to store the texts in the Library of Congress. That's the physical equivalent of this sort of thing. It's not something trivial.

Smells_Like_Tacos ago

The vast majority of techies that you know.

Morbo ago

I have been programming professionally for over 3 decades. I have never had the need to buy hard drives as a hobby. If you need to do this for your coding, then you're doing it wrong.

Smells_Like_Tacos ago

Whatever

TheBookWasBetter ago

...well if he is monitoring security vulnerabilities, he is probably not your average nut. Grade A autist we got here.

Morbo ago

Do you also believe israel is our greatest ally and that the West can't survive without diversity?

TheBookWasBetter ago

No, I do not believe those things.

Gopherurself ago

Grade AA

NoisySilence ago

Why would a glow-nigger make such a claim? What do you think the intended reaction is?

Morbo ago

What if the archive contains disinfo all over it? How can anyone verify the contents are genuine? Why would he have this in the first place? This is a costly endeavor just on storage and hosting alone. It doesn't make sense for a regular person to be able to do this sort of thing.

NoisySilence ago

There are too many assumptions in your post. OP is obviously not a normal goat fag.

Morbo ago

OP is obviously not a normal goat fag

And that doesn't raise any suspicions for you?

Warnos44 ago

I dunno, if I had the know how and the money to spend to do so, it seems like a very conscientious thing to do for human kind, especially after Kaczynski's tech critical manifesto.

Wahaha ago

If it's just text, you don't need much know-how. And even if you want to save the entire page with javascript, pictures and everything, it's still not that difficult, since there are tools floating around that will do just that. You just need something to crawl the entire Internet, something that saves everything and a harddrive big enough. Once you're done, repeat forever. This may sound complicated, but as there are tools that do exactly this it's basically three lines of code.

Morbo ago

But how could anyone validate that the archive isn't adulterated? What if there is a lot of disinfo in it because it has been altered for nefarious reasons? I would not trust it.

gazillions ago

You aren't supposed to trust what you read. When you read Aristotle you don't trust what he's saying. You think about it and decide for yourself which parts are agreeable and which parts aren't

You aren't supposed to trust whatever politician you vote for either. You end up with cult celebrities. China is still full of Maoists who believe Mao's deadly incompetence and malignancy was OK because they trusted him and still do today.

Q followers trust the plan when they're supposed to be judging the objective results of the guy they elected.

Read and judge everything for yourself because hopefully, you're an adult and not a breast feeding infant looking for someone to only ever tell you the truth you want to hear.

Morbo ago

You just argued that facts do not exist because it's all supposed to be up to your interpretation of things. That's not a world I want to live in. That's clown world. Like with science, facts should be verifiable and demonstrable and repeatable. They are not supposed to be open to interpretation or consensus. Can mathematics not be trusted because the rules for mathematics are written down? Is the solution to a mathematical formula subject to your interpretation of it? If so, Common Core has ruined you.

Dupinstein ago

I don’t think OP’s intention was to imply every interpretations is valid.

Consistency is the heart of truth. If a view of the world can explain all observations, all accounts of observation, and all other views in a manner that is consistent and does not result in logical contradictions, that view is true.

gazillions ago

Where did you get the idea that listening to someone had anything whatsoever to do with the known recipe of mathematics. They are not connected. You have to be able to recognise the difference between subjective and objective. If you expect human interaction to be objective, you are in a fantasy world and must be constantly disappointed and frustrated all the time. Do you expect a mathematical rating system to be applied to works of art? Maybe you're maladjusted. Maybe you're autistic. maybe you got caught up in the internet trend of people that wanted to be autistic because they heard the word was interchangeable with genius. It isn't.

Morbo ago

Where did you get the idea that listening to someone had anything whatsoever to do with the known recipe of mathematics.

From this:

You aren't supposed to trust what you read.

You're now changing your words. You said "read" in the above but "listening" in your response. That's being dishonest and moving the goal posts.

gazillions ago

Yeah right. Have fun with your game.

sirRantsalot ago

Creep alert! Stranger danger!

The_Duke_of_Dabs ago

Ho. Ly. Shit. Kaboom.

Gopherurself ago

@Ho-lee-fuk

TheWorstImaginable ago

I miss that guy

SecurityReasons ago

Bang ding ow

BlueDrache ago

Wai tu Lo

Gopherurself ago

We gon die

CameraCode0 ago

You are one interesting dude.

fuckfuckfuck1 ago

Q predicted this.

yurisrevenge ago

thanks comrade

engiebengie ago

Impressive

Ngrfgt ago

You should make a subverse so people can post findings

ArcherMcTaco ago

Thats really impressive. Honestly this is an amazing time capsul.

Wiglaf ago

Amazing. You're doing fantastic work if this is real. I can't imagine what kind of treasures or interesting statistics or things you can find digging through it. Stuff that has been memory holed, old news articles, etc.

Wonder who has been doing the snooping on your machine and why?

robot7247 ago

Stuff that has been memory holed, old news articles, etc.

I want to see the article on the Greenland vineyards found around 2000 by Danish archeologists.

It was up for a while then 404. After awhile I began to doubt I'd actually read it.

robot7247 ago

Stuff that has been memory holed, old news articles, etc.

I want to see the article on the Greenland vineyards found around 2000 by Danish archeologists.

It was up for a while then 404. After awhile I began to doubt I'd actually read it.

I want to see the article on the Greenland vineyards found around 2000 by Danish archeologists.

It was up for a while then 404. After awhile I began to doubt I'd actually read it.

Broc_Lia ago

Where the heck did you get it from?

GrandNagus ago

Dude is a pure bred autist, none of that artificially created vaccine autism, he probably has a hoarding issue and it just manifested as severe data hoarding.

No doubt this guy did it all by himself.

Jaegerjaques ago

Hundreds of years of selective breeding for only the most autistic of traits!

ChimpStenographer ago

E-mails?

DeputyPutt ago

IF this is real, it's just a home crawler that has sucked up a lot of text, no files or images.

I've considered doing something similar given how cheap things are these days.

Sheetz ago

Wow this is incredible!

shmuklidooha ago

52 TB? That seems a bit small. I'd think that it would be more along the lines of PB or even EB.

peacegnome ago

Text only could be low, especially if English (or latin alphabet only) only and compressed.

TheBudhha ago

Yeah. But explain 20plus TB of "edits".

peacegnome ago

Yeah, remember when a company having a T1 was the shit...

robot7247 ago

Mine had a fractional T1. We were happy, for a time...

Smells_Like_Tacos ago

Pretty sure he scraped just top level pages, first 50 or something. Let's ask.

ArielQflip ago

Agreed.

NarrativeControl ago

But explain 20plus TB of "edits"

He probably crawled the same texts at different points in time, several times. Those are the "edits": the differences between those texts. At least that's my guess.

TheBudhha ago

I have a funny feeling his files will magically become corrupt or go missing before he is able to post them.

Rotteuxx ago

Nonsense, he's a grade A builder. Nothing but quality to offer !

sirRantsalot ago

You sound pretty certain.....

NotHereForPizza ago

Well, I think Crensch has been right once or twice. Maybe it'll be a third time. Who knows?

sirRantsalot ago

All the creepiest crawlies are out today.

NotHereForPizza ago

huh?...

sirRantsalot ago

I was told you were a pedo.

NotHereForPizza ago

And? You just believe people for no reason?

sirRantsalot ago

I was drstrangegov, of old.

NotHereForPizza ago

That sounds right.

I don't know why you never saw I was genuine.

sirRantsalot ago

Because all of the people I spoke to that I spoke to on a daily basis didn't share that perspective.

NotHereForPizza ago

I'm sure these ambiguous people were very helpful to you. First, they lie to you, then they turn you in to a complete retard that can't think for yourself.

You know, people have tried to say some shit about me being a pedo ever since I first came to this site. Why? I think mostly because people saw that my name said something about pizza.

As it turns out, I was tipped off about Crensch and Vindica8tor being precisely the thing that they are - shills. Paid? Who knows. All I know is that they have a very particular agenda at this site that is manifesting itself very transparently.

This is the part where you say I'm deflecting. So, I'll address the same topic in my conclusion - the people telling you I'm a pedophile are wrong. I'm certainly not. I've worked for years to coordinate with other people, exchanging knowledge and doing at least somewhat technical research, to help corroborate open source evidence that exposes the very people you accuse me of being.

How about this: I'll give you a lead, which you can see pretty plainly for yourself, which exposes a great many instances of fake store fronts that are actually trafficking rings of some sort or child porn hosting exchanges, or some type of event. What you're looking for is a "Z". It's often in place of an "S", but it doesn't necessarily have to be. We're talking daycare centers, halfway houses, orphanages, immigrant facilities, etc. This is prevalent and seen simply without even taking a close look in to these things. I'll even offer you one more thing to really look at and ponder: why do so many of these sites have a "Members Only" area, but lack a registration page?

Now, stop listening to people that clearly offer you nothing.

sirRantsalot ago

Wow. Okay.

NotHereForPizza ago

Besides, Justin is clearly a retard.

sirRantsalot ago

Is this got to be like a fyre festival sort of thing? That was quieted fast. Fancy lad.

NotHereForPizza ago

Just who do you think I am, exactly?

sirRantsalot ago

Wasn't directed at you at all. Just typed what I was thinking.

sirRantsalot ago

He seems to be an okay guy. Bit obsessed with forks.

NotHereForPizza ago

I wonder why.

sirRantsalot ago

These are good boys. Maybe they're misguided.

NotHereForPizza ago

I'm sure good people lie about other people being pedophiles.

ItsOk2bArian ago

Soooooo, yes to the pedo question? Im just trying to follow along

NotHereForPizza ago

Maybe you missed the part where I'm certainly not a pedo and was also able to convey particular methods of pedophiles.

Are you just here for the spin?

sirRantsalot ago

Good people lie too and you know it.

sirRantsalot ago

Why? I really have no idea.

sirRantsalot ago

They gave false testimony, laid out a convincing argument. I'm kind of dumb, too. But nobody believes it because I have a large vocabuditty.

ShakklezthaKlown ago

why do you sound like a cop playing dumb?

NotHereForPizza ago

They gave false testimony and laid out a convincing argument.

playing?

ShakklezthaKlown ago

edited

NotHereForPizza ago

what?

sirRantsalot ago

I'm a nobody. Just a hyperborean wanderer, marvelling at what I see.

ShakklezthaKlown ago

something in the air.

NotHereForPizza ago

You shouldn't just believe people on this website. They've clearly lied to you.

sirRantsalot ago

They're the people that made the website.

NotHereForPizza ago

That's cute.

sirRantsalot ago

The sbbh people.

NarrativeControl ago

Yeah, I hope he has his opsec completely up to the task. Any tiny mistake like exposing his IP address for a second and he's screwed.

BearDolphin1488 ago

Shukran!!! Shaloms!!!

Lavender7 ago

I have a big cock.

ViperCarbz ago

Someone took the blue pill.

Nosferatjew ago

And you shoot 10,000 bullets a day.

6MAmZPaZ ago

per second ... caliber-magazine-clip

moirai11 ago

I approve this. I hope it's true.

virge ago

Magnet will be added once the appropriate precautions are in place.

10GB sequential bonded uplink. Security layering has been underway for the past 2 months.

SearchVoatBot ago

This comment was linked from this v/SoapboxBanhammer submission by @MadJackChurchill.

Posted automatically (#63861) by the SearchVoat.co Cross-Link Bot. You can suppress these notifications by appending a forward-slash(/) to your Voat link. More information here. (@virge: Click here to suppress your crosslink notifications from @MadJackChurchill)

goatboy ago

I want you to have my babies, you sexy minx you!

Blompf ago

Yes please.... Archives are an absolute necessity, but I do not have that much storage right now.

ArielQflip ago

Get this to a Austist you trust! Dam!!!!

That's fucking impressive if true!!!!

You seeding where?

Gopherurself ago

A FUCKIN MEN

Muh-Shugana ago

Seriously, pol could crack this wide fucking open, and there are plenty of takers.

Splooge ago

Feels like we're on the... @virge... of something big here.

I'll see myself out.

Gopherurself ago

Haha hell yeah those niggers would

TheTrigger ago

This. Post a link. I have seedboxes. If legit, why not.

Metanoiac ago

That is fucking impressive man. Did you make it yourself?

HighEnergyLife ago

Whoa

SurfinMindWaves ago

Fascinating. How do you navigate through it all?

toobaditworks ago

Netscape Navigator of course.

Wowbagger ago

seems like this is a job for Hadoop

virge ago

Overall, poorly.

auchtung ago

Can you blast through it easily with fgrep?

kissaki ago

xDDD

I would say so.

Well done and congrats backing up the entire internet btw! That's some weapons grade autism right there fren.

sirRantsalot ago

Holy shit..... godspeed fella

AlphaOmega ago

I let out a belly laugh. I can only

Imagine.

LiamOdinThomas ago

Thats amazing!