You are viewing a single comment's thread.

view the rest of the comments →

B3bomber ago

What's your complete list of archive sites? I want to make sure I don't lose the base URLs for whatever I find to plug new articles in so shit sites don't get traffic.

derram ago

sites = ["theguardian.com", "salon.com", "gizmodo.com", "kotaku.com", "buzzfeed.com", "vice.com", "boingboing.net", 'huffingtonpost.com', 'jezebel.com', 'washingtonpost.com', 'slate.com', 'arstechnica.co', 'wired.com', 'esquire.com', 'polygon.com', 'recode.net', 'curbed.com', 'sbnation.com', 'vox.com', 'eater.com', 'cracked.com', 'engadget.com', 'destructoid.com', 'eurogamer.net', 'gameinformer.com', 'joystiq.com', 'pcgamer.com', 'rockpapershotgun.com', 'theverge.com', 'wehuntedthemammoth.com', 'dailykos.com', 'thedailybeast.com', 'gamasutra.com', 'reddit.com', 'dailydot.com', 'pcgamesn.com', 'AusGamers.com', 'nydailynews.com', 'vocativ.com', 'telegraph.co.uk', 'twitter.com', 'slashdot.org', 'ign.com', 'kotaku.co.uk', 'kotaku.jp', 'gamespot.com', 'gizmodo.co.uk', 'themarysue.com', 'rawstory.com', 'geekparty.com', 'io9.com', 'jalopnik.com', 'gawker.com', 'lifehacker.com', 'deadspin.com', 'silverstringmedia.com', 'feministfrequency.com', 'xojane.com', 'uproxx.com', 'pcauthority.com.au', 'newyorker.com', 'feministing.com', 'facebook.com', 'bostonmagazine.com', 'nymag.com', 'time.com', 'rationalwiki.org', 'cnn.com', 'deadline.com', 'cinemablend.com', 'bostonglobe.com', 'digiday.com', 'fastcodesign.com', 'newsweek.com', 'pcmag.com', 'fortune.com', 'latimes.com', 'dailymail.co.uk', 'indieweb.org', 'techcrunch.com', 'ycombinator.com', 'nationalpost.com', 'newrepublic.com', 'thenation.com', 'alternet.org', 'independent.co.uk', 'theregister.co.uk', 'motherjones.com', 'pbs.org', 'nytimes.com', 'playboy.com', 'mediamatters.org', 'washingtontimes.com', 'theatlantic.com', 'stuff.co.nz', 'rollingstone.com', 'wsj.com', '.mic.com', '//mic.com', 'lawjournalpress.com', 'statescape.com', 'wcco.com', 'codeandtheory.com', 'theintercept.com', 'usatoday.com', 'LWN.net','noahpinionblog', 'nypost.com','techspot.com', 'businessinsider.com', 'reuters.com', 'thestar.com','cbc.ca', 'heatst.com', 'ibtimes.co.uk', 'politifact.com', 'theonion.com','theroot.com', 'venuspatrol.com', 'offworld.com', 'birthmoviesdeath.com', 'giantbomb.com', 'pastemagazine.com', 'rockpapershotgun.com', 'vg247.com', '.zam.com', '//zam.com', 'haaretz.co', 'blackmattersus.com', 'masslive.com', 'mediaite.com', 'politico.eu', 'somethingawful.com', 'culturess.com', 'inverse.com', 'bustle.com', 'imgur.com', 'insidehighered.com', 'complex.com', 'thinkprogress.org','extranewsfeed.com', 'mashable.com', 'affinitymagazine.us', 'newstatesman.com', 'scientificamerican.com', 'thrillist.com', 'decider.com', 'boiledleather.com', 'observer.com', 'lennyletter.com', 'realclearpolitics.com', 'huffpost.com', 'reallifemag.com', 'wonkette.com', 'fossforce.com', 'gamesindustry.biz', 'thehill.com', 'videogamer.com', 'pitchfork.com', 'resetera.com', 'newstatesman.com', 'weeklystandard.com', 'bleedingcool.com', 'gamerevolution.com', 'buzzfeednews.com', 'gfycat.com', 'youtube.com', 'youtu.be', 'hooktube.com', 'msn.com']

B3bomber ago

TY very much!

derram ago

Ah, no those are the sites the bot targets for archiving.

As for the sites it uses:

https://archive.ph/ for general archives.

https://catbox.moe/ for images.

https://invidio.us/ for youtube.

https://nitter.net/ for twitter.

B3bomber ago

Still useful, thank you for this list. Hopefully you travel along wherever we end up:)