Posted on 2 Comments

The Wayback Machine and the Quest for Deleted Fics

What is the Wayback Machine?

The Wayback Machine is the time machine used by Peabody and Sherman in “Rocky and Bullwinkle.” It’s also the nickname of The Internet Archive (https://web.archive.org/) which, since the late ’90s, has crawled the internet and just. Archived everything it finds. (You can read their history here). People now can enter pages they want to save (I used it to preserve some censored Chinese gay books, for example, entering all the URLs myself to be sure that Wayback captured them), and I don’t even know how else it finds stuff, but it’s pretty amazing. How amazing?

Image

This is their capture of my Tripod anime webpage from when I was in college. Some of the graphics are missing, yeah, but like. I made this website in fricken 1999, and stopped maintaining it in 2001 or 2002. Back then my e-mail address was still “unforth@penpen.com” and webrings were a thing and I was well known for creating Winamp skins in Jasc. That it’s there at all is pretty fucking incredible.

Who cares about your old anime page?

Other than me? No one. BUT. Wayback’s “catch all, save all, store all” approach to archiving means it’s an invaluable tool for finding deleted fic. For example, here’s their capture of “Rock Salt and Feathers,” which was (as far as I know) the first Destiel-specific fic archive made on the internet, and many of the earliest Destiel fics were posted there or x-posted there from LJ.

Image

The owner deleted it in 2010, taking all the fics with it, but many can still be accessed – and saved by my project, and read by anyone who wants to – because they’re in Wayback.

Okay, that’s way more interesting. How do I use Wayback to find stuff like that?

The key to using the Wayback Machine to find old and/or deleted fics is that you need the original url. Thus, teaching someone how to use Wayback to find deleted fics ends up mostly being about teaching someone tricks for finding ancient urls for fics that have been deleted (and occasionally when you find the url you actually discover the fic isn’t deleted at all, which is always nice!). Once you have the URL, the “how to use” part is easy, you just go to web.archive.org and enter the url in the search box.

Image

The bar graph of years shows every time Wayback Machine “captured” (archived) the specific page at that url. Often, each of these captures will be different, especially for websites that update regularly (like an archive or an author’s works page). When you click on a year, you’ll get a calendar, and then you just pick the date and time you want (I highlighted April 18th, 2009, as an example, and because it was my dad’s 68th birthday so why not? It’s also about a month before I personally started watching SPN, ah, memories…). Once you’ve picked the capture you want, it’ll load the next page and show you a capture of it – so here’s a (different than above) capture of Rock Salt and Feathers, dating to within a week of when the website was first founded! The same bar graph is now up top, and you can click on the bar you want to jump to that date and see how the website changed over time – so this capture on April 18th, 2009, is pretty bare bones; by the time of the May capture I screen capped above, things have moved along!

Further, once you’re in an archive of a deleted webpage you can (or at least, you can try) to navigate it as normal, just…all within Wayback’s interface. So like, on this page, I can access their list of new works (and find different ones by trying the different captures)…

…and I can even read them!

Uh oh, better watch out for those 4.20 spoilers. Anyway, the point is – if you’ve got the original URL, you can use it to load a deleted page into Wayback, and then navigate that website as normal…at least up until you try a link that Wayback didn’t archive, and then you’ll hit a “sorry, we don’t have that one” page (I’m not gonna screen cap cause at this rate I’ll hit Tumblr’s image limit in about 2 more minutes). Not everything will be there, ever. Rock Salt and Feathers is unusually well-preserved; when I did a deep-dive and spent three days trying to find things there, I was able to preserve nearly 90% of all the fic I know of that was posted there, and some of the rest I was able to find by tracking down alts for the people who posted there – many (though not all!) had x-posted their works to LJ, and later some ALSO x-posted to AO3, once AO3 existed (Rock Salt and Feathers predates the existence of AO3 by about 6 months).

So, as you can see – using Wayback is the easy part (at least until it isn’t – more on that later…it’s easy on a simple page like Rock Salt and Feathers, hence my using it for examples, but it can get hella complicated for more modern, dynamic websites like AO3). The hard part?

Where am I supposed to get the original URL for a fic that’s been deleted for 5 years or a decade or more?

Google search is your friend (or your preferred search engine I guess, but I always use Google). If you know the username and the exact title, it’s easy – especially using quotes, which is also your friend. So, for example, I couldn’t remember the URL for Rock Salt and Feathers and I didn’t actually have it saved, so I just googled “rocksalt and feathers” (in quotes). It prompt got mad at me and told me rock salt should be two words, and so I changed it, and sure enough the first result was an ancient LJ post that included the links I needed. Which is to say, what you’re really looking for isn’t the “thing itself,” but rather other websites that reference the thing in question. For works that were originally posted on LJ, FF.net, personal websites like Rock Salt and Feathers, or elsewhere, ancient rec lists tend to be winners for finding the links. Learning some search tricks can also help – like, if you don’t know the exact title, try variations, or try just the part you’re sure of. If you remember a quote, try searching for that. If the title is something super common, try adding the author name or, if you don’t know it, search for it using “(name of fic)” destiel. Anything you can think of, remember, etc., will help. Sometimes, you just get as close as you can, and then look through the results, and often there’ll be something close that even if it’s not right, will lead you to a resource that’ll help.

Alternatively, again for older works, searching for a different work that you know was released around the same time. So, like, looking for a fic by…idk…Fossarian? Or cautionzombies? Try search for aesc, or bauble, or obstinatrix, or annundriel – someone else who was active when Fossarian and cautionzombies were. (Obviously knowing some Destiel fandom history helps in this case, but there are enough fandom olds around that even if you don’t know this info, learning it is an ask away). Especially, try searching for a contemporary whose works are still up, because you can get titles for those more easily (for example, in this case, aesc, annundriel and obstinatrix all have some works cross posted on AO3, so finding the titles is easier, and then you just…keep going til you find what you want). You can also try looking for works where they were betas or editors or gift-recipients, and/or you can kinda…map out…their old friends groups, by seeing who commented where. For example, looking for links to cautionzombies stuff? cautionzomes and annundriel were friends, which I learned by poking around a fuck-ton, and annundriel’s accounts are still up, and some old cautionzombie links can be found in annundriel’s journals. The links don’t work but that’s not the point, you just need something to plug into Wayback!

And, as a side note – just because an old LJ link is dead, don’t assume that the work is lost! Many of those authors x-posted onto AO3 once they had AO3 accounts (heck, Gedry was continuing to back up works to AO3 as recently as last year), and even among those who didn’t (such as annundriel or CloudyJenn, who each only backed up a few) they often simply ported their accounts to Dreamwidth, so you can find their works just by reformatting their LJ url (username.livejournal.com) to a dreamwidth url (username.dreamwidth.org – works for me too, if you want to see the awful shit I wrote in 2005). Also, sometimes you’ll find they x-posted to FF.net but not AO3 (which, granted, presents FF.net own array of challenges for backing up, but that’s for a different post – drop me an ask if you want me to write that up sooner rather than later, otherwise I’ll just do it whenever I remember). All of which is to say – before you assume a dead link means a deleted work you can save yourself some trouble (and some heartbreak, Wayback isn’t great for LJ in general because of how LJ posts and blogs were structured) it’s worth your while to take a little extra time and check – okay, was it x-posted? Did the person have alternate usernames they used on different platforms? Did they have a writing community on LJ where they posted (for example, a lot of authors posted their works directly to deancasbigbang.livejournal.com or deancas-xmas.livejournal.com, and also a lot of authors made communities even just for themselves, and those communities remained even when they deleted their personal accounts). Even if you find they deleted across all platforms, it’s easier to find full works from AO3 or FF.net on Wayback than it is to find works from LJ, so it’s worth a try. And, honestly, with really old stuff? Finding the old work x-posted somewhere, or just asking someone like me, or the folks at @destielfanfic, is more likely to find it for you than putting an LJ url into Wayback, though in a pinch that of course is an option too.

Unforth, stop babbling about LJ, I care about deathbanjo, or apokteino, or TamrynEradni, or…

…or anyone who posted on AO3 exclusively, and deleted more recently, yeah, I get it. Of course, the tricks for finding the urls remain similar – rec lists are your friends! But, for AO3, there’s another super handy trick. It doesn’t always work, but it’s by far the best place to start.

Go to @ao3feed-destiel.

Search for the author’s name, and/or the fic title, and/or anything you can remember about the fic.

Since mid-2013, the Destiel AO3 feed Tumblr has logged probably around 75% of all the Destiel that’s been posted. There ARE gaps – works that weren’t initially tagged Destiel, or times when the feed was down and just caught nothing, or “oops the author changed their name four times and I don’t know which one they were using when they posted That Fic,” or “there are three people with very similar usernames” or “the fic is called ‘carry on’ and there are a bajillion fics with the same title.” It’s not perfect, but as a first step it’s essential. Because, whatever you find, it’ll have:

The link to the original AO3 post

The link to the author’s name page at the time

The exact date and time it was originally posted

The original title, tags, etc.

If the work was in a series, the series link

And all of the links can be put into Wayback to help you find The Thing You Want. So, to use a recent example from someone I know doesn’t mind having their stuff distributed (or, in this case, discovered on Wayback)…

When you click on the tinyurl, you get an AO3 error page, but, more importantly, in the enter-the-url bar, you get the original url for the fic! Which, in this case is:

http://archiveofourown.org/works/8447584

And then you can go over to Wayback, and…

Well, lookie there, it’s the fic that HazelDomain locked! (Note that you’ll get a “do you agree to the terms of use” and potentially other pop-ups. Just say yes and click through, there’s no way to avoid them because there’s no way to access these pages in Wayback as if you are “logged in as you,” so the notifications and, in the case of Mature and Explicit works, the “you must be 18+ to proceed…” warnings will pop up every single time (and the 18+ one will cause you depressing issues, which in general just make Mature and Explicit deleted works MUCH harder to find, more on that later, yes this post is really gonna be that long, sorry…)

Now, suppose you weren’t looking for this fic by HazelDomain, but instead were looking for one that ao3feed-Destiel didn’t have on their list. Well, now is when that link to HazelDomain username comes in handy!

http://archiveofourown.org/users/HazelDomain/pseuds/HazelDomain

You can put this directly into Wayback, and it’ll show HazelDomain’s home page or, alternatively, if you loaded the fic above (for example) you can just click where it says HazelDomain below the title, and you’ll get to go to their main page, which’ll list their most recent works (on the date that the capture was taken) and some other links. Tada! You’ve found HazelDomain fics on Wayback.

(Side note on all of this: AO3 links are stable and permanent, which means that they do not change even if the nature of a fic changes. If the fic’s posting date is edited? If the author changes their username? If the title changes? If it’s added or removed from a series or a collection? If it’s orphaned or added to an anonymous collection? The link will never change. That’s how I know that the so-called “orphaned” version of With Understanding is actually a fake – it doesn’t have the same URL as the actual version of With Understanding that apokteino posted. So, if you find a link to a work and it turns out that work has only been orphaned, not deleted, that link will still work! For example…

https://archiveofourown.org/works/13063581

One of sir_kingsley’s link, with the exact same link it had before it was orphaned!)

Okay, but the one I want isn’t on the author’s page even after I checked!

As I mentioned, a basic old site like Rock Salt and Feathers? Very easy to use on Wayback. A complex website like AO3? Much more messy, which means there are a bunch of tricks you can use to try to “get at” the data. There’s always the chance it’s not there at all; a random ficlet by a little known author? Unlikely to have made it into Wayback, unfortunately, especially if the ficlet was Mature or Explicit rated. But, there are bunch of things you can try, and there’s never any guessing which will work until you try. When I’m looking for something that’s been deleted? I try them all.

Trick 1: The “/pseud” trick.

See how in HazelDomain’s author link, it’s listed as “users/HazelDomain/pseuds/HazelDomain”? There’s a few tricks you can use related to this. First, on AO3, both “users/(username)” and “users/(username)/pseud/(username)” function as links (even if the second instance of username isn’t actually a pseud and is just a repeat of the same username, as in the HazelDomain example). As such, they are different urls for Wayback machine searching purposes. Sometimes, when you search “user/(username)” you’ll get results but get none when you search for “users/(username)/pseuds/(username), and vice verse. To Wayback, these are two completely different urls, so you have to check them individually – AO3 knows internally that these links route to the same place but Wayback is just basically taking screen caps (well, HTML text caps) so it doesn’t know they’re equal – so check both!

Trick 2: The “they changed usernames” trick.

If you know that an author changed usernames, try plugging every single one into those “user/(username)” and “user/(username)/pseud/(username)” links. Is it a lot of work? Yes. How bad did you want that fic, again?

(side note: having trouble figuring out if they had alternate usernames? Yeah, it’s a nightmare. Checking old rec lists is one way to find out. If the work is in a series, there’s also a trick – even if the person changes username, the “Series created by: (username)” thing at the top will still show the username they had when they created the series. Or, if they had a fic with a really unusual title, try doing a google search for that title specifically, even if it’s not the one you’re looking for, because the odds that two people used that crazy-specific title are low, and you’ll be able to see results that might give the different name. Or-or, as yet another option…my master spreadsheet lists every alternate name for a given user that I know of…for example, deathbanjo has also been loneprairies, beenghosting, and tumbleweeds. Also note – unlike WORK links, which are stable even if the person changes their username, orphans, etc., “user/(username)” links are NOT stable. If you search for, idk, bellacatbee…

https://archiveofourown.org/users/bellacatbee

…you’ll get an error, even though fairychangeling is bellacatbee and still active…

https://archiveofourown.org/users/fairychangeling

Trick 3: The “/works” trick.

Hope you’re not done giving those “users/(username)” and “users/(username)/pseuds/(username)” links a work out, because you’re not done yet! Those links will just give you their home page, which will only list their 6 (…I think it’s 6???) most recent works. And then you click on “works” at the side and…oh no there’s nothing there! Whelp, whichever link Wayback tried to use (“users/(username)/works” or “users/(username)/pseuds/(username)/works”) …try the other! And then try it for all their username changes, if they had any! Getting frustrated yet? If you’re lucky you’ll have Found The Thing and you can stop, but if you haven’t, we’re not done yet, cause yes, there’s more…

Trick 4: The “?fandom_id=27” trick.

So, I’m writing this guide specifically for Destiel, so this trick is being shared in the SPN-specific format. Every single fandom on AO3 has a fandom ID number. Supernatural’s is 27. If you’re looking for a different fandom, you’ll just have to find it’s number – you can do this by going to any author you know wrote for that fandom, going to their home/main page (users/(username) or users/(username)/pseuds/(username)) and clicking on the fandom – the results will show the fandom_id in the link. So, like, I’ve still got fairychangeling’s page open, Thor is fandom_id 245368, MCU is 414093, Good Omens is 114591, etc. Again, these IDs are stable – fandom_id=27 will ALWAYS be Supernatural, no matter who the writer is. AND, since Wayback treats every single one of these urls as unique, even if “users/(username)/works”/”users/(username)/pseuds/(username)/works” don’t work, “users/(username)/works?fandom_id=27” or “users/(username)/pseuds/(username)/works?fandom_id=27” might. And you know what comes next – yes, it’s try every variation again!

Trick 5: check every capture!

Captures on Wayback are a moment in time, which means there’s always a chance that each one will be different. Trying to find a work that a user wrote in 2011, but Wayback /works is only showing works from 2021 on the first page, and going to page 2 produces a dead link? Try going to the oldest capture. Try going through every single capture, until you find the title you want, if you find the title you want. The /works page wasn’t captured at all? Go through every old version of their main page, and see if there’s any version of it where the story you want was in the 6 most recent works they posted. Etc. Try every capture on every variation of the /users/(username) links. Test and test and test until you either find it or you’ve exhausted your options.

Finding lost fics is about patience and about exhausting every option before you give up. All these small variations that look like nothing? Are another chance that Wayback may have captured the work. Skipping one isn’t gonna do you any favors. There’s never a guarantee. Lots is simply not there. But – more is there then you’ll think if you just try one link then give up.

But I’m not looking for a list of their works, I’m looking for a specific work!

The above tricks are what I use when, for example, I’ve just heard a person deleted their account, and I’m trying to build as complete a list as possible of the works that have been deleted. Further, even if Wayback hasn’t captured the actual work, the /(username) page and the /works page will have the links. Sometimes, those links will help you discover the work was orphaned or moved to anon instead of actually deleted. Other times, you’ll click it, and bam, the fic will be right there in Wayback! Still other times, it won’t be…or at least not apparently. But, sure enough, there are tricks around that too. Before you give up and assume a fic isn’t in Wayback at all, you can try…

Trick 1: Remove the chapter part of the link

So, you’ve got the link to your fic – lets use, idk, “Carry On” by TamrynEradani (I haven’t actually tested this as an example yet, hopefully it works lmao for everything I need to do here… lmao).

The original link to Carry On (found on ao3feed-destiel):

http://archiveofourown.org/works/775352/chapters/1458361

AO3 assigns every work a unique number AND every chapter a unique number. If you put in a work without the “/chapters/####” part in AO3, it auto-routes you to chapter 1 and fills in the chapter number. But, not to beat this dead horse again – Wayback doesn’t know how to do that! It’s entirely literally. It captures only the link, exactly as the link was fed to it. Thus, if you put that link into Wayback? It gets no results. BUT, if you remove the “/chapters/1458361” part (it actually DID loop me to the chapter ID, but when I put it in WITH the Chapter ID, it found nothing – welcome to the joys and vagaries of searching for deleted fics in Wayback…)?

There’s Carry On…at least sort of! Because yes, there’s still a problem – that pesky “Proceed” button. Because you can’t log into Wayback as if it were AO3, and Wayback is (again) literal, you can often end up in annoying cycle where (with Mature and Explicit works) you just get looped back to the “Proceed” page over and over again. There are a couple ways you can try to bypass this.

Trick 2: Check past captures!

Are we learning yet? Yep, this is a repeat. Often, going through every capture will find one or more where, for whatever reason, the Proceed page just…isn’t in the way. I have no idea why that’s the case, but it works – it’s how I opened that HazelDomain fic above, for example. And, it works for Carry On, too – when I tried a different capture of the exact same URL?

There it is!

However, even if that doesn’t work, you still have recourse.

Trick 3: the “?view_adult_work=true” trick.

When you hit that “Proceed” button, AO3 auto-adds on “?view_adult_work=true” but (hits the horse with a stick again) Wayback doesn’t know that necessarily, unless you tell it. So, you can sometimes bypass the endless-loop-of-proceed problem by giving it the direct link instead. In this case…

http://archiveofourown.org/works/775352?view_adult_work=true”

or

http://archiveofourown.org/works/775352/chapters/1458361?view_adult_work=true

(this trick actually DOESN’T work with Carry On, but it DOES work sometimes, especially with one shot mature/explicit works. That said, the “check every capture” trick works more often, so definitely try that first).

Okay, so…getting somewhere, but! Carry On is 34 chapters, and this one I’ve found in Wayback (it’s here by the way – Wayback links? Also stable. https://web.archive.org/web/20131126180609/http://archiveofourown.org/works/775352/chapters/1458361) is showing just the first chapter. And when I try to go to Chapter 2? It gets caught up in that goddamn “Proceed for 18+” thing again, and there are only two captures now, and WHAT DO?”

Trick 4: The “?view_full_work=true” trick

There are two ways to implement this trick. One is easy – when you’re on the page in Wayback, you see that “Entire Work” button over the tags box? YA JUST CLICK IT! It’s like magic! At least, it’s magic when it works. (It does, in this case – if you want to read all of Carry On and don’t want to track it down? https://web.archive.org/web/20130911072416/http://archiveofourown.org/works/775352?view_full_work=true tada!)

And see the difference there? it’s the same link, just with ?view_full_work=true added to the end! So, if you’ve found yourself in a position where you can’t get by the “Proceed” loop, OR where you try to go to Chapter 1, try every link variation, and get nothing? You can always still try:

http://archiveofourown.org/works/######?view_full_work=true

Because there’s always a chance that Wayback captured that even if it didn’t capture the other variations.

Unforth…I’ve read all this…I’ve tried everything…I still couldn’t find the thing! What can I dooooooo….

At this point? You’ve mostly exhausted what you can try in Wayback. But! Wayback actually isn’t the only way to find a lost fic, it’s just the most obvious and most easily used by the public. There are a few others!

1. I already tagged @/destielfanfic, so I won’t again, but they’re a great resource for finding deleted fics that authors have said “yes you may distribute,” and they’ve also got a list of authors who’ve indicated “no.” I used their lists as the base for mine (and their head mod and I trade notes, and fics, semi-regularly and have for years). So, I mentioned Fossarian above? Well, you can find Fossarian fics for download by going to destielfanfic, searching for author Fossarian, and going through the links – for example, “All the Hours Wound” is available in ePub format right here!

2. If you’re willing to delve into Livejournal, spnstoryfinders (https://spnstoryfinders.livejournal.com/) is a still-active community that helps find all sorts of missing SPN stories (not just Destiel) and often posts will have links for x-posts, help with finding alts/different names people have used, or have people volunteering to distribute if contacted. Honestly, personally, I’m too shy to actually contact those people, and even if you’re braver than I if they haven’t posted since 2015 it’s anyone’s guess if you’ll still be able to reach them, but it’s always worth a try!

3. Me. Ask me. Even if it’s not on my list. Drop me a note. I know tricks, as you can see, and I’m just really experienced at this point. I’ve been doing this for years. And, even if I did list most of the tricks I know above, I probably forgot something, and I also have the time (…well, sometimes I do, like when I’m not spending 2.5 hrs writing blog posts about how to use Wayback lmao), and I might know pseuds for a person you don’t, and I have contacts who have collections, and, and, and…

4. Speaking of collections, the Profound Bond Discord mods graciously gave my archive a chat room (it’s #fic-archive-project in the collections section of the server). AND, people who are on that server who have large private fic collections can opt to give themselves the @/archivist role, and when things get deleted or when we look for things, even if I can’t find it, I can tag the other archivists and see if anyone else has it. When I exhaust MY options? That’s where I go. So. You can too, you don’t need me to mediate that, just join the Discord.

5. There’s a smaller, Wayback-esque archive webpage called http://archive.is/. It has way less in it, but I’ve occasionally had luck on there finding LJ stuff that Wayback didn’t have.

6. As a last ditch, you can always try Google. For example, if I google: tamryneradani “carry on” destiel download – the only damn result (I made this search up off the top of my head without testing it so I’m glad it worked lmao) is shiphitsthefans’s master post of TamrynEradani fic which includes download links, because Tamryn made it clear from the moment they deleted that they didn’t mind distribution (I was here then, which is how I know that…). So, like, literally, you want to read Carry On, yes I linked it above on Wayback but you can also just download the e-book from this post. There are all kinds of things in all kinds of pokey places on the internet. There’s a small old archive that got permission from LJ authors to PDF their works and posted about it, with links, on Tumblr, and now a lot of those originals are deleted (I don’t have the link sorry, I didn’t bother to save it after I downloaded everything they had) but the Tumblr posts are still up and the DL links that still work. There’s master posts for fics that have been deleted but the master post still has a functional link to a full PDF. Stuff is everywhere and you don’t know unless you check.

There’s so, so, so much Destiel, and so much as been deleted over the years. When you look, sometimes you’ll strike gold right away by just plugging the link into Wayback and YAY THERE’S THE THING, and sometimes you’ll spend an hour looking and think you finally finally have it and get so close and that last PDF link on the last place you had to check after everything else didn’t pan out will be broken and you’ll kind of want to burn down the internet, but…you’ll know you tried.

This is how I built this archive – that, and downloading as much as possible before it was deleted, so that once it was gone, I didn’t have to find it, cause I already had it. Basically every fic marked as “deleted and looking for copy” on my list? I tried all of this and still couldn’t find it. Not always – sometimes I just don’t have time – but. When I have the time, I check, and I even occasionally check again, just in case I missed something the first time. This is how it goes. You try, and you hope, and sometimes you’ll succeed, and sometimes you won’t. It’s hard, but if you want the fic bad enough…you do the thing.

So. This is my general tutorial on how to use Wayback. What you do with that information is up to you. Don’t ask me for help finding links for things I’ve said I won’t distribute, but if you’re willing to do the leg work and try the above strategies…well, authors can’t do much about Wayback, they lost that level of control the instant they posted their works, and it’s there to be accessed by anyone who knows how (if it’s there at all, anyway, which, well, sometimes it is, sometimes it isn’t).

Now you know.

Go forth and get the fic.

(And if you know of, or learn, some tricks I don’t know? PLEASE DO TELL! I am always ready to learn more!)