coaltide | Coal All The Way

Entry tags:

2019

Coal All The Way

yuletide |

yuletide_admin | Writing Post
2019 Collection | 2019 Madness
2019 App | Letter Post (LJ)

Flat / Recent | Top-Level (Last) | Search

Flat | Top-Level Comments Only

https://treatless.herokuapp.com/

Hello, so this is a very very rough draft right now. Some points:

- It's easy to check for unrevealed gifts, so this is limited to unrevealed collections right now. Might change that later, but that'll be slower.
- Performance is already kinda slow, probably will do more work on that near CB since that seems a decent size to test.
- Probably gonna noodle around with making the list sortable eventually.

If you have feature requests that would be useful for finding/writing treats, let me know and I can try to figure something out!

Nice! /coadie cheer

I am honestly too lazy to host a server (that isn't just static), but it's nice seeing someone do it for challenges.

SA pssst your Cristmas example breaks for me

CYRT

Is it? It's working for me, though it does take forever. Do you get a 500 error or just stalling out?

Application error, 503 POST error to https://treatless.herokuapp.com/search

Feels like a timeout to me, since spinny spinny, but I imagine you're doing quite a bit of paging work in the background so no real idea.

SC

Ah, your example has Critmas capitalised; it works with lowercase and only lowercase, methinks.

On this note, welcome to my life. I'm so happy to see another coder around, flailing at AO3's lack of API in the same cesspool of suffering as me :3

CYRT

Ah, my logs actually say Too Many Requests error, will somehow have to handle that gracefully at some point. I TRULY DON'T KNOW IF THERE'S A BETTER WAY TO DO THIS THAN SCRAPING EVERYONE'S GIFT PAGES, AO3 please throw me a fucking bone (and by bone I mean "can it be harder to query for a collection any given fic is in").

I don't know how you're handling the massive amount of data with e.g. Yuletide requests, did you just download it all one time so it's all in a DB on the back end?

This, my dear friend, is why I haven't set up a dynamic spin to win click to generate your own exchange app meta-app yet...

With closed signups that only reveal prompts after like Yuletide, yes. I dump the tagset before signups, then dump the prompts when they are revealed.

With the running signup exchanges like ToT and CB, I leave my computer turned on because I am cheap as hell and run the scraping as a cron job. It is a sad trumpet life.

Every time I get close to getting a meta-app stood up, I lose steam because I'm mostly tired from coding 24/7 in real life. Maybe this year... I'm pretty close. Just need to stand up an actual backend server for the running signups exchanges.

Sans API, to the main point again, there's really no good way other than working smart. I page incrementally so I don't lose data. You might want to chuck already-scraped exchanges into a backend and only update them max once idk every 15 minutes or so.

And yeah, with unrevealed exchanges AFAIK you are shit outta luck. I'm better off with Yuletide and some of the other exchanges I help with because mods are willing to give me mod rights so I can at least get the works pages to show usernames, but you have to build a certain level of trust with them that you won't go on a Delete All Assignments accidental click rampage.*

* No shade to Morbane, I literally have had nightmares of accidentally coding the app and blowing up Yuletide because I fatfingered something in the code. I am very, very, VERY paranoid.

NC

You can get a list of someone's gifts in a particular collection with a URL like this one:

https://archiveofourown.org/users/{LOGIN}/gifts?collection_id={COLLECTION}

(The page itself doesn't look right, so I suspect it's not supposed to exist. But it does, so I personally use it to count gifts. Not sure if it'd be an improvement over the normal gift pages for you, but I thought I'd mention it since you mentioned querying for which collections a fic is in.)

CYRT

Ooooh, I can use this to count gifts for revealed collections faster, that's really useful! Thanks coalie!

If you could scrape the requests and combine so we could see what fandoms treatless people have requested that'd be dope. Although as a mod I would prefer to have that be an option that could be selected rather than a default, because I might want to post the list without the fandoms for whatever reason.

Also as a mod the ability to contact you to have this tool exclude people (defaulters, mostly) by default for everyone would be nice? From a work-that-I-have-to-do standpoint I don't mind having to delete people off my of official list like I usually do, but I worry it will bring further scrutiny on defaulters if it's really easy to compare my list with this tool's list and see who's missing.

CYRT

Scraping requests for fandoms might be a way bigger project, but I'll look into it!

Removing defaulters from the list should be doable, I'll think on what the best way to do that is (might just end up with having mods email me, tbh).

Flat | Top-Level Comments Only

Coal All The Way

Treatless App Draft

Re: Treatless App Draft

Re: Treatless App Draft

Re: Treatless App Draft

Re: Treatless App Draft

Re: Treatless App Draft

Re: Treatless App Draft

Re: Treatless App Draft

Re: Treatless App Draft

Re: Treatless App Draft

Re: Treatless App Draft

Re: Treatless App Draft

Re: Treatless App Draft

Re: Treatless App Draft