Page 1 of 1
Canonical web resource of versions
Posted: Tue May 12, 2020 9:03 pm
by Hanse00
This is part question part idea, as I don't know if it's already easily attainable somewhere.
As someone who runs a server using a Docker container I've created myself, I'd love a way to automate the process of building a new container, when a new version of the game is out.
My best idea currently is scraping the HTML of
https://factorio.com/download-headless, but it would be ideal if there was some endpoint that would expose the list of versions, in a machine readable format, eg. JSON, XML, whatever.
Ideally the data should look something like (Loosely using python data structures as pseudo-code):
Code: Select all
{
[
tag: "latest",
version: "0.18.24",
],
[
tag: "stable",
version: "0.17.79",
],
[
tag: "0.17.79",
version: "0.17.79",
],
[
tag: "0.17.74",
version: "0.17.74",
],
}
This would make it possible to tell in advance, which version the latest stable and experimental are, and which versions are available in general.
Since the game itself has a way of knowing that a new version is available, I presume some sort of infrastructure along these lines already exists?
I believe an official solution would be significantly easier than scraping the HTML, since the data is already there somewhere on the backend, and being converted into HTML. Instead of trying to convert it back again, having the original data would be great.
Re: Canonical web resource of versions
Posted: Tue May 12, 2020 9:35 pm
by Koub
Re: Canonical web resource of versions
Posted: Tue May 12, 2020 9:42 pm
by Jap2.0
https://updater.factorio.com/get-available-versions will give you a list of possible update paths, and
https://updater.factorio.com/get-download-link will get you a link to download something or other (you have to send that some parameters but I'm not quite sure what). See
https://github.com/narc0tiq/factorio-updater, and maybe
https://github.com/Bisa/factorio-init.
Re: Canonical web resource of versions
Posted: Tue May 12, 2020 9:46 pm
by Hanse00
Yes!
It's a great start, although I think it could do with a few modifications for these kinds of purposes.
It lists which version is currently "stable", but not explicitly which is "latest".
Secondly it lists the update paths (Which makes sense for the purpose it's made for), but it doesn't (as far as I can tell) list canonically which version are currently considered "available" (That is, the versions listed at
https://factorio.com/download-headless).
Re: Canonical web resource of versions
Posted: Thu May 14, 2020 5:13 am
by ssilk
The latest is currently the “experimental” tab of that last link from above:
https://factorio.com/download-headless/experimental
Re: Canonical web resource of versions
Posted: Thu May 14, 2020 5:35 am
by Koub
@Ssilk : that's exactly what OP tries to avoid : having to parse an HTML page.
Hanse00 wrote: ↑Tue May 12, 2020 9:03 pm
My best idea currently is scraping the HTML of
https://factorio.com/download-headless, but it would be ideal if there was some endpoint that would expose the list of versions, in a machine readable format, eg. JSON, XML, whatever.
Re: Canonical web resource of versions
Posted: Thu May 14, 2020 5:55 am
by ssilk
It was meant as a temporary solution. html is machine readable (just need the right lib to parse it) dunno which language he’s (she?) programming, but this kind of simple list is parseable with simple regex-stacking.
It’s a BTN solution - of course.
Re: Canonical web resource of versions
Posted: Thu May 14, 2020 9:44 am
by SyncViews
Very much a last resort though as websites generally expect to be able to change the document structure freely, and even if they did care, hard to tell what will or won't break various parser cases.
Fully parsing HTML can be a bit of a pain since it is not valid XML, doable though, or a regex on the source text.
Re: Canonical web resource of versions
Posted: Thu May 14, 2020 4:45 pm
by Hanse00
ssilk wrote: ↑Thu May 14, 2020 5:55 am
It was meant as a temporary solution. html is machine readable (just need the right lib to parse it) dunno which language he’s (she?) programming, but this kind of simple list is parseable with simple regex-stacking.
It’s a BTN solution - of course.
That’s definitely a possible temp solution.
As others have pointed out, I’d like to avoid it though. The page may change over time, but the actual data that’s being used to generate that page on the backend, probably won’t.
Without knowing the details of how the team handles their web-development, it’s impossible to make any clear predictions, but in general it should be a simple ask. There clearly already is a JSON file out there which tracks the upgrade paths, it just needs to also track and label which one is the “latest” version.
But I don’t know who at Wube handles that part of the business, so I don’t have anyone to ask directly.
Re: Canonical web resource of versions
Posted: Thu May 14, 2020 4:50 pm
by Bilka
Re: Canonical web resource of versions
Posted: Thu May 14, 2020 4:54 pm
by Hanse00
Yes!
Is there any documentation explaining what “alpha” means in this context? Is it the full game client?
Re: Canonical web resource of versions
Posted: Thu May 14, 2020 5:00 pm
by Bilka
Hanse00 wrote: ↑Thu May 14, 2020 4:54 pm
Is there any documentation explaining what “alpha” means in this context? Is it the full game client?
It means the same as it does on
https://factorio.com/download - "Alpha is the standard full-featured build."
Re: Canonical web resource of versions
Posted: Thu May 14, 2020 5:02 pm
by Hanse00
Bilka wrote: ↑Thu May 14, 2020 5:00 pm
Hanse00 wrote: ↑Thu May 14, 2020 4:54 pm
Is there any documentation explaining what “alpha” means in this context? Is it the full game client?
It means the same as it does on
https://factorio.com/download - "Alpha is the standard full-featured build."
Awesome!
That’s exactly what I was looking for
On another note, do we have this, and other possible API endpoints (Such as the one with the upgrade paths) documented on the Wiki? I didn’t find anything, although I only searched briefly. It would be a good idea to have these documented for other users.
Are there any guidelines on rate-limiting etc? I don’t intend to hit the endpoint too often, but in general it’s another thing that would be good to share with people.
Re: Canonical web resource of versions
Posted: Thu May 14, 2020 5:22 pm
by Bilka
Hanse00 wrote: ↑Thu May 14, 2020 5:02 pm
On another note, do we have this, and other possible API endpoints (Such as the one with the upgrade paths) documented on the Wiki? I didn’t find anything, although I only searched briefly. It would be a good idea to have these documented for other users.
Yep, I grabbed the link from
https://wiki.factorio.com/Download_API. It didn't have the get-available-versions link, so I also added that just now.
Re: Canonical web resource of versions
Posted: Thu May 14, 2020 5:57 pm
by Jap2.0
That doesn't have
https://updater.factorio.com/get-download-link either, but I don't know how exactly that's used (...because it's not documented there).
Actually, looking more, it might be something along the lines of
Code: Select all
https://updater.factorio.com/get-download-link?username=xxxx&token=xxxx&package=something&from=a&to=b&apiVersion=2
but I've done no testing there. (From
https://github.com/narc0tiq/factorio-up ... actorio.py)
That said, are the updater.factorio.com APIs supposed to be public (as neither of them were documented and they're a different subdomain)?
Re: Canonical web resource of versions
Posted: Thu May 14, 2020 6:02 pm
by Bilka
Jap2.0 wrote: ↑Thu May 14, 2020 5:57 pm
That said, are the updater.factorio.com APIs supposed to be public?
They are not ""public""/official. That's why all the "documentation" on them is done by community members who reverse engineered it.
(As far as I understand, there is no problem with using the API's per se. They just don't have official docs/changelogs/stability guarantees.)
Re: Canonical web resource of versions
Posted: Thu May 14, 2020 6:04 pm
by Jap2.0
Bilka wrote: ↑Thu May 14, 2020 6:02 pm
They are not ""public""/official. That's why all the "documentation" on them is done by community members who reverse engineered it.
Community members... like the wiki admin on Wube's payroll
