Canonical web resource of versions

Post your ideas and suggestions how to improve the game.

Moderator: ickputzdirwech

Post Reply
Hanse00
Long Handed Inserter
Long Handed Inserter
Posts: 97
Joined: Mon Feb 25, 2013 6:07 pm
Contact:

Canonical web resource of versions

Post by Hanse00 »

This is part question part idea, as I don't know if it's already easily attainable somewhere.

As someone who runs a server using a Docker container I've created myself, I'd love a way to automate the process of building a new container, when a new version of the game is out.

My best idea currently is scraping the HTML of https://factorio.com/download-headless, but it would be ideal if there was some endpoint that would expose the list of versions, in a machine readable format, eg. JSON, XML, whatever.

Ideally the data should look something like (Loosely using python data structures as pseudo-code):

Code: Select all

{
  [
    tag: "latest",
    version: "0.18.24",
  ],
  [
    tag: "stable",
    version: "0.17.79",
  ],
  [
    tag: "0.17.79",
    version: "0.17.79",
  ],
  [
    tag: "0.17.74",
    version: "0.17.74",
  ],
}
This would make it possible to tell in advance, which version the latest stable and experimental are, and which versions are available in general.

Since the game itself has a way of knowing that a new version is available, I presume some sort of infrastructure along these lines already exists?
I believe an official solution would be significantly easier than scraping the HTML, since the data is already there somewhere on the backend, and being converted into HTML. Instead of trying to convert it back again, having the original data would be great.

Koub
Global Moderator
Global Moderator
Posts: 6788
Joined: Fri May 30, 2014 8:54 am
Contact:

Re: Canonical web resource of versions

Post by Koub »

Koub - Please consider English is not my native language.

Jap2.0
Smart Inserter
Smart Inserter
Posts: 2338
Joined: Tue Jun 20, 2017 12:02 am
Contact:

Re: Canonical web resource of versions

Post by Jap2.0 »

https://updater.factorio.com/get-available-versions will give you a list of possible update paths, and https://updater.factorio.com/get-download-link will get you a link to download something or other (you have to send that some parameters but I'm not quite sure what). See https://github.com/narc0tiq/factorio-updater, and maybe https://github.com/Bisa/factorio-init.
There are 10 types of people: those who get this joke and those who don't.

Hanse00
Long Handed Inserter
Long Handed Inserter
Posts: 97
Joined: Mon Feb 25, 2013 6:07 pm
Contact:

Re: Canonical web resource of versions

Post by Hanse00 »

Koub wrote:
Tue May 12, 2020 9:35 pm
Something like this : https://updater.factorio.com/get-available-versions ?
Yes!

It's a great start, although I think it could do with a few modifications for these kinds of purposes.

It lists which version is currently "stable", but not explicitly which is "latest".
Secondly it lists the update paths (Which makes sense for the purpose it's made for), but it doesn't (as far as I can tell) list canonically which version are currently considered "available" (That is, the versions listed at https://factorio.com/download-headless).

User avatar
ssilk
Global Moderator
Global Moderator
Posts: 12743
Joined: Tue Apr 16, 2013 10:35 pm
Contact:

Re: Canonical web resource of versions

Post by ssilk »

The latest is currently the “experimental” tab of that last link from above:
https://factorio.com/download-headless/experimental
Cool suggestion: Eatable MOUSE-pointers.
Have you used the Advanced Search today?
Need help, question? FAQ - Wiki - Forum help
I still like small signatures...

Koub
Global Moderator
Global Moderator
Posts: 6788
Joined: Fri May 30, 2014 8:54 am
Contact:

Re: Canonical web resource of versions

Post by Koub »

@Ssilk : that's exactly what OP tries to avoid : having to parse an HTML page.
Hanse00 wrote:
Tue May 12, 2020 9:03 pm
My best idea currently is scraping the HTML of https://factorio.com/download-headless, but it would be ideal if there was some endpoint that would expose the list of versions, in a machine readable format, eg. JSON, XML, whatever.
Koub - Please consider English is not my native language.

User avatar
ssilk
Global Moderator
Global Moderator
Posts: 12743
Joined: Tue Apr 16, 2013 10:35 pm
Contact:

Re: Canonical web resource of versions

Post by ssilk »

It was meant as a temporary solution. html is machine readable (just need the right lib to parse it) dunno which language he’s (she?) programming, but this kind of simple list is parseable with simple regex-stacking.

It’s a BTN solution - of course.
Cool suggestion: Eatable MOUSE-pointers.
Have you used the Advanced Search today?
Need help, question? FAQ - Wiki - Forum help
I still like small signatures...

SyncViews
Filter Inserter
Filter Inserter
Posts: 295
Joined: Thu Apr 21, 2016 3:17 pm
Contact:

Re: Canonical web resource of versions

Post by SyncViews »

Very much a last resort though as websites generally expect to be able to change the document structure freely, and even if they did care, hard to tell what will or won't break various parser cases.

Fully parsing HTML can be a bit of a pain since it is not valid XML, doable though, or a regex on the source text.

Hanse00
Long Handed Inserter
Long Handed Inserter
Posts: 97
Joined: Mon Feb 25, 2013 6:07 pm
Contact:

Re: Canonical web resource of versions

Post by Hanse00 »

ssilk wrote:
Thu May 14, 2020 5:55 am
It was meant as a temporary solution. html is machine readable (just need the right lib to parse it) dunno which language he’s (she?) programming, but this kind of simple list is parseable with simple regex-stacking.

It’s a BTN solution - of course.
That’s definitely a possible temp solution.

As others have pointed out, I’d like to avoid it though. The page may change over time, but the actual data that’s being used to generate that page on the backend, probably won’t.

Without knowing the details of how the team handles their web-development, it’s impossible to make any clear predictions, but in general it should be a simple ask. There clearly already is a JSON file out there which tracks the upgrade paths, it just needs to also track and label which one is the “latest” version.

But I don’t know who at Wube handles that part of the business, so I don’t have anyone to ask directly.

Bilka
Factorio Staff
Factorio Staff
Posts: 2864
Joined: Sat Aug 13, 2016 9:20 am
Contact:

Re: Canonical web resource of versions

Post by Bilka »

Is this what you are looking for? https://factorio.com/api/latest-releases
I'm an admin over at https://wiki.factorio.com. Feel free to contact me if there's anything wrong (or right) with it.

Hanse00
Long Handed Inserter
Long Handed Inserter
Posts: 97
Joined: Mon Feb 25, 2013 6:07 pm
Contact:

Re: Canonical web resource of versions

Post by Hanse00 »

Bilka wrote:
Thu May 14, 2020 4:50 pm
Is this what you are looking for? https://factorio.com/api/latest-releases
Yes!

Is there any documentation explaining what “alpha” means in this context? Is it the full game client?

Bilka
Factorio Staff
Factorio Staff
Posts: 2864
Joined: Sat Aug 13, 2016 9:20 am
Contact:

Re: Canonical web resource of versions

Post by Bilka »

Hanse00 wrote:
Thu May 14, 2020 4:54 pm
Is there any documentation explaining what “alpha” means in this context? Is it the full game client?
It means the same as it does on https://factorio.com/download - "Alpha is the standard full-featured build."
I'm an admin over at https://wiki.factorio.com. Feel free to contact me if there's anything wrong (or right) with it.

Hanse00
Long Handed Inserter
Long Handed Inserter
Posts: 97
Joined: Mon Feb 25, 2013 6:07 pm
Contact:

Re: Canonical web resource of versions

Post by Hanse00 »

Bilka wrote:
Thu May 14, 2020 5:00 pm
Hanse00 wrote:
Thu May 14, 2020 4:54 pm
Is there any documentation explaining what “alpha” means in this context? Is it the full game client?
It means the same as it does on https://factorio.com/download - "Alpha is the standard full-featured build."
Awesome!

That’s exactly what I was looking for :)

On another note, do we have this, and other possible API endpoints (Such as the one with the upgrade paths) documented on the Wiki? I didn’t find anything, although I only searched briefly. It would be a good idea to have these documented for other users.

Are there any guidelines on rate-limiting etc? I don’t intend to hit the endpoint too often, but in general it’s another thing that would be good to share with people.

Bilka
Factorio Staff
Factorio Staff
Posts: 2864
Joined: Sat Aug 13, 2016 9:20 am
Contact:

Re: Canonical web resource of versions

Post by Bilka »

Hanse00 wrote:
Thu May 14, 2020 5:02 pm
On another note, do we have this, and other possible API endpoints (Such as the one with the upgrade paths) documented on the Wiki? I didn’t find anything, although I only searched briefly. It would be a good idea to have these documented for other users.
Yep, I grabbed the link from https://wiki.factorio.com/Download_API. It didn't have the get-available-versions link, so I also added that just now.
I'm an admin over at https://wiki.factorio.com. Feel free to contact me if there's anything wrong (or right) with it.

Jap2.0
Smart Inserter
Smart Inserter
Posts: 2338
Joined: Tue Jun 20, 2017 12:02 am
Contact:

Re: Canonical web resource of versions

Post by Jap2.0 »

Bilka wrote:
Thu May 14, 2020 5:22 pm
]
Yep, I grabbed the link from https://wiki.factorio.com/Download_API. It didn't have the get-available-versions link, so I also added that just now.
That doesn't have https://updater.factorio.com/get-download-link either, but I don't know how exactly that's used (...because it's not documented there).

Actually, looking more, it might be something along the lines of

Code: Select all

https://updater.factorio.com/get-download-link?username=xxxx&token=xxxx&package=something&from=a&to=b&apiVersion=2
but I've done no testing there. (From https://github.com/narc0tiq/factorio-up ... actorio.py)

That said, are the updater.factorio.com APIs supposed to be public (as neither of them were documented and they're a different subdomain)?
There are 10 types of people: those who get this joke and those who don't.

Bilka
Factorio Staff
Factorio Staff
Posts: 2864
Joined: Sat Aug 13, 2016 9:20 am
Contact:

Re: Canonical web resource of versions

Post by Bilka »

Jap2.0 wrote:
Thu May 14, 2020 5:57 pm
That said, are the updater.factorio.com APIs supposed to be public?
They are not ""public""/official. That's why all the "documentation" on them is done by community members who reverse engineered it.

(As far as I understand, there is no problem with using the API's per se. They just don't have official docs/changelogs/stability guarantees.)
I'm an admin over at https://wiki.factorio.com. Feel free to contact me if there's anything wrong (or right) with it.

Jap2.0
Smart Inserter
Smart Inserter
Posts: 2338
Joined: Tue Jun 20, 2017 12:02 am
Contact:

Re: Canonical web resource of versions

Post by Jap2.0 »

Bilka wrote:
Thu May 14, 2020 6:02 pm
They are not ""public""/official. That's why all the "documentation" on them is done by community members who reverse engineered it.
Community members... like the wiki admin on Wube's payroll ;)
There are 10 types of people: those who get this joke and those who don't.

Post Reply

Return to “Ideas and Suggestions”