Page 1 of 1

Download CAPTCHA prevents automated downloads

Posted: Fri Sep 07, 2018 5:00 am
by keb
Previously, I was able to download updates using curl. The new CAPTCHA on the login page now prevents this.

Is there a ReST API endpoint on the mod portal that presents the latest binary distributions of the client? If not, could we have one? I think using the token stored in player-data.json is an appropriate way to support automated downloads.

Thanks in advance.

Re: Download CAPTCHA prevents automated downloads

Posted: Fri Sep 07, 2018 10:20 am
by Jap2.0
I mean, that's the point of a captcha :P.

But yes, I agree that having an automated way to download it would probably be nice, and using that would prevent some spam and, if need be, make it possible to identify who's misusing it.

Re: Download CAPTCHA prevents automated downloads

Posted: Fri Sep 07, 2018 12:31 pm
by Baughn
The purpose of this is to make Factorio possible to install on NixOS. It has to be automated; the package builder (by design) doesn't allow interactive actions, and the license obviously doesn't allow redistribution.

Previously we've been able to get away with a login script using curl, but that falls afoul of the captcha now. I hope you can work with the NixOS community to find an alternative.

Re: Download CAPTCHA prevents automated downloads

Posted: Fri Sep 07, 2018 12:57 pm
by Zavian
Well one possibility is to have the user manually download the tar.xz, then point the package build script at that local file.

Re: Download CAPTCHA prevents automated downloads

Posted: Fri Sep 07, 2018 2:16 pm
by Sanqui
The CAPTCHA was a necessary measure due to brute force attacks.
We'll discuss the options internally and let you know. I'm interested in supporting automated downloads via an API.

Re: Download CAPTCHA prevents automated downloads

Posted: Fri Sep 07, 2018 3:00 pm
by nuhll
getlatestbeta.php?user=XXX&pw=YYYY
getlateststable?user=XXX&pw=YYYY


It could be so easy, user and pw can ofc be changed by the player id.
Against Bruteforce only chance is to make it too slow. But better is use some sort of API key which is too long to bruteforce.

Re: Download CAPTCHA prevents automated downloads

Posted: Fri Sep 07, 2018 10:06 pm
by Jap2.0
nuhll wrote:getlatestbeta.php?user=XXX&pw=YYYY
getlateststable?user=XXX&pw=YYYY


It could be so easy, user and pw can ofc be changed by the player id.
Against Bruteforce only chance is to make it too slow. But better is use some sort of API key which is too long to bruteforce.
Is it just me or does having the username and password in plain text in the URL seem like a bad idea?

Re: Download CAPTCHA prevents automated downloads

Posted: Sat Sep 08, 2018 6:15 am
by Koub
That's true in HTTP, but then using HTTP is the worst idea.
In HTTPS (with proper HTTPS), your communication should be encrypted, including the URLs, and whatever the way, passwords are sent as plaintext to websites most of the time. Only drawback is the fact this gets in your browser history, so other people who can access your computer might see it.

Re: Download CAPTCHA prevents automated downloads

Posted: Sat Sep 08, 2018 6:34 pm
by daniel34
Factorio already has an API key - the service token, which keb already mentioned in the first post. It is a 30-character hexadecimal string stored in player-data.json (after the first in-game authentication using username and password).

It is used for Factorio updates and for hosting a server, where you can use it in the server-settings.json file instead of username and password. I think (but I have never checked for myself) that it is also used to download mods from the mod portal in-game.

Using a token instead of a password has the advantage that it is restricted as to what you can do with it; you can use it to download the game or mods and host a server, but you are unable to login to factorio.com and view/change account settings. The account owner can invalidate the current token on the Profile page, making it useless.

I'm not sure how up-to-date it is, but the wiki actually has a link to a service that returns that service token: https://wiki.factorio.com/Web_authentication_API
EDIT: Also see Unofficial Factorio Web API Documentation [Development-Tools]

Re: Download CAPTCHA prevents automated downloads

Posted: Sun Sep 09, 2018 3:20 pm
by nullvoid
That seems promising, but there doesn't appear to be a publicly known way to use this api to download whole versions.

In particular, while get-available-versions reports what the stable version is for each platform, you can't use it with get-download-link like you can for the from/to pairs.

Re: Download CAPTCHA prevents automated downloads

Posted: Mon Sep 10, 2018 11:14 am
by HanziQ
You can now download releases using 'https://factorio.com/get-download/0.16. ... ken={token}' similar to the mod portal.

Re: Download CAPTCHA prevents automated downloads

Posted: Mon Sep 10, 2018 11:34 am
by nullvoid
Wonderful, Thank you Dev Team :)

Do the other Platforms have similar endpoints (And/Or Will be getting)? The original impetus for this topic was for a Linux system. I tried using both the platforms used in the update list API ("core-linux64", etc), and just "linux64-manual" but they both return a 404.

Edit: I have found the list of identifiers at https://wiki.factorio.com/Mod_portal_API, and now see that the link is exactly the same as before, but with the login info included

Re: Download CAPTCHA prevents automated downloads

Posted: Wed Sep 12, 2018 1:13 am
by keb
HanziQ wrote:
Mon Sep 10, 2018 11:14 am
You can now download releases using 'https://factorio.com/get-download/0.16. ... ken={token}' similar to the mod portal.
Thanks, I appreciate the quick turnaround, but this seems to only work for a couple win64 releases. I'm getting 404s on
0.16.51/headless/win64-manual
0.16.51/alpha/linux64-manual
0.16.51/headless/linux64-manual
0.16.51/demo/linux64-manual
0.14.23/alpha/linux32-manual

Am I using the wrong target name for linux? Win64 alpha and demo are the only combinations that actually work.

Re: Download CAPTCHA prevents automated downloads

Posted: Wed Sep 12, 2018 9:06 am
by HanziQ
It's just the links from this page https://factorio.com/download

Re: Download CAPTCHA prevents automated downloads

Posted: Wed Sep 12, 2018 12:41 pm
by keb
Oh, okay. Thanks.

Re: Download CAPTCHA prevents automated downloads

Posted: Sat Sep 15, 2018 2:29 am
by keb
One last thing: Could you return a 401 error if the provided credentials are bad? Right now it returns 302 FOUND and redirects to /login, so curl can't distinguish that page from a legit download.

Re: Download CAPTCHA prevents automated downloads

Posted: Mon Sep 17, 2018 12:24 pm
by Sanqui
keb wrote:
Sat Sep 15, 2018 2:29 am
One last thing: Could you return a 401 error if the provided credentials are bad? Right now it returns 302 FOUND and redirects to /login, so curl can't distinguish that page from a legit download.
Done, except the error is 403. 401 may only be used with HTTP auth.