Download CAPTCHA prevents automated downloads

Anything that prevents you play the game properly. Do you have issues with paying for the game, downloading it or properly running it on your computer? Let us know here.
Post Reply
keb
Burner Inserter
Burner Inserter
Posts: 8
Joined: Sun Jun 26, 2016 2:34 pm

Download CAPTCHA prevents automated downloads

Post by keb » Fri Sep 07, 2018 5:00 am

Previously, I was able to download updates using curl. The new CAPTCHA on the login page now prevents this.

Is there a ReST API endpoint on the mod portal that presents the latest binary distributions of the client? If not, could we have one? I think using the token stored in player-data.json is an appropriate way to support automated downloads.

Thanks in advance.

Jap2.0
Smart Inserter
Smart Inserter
Posts: 1741
Joined: Tue Jun 20, 2017 12:02 am

Re: Download CAPTCHA prevents automated downloads

Post by Jap2.0 » Fri Sep 07, 2018 10:20 am

I mean, that's the point of a captcha :P.

But yes, I agree that having an automated way to download it would probably be nice, and using that would prevent some spam and, if need be, make it possible to identify who's misusing it.
There are 10 types of people: those who get this joke and those who don't.

Baughn
Long Handed Inserter
Long Handed Inserter
Posts: 54
Joined: Wed Jan 20, 2016 4:44 pm

Re: Download CAPTCHA prevents automated downloads

Post by Baughn » Fri Sep 07, 2018 12:31 pm

The purpose of this is to make Factorio possible to install on NixOS. It has to be automated; the package builder (by design) doesn't allow interactive actions, and the license obviously doesn't allow redistribution.

Previously we've been able to get away with a login script using curl, but that falls afoul of the captcha now. I hope you can work with the NixOS community to find an alternative.

Zavian
Smart Inserter
Smart Inserter
Posts: 1122
Joined: Thu Mar 02, 2017 2:57 am

Re: Download CAPTCHA prevents automated downloads

Post by Zavian » Fri Sep 07, 2018 12:57 pm

Well one possibility is to have the user manually download the tar.xz, then point the package build script at that local file.

User avatar
Sanqui
Factorio Staff
Factorio Staff
Posts: 46
Joined: Mon May 07, 2018 7:22 pm

Re: Download CAPTCHA prevents automated downloads

Post by Sanqui » Fri Sep 07, 2018 2:16 pm

The CAPTCHA was a necessary measure due to brute force attacks.
We'll discuss the options internally and let you know. I'm interested in supporting automated downloads via an API.
ovo

nuhll
Filter Inserter
Filter Inserter
Posts: 576
Joined: Mon Apr 04, 2016 9:48 pm

Re: Download CAPTCHA prevents automated downloads

Post by nuhll » Fri Sep 07, 2018 3:00 pm

getlatestbeta.php?user=XXX&pw=YYYY
getlateststable?user=XXX&pw=YYYY


It could be so easy, user and pw can ofc be changed by the player id.
Against Bruteforce only chance is to make it too slow. But better is use some sort of API key which is too long to bruteforce.

Jap2.0
Smart Inserter
Smart Inserter
Posts: 1741
Joined: Tue Jun 20, 2017 12:02 am

Re: Download CAPTCHA prevents automated downloads

Post by Jap2.0 » Fri Sep 07, 2018 10:06 pm

nuhll wrote:getlatestbeta.php?user=XXX&pw=YYYY
getlateststable?user=XXX&pw=YYYY


It could be so easy, user and pw can ofc be changed by the player id.
Against Bruteforce only chance is to make it too slow. But better is use some sort of API key which is too long to bruteforce.
Is it just me or does having the username and password in plain text in the URL seem like a bad idea?
There are 10 types of people: those who get this joke and those who don't.

Koub
Global Moderator
Global Moderator
Posts: 3870
Joined: Fri May 30, 2014 8:54 am

Re: Download CAPTCHA prevents automated downloads

Post by Koub » Sat Sep 08, 2018 6:15 am

That's true in HTTP, but then using HTTP is the worst idea.
In HTTPS (with proper HTTPS), your communication should be encrypted, including the URLs, and whatever the way, passwords are sent as plaintext to websites most of the time. Only drawback is the fact this gets in your browser history, so other people who can access your computer might see it.
Koub - Please consider English is not my native language.

daniel34
Global Moderator
Global Moderator
Posts: 2757
Joined: Thu Dec 25, 2014 7:30 am

Re: Download CAPTCHA prevents automated downloads

Post by daniel34 » Sat Sep 08, 2018 6:34 pm

Factorio already has an API key - the service token, which keb already mentioned in the first post. It is a 30-character hexadecimal string stored in player-data.json (after the first in-game authentication using username and password).

It is used for Factorio updates and for hosting a server, where you can use it in the server-settings.json file instead of username and password. I think (but I have never checked for myself) that it is also used to download mods from the mod portal in-game.

Using a token instead of a password has the advantage that it is restricted as to what you can do with it; you can use it to download the game or mods and host a server, but you are unable to login to factorio.com and view/change account settings. The account owner can invalidate the current token on the Profile page, making it useless.

I'm not sure how up-to-date it is, but the wiki actually has a link to a service that returns that service token: https://wiki.factorio.com/Web_authentication_API
EDIT: Also see Unofficial Factorio Web API Documentation [Development-Tools]

nullvoid
Burner Inserter
Burner Inserter
Posts: 15
Joined: Sat May 03, 2014 9:53 pm

Re: Download CAPTCHA prevents automated downloads

Post by nullvoid » Sun Sep 09, 2018 3:20 pm

That seems promising, but there doesn't appear to be a publicly known way to use this api to download whole versions.

In particular, while get-available-versions reports what the stable version is for each platform, you can't use it with get-download-link like you can for the from/to pairs.

User avatar
HanziQ
Factorio Staff
Factorio Staff
Posts: 630
Joined: Fri Mar 27, 2015 7:07 am

Re: Download CAPTCHA prevents automated downloads

Post by HanziQ » Mon Sep 10, 2018 11:14 am

You can now download releases using 'https://factorio.com/get-download/0.16. ... ken={token}' similar to the mod portal.

nullvoid
Burner Inserter
Burner Inserter
Posts: 15
Joined: Sat May 03, 2014 9:53 pm

Re: Download CAPTCHA prevents automated downloads

Post by nullvoid » Mon Sep 10, 2018 11:34 am

Wonderful, Thank you Dev Team :)

Do the other Platforms have similar endpoints (And/Or Will be getting)? The original impetus for this topic was for a Linux system. I tried using both the platforms used in the update list API ("core-linux64", etc), and just "linux64-manual" but they both return a 404.

Edit: I have found the list of identifiers at https://wiki.factorio.com/Mod_portal_API, and now see that the link is exactly the same as before, but with the login info included

keb
Burner Inserter
Burner Inserter
Posts: 8
Joined: Sun Jun 26, 2016 2:34 pm

Re: Download CAPTCHA prevents automated downloads

Post by keb » Wed Sep 12, 2018 1:13 am

HanziQ wrote:
Mon Sep 10, 2018 11:14 am
You can now download releases using 'https://factorio.com/get-download/0.16. ... ken={token}' similar to the mod portal.
Thanks, I appreciate the quick turnaround, but this seems to only work for a couple win64 releases. I'm getting 404s on
0.16.51/headless/win64-manual
0.16.51/alpha/linux64-manual
0.16.51/headless/linux64-manual
0.16.51/demo/linux64-manual
0.14.23/alpha/linux32-manual

Am I using the wrong target name for linux? Win64 alpha and demo are the only combinations that actually work.

User avatar
HanziQ
Factorio Staff
Factorio Staff
Posts: 630
Joined: Fri Mar 27, 2015 7:07 am

Re: Download CAPTCHA prevents automated downloads

Post by HanziQ » Wed Sep 12, 2018 9:06 am

It's just the links from this page https://factorio.com/download

keb
Burner Inserter
Burner Inserter
Posts: 8
Joined: Sun Jun 26, 2016 2:34 pm

Re: Download CAPTCHA prevents automated downloads

Post by keb » Wed Sep 12, 2018 12:41 pm

Oh, okay. Thanks.

keb
Burner Inserter
Burner Inserter
Posts: 8
Joined: Sun Jun 26, 2016 2:34 pm

Re: Download CAPTCHA prevents automated downloads

Post by keb » Sat Sep 15, 2018 2:29 am

One last thing: Could you return a 401 error if the provided credentials are bad? Right now it returns 302 FOUND and redirects to /login, so curl can't distinguish that page from a legit download.

User avatar
Sanqui
Factorio Staff
Factorio Staff
Posts: 46
Joined: Mon May 07, 2018 7:22 pm

Re: Download CAPTCHA prevents automated downloads

Post by Sanqui » Mon Sep 17, 2018 12:24 pm

keb wrote:
Sat Sep 15, 2018 2:29 am
One last thing: Could you return a 401 error if the provided credentials are bad? Right now it returns 302 FOUND and redirects to /login, so curl can't distinguish that page from a legit download.
Done, except the error is 403. 401 may only be used with HTTP auth.
ovo

Post Reply

Return to “Technical Help”