TLS problems & packaging for NixOS

[Baughn]

Hi,

I'm working on packaging Factorio for NixOS.

Because I love pain, I decided to teach it how to download Factorio, rather than depending on the user to supply a tarball, and that... *sort of* worked, but not quite; I'm having TLS problems. The procedure goes somewhat like this:

> curl -c cookies -d 'username-or-email=Baughn&password=<REDACTED>' https://factorio.com/login
> curl -b cookies -L https://www.factorio.com/get-download/0 ... ha/linux64

The login works, as does the redirect, but the certificate you're using for eu2.factorio.com is failing to validate against the standard Linux keyring. It does work with Chrome, and okay, I could probably bundle the exact CA cert needed, but I thought you might want to know. What I'm actually going to use is the -k option, to disable verification; this is perfectly safe, as the Nix package repository will include the sha256 hash of the tarball anyway.

Some questions, though.
> Are you fine with this, or is there a different download procedure you'd like me to use?
> Will the above curl commands remain usable in the long term?
> Is there a flag I can pass factorio to let it know that automatic updates will not be working? Although I'm going to make sure that scenario downloads do.

And because I really do love pain,
> Do you have plans for a mod repository? Because it'd be awesome if I could configure the mod setup with Nix expressions as well, although that's probably something I should worry about once I've actually gotten the binary running.

[Baughn]

One more question:

Can I set system-read-path from the command line, using a flag or envvar? Because the data is not going to be in /usr/share. /usr/share doesn't exist.

[prg]

One more question:

Can I set system-read-path from the command line, using a flag or envvar? Because the data is not going to be in /usr/share. /usr/share doesn't exist.

You can specify whatever read- and write-data paths you want in the config file. A default config can be generated with use-system-read-write-data-directories=false in config-path.cfg to work around the missing /usr/share, then adjust the paths to your liking. If the config also can't be found in the expected location, you can pass a different one with -c or --config.

[Baughn]

I figured it out, by looking at the config file from an old pre-Nix backup.

[cube]

Hi,

I'm working on packaging Factorio for NixOS.

Because I love pain, I decided to teach it how to download Factorio, rather than depending on the user to supply a tarball, and that... *sort of* worked, but not quite; I'm having TLS problems. The procedure goes somewhat like this:

> curl -c cookies -d 'username-or-email=Baughn&password=<REDACTED>' https://factorio.com/login
> curl -b cookies -L https://www.factorio.com/get-download/0 ... ha/linux64

The login works, as does the redirect, but the certificate you're using for eu2.factorio.com is failing to validate against the standard Linux keyring. It does work with Chrome, and okay, I could probably bundle the exact CA cert needed, but I thought you might want to know. What I'm actually going to use is the -k option, to disable verification; this is perfectly safe, as the Nix package repository will include the sha256 hash of the tarball anyway.

Some questions, though.
> Are you fine with this, or is there a different download procedure you'd like me to use?
> Will the above curl commands remain usable in the long term?
> Is there a flag I can pass factorio to let it know that automatic updates will not be working? Although I'm going to make sure that scenario downloads do.

And because I really do love pain,
> Do you have plans for a mod repository? Because it'd be awesome if I could configure the mod setup with Nix expressions as well, although that's probably something I should worry about once I've actually gotten the binary running.

That is pretty cool.

- The download method is ok with us, but it might be changed to use a separate server for authentication. I'm not sure what else would have to change in your script.
- We don't have a way to disable updates from command line yet. Do you think it's necessary? You can disable them from config, but that's probably not enough.
- Yes, mod repository is currently being developed :-). We are planning to eventually have mod downloading integrated in Factorio (mainly to improve compatibility when connecting to MP games), so adding mods as packages might not be necessary. But the mod database will have a reasoneble API, so you shouldn't have too many problems with it.
- You might want to change the value use-system-read-write-data-directories in <executable-path>/../../config-paths.cfg -- it sets the default paths when a new config is created.

[Baughn]

I've *almost* got it working now. Just one slight glitch remaining...

Code: Select all

  17.254 Error GlobalContext.cpp:881: Couldn't write /nix/store/v5m978fprjn2fq4lavarz37158958hjl-factorio-0.12.20/share/factorio/config-path.cfg: boost::filesystem::copy_file: Read-only file system: "/home/svein/.factorio/config.ini", "/nix/store/v5m978fprjn2fq4lavarz37158958hjl-factorio-0.12.20/share/factorio/config-path.cfg"

I'm going to resolve this by moving config-path.cfg to the user's homedir, overwriting it on each startup, but it sure looks like writing config.ini fails if it can't first write config-path. You might want to do something about that. Or not, if you like.

Disabling updates from the command line isn't critical, but any user attempt to use the function will fail. Which is a bad experience.

EDIT: Yeah, I misinterpreted that error. It's a copy, not a multi-file write. D'oh.

[Baughn]

Okay, so overwriting the config file on every startup wasn't a good idea; the user's configuration gets mixed in with the paths I'm setting. So what I ended up doing was to edit that config file with sed, instead of overwriting it. I don't like this approach.

(Can I have a flag for setting the data directory? That would simplify this immensely.)

Regardless, it's working...

Code: Select all

   0.000 2016-01-23 00:35:39; Factorio 0.12.20 (Build 17489, linux64)
   0.000 Operating system: Linux
   0.000 Program arguments: "/nix/store/cb59xfa2pil04a0fm3v2qdz91chk9af4-factorio-0.12.20/bin/factorio.x64" "-c" "/home/svein/.factorio/config.cfg" 
   0.000 Read data path: /nix/store/cb59xfa2pil04a0fm3v2qdz91chk9af4-factorio-0.12.20/share/factorio/data/.
   0.000 Write data path: /home/svein/.factorio
   0.000 Binaries path: /nix/store/cb59xfa2pil04a0fm3v2qdz91chk9af4-factorio-0.12.20
   0.010 Available display adapters: 1
   0.010  [0]: resolution 2560x1440px at [0,0]
   0.010 Create display on adapter 0. Size 1280x720 at position [630, 342]. Monitor 0
   0.082 Initialised OpenGL:[0] GeForce GTX 970/PCIe/SSE2; driver: 4.5.0 NVIDIA 352.63
   0.093 Graphics options: [FullScreen: false] [VSync: true] [UIScale: 100%] [MultiSampling: OFF] [Graphics quality: normal] [Video memory usage: high] [Light scale: 20%] [Screen: 0]
   0.116 Loading mod core 0.0.0 (data.lua)
   0.117 Loading mod base 0.12.20 (data.lua)
   0.410 Initial atlas bitmap size is 16384
   0.411 Created atlas bitmap 16384x5509
   8.096 Factorio initialised
  14.686 Info Scenario.cpp:123: Map version 0.12.20-1
  25.294 Goodbye
➜  nixpkgs git:(factorio)

If you're curious, you can look at the crazy rube goldberg machine I created to make it run.
https://github.com/NixOS/nixpkgs/pull/12561

[ssilk]

Moved to tools, leaving a link in technical help. I recommend to edit the first post with the current solution. Nice work.

[cube]

Okay, so overwriting the config file on every startup wasn't a good idea; the user's configuration gets mixed in with the paths I'm setting. So what I ended up doing was to edit that config file with sed, instead of overwriting it. I don't like this approach.

(Can I have a flag for setting the data directory? That would simplify this immensely.)

Regardless, it's working...

Code: Select all

   0.000 2016-01-23 00:35:39; Factorio 0.12.20 (Build 17489, linux64)
   0.000 Operating system: Linux
   0.000 Program arguments: "/nix/store/cb59xfa2pil04a0fm3v2qdz91chk9af4-factorio-0.12.20/bin/factorio.x64" "-c" "/home/svein/.factorio/config.cfg" 
   0.000 Read data path: /nix/store/cb59xfa2pil04a0fm3v2qdz91chk9af4-factorio-0.12.20/share/factorio/data/.
   0.000 Write data path: /home/svein/.factorio
   0.000 Binaries path: /nix/store/cb59xfa2pil04a0fm3v2qdz91chk9af4-factorio-0.12.20
   0.010 Available display adapters: 1
   0.010  [0]: resolution 2560x1440px at [0,0]
   0.010 Create display on adapter 0. Size 1280x720 at position [630, 342]. Monitor 0
   0.082 Initialised OpenGL:[0] GeForce GTX 970/PCIe/SSE2; driver: 4.5.0 NVIDIA 352.63
   0.093 Graphics options: [FullScreen: false] [VSync: true] [UIScale: 100%] [MultiSampling: OFF] [Graphics quality: normal] [Video memory usage: high] [Light scale: 20%] [Screen: 0]
   0.116 Loading mod core 0.0.0 (data.lua)
   0.117 Loading mod base 0.12.20 (data.lua)
   0.410 Initial atlas bitmap size is 16384
   0.411 Created atlas bitmap 16384x5509
   8.096 Factorio initialised
  14.686 Info Scenario.cpp:123: Map version 0.12.20-1
  25.294 Goodbye
➜  nixpkgs git:(factorio)

If you're curious, you can look at the crazy rube goldberg machine I created to make it run. :P
https://github.com/NixOS/nixpkgs/pull/12561

It shouldn't be necesary to set config path from commandline.
You should only have the <executable-path>/../../config-path.cfg file (in the previous post i had an extra S there, sorry) with content like this:

Code: Select all

config-path=~/.factorio/config
use-system-read-write-data-directories=true

The use-system-read-write-data-directories flag then sets the default read and write data to /usr/share/factorio and ~/.factorio .
If you need to change any of them, you can then pregenerate a config file (~/.factorio/config/config.ini) that contains something like the following:

Code: Select all

[path]
read-data=any_path
write-data=any_path

Also while you can't disable updates now, at least disabling them by default with

Code: Select all

[other]
check_updates=false

in config file might help.

[cube]

Actually there's a problem -- "~" doesn't get expanded. If you just don't put the config-path.cfg there, it will use a default which is using system directories and config in ~/.factorio/config. But this is ugly and should change. I'll add it to our list.

[Baughn]

Moved to tools, leaving a link in technical help. I recommend to edit the first post with the current solution. Nice work.

As per NixOS standards, the current solution for installing Factorio on NixOS is to type "nix-env -i factorio".

None of what we went through in this post is necessary. The code is merged into the Nix package repository.