Clusterio 2.0 Developments

[Hornwitser]

I've spent the last 9 months rewriting the Clusterio code with goals of making it easier to develop for, easier to use and manage at scale, and of course have it be able to scale to up 100 servers in a cluster. The new code has gotten to the point that it is able to run a cluster again, and while I still have lots of things to (re-)implement I want to hear from the modding community about your interest in using Clusterio in mods, scenarios and community games.

But first I need to clear up some confusion about what Clusterio is. Clusterio 2.0 is a Node.js backend for Factorio servers that allows in-game lua code to talk to JavaScript code running outside the game. The Clusterio project started out as a Factorio mod + Node.js program to teleport items between servers using giant teleoporter chests. Since then the focus of Clusterio has shifted from being a teleporter chest mod to being just the backend that allows mods like the teleporter chest mod to work. So to avoid more confusion from this change in direction the in-game Clusterio mod that provides those teleporter chests is therefore being renamed to Subspace Storage.

With that out of the way, the things I want to hear about is what you want out of the cross server toolbox that is Clusterio. Right now the things that I have (re-)implemented into Clusterio 2.0 is

• A Factorio server process manager that handles starting, stopping, connecting to the RCON interface, and reading the stdout of Factorio servers.
• Slaves that connect to the master server of the cluster and can run multiple Factorio servers called instances for the cluster.
• Centralized management of instances in the cluster. The master server controls what instances to start, and what configuration to use for them. Management is currently only available through a command line tool, but a more user friendly web interface is also planned.
• Plugin system that allows JavaScript modules to be loaded by Clusterio, with hooks for lifecycle events, and APIs for sending RCON commands as well as sending data to other instances running on the cluster.
• WebSocket based communication layer between slaves, the master and the command line tool that gives an efficient bi-directonal transport between them.
• Save patcher that puts lua code packed into modules into the level/scenario, making it easy to to "soft" mod freeplay games with lua code from a selection of pre made modules and/or custom written ones.
• Actual documentation. As sad as it is to say, the documentation for Clusterio before 2.0 was next to non-existent. I still have a long way to go before I would call the documentation for great, but be sure to check out the Developing for Clusterio and Writing Plugins documents in the docs/ folder of the repository for what I have written up to now. Needless to say, this is a work in progress.

All in all what I would like to see is Clusterio being used for more things than just the chest teleporters and the yearly Clusterio event. Those are certainly cool, but the ease at which novel Factorio server integrations can be implemented on top of Clusterio 2.0 is one I hope doesn't go unnoticed.

Please note that Clusterio 2.0 is still a work in progress alpha, and the APIs provided are subject to breaking changes

[psihius]

We also have a Discord where we do all types of collaboration with the development of Clusterio: https://discord.gg/jdRAYaG

[E-37]

The added documentation will be really nice. It took me a whole afternoon to figure out how to get Clusterio running and I don't know where to begin to make a plugin.
I have been using Clusterio just to build singleplayer megabases for a while now and it has worked really well. If there was a very simple way to install it (like a 'double click me to create a new cluster' executable) I bet that there could be a lot of use for it in singleplayer.
As for things to add, the ability to sync mod data between servers would be nice. I installed SpaceX on one of my clusters. You need to send many different items to space, but that progress is separate for each server. Having it the same on all servers would be nice. That might not be possible though.

I'm looking forward to seeing all the cool things that will be done!

[psihius]

The added documentation will be really nice. It took me a whole afternoon to figure out how to get Clusterio running and I don't know where to begin to make a plugin.
I have been using Clusterio just to build singleplayer megabases for a while now and it has worked really well. If there was a very simple way to install it (like a 'double click me to create a new cluster' executable) I bet that there could be a lot of use for it in singleplayer.
As for things to add, the ability to sync mod data between servers would be nice. I installed SpaceX on one of my clusters. You need to send many different items to space, but that progress is separate for each server. Having it the same on all servers would be nice. That might not be possible though.

I'm looking forward to seeing all the cool things that will be done!

We actually have an alpha version of science progress sync working that is shared across the cluster, but SpaceX has its own big part of code that probably can't be done without explicit clusterio support from SpaceX, so the best idea is to do production on different servers, but launch stuff only on one so SpaceX has it's code run properly and state handled correctly.

[E-37]

We actually have an alpha version of science progress sync working that is shared across the cluster, but SpaceX has its own big part of code that probably can't be done without explicit clusterio support from SpaceX, so the best idea is to do production on different servers, but launch stuff only on one so SpaceX has it's code run properly and state handled correctly.

I was thinking of something along the lines of Clusterio being able to sync mods global variables across servers. Now that I think about it, I imagine that having their global variables changed from underneath them probably wouldn't go down well with a lot of mods.