Export data to json

[pinoooh]

Hi all,

My (grid-structured) base is getting larger and larger, and I have trouble keeping up with the trains (>3700) and the available train stations (>2250). Now and then I want to know which stations need more trains assigned, or where supply is not keeping up etc. The available list in factorio is working rather well, but it's quite cumbersome to scroll through all stations and comparing data manually.

So, my idea is to export some data to json periodically. The json-files can be fed to some external (custom) tool and there I can extract/display the data I need, point out bottlenecks etc. As a start I created a (very, very) simple mod which now only exports the active entity count, just as a start.

Code: Select all

local json = require "lib/json"
local default_export_file = "export.json"

local function onExportCommand(event)
    local output = {}

    output.active_entities_count = game.get_active_entities_count()

    local file = event.parameter or default_export_file
    
    game.remove_path(file)
    game.write_file(file, json.encode(output), true)

    game.players[event.player_index].print("Export successful")
end

commands.add_command("export", nil, onExportCommand)

This gives me a export.json file containing a single line:

Code: Select all

{"get_active_entities_count":120949}

btw I am using the rxi/json.lua library, see here: https://github.com/rxi/json.lua

Now for starters, I would like to have a list of all stations (grouped by name) and the amount of trains, just as in the train overview within the game. E.g.
- Copper cable [150] 300/300
- Copper cable ore [69] 123/136

I've searched through these forums a bit, but I cannot find how to do this (and lua is new to me too). Do I have to use the game.get_train_stops() reference?
Any suggestions on how to do this?

Thanks in advance!

[Hornwitser]

btw I am using the rxi/json.lua library, see here: https://github.com/rxi/json.lua

You should be using game.table_to_json() rather than a lua json library as it will be much faster for large data structures due to it being implemented in C++.

Now for starters, I would like to have a list of all stations (grouped by name) and the amount of trains, just as in the train overview within the game. E.g.
- Copper cable [150] 300/300
- Copper cable ore [69] 123/136

I've searched through these forums a bit, but I cannot find how to do this (and lua is new to me too). Do I have to use the game.get_train_stops() reference?
Any suggestions on how to do this?

Yes, that looks like where you would start. Then you can use entity.trains_count and entity.trains_limit to see if the train stop is requesting trains and receiving it.

E.g. something like this

Code: Select all

local train_stops_by_name = {}
for _, train_stop in ipairs(game.get_train_stops()) do
    if not train_stops_by_name[train_stop.backer_name]
        train_stop_by_name[train_stop.backer_name] = {}
    end
    table.insert(train_stop_by_name[train_stop.backer_name], train_stop)
end

output.train_stops = {}
for name, train_stops in pairs(train_stops_by_name) do
    local fulfilled = 0
    for _, stop in ipairs(train_stops) do
        if stop.trains_count >= stop.trains_limit then
            fulfilled = fulfilled + 1
        end
    end
    output.train_stops[name] = {
    	trains = #train_stops[1].get_train_stop_trains(),
    	fulfilled_stops = fulfilled,
    	stops = #train_stops,
    }
end

should be able to give a basic idea of what you need to gather the numbers you need. Note that I didn't test this code so it probably contains typos and bugs. It may also give you different numbers than the in-game list does as I'm not exactly sure how it calculates those numbers.

[pinoooh]

Wow! This is exactly what I was looking for! Thanks a lot!!!

Had to do make some minor adjustements, but this works:

Code: Select all

local default_export_file = "export.json"

local function onExportCommand(event)
    local output = {}

    -- basic info
    output["active_entities_count"] = game.get_active_entities_count()

    -- trains
    local train_stops_by_name = {}
    for _, train_stop in ipairs(game.get_train_stops()) do
        if not train_stops_by_name[train_stop.backer_name] then
            train_stops_by_name[train_stop.backer_name] = {}
        end
        table.insert(train_stops_by_name[train_stop.backer_name], train_stop)
    end

    output.train_stops = {}
    for name, train_stops in pairs(train_stops_by_name) do
        local fulfilled = 0
        for _, stop in ipairs(train_stops) do
            if stop.trains_count >= stop.trains_limit then
                fulfilled = fulfilled + 1
            end
        end
        output.train_stops[name] = {
            trains = #train_stops[1].get_train_stop_trains(),
            fulfilled_stops = fulfilled,
            stops = #train_stops,
        }
    end

    -- export to file
    local file = event.parameter or default_export_file
    
    game.remove_path(file)
    game.write_file(file, game.table_to_json(output), true)

    game.players[event.player_index].print("Export successful")
end

commands.add_command("export", nil, onExportCommand)

The export looks like this now:

Code: Select all

{
    "active_entities_count": 120917,
    "train_stops": {
        "copper plate": {
            "trains": 330,
            "fulfilled_stops": 118,
            "stops": 118
        },
        "copper plate ore": {
            "trains": 330,
            "fulfilled_stops": 35,
            "stops": 64
        },
        "iron plate": {
            "trains": 305,
            "fulfilled_stops": 126,
            "stops": 126
        },
        "iron plate ore": {
            "trains": 305,
            "fulfilled_stops": 23,
            "stops": 56
        },
        ... (many more here)
   }
}

Thanks again!