Automatic mod tests
Posted: Thu May 03, 2018 2:54 am
Not sure if this is the correct place to post this but it seems to fit here the best.
This is a rough idea of how i think automatic tests in factorio should work so i just hope to get the general idea across.
What is this request about?
One of the things i don't like about factorio modding is the fact that i have to do all the testing myself.
After i have changed something i need to do a lot of manual testing to verify that everything still works.
So this request is about adding an automated way to run mod tests in factorio.
What should be testable?
A test should be able to emulate a player. That means a test should run a world and be able to interact with it.
The test should also have access to the mods global table so the test can verify that the state of the mod is correct.
It should also be able to call the functions inside the mod so they can be tested as well.
How should it work?
Just like we have data.lua and control.lua that are special files, test.lua should be special.
in test.lua you can add test by writing something like
A test will have three special events that are used to either setup the test or to test the mod. All the events has access to the game and mods global table.
When the world has been created it calls test.before_mod_load, then runs the mod, then test.after_mod_load.
After that it runs the game normally until the test ends or for 10000 ticks(test needs a limit in case of infinitely runing tests).
A test file could look like this
How to run the tests
I would image it to be a console command.
The console runs all the tests for the mod and prints out whether each tests has succeded or not.
If a test throws an error then the error is displayed.
This is a rough idea of how i think automatic tests in factorio should work so i just hope to get the general idea across.
What is this request about?
One of the things i don't like about factorio modding is the fact that i have to do all the testing myself.
After i have changed something i need to do a lot of manual testing to verify that everything still works.
So this request is about adding an automated way to run mod tests in factorio.
What should be testable?
A test should be able to emulate a player. That means a test should run a world and be able to interact with it.
The test should also have access to the mods global table so the test can verify that the state of the mod is correct.
It should also be able to call the functions inside the mod so they can be tested as well.
How should it work?
Just like we have data.lua and control.lua that are special files, test.lua should be special.
in test.lua you can add test by writing something like
Code: Select all
--test.lua--
test.add_test("test signal detection", "tests/signal-detection.lua")
test.add_test("test train teleporting", "tests/train-teleporting.lua", {wagonCount = 2, wagonContent = {"iron_ore" = 27}})
- test.add_test(test_name, test_path, data)
- test_name: the name of the test. Used to identify the test.
- test_path: A path leading to a test file.
- data(optional): A table of some data you want available in the test. Can be accessed with test.data inside the test. Prevents duplicating test script when they are very similar.
A test will have three special events that are used to either setup the test or to test the mod. All the events has access to the game and mods global table.
- test.before_mod_load(function)
- function: A function that's executed before the mod is loaded. Can be used to place entities/items in the world as preparations to the test.
- test.after_mod_load(function)
- function: A function that's executed right after the mod has loaded and run all its first event(on_load, on_init, etc). Can be used to verify that a mod loads correctly on an existing or new map.
- test.add_on_tick(tick, function)
- tick: Specifies which tick the function should run on.
- function: The function that's executed on that tick. The function is executed after the mod has run for that tick.
- test.end_test()
- Also a way to end the test.
When the world has been created it calls test.before_mod_load, then runs the mod, then test.after_mod_load.
After that it runs the game normally until the test ends or for 10000 ticks(test needs a limit in case of infinitely runing tests).
A test file could look like this
Code: Select all
test.before_mod_load(function()
game.surfaces[1].create_entity({name = "steel-chest", position = {10, 10}})
game.surfaces[1].create_entity({name = "wood-chest", position = {11, 10}})
end)
test.after_mod_load(function()
assert(#global.steelchests == 1, "Expected 1 steel chest but there was "..tostring(#global.steelchests).." steel chests instead")
assert(global.steelchests[1].name == "steel-chest", "Chests added the steelchest table was not a steel chest but instead a ".. global.steelchests[1].name)
assert(#global.woodchests == 1, "Expected 1 wood chest but there was "..tostring(#global.woodchests).." wood chests instead")
assert(global.woodchests[1].name == "wood-chest", "Chests added the steelchest table was not a wood chest but instead a ".. global.woodchests[1].name)
assert(#global.anychests == 2, "Expected two chests but there was instead "..tostring(#global.anychests).." chests")
--call method from the mod
local destroyedChests = destroyWoodenChests()
assert(destroyedChests == 1, "Expcted to destroy 1 wooden chest but "..tostring(destroyedChests).." wooden chests were destroyed instead");
end)
test.add_on_tick(1, function()
assert(#global.woodchests == 0, "Expected 0 wood chest but there was "..(#global.woodchests).." wood chests instead")
--bla bla bla more checks and so on
test.end_test()
end)
I would image it to be a console command.
Code: Select all
--run tests for a single mod
factorio.exe --test random_mod_1.0.1
--run tests for all mods
factorio.exe --test_all
If a test throws an error then the error is displayed.