Automatic mod tests

Place to ask discuss and request the modding support of Factorio. Don't request mods here.
Post Reply
keyboardhack
Filter Inserter
Filter Inserter
Posts: 475
Joined: Sat Aug 23, 2014 11:43 pm
Contact:

Automatic mod tests

Post by keyboardhack »

Not sure if this is the correct place to post this but it seems to fit here the best.
This is a rough idea of how i think automatic tests in factorio should work so i just hope to get the general idea across.


What is this request about?
One of the things i don't like about factorio modding is the fact that i have to do all the testing myself.
After i have changed something i need to do a lot of manual testing to verify that everything still works.
So this request is about adding an automated way to run mod tests in factorio.

What should be testable?
A test should be able to emulate a player. That means a test should run a world and be able to interact with it.
The test should also have access to the mods global table so the test can verify that the state of the mod is correct.
It should also be able to call the functions inside the mod so they can be tested as well.

How should it work?
Just like we have data.lua and control.lua that are special files, test.lua should be special.
in test.lua you can add test by writing something like

Code: Select all

--test.lua--
test.add_test("test signal detection", "tests/signal-detection.lua")
test.add_test("test train teleporting", "tests/train-teleporting.lua", {wagonCount = 2, wagonContent = {"iron_ore" = 27}})
  • test.add_test(test_name, test_path, data)
    • test_name: the name of the test. Used to identify the test.
    • test_path: A path leading to a test file.
    • data(optional): A table of some data you want available in the test. Can be accessed with test.data inside the test. Prevents duplicating test script when they are very similar.
Each test has a test file associated with that test.
A test will have three special events that are used to either setup the test or to test the mod. All the events has access to the game and mods global table.
  • test.before_mod_load(function)
    • function: A function that's executed before the mod is loaded. Can be used to place entities/items in the world as preparations to the test.
  • test.after_mod_load(function)
    • function: A function that's executed right after the mod has loaded and run all its first event(on_load, on_init, etc). Can be used to verify that a mod loads correctly on an existing or new map.
  • test.add_on_tick(tick, function)
    • tick: Specifies which tick the function should run on.
    • function: The function that's executed on that tick. The function is executed after the mod has run for that tick.
  • test.end_test()
    • Also a way to end the test.
A test should(to begin with) simply run an empty world with only dirt tiles and a player in it.
When the world has been created it calls test.before_mod_load, then runs the mod, then test.after_mod_load.
After that it runs the game normally until the test ends or for 10000 ticks(test needs a limit in case of infinitely runing tests).

A test file could look like this

Code: Select all

test.before_mod_load(function()
	game.surfaces[1].create_entity({name = "steel-chest", position = {10, 10}})
	game.surfaces[1].create_entity({name = "wood-chest", position = {11, 10}})
end)

test.after_mod_load(function()
	assert(#global.steelchests == 1, "Expected 1 steel chest but there was "..tostring(#global.steelchests).." steel chests instead")
	assert(global.steelchests[1].name == "steel-chest", "Chests added the steelchest table was not a steel chest but instead a ".. global.steelchests[1].name)
	
	assert(#global.woodchests == 1, "Expected 1 wood chest but there was "..tostring(#global.woodchests).." wood chests instead")
	assert(global.woodchests[1].name == "wood-chest", "Chests added the steelchest table was not a wood chest but instead a ".. global.woodchests[1].name)
	
	assert(#global.anychests == 2, "Expected two chests but there was instead "..tostring(#global.anychests).." chests")
	
	--call method from the mod
	local destroyedChests = destroyWoodenChests()
	assert(destroyedChests == 1, "Expcted to destroy 1 wooden chest but "..tostring(destroyedChests).." wooden chests were destroyed instead");
end)

test.add_on_tick(1, function()
	assert(#global.woodchests == 0, "Expected 0 wood chest but there was "..(#global.woodchests).." wood chests instead")
	--bla bla bla more checks and so on

	test.end_test()
end)
How to run the tests
I would image it to be a console command.

Code: Select all

--run tests for a single mod
factorio.exe --test random_mod_1.0.1
--run tests for all mods
factorio.exe --test_all
The console runs all the tests for the mod and prints out whether each tests has succeded or not.
If a test throws an error then the error is displayed.
Waste of bytes : P

Rseding91
Factorio Staff
Factorio Staff
Posts: 10760
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Automatic mod tests

Post by Rseding91 »

This sounds like it could be mildly useful but it also sounds like it will break very easily without adding much utility for the vast majority of mod devs who won't use it.

You can roughly get the same results by just setting up your mod to run on player created and click "new game". You don't have to restart the game for changes to control.lua to take effect - just the map.
If you want to get ahold of me I'm almost always on Discord.

Post Reply

Return to “Modding interface requests”

Who is online

Users browsing this forum: No registered users