[0.8.3] game crashes

[BurnHard]

When walking east near to the spawner the game crashes.

Savegame: https://dl.dropboxusercontent.com/u/518 ... sh_save.7z

[kovarex]

Thank you for the report.
For a quick fix of your, you can type this into the console

Code: Select all

game.regeneratetiles()

What version did you start this map with?

[BurnHard]

Thx, that fixed it. I think it had been 0.8.1 when I started this map.

[kovarex]

Finally,
the bug causing this was found and exterminated.
Fixed for 0.8.4.

Thank you for the report again

[BurnHard]

... keepin' you busy

[kovarex]

... keepin' you busy

I can't stress enough how much are players like you helping, when you are willing to test the experimental releases.
If we had to find all these bugs ourselves, we would either had stable releases "not so stable" or we would have to spend a lot of time by testing which would mean less new features and content.

Anyway, one or two more bugs fixed and we are ready for the 0.8.4. The chances for it to be stable are raising

[ssilk]

From the players side it must be said, that you managed always to bring out a more or less stable version, which is playable, even with bugs. 0.8 is a small exception in this line, but there where always ways to go around the bugs.


And the problem is really the time needed to test some stuff. Well, I've seen some projects going totally down the drain (in one I was, we just made the curve in the last second), because they didn't make their software testable. And testable means much more than unit-tests.

What do you think about automated integration tests? I mean you have unit tests, but I don't know, if you have also integration tests?

If not: an integration test for Factorio would be like a special map and the character runs automatically around like a robot and does things, and if the expected behavior comes out, we have tested some functionalities. Simple tests like there must be 1000 iron in a chest or a train is at station B or or more complicated like when pressing here there must happen that. Or, or, or.

Which leads to much faster testing, which let you find earlier the bugs, which leads to faster programming.

The point, why I'm diving into this is, that I mean, that testing like so isn't currently practicable. Too complicated.
What's needed are hundreds (!) very small to very large "scenarios", which tests very different aspects.

And thinking of that brings me back to blueprints! Because I think that you are working on that and this would be a great technique to create all this small things and repeat an existing test case over and over, with this small but important differences to test all this border-cases. Think for example to the inserters, there are four orientations, and a dozen types of entities you can pick up and place. Not to think about priorities. All that should be tested in a way, that even a minimum number of change in throughput or behavior should be recognized.

Or in other words: if you built blueprints into Factorio, make it useable from very small to very large scales. Large: 200x200 tiles minimum. Because then we (the players) can provide you with maps, where one aspect of Factorio is tested over and over for hundreds of cases. Make it possible to load a bunch of maps and letting them run for some time/event. Catch the the test-results. (If you would built all that also into the map editor, you lay the base for modders to create game scenarios!)
Make it possible to built maps from some smaller maps or blueprints. Let's having "layers" (e. g. one layer for the ground, one for the entities and one for the items), so that I can "reset" a map by loading the item layer of an older save, even, if the belt layout has been changed.

And half a dozen of more ideas... I stop here, because my experience says, that you might already do such stuff and that it is too long otherwise. But I'm willing to explain, if there are questions.

And in danger to repeat me: the big advantage would be to have tests, repeatable tests, which will help you to keep the stability (fix one, break another) and to test new features much faster.

[slpwnd]

What do you think about automated integration tests? I mean you have unit tests, but I don't know, if you have also integration tests?

If not: an integration test for Factorio would be like a special map and the character runs automatically around like a robot and does things, and if the expected behavior comes out, we have tested some functionalities. Simple tests like there must be 1000 iron in a chest or a train is at station B or or more complicated like when pressing here there must happen that. Or, or, or.

Yes, integration is really the problem. We were thinking about this couple of times. The maps would have to be generated from Lua because now and then we do agressive migrations that would require rebuilding the test maps. When we were thinking about this in the past we always came to the conclusion that there is not the time to do that (at that point in time). But maybe we can start working on this incrementally. Spend some time now with the testing framework and then add couple of tests and see how it goes. And then keep adding tests as we go (when we fix a bug or add a feature). We will see:)

[ssilk]

Yes, integration is really the problem. We were thinking about this couple of times. The maps would have to be generated from Lua because now and then we do agressive migrations that would require rebuilding the test maps.

Well, don't automate everything. That was the point with the blueprints: a small piece of map (and script), which does one very special test case.
Integrate it into a bigger map (lua), to have all tests for this function on one map.

I try to explain. I made belt testing maps like so: I filled a belt with items and then inserted the items from this belt into chests (more complicated, but not yet) and measured how many items are transported within a minute over this "test drive".
But this was only possible for one type of belt. The next type of belt took the same time to built up. But that's what I mean: with blueprints this is much easier, because I can split this scenario into functions (pre-belt filled with items, test drive, inserters and chests and counting). I just take the needed blueprints (or parts of maps) I need and put them together as I like. And so I can create all kinds of combinations within a minute and not within an hour.

Ok. I think this is too far away, too complicated for now, forgive me. I take another example, with a much better practical usage.

One part of test driven development is to generate for every bug a test. We have here some dozens of saved games, which produces/have produced errors. So this could be simple: when you find a bug, create a test case, which reproduces this test. And what can be easier, than using the map, which includes this bug? Now what we need is to copy that parts of that map, which include the error and then do something in it/let it run, which reproduces it.

...

Well and when you are here you need to think "how can I copy that parts of the map, which generate an error". This is heavy stuff. But if you find out how, then you took the biggest hurdle.

When we were thinking about this in the past we always came to the conclusion that there is not the time to do that (at that point in time). But maybe we can start working on this incrementally. Spend some time now with the testing framework and then add couple of tests and see how it goes. And then keep adding tests as we go (when we fix a bug or add a feature). We will see:)

That's exactly how it must be done. take a map with an error and think about how the bug can be copied from there into a piece test, which reproduces it.

[slpwnd]

Sure, but most of the test-cases (bugs) can be actually brought down to some relatively simple setups. And this is where I see the lua to step in - the map would be generated by the script (place rail here, place rail there, put the train here, etc.) and then executed with asserts.

[ssilk]

Ok. I see. yes, this is much better...

Then I would make a generator, which I can use to generate a lua program. Some function, input are coordinates of the map and output is a lua function, which generates that at other coordinates and eventually handles also the change in orientation or mirroring, if possible. Some targets: On disc as lua program, or into the copy buffer or as blueprint...

Btw. That's also the way, how blueprints can work: not to store it as map, but as lua-program...

[slpwnd]

Yes, deconstructing the map to set of lua commands would be nice. But it is a lot of work (basically writing a new serialization for every entity) ...

[ssilk]

Hm, serialization, yes. But isn't that just stupid copy&paste? Get all information about the piece of map, go through the structures and translate X to Y.
And I repeat: basically a blueprint is nothing else in my eyes... and the first use of that would be to test if the copy behaves identical to the original.