trying to make a mod, accidentally crashed my whole computer

[Rseding91]

No. We're not going to spend time working on trying to prevent mods from making infinite loops.

Your incompetence is preventing you from seeing the underlying issue here, more importantly, you don't seem to understand who's responsible for what. The underlying issue is not Lua developers creating inefficient mods. The underlying issue is the game being so shit at dealing with this that it crashes not only the game, but the entire device...

Did you even try to reproduce the issue? It doesn't crash the computer. In fact I'm typing this right now as the process is using 99% of my available RAM for the past 5 minutes and the computer's running just fine.

[ratchetfreak]

No. We're not going to spend time working on trying to prevent mods from making infinite loops.

Your incompetence is preventing you from seeing the underlying issue here, more importantly, you don't seem to understand who's responsible for what. The underlying issue is not Lua developers creating inefficient mods. The underlying issue is the game being so shit at dealing with this that it crashes not only the game, but the entire device...

Did you even try to reproduce the issue? It doesn't crash the computer. In fact I'm typing this right now as the process is using 99% of my available RAM for the past 5 minutes and the computer's running just fine.

Still having a runaway allocation loop is something that that can be tracked (assuming that the lua engine lets you track the allocations) and then if allocation numbers go insane you can abort cleanly without first having to bring the OS to its knees.

[DedlySpyder]

No. We're not going to spend time working on trying to prevent mods from making infinite loops.

Your incompetence is preventing you from seeing the underlying issue here, more importantly, you don't seem to understand who's responsible for what. The underlying issue is not Lua developers creating inefficient mods. The underlying issue is the game being so shit at dealing with this that it crashes not only the game, but the entire device...

Did you even try to reproduce the issue? It doesn't crash the computer. In fact I'm typing this right now as the process is using 99% of my available RAM for the past 5 minutes and the computer's running just fine.

Still having a runaway allocation loop is something that that can be tracked (assuming that the lua engine lets you track the allocations) and then if allocation numbers go insane you can abort cleanly without first having to bring the OS to its knees.

But then the problem becomes that Factorio has to constantly check for this thing. The game starts to chug with large factories, the more checks that the game makes, the smaller you can make a factory.

I'd rather Wube spends their time improving a beautiful game than stopping people from doing something dumb

[Divran]

No. We're not going to spend time working on trying to prevent mods from making infinite loops.

Your incompetence is preventing you from seeing the underlying issue here, more importantly, you don't seem to understand who's responsible for what. The underlying issue is not Lua developers creating inefficient mods. The underlying issue is the game being so shit at dealing with this that it crashes not only the game, but the entire device...

Did you even try to reproduce the issue? It doesn't crash the computer. In fact I'm typing this right now as the process is using 99% of my available RAM for the past 5 minutes and the computer's running just fine.

I wouldn't have posted the thread if my computer hadn't crashed. Are you using linux? Linux probably handles memory better than windows.

But then the problem becomes that Factorio has to constantly check for this thing. The game starts to chug with large factories, the more checks that the game makes, the smaller you can make a factory.

I'd rather Wube spends their time improving a beautiful game than stopping people from doing something dumb

Checking for memory usage takes pretty much no time at all. It would be the difference between having 999 999 999 robots instead of 1 000 000 000 or something

[Rseding91]

I wouldn't have posted the thread if my computer hadn't crashed. Are you using linux? Linux probably handles memory better than windows.

Windows 7 SP1 x64.

[Divran]

Alright guys I just tested this in linux, inside a VM. I deliberately created an infinite loop that placed new values into a table. It seems Factorio already has exactly the thing I've been trying to get you to add. After a few seconds, a popup appeared inside factorio telling me that I didn't have enough memory. The second time I tested it, the factorio process closed (maybe linux closed it automatically) without showing the popup. So it's possible that this check that creates the popup simply doesn't work properly, but the functionality is already there. So, Rseding91, this has now changed from me requesting a new feature, to me requesting you fix a feature that already exists. What do you say about that?

Currently I'm downloading a windows 10 iso to test that in vmware as well.

I wouldn't have posted the thread if my computer hadn't crashed. Are you using linux? Linux probably handles memory better than windows.

Windows 7 SP1 x64.

I'm using windows 10

EDIT: Windows iso is downloaded. Currently installing.

[bNarFProfCrazy]

@Divran

Are you able to reproduce the full computer crash on your system (where you encountered it last time)?

Could you provide a test plugin/mod that reproducible crashes your computer so others can test it as well?
(Please make the mod as light-weight as possible, so there isn't any testing overhead or distraction from the issue you encounter)
I had a broken RAM module once that crashed my computer when I tried to use a fixed but small section of it (usually in high load situations).

[Divran]

@Divran

Are you able to reproduce the full computer crash on your system (where you encountered it last time)?

Could you provide a test plugin/mod that reproducible crashes your computer so others can test it as well?
(Please make the mod as light-weight as possible, so there isn't any testing overhead or distraction from the issue you encounter)
I had a broken RAM module once that crashed my computer when I tried to use a fixed but small section of it (usually in high load situations).

I would prefer to not crash my computer again, which is why I'm installing windows 10 in a VM. It's finished installing now and I'm about to test it.
As you say, it's not entirely impossible that I do have a broken ram module. It would help if someone else could test it as well to be sure. I would be nothing but very happy if I'm proven wrong here and factorio already has the memory checks in place.

EDIT: I managed to get the windows 10 VM to crash. First time I ran the infinite loop, factorio showed the popup, and everything was fine. Then, without restarting factorio, I tried starting a new game. It showed the popup again. After that, I tried to open task manager inside the VM, to see how much memory factorio was using. Windows showed a popup saying that I didn't have enough space in the swap file to open task manager. I tried opening task manager again, and then the screen turned black. The VM had crashed. Kinda weird how that works, but you know, it's windows.
If my ram has an actual physical problem in it, would that not cause my computer to crash even if the VM was using the memory? It didn't crash my computer though, only the VM.

EDIT 2: I should clarify that factorio failed to clear its memory after the popups appeared, which may very well be the cause of the problem. The factorio process eats all of your memory and then keeps it forever, until you manually close factorio. Pretty sure this shouldn't happen.

[bobingabout]

Yeah, I think I've noticed some memory leaks in factorio myself as well.

And even thought a few times.. "Nah, surely factorio must have something in place to prevent memory leaks in the background".

For example... tables.

one of the things in my ores mod is a table for autoplace, the table holds the information required to create a proper autoplace using my functions.

this autoplace table is generated under a local. then once the function is complete... ore.autoplace = autoplace.

what that last instruction does is replace the table in the ore definition table (used to create the entity), with the local. in this case, both ore.autoplace and local autoplace contain a table. when I do ore.autoplace = autoplace, I see that ore.autoplace simply has it's pointer changed to point to the local variable.

Now here is my issue. If it is true that it is POINTING to a local variable, then when the function ends, the local should deconstruct cleanly and not exist anymore, therefore ore.autoplace should point to an invalid object, which isn't the case.

Therefore, you must assume one of 3 scenarios:
1. The instruction doesn't just change the pointer, it clones the entire table in this one instruction and deletes the old one, so when the function ends, the original table is deconstructed cleanly.
2. The instruction does just change the pointer, but the only reason why it doesn't break is because the local table isn't being deconstructed when the function ends.
3. Highly unlikely, but it's possible the game actually tracks which tables are being used by what, so it knows that this specific table is still being used, and isn't collapsed when the function ends, where other tables that might of been created, and the one originally in ore.autoplace that is replaced with the local table created in this fuction are collapsed and deconstructed cleanly.

Now, given that memory leaks are being observed, I'd rule out scenario 3, I mean, a highly unlikely scenario to begin with, I sure as hell wouldn't want to try programming that in if this was my game. This leaves us to either scenario 1 or 2. Given the fact that table.deepcopy exists, a function that appears to explicitly copy a table, my inclination is that scenario 2 is correct, meaning tables are not cleanly destroyed when not used anymore.

I know for a fact that my techsave mod, the one that tracks the progress of a research and restores it when swapping active researches, constantly creates and replaces new tables, this was done because I couldn't figure out a clean way to remove a table entry via it's name (table.remove wants an index number, when all I have is the index name), so instead recreates the entire table minus this entry, then saves that instead. The instances of the recreated tables is actually few (Every time research completes, every time a force is merged, and possibly during version change if researches have been removed from the game), but knowing my mods contribute to memory leakage is disturbing.



Now to put this in context of this discussion. you run an infinite loop that creates an endless table, end the game, but don't exit the game, and start a new game, well, the table still exists from the previous session because it wasn't deconstructed cleanly, and is just leaking memory.

[prg]

Now here is my issue. If it is true that it is POINTING to a local variable, then when the function ends, the local should deconstruct cleanly and not exist anymore, therefore ore.autoplace should point to an invalid object, which isn't the case.

This is not C++. Making a variable local changes where its name is visible, it does not get scheduled for deconstruction when it falls out of scope.

3. Highly unlikely, but it's possible the game actually tracks which tables are being used by what, so it knows that this specific table is still being used, and isn't collapsed when the function ends, where other tables that might of been created, and the one originally in ore.autoplace that is replaced with the local table created in this fuction are collapsed and deconstructed cleanly.

You've just discovered how garbage collectors work.

Now to put this in context of this discussion. you run an infinite loop that creates an endless table, end the game, but don't exit the game, and start a new game, well, the table still exists from the previous session because it wasn't deconstructed cleanly, and is just leaking memory.

Trying to stuff an infinite amount of data into a table is not a memory leak. Also if you've got an infinite loop in your Lua code, you can't exit the game and start a new one anymore because it's just frozen.

[ratchetfreak]

Therefore, you must assume one of 3 scenarios:
1. The instruction doesn't just change the pointer, it clones the entire table in this one instruction and deletes the old one, so when the function ends, the original table is deconstructed cleanly.
2. The instruction does just change the pointer, but the only reason why it doesn't break is because the local table isn't being deconstructed when the function ends.
3. Highly unlikely, but it's possible the game actually tracks which tables are being used by what, so it knows that this specific table is still being used, and isn't collapsed when the function ends, where other tables that might of been created, and the one originally in ore.autoplace that is replaced with the local table created in this fuction are collapsed and deconstructed cleanly.

4. the lua engine uses a garbage collection algorithm that runs "later" to clean up non-referenced tables. Heuristics in the algorithm make it only run after X amount of memory is used.

Is the lua engine designed to have multiple instances of lua running? Most applications would have only one lua environment active but factorio has one per mod (AFAIK). So if the heuristics are designed for only 1 instance then the GC could be not eager enough.

[ArderBlackard]

Now here is my issue. If it is true that it is POINTING to a local variable, then when the function ends, the local should deconstruct cleanly and not exist anymore, therefore ore.autoplace should point to an invalid object, which isn't the case.

This is not C++. Making a variable local changes where its name is visible, it does not get scheduled for deconstruction when it falls out of scope.

Right, the table memory will be freed upon garbage collector run if there are no variables referencing it. So, Bob, you will end with ore.autoplace pointing to the new table (which was previously also pointed to by a local variable) and the old table will be referenced by noone so it eventually will be collected by GC.

[bobingabout]

Now to put this in context of this discussion. you run an infinite loop that creates an endless table, end the game, but don't exit the game, and start a new game, well, the table still exists from the previous session because it wasn't deconstructed cleanly, and is just leaking memory.

Trying to stuff an infinite amount of data into a table is not a memory leak. Also if you've got an infinite loop in your Lua code, you can't exit the game and start a new one anymore because it's just frozen.

Except in the specific example given(By the OP), the computer crashs because factorio is taking up more memory than the system can handle, as a result of the table not being cleared.

[prg]

Except in the specific example given(By the OP), the computer crashs because factorio is taking up more memory than the system can handle, as a result of the table not being cleared.

Why should a table be cleared that's still being used?

[ArderBlackard]

Now to put this in context of this discussion. you run an infinite loop that creates an endless table, end the game, but don't exit the game, and start a new game, well, the table still exists from the previous session because it wasn't deconstructed cleanly, and is just leaking memory.

Trying to stuff an infinite amount of data into a table is not a memory leak. Also if you've got an infinite loop in your Lua code, you can't exit the game and start a new one anymore because it's just frozen.

Except in the specific example given(By the OP), the computer crashs because factorio is taking up more memory than the system can handle, as a result of the table not being cleared.

It's actually not a memory leak as the memory is used by allocated tables. If they were released - garbage collector would eventually free the memory.

[Divran]

Now here is my issue. If it is true that it is POINTING to a local variable, then when the function ends, the local should deconstruct cleanly and not exist anymore, therefore ore.autoplace should point to an invalid object, which isn't the case.

Nah, this is just the way Lua works (other languages may work this way as well). Any reference to a variable keeps that variable alive. You'd need to set both the local and the table's reference to nil to have the garbage collector collect it.

EDIT: Other people have already explained exactly this, ignore me lalalaa

EDIT 2: To get lua to behave the way you want, you can use a metatable. Metatables in Lua have the ability to configure a table to become a weak table. Any references a weak table has to other values will be ignored by the garbage collector, so if the only references to a variable is a bunch of weak ones, then the variable will be collected.

[Divran]

Okay so now that we've established that this actually is a bug (the check for memory usage already exists in factorio, only problem is it doesn't work most of the time, in windows 10. Doesn't work in linux either but there it isn't a problem because linux automatically terminates the process when necessary), and not a feature request nor a modding help request, could we get this thread moved back to the proper subforum (bug reports)? I feel it has a larger chance of being seen by developers then, and eventually getting fixed.

[bNarFProfCrazy]

Have you provided any prove or material for reproducing the issue?

[Divran]

That's not necessary. All you need to do to reproduce it is to make something in a mod that eats all your memory (for example, a loop that keeps adding more values to a table). Then, create a new game. If it doesn't crash at first, create another new game without restarting factorio. Then you should be completely out of ram. Now it should either crash your entire computer immediately, or you may need to attempt to launch any other app (such as task manager, which is what I did once when it happened) to use that tiny bit of too much ram. I've tried this both on my actual computer (with windows 10 home 64bit) and in two VMs running in VMWare Player: ubuntu VM (don't have the version number right now) and windows 10 home 64bit VM. All three have crashed in almost the same way. Only difference is that linux automatically terminated the factorio process instead of crashing the whole VM, because linux is smarter than windows about things like this.

[bNarFProfCrazy]

That's not necessary.

You are missing the point.
You encountered a bug.
You have to offer if possible (without redundancy)

• a crash log file
• a screenshot
• a description or material how to reproduce it (done)
• ...

The devs have already tested your issue and they don't have that issue, so the ball is on your side once again.
Now you need prove that the issue exists on other systems than your own.
Maybe the issue is in the mod you created and can only produced in a specific way, that the devs didn't use during testing. Who knows? Thats why I asked you to provide the mod so that others can confirm the bug and to give the devs a chance to look at it.

In code there are "sideeffects" that "magically" do some stuff you don't expect, but without code its hard to reproduce them. I know lots of developers myself. If you have nothing in your hand then the theoretical chance of finding the right spot alone (without a solution) is worse than the lottery "6 out of 49 (+super number)" if you pass the 140MB border.

If 1 or 10 people encounter the issue (in rare cases) its not worth adding something to the game.
If 1% or 10% of the players encounter the issue then the devs have to consider what they are doing.

But currently it is only you who has that issue. That is sad of course, but its your task to find others.
If this would be a common issue then more people would have come forward yet.