[MOD 0.13] EfficienSee : optimise your factory !

[pyrolytic_tungsten]

Never noticed factorio had a multi threaded menu option...

honestly I don't know, but when I look at my 4 cores charge, on a big factory, it seems that 4 cores are working equally around 30-40%

Side threads are used to handle few things but main simulation is limited to single thread.
I'm not 100% sure if render uses it's own thread or not.

Factorio is threaded like this:

This is detailed at https://www.factorio.com/blog/post/fff-70.

[binbinhfr]

@pyrolytic_tungsten : about this different (interesting) subject, could you please exchange by PM or on another thread. Thx guy.

[depeter]

Since the most recent update I did (not sure what exact version I had before, previous update was max 2 days ago), I'm getting an error when loading a game. I do have a ton of other mods active, so it might not be just this mod that's causing it, but disabling this mod is enough to remove the issue.

Saving a game with this mod active, and then loading it again, gives the following error after the loading progress bar is complete, just before it would normally switch to the normal game-playing view :

Code: Select all

Error while running deserialization: [string "do local _={ticks=87,enabled=true,force_mem={..."]:1: too many C levels (limit is 200) in main function near "802384732948701238470123""

If you can't reproduce it (suggesting it might indeed be due to some interaction with another mod), let me know, and I'll try to narrow it down a bit more

[binbinhfr]

@depeter : wao, this one is weird, because it seems to involve factorio memory itself. But it has to do with my mod, because force_mem is one of my variable.

Are you running solo or MP ? What factorio version ? Did you try very last version of my mod (1.0.15) ?

Would it be possible to upload somewhere you faulty map, with the exact set of mods so that I can reproduce at home ?

Does someone else have this problem ?

EDIT : maybe you can try to open your map without my mod installed. Save it (to wipe all data concerning my mod).
Then install very last very of my mod. And reopen map to reset fresh data. And then tell me...

[depeter]

I'm running the latest Factorio version, in single player mode, and the latest version of your mod. I'll make a zip file and upload it somewhere, PM'ed you the link. Zip contains : full mod directory (with your mod active initially), save made with your mod (that triggers the error when loading with your mod), save made while your mod was toggled off (that doesn't trigger the error when loading without your mod). You can then use the latter (after toggling your mod off) to easily see what all the mods add.

I just remembered seeing something about a 200 limit error with another mod relating to trees, so you might check with TheSAguy and/or look into what he did, it seems related to amount of local variables (so maybe in your case, amount of recepies added by mods, or amount of items I start the game with, or amount of different items you're tracking, etc)
viewtopic.php?f=93&t=17548&hilit=200+li ... 60#p186519
viewtopic.php?f=93&t=17548&hilit=200+li ... 40#p184673
viewtopic.php?f=25&t=26915&p=170533&hil ... ee#p170533

[binbinhfr]

Did you try this :

EDIT : maybe you can try to open your map without my mod installed. Then save the map (to wipe all data concerning my mod).
Then install very last very of my mod. And reopen map to reset fresh data. And then tell me...

I saw it was suggested on another post

[Jupiter]

I have a feature request, a question and 2 possible a bug reports.

In the object screen (the lower part of the main screen) there are number concerning the number of machines that create the selected object. active assemblers/total assemblers with that recipe/total assemblers.
Question: Why is that last number shown for every item and every recipe? The number is the same for all right? Maybe you can just show it just once somewhere on the mainscreen instead of repeat it dozens of times.
Possible bug 1: for most (but not all) of the recipes for which I actually have one or multiple assemblers the mainscreen says that I have 0 assemblers for them. Even if the assembers are active the numbers stay 0. The weird part is that those recipes are still shown even with "show unused recipes" turned off.
Possible bug 2: For some reason, if I open the mainscreen the "show machine status" is turned on but there is no actual on-map info shown. If I toggle the button off and on then the info does show.

Both bugs are visible in this img:


Feature request:
Could you please add some numbers that show the theoretical maximal consumption and production of every recipe/item (well, only the recipes that are shown of course)? I would use this to make sure that I have enough (and not way more than enough) assemblers creating items used in other recipes. For example, I'm renovating my oil processing and I want to scale my plastic and acid production to the size of my red and blue circuit production and all other consumers of those products. I want to make sure that I can supply enough of both to have all my plastic/acid consumers working at the same time. It is possible to do this by hand but it would be a hassle.

[binbinhfr]

Hi,

> In the object screen (the lower part of the main screen) there are number concerning the number of machines that create the selected object. active assemblers/total assemblers with that recipe/total assemblers.
Question: Why is that last number shown for every item and every recipe? The number is the same for all right? Maybe you can just show it just once somewhere on the mainscreen instead of repeat it dozens of times.

These numbers are : active assemblers with that recipe / total assemblers with that recipe / total assemblers in game (all recipes)
same things with furnaces.

> for most (but not all) of the recipes for which I actually have one or multiple assemblers the mainscreen says that I have 0 assemblers for them. Even if the assembers are active the numbers stay 0. The weird part is that those recipes are still shown even with "show unused recipes" turned off.

Note that these numbers are refreshed only if the corresponding machines are tagged on map. It's to avoid over calculations. When not refreshing, these numbers are greyed (you can see it on the screencopy, where only the green circuit is attged on map, and thus its numbers are refreshed) The problem with this mod is that I cannot constantly calculate every machine and every recipe. It would be too much for the cpu. If you read above, there already a user that has numerous mods that has memory problems... So I'll have to rewrite the mod inside again to solve this, and make some choices.

> For some reason, if I open the mainscreen the "show machine status" is turned on but there is no actual on-map info shown. If I toggle the button off and on then the info does show.

For the same reason, I cannot track which family of machines are already "tagged" on map. So this "show all status" button is not always accurate, and sometimes you have to click it twice...

> Could you please add some numbers that show the theoretical maximal consumption and production of every recipe/item

I think you are looking for the "dP/dC norm" numbers that I show in the recipe pane : If no ingredients are used in another recipe, to be perfect, the recipe normalised numbers should be equal. On your screencopy, you should have -68 -68 +68, or -244 -244 +244, meaning that the consumed objects are balanced with the produced objects. But I suppose that you iron-plate is used in another recipe too, so it's only an indication... The problem is that items are used in many recipes, so these theoritical rates are not easy to compute...

[Jupiter]

These numbers are : active assemblers with that recipe / total assemblers with that recipe / total assemblers in game (all recipes)
same things with furnaces.

I understand what they are. I am proposing to remove that last number (total number of assemblers) from the table and simply show it once somewhere at the top of the mainscreen or something. There is no point in showing the same number a 100 times. Well, maybe this wasn't a question but a feature request as well.

Note that these numbers are refreshed only if the corresponding machines are tagged on map. It's to avoid over calculations. When not refreshing, these numbers are greyed (you can see it on the screencopy, where only the green circuit is attged on map, and thus its numbers are refreshed) The problem with this mod is that I cannot constantly calculate every machine and every recipe. It would be too much for the cpu. If you read above, there already a user that has numerous mods that has memory problems... So I'll have to rewrite the mod inside again to solve this, and make some choices.

Ah oke, yes now the numbers do update.

For the same reason, I cannot track which family of machines are already "tagged" on map. So this "show all status" button is not always accurate, and sometimes you have to click it twice...

It would be nice if you could make it so that the visual status of the toggle buttons actually corresponds with whether the on-map info is shown.

I think you are looking for the "dP/dC norm" numbers that I show in the recipe pane : If no ingredients are used in another recipe, to be perfect, the recipe normalised numbers should be equal. On your screencopy, you should have -68 -68 +68, or -244 -244 +244, meaning that the consumed objects are balanced with the produced objects. But I suppose that you iron-plate is used in another recipe too, so it's only an indication... The problem is that items are used in many recipes, so these theoritical rates are not easy to compute...

It would be awesome if you could manage to implement it. It's a one time calculation anyway. It's not something that you would need to track in realtime. These numbers only need to be updated when the player is placing new assemblers (and assigning recipes to them). I would also understand if you choose not to, please don't feel obligated.
PS: I just realized that the whole story of theoretical consumption gets waaaay more complicated if you start thinking about modules and beacons. hmmmm......

[binbinhfr]

> I am proposing to remove that last number (total number of assemblers) from the table and simply show it once somewhere at the top of the mainscreen

yes, I'll do this.

> It would be nice if you could make it so that the visual status of the toggle buttons actually corresponds with whether the on-map info is shown.

Alas as I told you, it's very cpu consuming and difficult to track. Because machines can be destroyed, change their recipe (furnace can change their recipe at any moment). So I cannot track every machine in realtime at any moment. And I need this to update this button accurately.

> I just realized that the whole story of theoretical consumption gets waaaay more complicated if you start thinking about modules and beacons. hmmmm......

Ah yes, it's way too complicated and not a one time calculation. These theorical numbers are dependant on your factory settings, assembler types, beacon, module, etc... You have spreadsheet on the internet to calculate this, I think. But later, when my mod will be stable, I will think about it.

You must also understand that the production/consumption numbers are for one item in general, not one item per recipe.
For example, I display the overall production/consumption of iron-plate, even if you just show the green circuit recipe. I do not display the iron-plate consumption of this particular recipe. This number is not available in factorio API.

[Peppe]

@depeter : wao, this one is weird, because it seems to involve factorio memory itself. But it has to do with my mod, because force_mem is one of my variable.

Are you running solo or MP ? What factorio version ? Did you try very last version of my mod (1.0.15) ?

Would it be possible to upload somewhere you faulty map, with the exact set of mods so that I can reproduce at home ?

Does someone else have this problem ?

EDIT : maybe you can try to open your map without my mod installed. Save it (to wipe all data concerning my mod).
Then install very last very of my mod. And reopen map to reset fresh data. And then tell me...

I got this same error on my save as well. Loaded without your mod and the save worked again. Saving then without your mod and then adding your mod back seems to also be working.

Was not sure it was your mod since I changed some other mods at the same time and usually factorio is good about telling you which mod is causing an error. Was planning to investigate today when I saw the same error here.

[binbinhfr]

I published a v1.0.16 to do an automatic revert to 1.0.13, in order to prevent further errors.

please update and be patient Thats' a big piece and testings are not easy. But we'll soon obtain a stable version, with the help of courageous beta-testers

I am currently rewriting the core.

[binbinhfr]

please try 1.0.20 ; I hope we are going more stable now. The size of the scprit.dat is far reduced now that
I do not store everything forever. A little more work while loading objects and recipes, but nothing too lagy
while I tested it.

1.0.20 - now, in the GUI, you can click the main recipe or object buttons with an object in
your hand to browse the corresponding recipe or item.
- correct a possible gui problem with recipe duplicated in ingredients and products list.
1.0.19 - minor gui change
1.0.18 - due to big memory problems with some users using many mods and recipes,
I had to rewrite all the core once again, with less stored data.
- new way to manage/update machine counts.
- delete total machine counts from recipe line and put them once on the bottom line

[binbinhfr]

Due to past memory problems, to clean your map file from possible unwanted data, I recommand unsintalling the mod, loading your map, saving it to clean data.
Then reinstalling the mod.

The better would be to start from an old map save that never saw the mod before

(Sorry but it seems that factorio API is not made to store vast amount of data with various indexes type, but I was not aware of that when starting this mod)

[Peppe]

Due to past memory problems, to clean your map file from possible unwanted data, I recommand unsintalling the mod, loading your map, saving it to clean data.
Then reinstalling the mod.

The better would be to start from an old map save that never saw the mod before

(Sorry but it seems that factorio API is not made to store vast amount of data with various indexes type, but I was not aware of that when starting this mod)

Could you use a migration script to clean-out the data structures? Or a uninstall version of you mod that just loads and cleans whatever data you are concerned about out?

[binbinhfr]

Could you use a migration script to clean-out the data structures? Or a uninstall version of you mod that just loads and cleans whatever data you are concerned about out?

I already clean all these unwanted data in the on_config event of last mod versions.
This should be ok.
But the uninstall / reinstall trick is the best IMHO, to get a totally fresh start.
I never heard about "uninstall version of mod" : for me, if you uninstall a mod, factorio deletes all the data that was concerning it.
Do you have problems with the mod ? It works for me on my base, but as some people with a lot of mods/recipes encountered some problems in the past, I prefer to put that warning...

[Peppe]

Could you use a migration script to clean-out the data structures? Or a uninstall version of you mod that just loads and cleans whatever data you are concerned about out?

I already clean all these unwanted data in the on_config event of last mod versions.
This should be ok.
But the uninstall / reinstall trick is the best IMHO, to get a totally fresh start.
Do you have problems with the mod ? It works for me on my base, but as some people with a lot of mods/recipes encountered some problems in the past, I prefer to put that warning...

You just had me concerned when you said my save my be dirty with older version data structures/indexes since I don't have a pure pre-efficiensee mod save at this point.

The latest version seem good on performance. I don't know that I have saved/loaded with every version step. Do you only cleanup the previous versions? I feel like maybe the save takes longer to load since adding the early versions of EfficienSee, but other things have changed since and may base/world has grown as well, so may be unrelated.

[binbinhfr]

The loading time is always increased when you change you mods list : because in this case, Efficiensee has to update its recipes list and also to rebuild assemblers/furnaces/drills list in case some of them disappeared with a mod uninstall. I need those list to calculate statistics on recipes/machines/items, etc...
But if you just load a game with no mod changes, efficiensee does not steal any time.
If you want to be safe, do the uninstall/save/reinstall trick. It will clean all.

And honestly, you should always create a backup copy of your map from times to times, because even Factorio is not always stable when there is a version change, so it is safer, in order to be able to revert...

[Litcube]

I downloaded this because I'm looking for an in game interface to see ''ingredient of' in-game. Doesn't seem like it's intended for that purpose, though. Still, it's a pretty slick utility.

It would be neat to see all recipes that use a selected item for planning a bus.

[binbinhfr]

I downloaded this because I'm looking for an in game interface to see ''ingredient of' in-game. Doesn't seem like it's intended for that purpose, though. Still, it's a pretty slick utility.
It would be neat to see all recipes that use a selected item for planning a bus.

Oh yes of course you can do that.
You have to check the "unused recipe" checkbox to display "product_of" and "ingr_of" of items that have not been produced yet. Using this, you can navigate the whole production tree, from items to recipes, even if they are not used or researched yet.

And I will soon add a recipe and item list to pickup diretly from the list.

screen21.jpg (50.94 KiB) Viewed 6809 times