Page 1 of 3

[MOD 0.13.x] Interface Chest - 1.13.2

Posted: Sat Apr 02, 2016 5:01 am
by Peppe
Type: Mod
Name: Interface Chest
Description: Inline/Slipstream chests to push/pull items to belts. Chest connects to adjacent inventories to: Pull from adjacent inventory and push to belt or Pull from belt and push to adjacent inventory, inline lane balancer, and a trashcan.
License: MIT
Version: 1.13.0
Release: 2016-06-27
Tested-With-Factorio-Version: 0.13.xx
Category: Item
Tags: chest, loader, inline, slipstream, trash can, void, train, logistic, balancer
Download-Url: https://mods.factorio.com/mods/Peppe/InterfaceChest
Website: https://github.com/Peppe-KSP/InterfaceChest/
License
Long description
Pictures
Version history
Thanks

Re: [MOD 0.12.x] Interface Chest - 1.0.0

Posted: Sat Apr 02, 2016 5:27 am
by Supercheese
Cool implementation of the loader, but sadly they are somewhat bad for game performance. Running just 4 interface chests costs ≈0.35 in time_used, and this just gets more painful as you add more chests. :(

Re: [MOD 0.12.x] Interface Chest - 1.0.0

Posted: Sat Apr 02, 2016 5:34 am
by Peppe
Supercheese wrote:Cool implementation of the loader, but sadly they are very bad for game performance. Running just 4 interface chests costs ≈0.35 in time_used, and this just gets more painful as you add more chests. :(
Where is time used?

This was my first iteration to survive a full train station implementation, so I thought I was onto something decent... tried to throttle and offset when chests update -- at least that was my intention, but not super familiar with LUA or factorio's api/engine.

Re: [MOD 0.12.x] Interface Chest - 1.0.0

Posted: Sat Apr 02, 2016 8:14 am
by Supercheese
Under the debug info menu (F4) there is "show_time_used_percent", which gives you some measure of how much CPU your mod is taking up (mod-InterfaceChest). If the number there goes in excess of ≈2.0 (or thereabouts, this is just my gut instinct giving an arbitrary value), it is an indicator that it might start hurting overall game performance.

For example, I used 16 interface chests and the value increased to a steady ≈1.25. Tolerable. 36 interface chests puts it around 2.5. While your game can survive this, if you start using them to unload all your trains, it can bog down.

It's not that terrible, as I made it sound in my first post, this is true. It's likely not as bad as using inserters on Storage Warehouses, but that isn't as easily viewable in the Script Update section ;)

Re: [MOD 0.12.x] Interface Chest - 1.0.0

Posted: Sat Apr 02, 2016 12:54 pm
by Peppe
Thanks for the info gives me a number to compare changes against. Tried a couple alternate implementations and had no impact. Then tried to remove all the get chest contents and the alternative nearly doubled the time used. But now I can atleast try things and see a result other than UPS.

Is the number different for each machine, like I have 50+ active chests in my world and it is ~2. Could be doing different things than your tests, or could be my CPU is faster so I need more to see an impact? If I had a slower CPU would my used time be higher for the same base?

So I assume eventually my time used could get too high and ups drops and then frames drop and that is when the game is less playable/choppy.

Need to isolate each part of the script and see what is eating the most cycles.

Impressed the game uses so little for most tasks on that f5 board. It is me and entity's as the only things above 1.5 on my f5. Entity is I assume all the vanilla belts, buildings, robots, and inserters?

Re: [MOD 0.12.x] Interface Chest - 1.0.0

Posted: Sat Apr 02, 2016 1:21 pm
by Choumiko
Without having used it, i can maybe give some hints:

You seem to use LuaSurface:find_entities_filtered up to 24 times per chest per tick (12 times for getInputBelts, 12 times for getOutputBelts) in the worst case, i guess you could save a lot of time if you do this in the various events related to placing/mining/rotating/dying entities and save them along with the InterfaceChests in global. It's unlikely that they change every tick, so the more complex work might be worth the effort.

Maybe as a first test change the code so it only looks north for input and only south for output and place a few dozen chests along with belts, etc and observe the timing of your mod.

Re: [MOD 0.12.x] Interface Chest - 1.0.0

Posted: Sat Apr 02, 2016 9:34 pm
by Peppe
Choumiko wrote:Without having used it, i can maybe give some hints:

You seem to use LuaSurface:find_entities_filtered up to 24 times per chest per tick (12 times for getInputBelts, 12 times for getOutputBelts) in the worst case, i guess you could save a lot of time if you do this in the various events related to placing/mining/rotating/dying entities and save them along with the InterfaceChests in global. It's unlikely that they change every tick, so the more complex work might be worth the effort.

Maybe as a first test change the code so it only looks north for input and only south for output and place a few dozen chests along with belts, etc and observe the timing of your mod.
While it was throttled to not do the tile search every tick I am refactoring that now. In first pass on performance I have 50% improvement :) 200 chests went from 8-9 to 4ish. I could also slow the chests down from ~compressed redbelt speed to yellow belt speed and reduce the load further.

Instead of filtered search I am now just getting whatever is on the tile and looking through my known list of types my chest can work with and see what it is:
Something like:

Code: Select all

	local entity = game.get_surface(1).find_entities(area)
	if entity[1] then
		if (entity[1].type == "transport-belt" or entity[1].type == "splitter" or (entity[1].type == "transport-belt-to-ground" and  entity[1].belt_to_ground_type == undergroundType)) and  entity[1].direction == direction then
				return entity[1] 
		end
	end
One weird thing was checking the force of the item seemed to add 1 use-time itself, so for now -- worry about supporting all use cases later.

I'm going to probably pull the surface search up another layer or two and try to get the get_surface calls down as they seem high cost. I could see getting a whole 3x3 area with the chest at the center and then using the result to figure out what the chest should be doing.

I looked over the slipstream mod and it looks like he got through/around this by caching the input/output location on placement. Could be an option. Not sure what his performance ended up at.

Re: [MOD 0.12.x] Interface Chest - 1.0.0

Posted: Sat Apr 02, 2016 9:56 pm
by orzelek
From what I remember you can't store entity as table key but you can store it in the table.
So if you store the belts around the chest along with it in a table then you won't need to call any searches until something changes.
So tracking of the built/bot constructed and similar deconstructed events plus the valid field of entities in question should mean you don't need to do any searches.

Re: [MOD 0.12.x] Interface Chest - 1.0.1

Posted: Sun Apr 03, 2016 10:35 pm
by Peppe
Updated the mod. Should upgrade from 1.0.0 if anyone is using it. Some chests in 1.0.0 might lose their entry in the master list and need to be replaced.

Dug in the world of lua optimization.... refactored most of the mod to focus on performance and slowed the output rate down a hair (6 times a second vs 10). Output per side is basically double 2 per cycle, so 12 items a second or ~yellow belt throughput.

In a 200 chest torture test it went from 8 use time down to 1.5. In the end changing the output speed was worth ~1 use time across 200 chests. I think slowing them down was a reasonable tradoff to get under ~2 use time for 200 chests in 100% active duty time.

In a real base test ~90 chests is using .4-.5 which is about the same amount as the electric network for the base (25 MW, 400 solar+30 steam). Chest use less than my torture test becuase they can get full/empty and have no work to do and idle that cycle.

Did not find a good event to watch for a train stopping at a chest, so chests next to a rail are on a 2 second auto polling interval. Impact is low and is fairly responsive to a train stopping/leaving.

All other chest input/output updates are event bases, so rotating, placing, or removing something in the 3x3 grid of the chest will trigger a scan of it's area and update it's inputs/outputs.

Re: [MOD 0.12.x] Interface Chest - 1.0.1

Posted: Mon Apr 04, 2016 3:13 am
by Supercheese
Very nice, this is a marked improvement. :)

Re: [MOD 0.12.x] Interface Chest - 1.0.2

Posted: Mon Apr 04, 2016 6:09 pm
by Peppe
1.0.2 Seperated out the trash can functionality into it's own item. Should work with blueprints/bots better now and avoid accidents...

Tried to add any missing ".valid" checks before doing anything with a chest. Could occur in 1.0.1 when removing the chest with an update scheduled.

Not sure if this was the best way to handle it, but on upgrading to 1.0.2 chests will migrate and it will do a one time search 5000 blocks from the map origin for my chests and index them. Chests outside that range would need to be broken/replaced.

Re: [MOD 0.12.x] Interface Chest - 1.0.2

Posted: Mon Apr 04, 2016 8:02 pm
by Adalonus
I'm very excited to use this mod. I'm downloading it right now.

Does it cost any energy to use it balance wise? If it's going to compress a full yellow belt maybe consider making it cost the energy equivalent of 6 fast inserter or, if possible, make extracting an item cost the kJ/item equivalent of a fast inserter.

Additionally, this item does not interface with other mod inventories, i.e. the Warehousing Mod.

Re: [MOD 0.12.x] Interface Chest - 1.0.2

Posted: Tue Apr 05, 2016 5:03 pm
by Peppe
Adalonus wrote:I'm very excited to use this mod. I'm downloading it right now.

Does it cost any energy to use it balance wise? If it's going to compress a full yellow belt maybe consider making it cost the energy equivalent of 6 fast inserter or, if possible, make extracting an item cost the kJ/item equivalent of a fast inserter.

Additionally, this item does not interface with other mod inventories, i.e. the Warehousing Mod.
Updated to 1.0.4.

Chests now use:
3kw idle (a little higher than 6 fast inserters).

Active output uses ~100kw per side (roughly 6 fast inserters doing constant work)

Active input uses ~50kw per side.

Can now pull from neighboring smart containers and warehouses.

---
It should already work for logistics warehouse and now smart warehouse? I will need to download and test the mod... it may be like trains and it's collision box would need to be checked outside the N, S, E, W grid I search for chests/belts in.

---

Updated to work with all warehouses. There is one position quirk that you can access the warehouse from the corner squares... not sure if it is worth fixing... don't want to lose how it works for trains currently and don't want to write anything too custom just for warehouses.

Re: [MOD 0.12.x] Interface Chest - 1.0.2

Posted: Tue Apr 05, 2016 7:32 pm
by Adalonus
Peppe wrote:
Adalonus wrote:I'm very excited to use this mod. I'm downloading it right now.

Does it cost any energy to use it balance wise? If it's going to compress a full yellow belt maybe consider making it cost the energy equivalent of 6 fast inserter or, if possible, make extracting an item cost the kJ/item equivalent of a fast inserter.

Additionally, this item does not interface with other mod inventories, i.e. the Warehousing Mod.
Updated to 1.0.4.

Chests now use:
3kw idle (a little higher than 6 fast inserters).

Active output uses ~100kw per side (roughly 6 fast inserters doing constant work)

Active input uses ~50kw per side.

Can now pull from neighboring smart containers and warehouses.

---
It should already work for logistics warehouse and now smart warehouse? I will need to download and test the mod... it may be like trains and it's collision box would need to be checked outside the N, S, E, W grid I search for chests/belts in.

---

Updated to work with all warehouses. There is one position quirk that you can access the warehouse from the corner squares... not sure if it is worth fixing... don't want to lose how it works for trains currently and don't want to write anything too custom just for warehouses.
I love how it works with trains. This is brilliant. Thank you!

Re: [MOD 0.12.x] Interface Chest - 1.0.4

Posted: Sat Apr 16, 2016 12:26 am
by gaza9422
This is great. I was waiting for the slipstream chest mod to update but this looks like an improved version. If this mod works with my mod pack, which I will test at a later date, may I have permission to add it to my pack?

Re: [MOD 0.12.x] Interface Chest - 1.0.4

Posted: Sat Apr 16, 2016 12:30 am
by Peppe
gaza9422 wrote:This is great. I was waiting for the slipstream chest mod to update but this looks like an improved version. If this mod works with my mod pack, which I will test at a later date, may I have permission to add it to my pack?
Sure. Anyone can use it/distribute fork however they see fit.

Re: [MOD 0.12.x] Interface Chest - 1.0.5

Posted: Sat Apr 16, 2016 2:00 am
by Peppe
Released, 1.0.5 (2016-04-15): Night time power fix

Just noticed in a test game that the interface chests shutoff their power draw at night since their power was based off an accumulator.
Switched the power to be based of a laser turret. So far no issues.

Re: [MOD 0.12.x] Interface Chest - 1.0.5

Posted: Sat Apr 16, 2016 7:28 am
by Sedar
Peppe wrote:Released, 1.0.5 (2016-04-15): Night time power fix

Just noticed in a test game that the interface chests shutoff their power draw at night since their power was based off an accumulator.
Switched the power to be based of a laser turret. So far no issues.
Thank you for the wery useful mod. :D

btw, how about to add the next grade of the interface chests with more speed of load/unload stuff. :roll:

Re: [MOD 0.12.x] Interface Chest - 1.0.5

Posted: Thu Apr 21, 2016 9:07 pm
by Jeri.c
Can you add like different types of chest that inputs/outputs at different speed or have it include 2-4 slots (increases every research) for speed modules. :mrgreen:

Re: [MOD 0.12.x] Interface Chest - 1.0.5

Posted: Fri Apr 22, 2016 12:51 am
by Peppe
Jeri.c wrote:Can you add like different types of chest that inputs/outputs at different speed or have it include 2-4 slots (increases every research) for speed modules. :mrgreen:
I'll do some performance tests and see if a faster chest would work after the optimizations i did. Initially the chests were tuned to be about twice as fast, but since this requires a lot of inventory checking through the API scripts it had the potential to slow down a game.