Modded Accumulators Which Can Charge/Be Charged By Other Accumulators?

[Rainboy]

Even with a variant of my second algorithm above, there's still some jitter in the power graph unless global throughput is EXACTLY divisible by one tesseract's throughput (which would basically never happen). The only way that I know of to deal with fractional tesseracts worth of throughput in that model is to periodically enable one, which will cause a spike in the graph for that tesseracts electric network.

I suppose, theoretically, I have a couple options for that:

1. I could designate a single tesseract to evaluate every tick on the surplus side (eg, the request side if total request > total supply, or vice versa) and use that to smooth out the curve. That seems rather annoying to implement.
2. I could accept extra throughput (effectively, round throughput up to the nearest whole number of input tesseracts) and allow the global buffer to grow unbounded.

Either of these algorithms would still have the significant downside of not providing even distribution when there are multiple source/destination electric networks. I'll give it some more thought. It's quite a complex problem to solve and, as you said, there are many compromises which could be made.

[eradicator]

(Here's another stream-of-thoughts style post...)

Meh, i just remembered that smoothing the graph in a brownout (under-supply) scenario is exactly the problem i solved back then with runtime adjustment of the flow_limit, and that part of the api was sacrificed when they optimized the electric network. I don't think the exact/over-supply scenario would be affected though and should thus be smooth. You'd basically be doing exactly the same thing as now, only less often, and compensate by storing more energy in EEI buffers which slowly flows in/out until the next update cycle.

Sacrificing the brownout scenario is a major sacrifice that i'd personally not want to do though. Unless one wants to promote it as an "early warning" feature :D.

Hm...another completely different and quite radical solution would be to enforce exactly one real tesseract per electric network. Basically the player would only build dummies, and one of them gets the actual tesseract as a hidden addon. Would completely remove visual feedback or automatic animations. You'd also suddenly have to deal with the structure of electricity networks which would be quite a large chunk of code i imagine. High effort high gain optimization...

It might also be possible to fake a change in the flow_limit by absuding the production/consumption capabilities of the EEI but that is only a vague theoretical concept even in my head, so no clue if or how that would work.

Yet another approach vector: If you're ready to sacrifice *perfect* input/output balancing and be ok with just approximate balancing, then you could internally use a paired approach where you assign each input tesseract to an output tesseract and vice versa. But ofc at that point you might as well do a straight "first come first serve" model and completely drop load-balancing.

So in conclusion i guess the main question is: What is the most important scenario you want to optimize for? In an over-supply case the balancing is useless overhead. In an under-supply scenario it's not, but that doesn't happen that often.

[eradicator]

And for some relaxation: Teleporting my accus in a circle is actually less than twice as expensive as your mod, but more importantly...actually works :o. It's amazing what kind of shit the engine can cope with.

Code: Select all

/c 
accus = {}
for _,accu in pairs(game.player.surface.find_entities_filtered{type='accumulator'}) do
  table.insert(accus,accu)
  end
local last_pos = accus[#accus].position
script.on_event(defines.events.on_tick,function(e)
  for i=1,#accus do 
    local pos = last_pos
    last_pos = accus[i].position
    accus[i].teleport(pos) end
  end)

[eradicator]

So i managed to make my accu sample impelentation significantly faster in the default scenario of more input than output capacity. At least i would assume that most sane players will build slightly more inputs to ensure output is always full... (looks like at least 50% but it also looks like V2 is bugged so measurement is meh. It's faster than the "delete all energy" approach...). In brownout scenarios it should be at least same speed, in complete blackout it's faster, but that's not really important :D.

I do this by storing up to 1 ticks worth of input+output_buffer_size energy in a lua side buffer and only fetch new energy when actually required. This means that often i can completely skip touching the input accus at all. This also lets me simplify the output in most of the time when i already know that i have enough energy for all outputs. This also means that building *more* input accus actually makes it run slightly *faster* if output accu count is the same.

Code: Select all

--ATTEMPT 2: Use a lua side virtual buffer
--do "upkeep" of input tesseracts via actual power consumption of EEI

/c

local function find_accus()
  global.accus = {inputs={},outputs={}}
  for _,accu in pairs(game.player.surface.find_entities_filtered{type='accumulator'}) do
    if accu.position.x < 0 then 
      table.insert(global.accus.inputs,accu)
    else
      table.insert(global.accus.outputs,accu)
      end
    end
  end
  
local function update_stuff()

  global.lua_energy = 0

  global.total_output_buffer_size = 0
  for _,accu in pairs(global.accus.outputs) do
    global.total_output_buffer_size =
      global.total_output_buffer_size + accu.electric_buffer_size
    end

  end
  
local function get_input_energy()
  local inputs = global.accus.inputs
  for i=1,#inputs do
    global.lua_energy = global.lua_energy + inputs[i].energy
    inputs[i].energy  = 0
    end
  end
  
--[[only used if satisfaction *guaranteed* to be 100%]]
local function simple_output_energy()
  local outputs = global.accus.outputs
  local nrg = global.lua_energy
  for i=1,#outputs do
    local this = outputs[i]
    nrg = nrg - (this.electric_buffer_size - this.energy)
    this.energy = this.electric_buffer_size    
    end
  global.lua_energy = nrg
  end
  
--[[used if satisfaction *might* be below 100%]]
local function balanced_output_energy()
  local total_demand = 0
  local outputs = global.accus.outputs
  local nrg = global.lua_energy
  for i=1,#outputs do
    total_demand = total_demand + (outputs[i].electric_buffer_size - outputs[i].energy)
    end
  if total_demand > 0 then --[[watch out for division by zero!]]
    local satisfaction = math.min(1,nrg/total_demand)
    for i=1,#outputs do
      local this = outputs[i]
      local demand = this.electric_buffer_size - this.energy
      local supply = demand * satisfaction
      this.energy = this.energy + supply
      nrg = nrg - supply
      end
    global.lua_energy = nrg
    end
  end
  
  
find_accus()
update_stuff()
  
local oversupply_factor = 1 --[[the larger this is the cheaper handling becomes when there is a *large* oversupply]]
  
script.on_event(defines.events.on_tick,function(e)

  if global.lua_energy < (global.total_output_buffer_size * oversupply_factor) then
    get_input_energy()
    end
    
  if global.lua_energy >= global.total_output_buffer_size then
    simple_output_energy()
    
  elseif global.lua_energy > 1 then --[[lua_energy can get *really* small, but often not 0]]
    balanced_output_energy()
    end
    
  if e.tick % 600 == 0 then
    print('lua_energy is ',global.lua_energy)
    print('total out buffer is ',global.total_output_buffer_size)
    end

  end)
  

[Rainboy]

(Here's another stream-of-thoughts style post...)

Meh, i just remembered that smoothing the graph in a brownout (under-supply) scenario is exactly the problem i solved back then with runtime adjustment of the flow_limit, and that part of the api was sacrificed when they optimized the electric network. I don't think the exact/over-supply scenario would be affected though and should thus be smooth. You'd basically be doing exactly the same thing as now, only less often, and compensate by storing more energy in EEI buffers which slowly flows in/out until the next update cycle.

Sacrificing the brownout scenario is a major sacrifice that i'd personally not want to do though. Unless one wants to promote it as an "early warning" feature .

I think that the brownout situation is exactly the same as the normal situation, except we switch which electric network has a 'surplus' and which has resource contention. In a normal situation, there is a surplus of supply which generates resource contention on the input electric network, making it difficult to eliminate jitter; in a brownout, it becomes a surplus of request which creates resource contention and jitter on the output electric network. Either way, one electric network looks completely smooth, and the other isn't.

Hm...another completely different and quite radical solution would be to enforce exactly one real tesseract per electric network. Basically the player would only build dummies, and one of them gets the actual tesseract as a hidden addon. Would completely remove visual feedback or automatic animations. You'd also suddenly have to deal with the structure of electricity networks which would be quite a large chunk of code i imagine. High effort high gain optimization...

This is a very interesting idea to me. I believe I'd also have to create a new entity for each possible throughput configuration, so internally there'd be a few hundred (at least) entities with each multiple of flow_limit and buffer_capacity (unless I can somehow change these at runtime? but that's the whole problem we're trying to solve: we CAN'T change flow_limit at runtime). I'd then have the base entity be a dummy one that's basically just a picture on the map, and I'd have to group tesseracts by electric network and promote one tesseract to have the combined stats of all the others on that network. I wonder if that actually helps anything having hundreds of entities floating around... does the engine handle that well? Does it realize that it only needs one entry in the atlas for all of them? etc. Like you said, it is a high effort, high gain optimization. I'd be willing to do it if I was reasonably sure the engine would handle it well, but I'm not convinced yet.

It might also be possible to fake a change in the flow_limit by absuding the production/consumption capabilities of the EEI but that is only a vague theoretical concept even in my head, so no clue if or how that would work.

Yet another approach vector: If you're ready to sacrifice *perfect* input/output balancing and be ok with just approximate balancing, then you could internally use a paired approach where you assign each input tesseract to an output tesseract and vice versa. But ofc at that point you might as well do a straight "first come first serve" model and completely drop load-balancing.

So in conclusion i guess the main question is: What is the most important scenario you want to optimize for? In an over-supply case the balancing is useless overhead. In an under-supply scenario it's not, but that doesn't happen that often.

A paired approach would make me change throughput somehow. Currently, both input and output tesseracts have the same throughput; however, there are both global costs as well as percentage costs which force you to have more input tesseracts than output tesseracts. I'd have to figure out how to factor those costs in to a paired approach like that. I'm not against the concept, but the reality would be a good bit of work, especially with the flat global cost in the way.

For me, I've been trying to optimize for balance and quality primarily, and normal throughput (eg, supply much greater than demand) secondly. As much as is reasonable, I've sacrificed CPU on the altar of things like smooth power graphs. I still do what I can to save CPU, but not when it degrades the quality of the mod (though I'm not at all against making settings which allow the user to degrade the quality to prioritize the CPU). I'm also not against any change which increases efficiency for the normal case, even at the cost of a similar efficiency loss during brown/blackout (eg, my second algorithm which needed to evaluate tesseracts far more often when throughput entered or exited brownout status and it began to exhaust/exceed the global buffer). At the end of the day, this is a mod I made for me and my friends because I didn't see anything else like it on the portal (you'll notice that the mod wasn't updated for nearly two years after initial release). Since we thought it useful, I've shared it with the community; but, its development will generally steer into supporting users that have generally good PCs and don't go completely crazy with megabases.

And for some relaxation: Teleporting my accus in a circle is actually less than twice as expensive as your mod, but more importantly...actually works . It's amazing what kind of shit the engine can cope with.

Code: Select all

/c 
accus = {}
for _,accu in pairs(game.player.surface.find_entities_filtered{type='accumulator'}) do
  table.insert(accus,accu)
  end
local last_pos = accus[#accus].position
script.on_event(defines.events.on_tick,function(e)
  for i=1,#accus do 
    local pos = last_pos
    last_pos = accus[i].position
    accus[i].teleport(pos) end
  end)

Man, that's a riot. This engine is hilarious sometimes.

So i managed to make my accu sample impelentation significantly faster in the default scenario of more input than output capacity. At least i would assume that most sane players will build slightly more inputs to ensure output is always full... (looks like at least 50% but it also looks like V2 is bugged so measurement is meh. It's faster than the "delete all energy" approach...). In brownout scenarios it should be at least same speed, in complete blackout it's faster, but that's not really important .

I do this by storing up to 1 ticks worth of input+output_buffer_size energy in a lua side buffer and only fetch new energy when actually required. This means that often i can completely skip touching the input accus at all. This also lets me simplify the output in most of the time when i already know that i have enough energy for all outputs. This also means that building *more* input accus actually makes it run slightly *faster* if output accu count is the same.

Code: Select all

--ATTEMPT 2: Use a lua side virtual buffer
--do "upkeep" of input tesseracts via actual power consumption of EEI

/c

local function find_accus()
  global.accus = {inputs={},outputs={}}
  for _,accu in pairs(game.player.surface.find_entities_filtered{type='accumulator'}) do
    if accu.position.x < 0 then 
      table.insert(global.accus.inputs,accu)
    else
      table.insert(global.accus.outputs,accu)
      end
    end
  end
  
local function update_stuff()

  global.lua_energy = 0

  global.total_output_buffer_size = 0
  for _,accu in pairs(global.accus.outputs) do
    global.total_output_buffer_size =
      global.total_output_buffer_size + accu.electric_buffer_size
    end

  end
  
local function get_input_energy()
  local inputs = global.accus.inputs
  for i=1,#inputs do
    global.lua_energy = global.lua_energy + inputs[i].energy
    inputs[i].energy  = 0
    end
  end
  
--[[only used if satisfaction *guaranteed* to be 100%]]
local function simple_output_energy()
  local outputs = global.accus.outputs
  local nrg = global.lua_energy
  for i=1,#outputs do
    local this = outputs[i]
    nrg = nrg - (this.electric_buffer_size - this.energy)
    this.energy = this.electric_buffer_size    
    end
  global.lua_energy = nrg
  end
  
--[[used if satisfaction *might* be below 100%]]
local function balanced_output_energy()
  local total_demand = 0
  local outputs = global.accus.outputs
  local nrg = global.lua_energy
  for i=1,#outputs do
    total_demand = total_demand + (outputs[i].electric_buffer_size - outputs[i].energy)
    end
  if total_demand > 0 then --[[watch out for division by zero!]]
    local satisfaction = math.min(1,nrg/total_demand)
    for i=1,#outputs do
      local this = outputs[i]
      local demand = this.electric_buffer_size - this.energy
      local supply = demand * satisfaction
      this.energy = this.energy + supply
      nrg = nrg - supply
      end
    global.lua_energy = nrg
    end
  end
  
  
find_accus()
update_stuff()
  
local oversupply_factor = 1 --[[the larger this is the cheaper handling becomes when there is a *large* oversupply]]
  
script.on_event(defines.events.on_tick,function(e)

  if global.lua_energy < (global.total_output_buffer_size * oversupply_factor) then
    get_input_energy()
    end
    
  if global.lua_energy >= global.total_output_buffer_size then
    simple_output_energy()
    
  elseif global.lua_energy > 1 then --[[lua_energy can get *really* small, but often not 0]]
    balanced_output_energy()
    end
    
  if e.tick % 600 == 0 then
    print('lua_energy is ',global.lua_energy)
    print('total out buffer is ',global.total_output_buffer_size)
    end

  end)
  

Wouldn't that generate a ton of jitter on the input side? If I have twice as much supply as demand, wouldn't every second tick consume one tick's worth of supply, generating a very large jump in throughput that tick, followed by no throughput on the tick after?

[eradicator]

Brownout is different than oversupply because in oversupply you don't need output balancing. Ofc this requires that you *know* the situation is oversupply. For the graph it's "spikes on one side or another" i guess, yea...

Wouldn't that generate a ton of jitter on the input side? If I have twice as much supply as demand, wouldn't every second tick consume one tick's worth of supply, generating a very large jump in throughput that tick, followed by no throughput on the tick after?

Thought about it some more and yea...there would be some spikes, but the larger they get the shorter and the more spread apart they are and vice versa, so they might get smoothed away by the graph. I was only thinking about performance when i wrote that though, and forgot to consider the effect on the graph. :p

There are a few more thought trails that could be followed, like using actual production/consumption with a predictive/adaptive transfer volume based on how much was used during the previous cycle, which would sacrifice the tesseracts ability to cover spikes in demand (i.e. laserturrents), but i don't have time to do a full test implementation of that right now.

The engine doesn't care about a few thousand additional entity prototypes (see end of FFF 301), but tracking *every* change to electric networks is not trivial, you'd have to track every pole de/construction, and most annoyingly: manual changes to copper wire. You can change the buffer_size at runtime, so if you used a ridiculously large flow limit (i.e. "1YW") you could control the effective flow limit by adjusting the buffer_size, so you might get away with only one extra prototype and one extra entity per network if you stay with the "once per tick" approach. It's kinda nice though because "handling them all as one entity" is exactly what the base game does with accus/solars. (The more i think about it the nicer this sounds performance-wise...)

[eradicator]

Hm. Maybe you don't even have to watch for copper cable changes with too much precision. You can simply mark the network as "dirty" and reconstruct dirty networks once per second or so. As you're already running every tick this could be done with a single "if some_networks_are_dirty then clean_stuff() end" statement, i.e. at almost no performance cost. And the only thing sacrificed would be that it *might* take some seconds until the mod reacts to manual network changes (i.e. seperating one electric network into two - power switches are already seperate so need no special handling).

[Rainboy]

electric_buffer_size :: double [RW]

Oh, you beautiful creature you. This is wonderful!

I am very much warming to the idea of handling everything through one entity per electric network. I've got some time next week and plan to sit down and see what I can do with the idea. Thanks again for all your help and ideas; I'm really excited about this one. Seems to solve all of the problems with none of the drawbacks (except code complexity, which I'm fine with if I can pull it off).

[eradicator]

I'll be looking forward to hearing if it works as expected. :D

[darkfrei]

Here was an old bug with solar panels, early you can connect one solar panel to two electrical networks and get twice more energy from it.

[Rainboy]

Some random thoughts for me to refer to later:

I think it should be relatively easy to deal with Tesseracts that are added and removed, but dealing with electric networks will probably prove much more annoying. For storage, I'll want a table of electric network ID -> list of tesseracts on that network. The first Tesseract on the list can be dubbed the 'real' tesseract and endowed with a non-zero buffer_capacity (equal to the sum of all tesseracts).

1. When a tesseract is added or removed, it's easy to go to the correct network (via entity.electric_network_id) and modify the list as well as set the buffer_capacity of the first entity. If the first entity on the list is ever removed, the next in line can easily and transparently be promoted to that status.
2. Splitting or combining networks by placing or mining power poles should be easy enough. When a pole is added, I can check that the first tesseract in each list has the correct ID (eg, map key == entity.electric_network_id) with the knowledge that adding a power pole will never split a network and I'm just looking for potential lists to merge. Removing a pole requires traversal of the entire list that matches that pole's electric_network_id to ensure nothing got split off.
3. I need to research how to detect copper cable muckary. What events might get fired when that happens?
4. Same as above for power switches. What events get fired when an electric network is split/combined via power switch?
5. Am I missing any ways (outside of modding activities like teleportation) that an entity can have its electric network ID change?

[eradicator]

Oh, btw, i think you're not handling forces yet?

A few additional thougts from me:

1. Or you might keep the "real" tesseracts in an entirely different list.
2. -
3. For example a custom hotkey with linked_game_control to input.build, checking for cursor_stack. Checking for changes too often is better than missing an event. Blueprints can't change copper diretctly (unlike circuit wire) so that should not be a problem. Mods might do undetectable shit so it might be a good idea to check for changes once every 1/5 minutes anyway, just to be 100% sure everything is right. Miniloaders (i think) has a custom implementation of on_wire_change (not sure how reliable), but as we don't care for circuit wires at all it's probably overkill.
4. As far as i know power switch networks are always split from the view of the modding API, enabling the switch doesn't change network ids.
5. I can think of:
   Vanilla detectable: construction, deconstruction, destruction
   Vanilla no-change: blueprints don't contain copper wire
   Vanilla unknown: undo, redo (i haven't yet done anything with this)
   Vanilla undetectable (afaik): removing entities in /editor mode
   Modded (detectable for non-shitty mods): ghost revival
   Modded (undetectable): connect_neighbour, disconnect_neighbour

[eradicator]

Any luck with implementing any of this?

[Rainboy]

I've been looking at it a little. I need to sit down and test the effects of power switches. Haven't had as much time for it as I was hoping.

[eradicator]

#power-pole, #teleport, #infinite-range, #invisible-wire (hopefully this'll help me find this post in a year or so :p).

Apparently you can connect powerpoles at arbitrary distances on the same surface. As power poles can (as far as i know) in 0.17 be made to not render wires at all, this might have some nice abuse potential with a 0.1 supply-range, 0.0 wire-range invisible power pole (to prevent connection to "real" networks). You'd only need a bunch of EEIs to consume the "upkeep", but the power transfer would be basically free (on the same surface). Just posting that here so i don't forget it myself later :p.

As a workaround, you can teleport the entity close, connect the wire, then teleport it back.