Friday Facts #209 - Optimisation is a way of life

[Jap2.0]

God I wish you and your team would go work on the multi-thousand-dollar CAD software I use at work. Imagine factorio...except that every time you add, remove, or edit any entity, there is a 1-2 second lag and a 0.1% chance of crashing. Then pretend you do that 40 hours a week for a job.

At least there are no biters... but I would gladly take them to make the crashes go away!

I think I just died.
Cause of death: Cringing.

[IronCartographer]

What would help is more tiers of bots- the better the bots, the fewer you need. Or you could make "stack inserter" version of logistic bots- bots that are slow, need huge amount of energy, only move items to storage or to that new chest but move one stack of items at a time.

Angel's Logistics mod adds this kind of bot, and it definitely has its advantages. A more vanilla-friendly solution might be to include infinite research for capacity and battery charge in the base game.

Maybe even add away for them to get damaged in nuke explosions?

It would make sense, but might cause them to take damage from grenades as well due to the damage type being explosion. What really bugs me is fire causing damage to bots flying overhead.

[Uristqwerty]

What if the bot optimization was an upgrade that had to be researched? Explained as something like a design optimized to fly above the reach of enemies, its position in the tech tree could balance between gameplay and the extremely-late-game when the UPS impact is most critical.

Alternatively, how plausible would it be to use the fast bot style until they get within a few chunks of an enemy or player? I have no idea how much complexity it would add to the game, so maybe it's a terrible idea. Perhaps it would lead to far too many lurking desync bugs.

[sowieso]

Those stone paths are beautiful. I never build stone paths in the game because they're not as fast as concrete, and I don't like how they look. If they looked like this, they're probably all that I would build!

How about making the stone path only faster for walking, the concrete paths faster for walking and driving. In this case I would use stone inside the base (where I can't navigate by car) and use concrete for larger distances. In this way, both would make sense.

[riking]

with other changes, it could get down to 16 if I wanted. But it might not be needed until smoke shows on the profiler again.

By changing the TinyVector for direction to an Angle16 that saves you another 2 bytes, with the rest of the commented changes that gets you down to 16 bytes right?

(Angle16 is the type used for directions in Super Mario 64: https://www.youtube.com/watch?v=rJjFXBFiyas , needs its own sin/cos/tan)

[hoho]

Well how about allowing mods to use separate CPU thread to run? I'm sure that factorisimo running in 8 more threads itself with each of mini factory which is world itself would allow us to create insanely big worlds. But that would require some connection that connects those separated worlds.

Problem is, mods tend to access game state data pretty much randomly both to read and write it. To make that thread safe, a whole bunch of limitations have to be made to how mods can access the data.

Also, problem isn't with Factorio being limited by computations. Factorio is limited by memory access latency and bandwidth. Going multicore will increase troubles with latency and it's quite unlikely that it'd help with bandwidth.

[BigDub]

I've been following Factorio for about two and a half years now. While I love to see the art improvements and I like to learn of how you've improved the engineering of the game I am still rather saddened that the game itself has barely changed. In terms of gameplay very few things have changed. Maybe my memory is failing me here, but I think the only real differences since I first played the game is that the rocket is actually implemented, there's modular armor, some new combat robots or special grenades or something were added (which I never even used), the size of the turrets changed, signal towers, and maybe a couple other things here and there. There was a lot of work done to have combinators and logic networks in the game, but by the time those become necessary, which I am still not convinced they ever do, you've already long passed what you needed to do to beat the game.

Basically, I've not seen anything interesting in terms of development of the gameplay itself. It saddens me a bit. I know you guys have done some amazing work on the engineering and art, but I am used to following games like Minecraft before Microsoft bought it. If I stopped following for a couple months I would come back and there would be a host of new features added to the game and the very way you played the game had changed. Maybe this is simply that Factorio is nearing the end of its development and that its game design has solidified, but that thought makes me sad. I feel like there are still many more things Factorio could do. More processes that could be explored.
I don't know. I really enjoyed playing Factorio as it exposed me to mining and manufacturing processes I had never considered before. I learned a great deal researching the real world processes that had been presented in a simplified state in Factorio. I guess I was hoping every once in a while I could come back and find something new to learn and play with.

You guys are doing some great work. Truly. There is some good engineering and art being done. I just don't see evidence of anyone working on the game design.

--Edit--
Just saw 187. Yes. This is what I want. More stuff like that. More systems, more things to learn. Nuclear power. Cool stuff.

[Faark]

I might be misunderstand the electric update... did the game so far really iterate over every (active) electric consumer every tick? I thought for sure they group similar ones together, like they do with solar panels & accumulators! Also since you really want to anyway group factories with the same blueprint, module setup & power grid, since that way you can update the production progress of a bunch of those with a single addition for the entire group (and only individually processing those that actually have finished an item) kinda like described in fff-176 with items transported on belts. Same with inserters... as long as they move from inventory to inventory (chest or factory; not belt or ground) how long that takes seems fully depended on inserter type & energy supply so as long as they have the same energy source they will finish in the same order, thus you can put them in a single queue when they start to move and only take them out once they finish. No iterating over other moving inserters, not even any kind of sorting involved. As usual you would have to put up with more complex source code in favor of performance. For a game like Factorio advertising huge factories those tradeoff might be worth it, especially when we place the same patterns over and over again in out factories.

In general I'd love to have a real profiling session with the game instead of just seeing "entities need 80ms". Anyone got some debug symbols to give Visual Studio? Or might there even be a way to have a peek at the source code? I'd love to dig around to find reasons for some non obvious quirks. Like why do roboports keep chunks active? I don't see any obvious reason, especially for those currently unused ones that provide bot range at my solar grid. Or how the game avoids iterating over my 18k laser towers to find targets (would just reversing it aka searching for turrets near moving enemies work well enough?). Or how well have they managed to optimize bots. I kinda hope the game doesn't iterate over all of those every tick just to update their position. In most cases you'd probably only need to update them once they run out of energy/arrive at their destination. At least as long as they satisfy a few conditions... stationary target, safe area, offscreen area. And yes, determining those is the challenging, interesting & fun part.

[Encrypted]

Hello,

I have an idea about the iteration over the electric network items. Because most of the time you have enough power the second loop can be skipped. So all machines have a default that they take 100% power in the first iteration you calculate the total power need. If you have enough you stop there. If you don't have enough you do the distribution iteration and change the power consumption. Most of the time you can thus skip a whole cycle

Cheers

[pleegwat]

Hello,

I have an idea about the iteration over the electric network items. Because most of the time you have enough power the second loop can be skipped. So all machines have a default that they take 100% power in the first iteration you calculate the total power need. If you have enough you stop there. If you don't have enough you do the distribution iteration and change the power consumption. Most of the time you can thus skip a whole cycle

Cheers

You could possibly even get rid of it completely by assigning power each tick on a round-robin basis. If you have 5 objects, and power for 3 of them, then the first tick the first three get power, the second tick the last 2 get power and the first gets power again, the third tick 2-4 get power, et cetera. Effectively, things still move 3/5ths of the time, but on a per-tick basis entities are either 'on' or 'off' and never in a partial operation state. A downside with my idea is that it could lead to jittery animations, but that could probably be fudged in the rendering pass (which affects far fewer entities).

[Aardwolf]

Nice optimizations!

Not a fan of the stone path personally. It looks very nice... for a fantasy RPG game.

Shouldn't things look grimy and scrappy in factorio? It looks odd with those industrial buildings on it too.

[dr_vm]

Optimizations are always welcome

As already mentioned, UPS-Efficient Robots would be very nice for Mega Bases but would unbalance the combat side of the game.

Assumed that it would be possible to have entity and smoke robots at the same time in the game, a new late game technology like "High flying Robots" may solve this ethical dilemma in a backward compatible way.
Also, since the altitude of the robot changes the player won't be surprised that the new type of robot flies out of combat or mining reach.

Thanks !

[TheTom]

I've been thinking about smoke. Is there any reason why smoke needs to be ticked at all?

Just store a static list of smoke start location (x,y in chunk) and tick number that it was placed there (which could be as low as 4 bytes per entity). All the logic can be in the render function: smoke age = current tick - placement tick; location is a pure function of age, start location, and the current wind vector. All the cleanup you have to do during the tick is advance the head of the circular buffer to invalidate the old smoke (should only be a single int per chunk).

I would dare saying you may even be able to do most of the calculations on the graphics card then

[Engimage]

Regarding the bots.
Personally I would prefer them all being untargetable/unminable. All the minability does to you:
- Lets you cheat charging
- Lets you pick up robots accidentally instead of other items
- Completely make your base uninteractible in dense bot builds
- You pick random items along with random bots clogging your inventory from time to time

I would prefer a tool to summon bots within a network to a selected roboport (just like requester chest but for active bots) so you could pick them up leaving roboports intact.

However I do not want bots to become invulnerable to biters... This is a very interesting and fair part of the game.

[TheTom]

While we are at it - are you now testing and optimizing also against AMD Zen? They sell like crazy and we would not mind having Factorio properly optimized for them.

Talking from a 16 core ThreadRipper 1950x which is amazing in it's reserves.

[Avezo]

What about allowing both bot network ways of working that would be selectable with all other map settings?

[DaemosDaen]

Posted by kovarex on 2017-09-21, all posts
I need to make a confession.
I'm addicted.
Addicted to optimization.

I could kiss you, and I'm fairly sure my wife would not mind.

It's something that programmers have lost sight in recent years.

[bobucles]

Where's my spiderbot?

Where's my spiderbot?

Where's my spiderbot?

Where's my spiderbot?

Where's my spiderbot?

Where's my spiderbot?

[impetus maximus]

Spidertron

[JCav]

I have no issues with logistics bots being invulnerable. I feel compelled to build my base perimeter convex at all times because of the annoyance of biters attacking robots that cannot smartly fly a safe path from one roboport to another.