Friday Facts #415 - Fix, Improve, Optimize

Regular reports on Factorio development.
FuryoftheStars
Smart Inserter
Smart Inserter
Posts: 2767
Joined: Tue Apr 25, 2017 2:01 pm
Contact:

Re: Friday Facts #415 - Fix, Improve, Optimize

Post by FuryoftheStars »

Mithaldu wrote:
Tue Jun 18, 2024 8:04 am
FuryoftheStars wrote:
Tue Jun 18, 2024 2:29 am
If a function becomes outdated and they'd want to remove normally
my complaint isn't even remotely about that though
You complained about bug fixes going into 2.0 instead of 1.1 because there are mod breaking changes in 2.0 that mod authors will slowly, if ever, port over for. The devs have been releasing a number of bugfixes for 1.1, still. But anything that is dependent on a change in 2.0 that isn't easy to backport or is something exclusively for 2.0, they are fixing and leaving in 2.0. As such, the only way to resolve this, really, is backwards compatibility, which leads exactly to what I was saying, as far as I see. If there's something else that you're seeing, then you may need to be more explicit.
My Mods: Classic Factorio Basic Oil Processing | Sulfur Production from Oils | Wood to Oil Processing | Infinite Resources - Normal Yield | Tree Saplings (Redux) | Alien Biomes Tweaked | Restrictions on Artificial Tiles | New Gear Girl & HR Graphics

User avatar
Mithaldu
Long Handed Inserter
Long Handed Inserter
Posts: 85
Joined: Sun Mar 13, 2016 5:39 pm
Contact:

Re: Friday Facts #415 - Fix, Improve, Optimize

Post by Mithaldu »

FuryoftheStars wrote:
Tue Jun 18, 2024 10:54 am
you may need to be more explicit
i was very explicit in my first post:
Mithaldu wrote:
Fri Jun 14, 2024 12:09 pm
it's really frustrating to see straight-up bugfixes that sound like they would easily apply to factorio v1, being limited to v2

SirSmuggler
Long Handed Inserter
Long Handed Inserter
Posts: 66
Joined: Tue May 24, 2016 1:55 pm
Contact:

Re: Friday Facts #415 - Fix, Improve, Optimize

Post by SirSmuggler »

Mithaldu wrote:
Tue Jun 18, 2024 11:52 am
FuryoftheStars wrote:
Tue Jun 18, 2024 10:54 am
you may need to be more explicit
i was very explicit in my first post:
Mithaldu wrote:
Fri Jun 14, 2024 12:09 pm
it's really frustrating to see straight-up bugfixes that sound like they would easily apply to factorio v1, being limited to v2
Since they have been releasing some bugfixes for 1.1 during the last year, I can only assume that they have good reason to not do all the bug fixes reported in FFFs. So what "sounds" like it should be easy to apply to 1.1 proboly isn't actually that easy... Would be my guess any way.

Koub
Global Moderator
Global Moderator
Posts: 7452
Joined: Fri May 30, 2014 8:54 am
Contact:

Re: Friday Facts #415 - Fix, Improve, Optimize

Post by Koub »

[Koub] Moderator-me here : please, stay on topic. Keep focused on the FFF, and stop targeting each other specifically in your discussion (or switch to private messages). Anything off topic past this post will be moderated eventually.
Koub - Please consider English is not my native language.

User avatar
Mithaldu
Long Handed Inserter
Long Handed Inserter
Posts: 85
Joined: Sun Mar 13, 2016 5:39 pm
Contact:

Re: Friday Facts #415 - Fix, Improve, Optimize

Post by Mithaldu »

SirSmuggler wrote:
Tue Jun 18, 2024 12:06 pm
Since they have been releasing some bugfixes for 1.1 during the last year, I can only assume that they have good reason to not do all the bug fixes reported in FFFs. So what "sounds" like it should be easy to apply to 1.1 proboly isn't actually that easy... Would be my guess any way.
if it isn't i'd love to see them talk about it :)

but as others have mentions, this particular fff was a little thin
Koub wrote:
Tue Jun 18, 2024 12:53 pm
thanks :)

ryanalpasta
Burner Inserter
Burner Inserter
Posts: 12
Joined: Sun Sep 03, 2023 8:06 pm
Contact:

Re: Friday Facts #415 - Fix, Improve, Optimize

Post by ryanalpasta »

As a professional physicist who codes a Lot, I hate to say it but I wish my group could hire you to fix our terrible code. You seem to have a gift for getting to the core of a complex problem and modifying it enough to make it better across the board. So hyped about 2.0 :)

Faark
Burner Inserter
Burner Inserter
Posts: 18
Joined: Tue Apr 25, 2017 8:33 pm
Contact:

Re: Friday Facts #415 - Fix, Improve, Optimize

Post by Faark »

That, combined with sorting the resulting rectangles meant I could do a simple binary search to check if a given task was within the network area.
Huh i always expected it to be the other way around... build tasks being associated to logistic networks, since their networks mostly don't change. Then selecting one could be, per network, a simple do-we-have-robots check + task queues per material. But i guess resolving intersecting networks in a deterministic manner isn't that easy either.

User avatar
mrudat
Fast Inserter
Fast Inserter
Posts: 248
Joined: Fri Feb 16, 2018 5:21 am
Contact:

Re: Friday Facts #415 - Fix, Improve, Optimize

Post by mrudat »

numzero wrote:
Mon Jun 17, 2024 1:17 pm
mrudat wrote:
Fri Jun 14, 2024 6:52 pm
Zavian wrote:
Fri Jun 14, 2024 6:14 pm
mrudat wrote:
Fri Jun 14, 2024 1:27 pm
For huge saves, it would be nifty if the delta from the last save could be sent instead of having to download the entire save, given that we know the last time any given player was last connected; perhaps a utility that builds and maintains a set of delta files between saves allowing any player to download a set of deltas to bring their local save up-to-date hopefully using only a fraction of the required bandwidth.
Differential or delta saves work well where 90+% of the data is static data that does not change between saves, but that isn't the case with Factorio.
From what I've read about saves, the majority of a save is (relatively) static. Decorative entities, trees, tiles and relatively static entities (power poles, idle turrets, walls) outnumber active entities (assemblers, inserters, pipes, ...); removing decoratives can shrink a save significantly, for example.

That said, I haven't checked what the actual % change between one save and the next is (yet), and it's entirely possible that, as you say, it's just not worth the effort.
I just did a quick check, on some old saves I found (29.0 and 30.6 MiB zips, a few hours apart). xdelta on those (fully uncompressed, 87.2 and 91.3 MiBs respectively) yielded a 3.3 MiB diff (not further compressible with anything I have on hand).
As far as I know xdelta/rsync both produce output designed to be sent over the network directly, i.e. already compressed.

If we only had to transmit ~10% of the data for a mp save, in theory (presuming applying the delta isn't horribly slow) that should mean joining a server goes 10x faster.

A good question is, is it 10%/hr, or does 10% of the save change between _every_ save? If the former, storing a series of delta files to apply in sequence would work, if the latter, using librsync to transfer the save would be best (as it will work without a copy of the client's old save on the server).

It does sound like, in either case, it might be well worth the effort to use delta compression in some way to make transferring mp saves faster.
numzero wrote:
Mon Jun 17, 2024 1:17 pm
That’s not the full story though. First, fully uncompressing a save isn’t exactly trivial as instead of using zip compression, most files are compressed with zlib and then are put into the zip. That’s weird as that’s what zip would do on its own.

Second, the compression is lower than it could be. Recompressing with gzip -9 resulted in 26.1 and 27.5 MiB archives. And with lzip -9 instead, I got 21.0 and 22.1 MiB respectively which is ~30% less than the original.

Third, the parts are compressed individually. That’s not very space-efficient though it my case, the gains weren’t large, just 1 MiB with lzip and marginal 0.1 MiB with gzip.
The (potential) double-compression and that there's room to squeeze more bytes out of the save sounds like a bug (or missing feature) to me. Edit: Perhaps there's room to change the save file format in 2.0?

Edit: Parts being compressed individually is part of the zip spec; .tar.gz archives multiple files, and then compresses the archive; if the file is damaged, anything after the damage is lost, and you need to uncompress the entire file before you can read it. Zip files compress then archive, if there's any corruption, often only the one file is lost, but it doesn't compress as well, on the other hand you can read individual files without uncompressing the entire archive.

GTexperience
Burner Inserter
Burner Inserter
Posts: 5
Joined: Mon Oct 09, 2023 11:56 pm
Contact:

Re: Friday Facts #415 - Fix, Improve, Optimize

Post by GTexperience »

I don't understand, why not make the updates per game tick dynamic? can u not incrementally increase the updates, till performance reaches a certain point? than stop there? This would allow faster pc's to run more updates. I use the mod and run about 10-20 updates. Which works totally fine for me.

bobucles
Smart Inserter
Smart Inserter
Posts: 1707
Joined: Wed Jun 10, 2015 10:37 pm
Contact:

Re: Friday Facts #415 - Fix, Improve, Optimize

Post by bobucles »

GTexperience wrote:
Wed Jul 03, 2024 6:56 pm
I don't understand, why not make the updates per game tick dynamic?
Ever played an unrealengine game with physics? The game skips a frame, and everything starts flying? Dynamic FPS is like that. It's also useless in this scenario, because everyone has to run the same calculations to avoid desync. Dynamic frame rate is the last thing in the entire universe capable of that.

User avatar
picklock
Fast Inserter
Fast Inserter
Posts: 159
Joined: Sat Nov 09, 2019 6:49 am
Contact:

Re: Friday Facts #415 - Fix, Improve, Optimize

Post by picklock »

It's great to see how the community is being addressed here and how issues are being tackled and problems solved, as well as how parts of the game that have been implemented for years are being optimized.
My Mods: Picklocks Fusion Power | Picklocks Inserter | Picklocks Lithium Polymer Accumulator | Picklocks rocket silo stats | Picklocks Set Inventory Filters

User avatar
Neutronium
Inserter
Inserter
Posts: 36
Joined: Thu Oct 19, 2023 4:16 pm
Contact:

Re: Friday Facts #415 - Fix, Improve, Optimize

Post by Neutronium »

For the new server option "auto-pause when a player is joining," is this only for dedicated servers, or will this functionality (hopefully) exist in in-game hosting as well? Hopefully its the latter.

Post Reply

Return to “News”