Friday Facts #415 - Fix, Improve, Optimize

[FuryoftheStars]

If a function becomes outdated and they'd want to remove normally

my complaint isn't even remotely about that though

You complained about bug fixes going into 2.0 instead of 1.1 because there are mod breaking changes in 2.0 that mod authors will slowly, if ever, port over for. The devs have been releasing a number of bugfixes for 1.1, still. But anything that is dependent on a change in 2.0 that isn't easy to backport or is something exclusively for 2.0, they are fixing and leaving in 2.0. As such, the only way to resolve this, really, is backwards compatibility, which leads exactly to what I was saying, as far as I see. If there's something else that you're seeing, then you may need to be more explicit.

[Mithaldu]

you may need to be more explicit

i was very explicit in my first post:

it's really frustrating to see straight-up bugfixes that sound like they would easily apply to factorio v1, being limited to v2

[SirSmuggler]

you may need to be more explicit

i was very explicit in my first post:

it's really frustrating to see straight-up bugfixes that sound like they would easily apply to factorio v1, being limited to v2

Since they have been releasing some bugfixes for 1.1 during the last year, I can only assume that they have good reason to not do all the bug fixes reported in FFFs. So what "sounds" like it should be easy to apply to 1.1 proboly isn't actually that easy... Would be my guess any way.

[Koub]

[Koub] Moderator-me here : please, stay on topic. Keep focused on the FFF, and stop targeting each other specifically in your discussion (or switch to private messages). Anything off topic past this post will be moderated eventually.

[Mithaldu]

Since they have been releasing some bugfixes for 1.1 during the last year, I can only assume that they have good reason to not do all the bug fixes reported in FFFs. So what "sounds" like it should be easy to apply to 1.1 proboly isn't actually that easy... Would be my guess any way.

if it isn't i'd love to see them talk about it

but as others have mentions, this particular fff was a little thin

thanks

[ryanalpasta]

As a professional physicist who codes a Lot, I hate to say it but I wish my group could hire you to fix our terrible code. You seem to have a gift for getting to the core of a complex problem and modifying it enough to make it better across the board. So hyped about 2.0

[Faark]

That, combined with sorting the resulting rectangles meant I could do a simple binary search to check if a given task was within the network area.

Huh i always expected it to be the other way around... build tasks being associated to logistic networks, since their networks mostly don't change. Then selecting one could be, per network, a simple do-we-have-robots check + task queues per material. But i guess resolving intersecting networks in a deterministic manner isn't that easy either.

[mrudat]

For huge saves, it would be nifty if the delta from the last save could be sent instead of having to download the entire save, given that we know the last time any given player was last connected; perhaps a utility that builds and maintains a set of delta files between saves allowing any player to download a set of deltas to bring their local save up-to-date hopefully using only a fraction of the required bandwidth.

Differential or delta saves work well where 90+% of the data is static data that does not change between saves, but that isn't the case with Factorio.

From what I've read about saves, the majority of a save is (relatively) static. Decorative entities, trees, tiles and relatively static entities (power poles, idle turrets, walls) outnumber active entities (assemblers, inserters, pipes, ...); removing decoratives can shrink a save significantly, for example.

That said, I haven't checked what the actual % change between one save and the next is (yet), and it's entirely possible that, as you say, it's just not worth the effort.

I just did a quick check, on some old saves I found (29.0 and 30.6 MiB zips, a few hours apart). xdelta on those (fully uncompressed, 87.2 and 91.3 MiBs respectively) yielded a 3.3 MiB diff (not further compressible with anything I have on hand).

As far as I know xdelta/rsync both produce output designed to be sent over the network directly, i.e. already compressed.

If we only had to transmit ~10% of the data for a mp save, in theory (presuming applying the delta isn't horribly slow) that should mean joining a server goes 10x faster.

A good question is, is it 10%/hr, or does 10% of the save change between _every_ save? If the former, storing a series of delta files to apply in sequence would work, if the latter, using librsync to transfer the save would be best (as it will work without a copy of the client's old save on the server).

It does sound like, in either case, it might be well worth the effort to use delta compression in some way to make transferring mp saves faster.

That’s not the full story though. First, fully uncompressing a save isn’t exactly trivial as instead of using zip compression, most files are compressed with zlib and then are put into the zip. That’s weird as that’s what zip would do on its own.

Second, the compression is lower than it could be. Recompressing with gzip -9 resulted in 26.1 and 27.5 MiB archives. And with lzip -9 instead, I got 21.0 and 22.1 MiB respectively which is ~30% less than the original.

Third, the parts are compressed individually. That’s not very space-efficient though it my case, the gains weren’t large, just 1 MiB with lzip and marginal 0.1 MiB with gzip.

The (potential) double-compression and that there's room to squeeze more bytes out of the save sounds like a bug (or missing feature) to me. Edit: Perhaps there's room to change the save file format in 2.0?

Edit: Parts being compressed individually is part of the zip spec; .tar.gz archives multiple files, and then compresses the archive; if the file is damaged, anything after the damage is lost, and you need to uncompress the entire file before you can read it. Zip files compress then archive, if there's any corruption, often only the one file is lost, but it doesn't compress as well, on the other hand you can read individual files without uncompressing the entire archive.

[GTexperience]

I don't understand, why not make the updates per game tick dynamic? can u not incrementally increase the updates, till performance reaches a certain point? than stop there? This would allow faster pc's to run more updates. I use the mod and run about 10-20 updates. Which works totally fine for me.

[bobucles]

I don't understand, why not make the updates per game tick dynamic?

Ever played an unrealengine game with physics? The game skips a frame, and everything starts flying? Dynamic FPS is like that. It's also useless in this scenario, because everyone has to run the same calculations to avoid desync. Dynamic frame rate is the last thing in the entire universe capable of that.

[picklock]

It's great to see how the community is being addressed here and how issues are being tackled and problems solved, as well as how parts of the game that have been implemented for years are being optimized.

[Neutronium]

For the new server option "auto-pause when a player is joining," is this only for dedicated servers, or will this functionality (hopefully) exist in in-game hosting as well? Hopefully its the latter.