Friday Facts #201 - 0.15 Stable, but not really

Regular reports on Factorio development.

Re: Friday Facts #201 - 0.15 Stable, but not really

Postby ratchetfreak » Wed Aug 16, 2017 12:45 pm

abordoli wrote:I'm auto-saving every 30 minutes and, tbh, I don't even know why I'd save that frequently. I DO gather that the longer "in between" saves, the longer the save operation will be. I'm quite betwixt as I'd rather saves not bother me at all, but then again..."a stich in time...saves nine" keeps nagging at me.
~B


Each auto save is independent from the previous save. It's a full save every time without any optimizations based on the previous save
ratchetfreak
Filter Inserter
Filter Inserter
 
Posts: 840
Joined: Sat May 23, 2015 12:10 pm

Re: Friday Facts #201 - 0.15 Stable, but not really

Postby bobucles » Wed Aug 16, 2017 2:01 pm

But saving taking 20-60 seconds is not uncommon.

No, that is exactly what uncommon means. I've seen plenty of beefy UPS killing bases in the middle of nowhere and they take no more than 10 seconds to save. Reaching a full second of save time is an accomplishment in of itself. If saving the map is approaching up to a minute that means:

- You have a literal potato toaster or
- A mod is grossly misbehaving

Neither of those problems are the dev's fault. In fact the first problem can't be solved with live action saving because, as you might expect, a toaster can barely run the game without distractions as is. Doubling the system requirements won't make a potato faster. It'll make the potato crash and burn. The second problem can't be solved by devs because they didn't make it. You'll have to work with the mod author to figure out just exactly what they're goofing up that causes save times to skyrocket.

I think I mentioned I had saves take 10+ minutes.

Congratulations on reaching levels of slow no one ever thought possible. Tell us your secrets.
bobucles
Filter Inserter
Filter Inserter
 
Posts: 723
Joined: Wed Jun 10, 2015 10:37 pm

Re: Friday Facts #201 - 0.15 Stable, but not really

Postby posila » Wed Aug 16, 2017 5:48 pm

NotABiter wrote:Oh, and fork() isn't available on Windows. My scheme works under all OSes. Why is anyone still talking about fork?

(The devs aren't going to do any of this so the discussion is not useful for that. But I was once again "triggered" by people posting *false* information.)

I like your proposal, I think we should definitely do it if Factorio 2 ever happens, but I don't see it happening in the current one.
posila
Factorio Staff
Factorio Staff
 
Posts: 2065
Joined: Thu Jun 11, 2015 1:35 pm

Re: Friday Facts #201 - 0.15 Stable, but not really

Postby mrvn » Thu Aug 17, 2017 8:43 am

bobucles wrote:
But saving taking 20-60 seconds is not uncommon.

No, that is exactly what uncommon means. I've seen plenty of beefy UPS killing bases in the middle of nowhere and they take no more than 10 seconds to save. Reaching a full second of save time is an accomplishment in of itself. If saving the map is approaching up to a minute that means:

- You have a literal potato toaster or
- A mod is grossly misbehaving

Neither of those problems are the dev's fault. In fact the first problem can't be solved with live action saving because, as you might expect, a toaster can barely run the game without distractions as is. Doubling the system requirements won't make a potato faster. It'll make the potato crash and burn. The second problem can't be solved by devs because they didn't make it. You'll have to work with the mod author to figure out just exactly what they're goofing up that causes save times to skyrocket.

I think I mentioned I had saves take 10+ minutes.

Congratulations on reaching levels of slow no one ever thought possible. Tell us your secrets.


Install AAI mods, mark a zone with 800k tiles. Enjoy.

I'm assuming the mod isn't misbehaving but serialization has to call some lua code for every one of those 800k marked tiles. That's bound to be slow no mastter what.
mrvn
Smart Inserter
Smart Inserter
 
Posts: 1334
Joined: Mon Sep 05, 2016 9:10 am

Re: Friday Facts #201 - 0.15 Stable, but not really

Postby Dave64738 » Sat Aug 19, 2017 3:08 pm

Regarding optimisations, have you considered using database techniques to optimise out that huge annoyance that is Autosave? A large production database cannot afford to lock up for several minutes every few minutes to write everything to disk; if you use an approach like the redo log stuff in Oracle you might be able to remove autosave alrogether.

Of course, there would be stuff that would need working out; there is no point restoring a game after being killed to the point immediately before you die, but you could perhaps restore to a player-specified time before that (maybe even visually rewind the gameplay so that the player can determine where to restart).

A redo log equivalent might also make some hard-to-find bugs easier to reproduce, if you are able to recover the last save plus the redo, you would get much closer to the bug before having to guess what happened next. Currently the only checkpoint is the last auto or manual save.
Dave64738
Inserter
Inserter
 
Posts: 22
Joined: Wed Apr 13, 2016 10:37 pm

Re: Friday Facts #201 - 0.15 Stable, but not really

Postby pleegwat » Sun Aug 20, 2017 11:32 am

Dave64738 wrote:Regarding optimisations, have you considered using database techniques to optimise out that huge annoyance that is Autosave? A large production database cannot afford to lock up for several minutes every few minutes to write everything to disk; if you use an approach like the redo log stuff in Oracle you might be able to remove autosave alrogether.

Of course, there would be stuff that would need working out; there is no point restoring a game after being killed to the point immediately before you die, but you could perhaps restore to a player-specified time before that (maybe even visually rewind the gameplay so that the player can determine where to restart).

A redo log equivalent might also make some hard-to-find bugs easier to reproduce, if you are able to recover the last save plus the redo, you would get much closer to the bug before having to guess what happened next. Currently the only checkpoint is the last auto or manual save.


While it's a nice idea, I fear that occasional full snapshots are still required, as replaying the redo log is likely to take just as much CPU as playing the game initially did. So if your megabase needs to load a point half an hour after the last snapshot, it will likely take half an hour to load the save.
pleegwat
Inserter
Inserter
 
Posts: 31
Joined: Fri May 19, 2017 7:31 pm

Re: Friday Facts #201 - 0.15 Stable, but not really

Postby mrvn » Mon Aug 21, 2017 9:41 am

pleegwat wrote:
Dave64738 wrote:Regarding optimisations, have you considered using database techniques to optimise out that huge annoyance that is Autosave? A large production database cannot afford to lock up for several minutes every few minutes to write everything to disk; if you use an approach like the redo log stuff in Oracle you might be able to remove autosave alrogether.

Of course, there would be stuff that would need working out; there is no point restoring a game after being killed to the point immediately before you die, but you could perhaps restore to a player-specified time before that (maybe even visually rewind the gameplay so that the player can determine where to restart).

A redo log equivalent might also make some hard-to-find bugs easier to reproduce, if you are able to recover the last save plus the redo, you would get much closer to the bug before having to guess what happened next. Currently the only checkpoint is the last auto or manual save.


While it's a nice idea, I fear that occasional full snapshots are still required, as replaying the redo log is likely to take just as much CPU as playing the game initially did. So if your megabase needs to load a point half an hour after the last snapshot, it will likely take half an hour to load the save.


But at least you don't have to do all the clicks again. And he did say "last save plus the redo".

I would really like it if one could load a save from say 10 minutes ago plus the redo and let it run. An then at some point, say 5 minutes in you hit stop and do something different and play again.
mrvn
Smart Inserter
Smart Inserter
 
Posts: 1334
Joined: Mon Sep 05, 2016 9:10 am

Re: Friday Facts #201 - 0.15 Stable, but not really

Postby jerocom » Mon Aug 21, 2017 2:19 pm

1.83 seconds to generate new map... and here am I, with my perfect PC, generating new map 3 minutes :D
jerocom
Burner Inserter
Burner Inserter
 
Posts: 7
Joined: Sun Jun 11, 2017 7:24 pm

Re: Friday Facts #201 - 0.15 Stable, but not really

Postby Tomik » Mon Aug 21, 2017 4:04 pm

jerocom wrote:1.83 seconds to generate new map... and here am I, with my perfect PC, generating new map 3 minutes :D

Hmmm. What CPU do you have? Maybe it is the problem that the Devs are optimizing the game on one set of CPU's and Software Drivers and once the update comes there will be many people complaining: "BUT IT DOESN'T LIKE WITH THE DEVS!"
User avatar
Tomik
Fast Inserter
Fast Inserter
 
Posts: 110
Joined: Mon Apr 18, 2016 3:30 pm
Location: Bohemia

Previous

Return to News

Who is online

Users browsing this forum: hitzu and 7 guests