Run from ramdisk

Anything that prevents you from playing the game properly. Do you have issues playing for the game, downloading it or successfully running it on your computer? Let us know here.
Camylarde
Burner Inserter
Burner Inserter
Posts: 11
Joined: Fri Mar 04, 2016 4:52 pm
Contact:

Run from ramdisk

Post by Camylarde »

Hi,

is there a way to move factorio to ramdisk?
I have:
1)Created a ramdisk h:/
2) Moved the steam part of the game to the root.
3) Created a sub-folder config-copy to have a separate location for roaming/factorio config data.
4) Applied the article How to move your data folder from this forum technical section faq.
5) Found the exe in the /bin folder

I still get error
---------------------------
Error
---------------------------
weakly_canonical: Incorrect function.

: "h:/config-copy/config\config.ini"
path1: h:/config-copy/config\config.ini
---------------------------
OK
---------------------------

despite the file confirm.ini being there in the path. Notable is also that if the file isn't there the new one is not created as the article says it should be...

Any suggestions?

posila
Factorio Staff
Factorio Staff
Posts: 5203
Joined: Thu Jun 11, 2015 1:35 pm
Contact:

Re: Run from ramdisk

Post by posila »

Camylarde wrote:
Tue Feb 11, 2020 7:05 pm
weakly_canonical: Incorrect function.
Hello, the error you are seeing originates from std::filesystem standard C++ library (which might or might not get it from underlying calls to WinAPI).
Can you please
1) post exact build version of Windows you are running (Win + R and type in winver, or just post Factorio log).
2) decribe steps you used to create the ramdisk
3) post your modified config.ini and command line parameters you use to launch Factorio?

User avatar
Optera
Smart Inserter
Smart Inserter
Posts: 2920
Joined: Sat Jun 11, 2016 6:41 am
Contact:

Re: Run from ramdisk

Post by Optera »

For such experiments I'd start by downloading the portable version directly from factorio.com instead of using the steam version.

User avatar
hhrhhr
Inserter
Inserter
Posts: 36
Joined: Mon Feb 18, 2019 11:02 pm
Contact:

Re: Run from ramdisk

Post by hhrhhr »

Factorio 0.18.26 and 0.18.26, unzipped to ramdisk (created with OSFMount):

Code: Select all

I:\Factorio_0.18.22>ver

Microsoft Windows [Version 10.0.18363.657]

I:\Factorio_0.18.22>bin\x64\factorio.exe -v

I:\Factorio_0.18.22>   0.000 Error Util.cpp:83: weakly_canonical:  .
: "I:\Factorio_0.18.22\bin\x64/../../config\config.ini"
path1: I:\Factorio_0.18.22\bin\x64/../../config\config.ini
   0.000 Error MessageDialog.cpp:140: Failed to widen string: bad conversion
   0.000 Goodbye
No logs appear after launch.

P.S.
If you select the "Physical Drive Emulation" (instead "Logical") option when creating a virtual disk, then no errors occur. This creates an almost "real" disk, which is visible in the Windows "Disk management".

Rseding91
Factorio Staff
Factorio Staff
Posts: 13372
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Run from ramdisk

Post by Rseding91 »

A little off topic: why do you want to run it from a ramdisk?
If you want to get ahold of me I'm almost always on Discord.

User avatar
hhrhhr
Inserter
Inserter
Posts: 36
Joined: Mon Feb 18, 2019 11:02 pm
Contact:

Re: Run from ramdisk

Post by hhrhhr »

In my case, the data directory contains almost 10 thousand files. Even after several launches and caching of files by the operating system, the launch of the game takes 10-12 seconds. And when using a RAM disk - only 2-3 seconds. When debugging mods / scripts, this is pretty significant.
Аnother example: the game.save_atlas() function works on a regular disk for 5-6 minutes (at maximum quality), on a RAM disk - no more than a couple of minutes.

Rseding91
Factorio Staff
Factorio Staff
Posts: 13372
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Run from ramdisk

Post by Rseding91 »

hhrhhr wrote:
Sun May 24, 2020 9:47 am
In my case, the data directory contains almost 10 thousand files. Even after several launches and caching of files by the operating system, the launch of the game takes 10-12 seconds. And when using a RAM disk - only 2-3 seconds. When debugging mods / scripts, this is pretty significant.
Аnother example: the game.save_atlas() function works on a regular disk for 5-6 minutes (at maximum quality), on a RAM disk - no more than a couple of minutes.
Can you give a log file after loading to the main menu with all those mods? I've never seen things run at that speed.

Also, do you have a spinning hard drive, solid-state, or M2 solid state?
If you want to get ahold of me I'm almost always on Discord.

User avatar
hhrhhr
Inserter
Inserter
Posts: 36
Joined: Mon Feb 18, 2019 11:02 pm
Contact:

Re: Run from ramdisk

Post by hhrhhr »

I meant a clean game without mods. At least at 0.18.26, the data directory contains exactly 9999 files and 1503 directories.
Yes, on powerful desktops with fast SSDs, the game’s launch time is not so exciting. But sometimes I have to run the game on an old HP laptop (with Radeon HD4250 video and 256 MB of video memory ;). Installing an SSD is useless, the controller is slow. But I can put 8-16 GB of RAM and use half to create a virtual disk.

Here is the log of the launch of a clean game from RAM-disk with such settings:

Code: Select all

max-texture-size = 8192
graphics-quality = normal
high-quality-animations = false
high-quality-shadows = false
high-quality-terrain = false
texture-compression-level = low-quality

Code: Select all

...
   3.922 Custom mipmaps uploaded.
   3.922 Verbose AtlasSystem.cpp:910: Atlas memory size: 245.54MB; 0.00MB (virtual)
   3.922 Verbose AtlasSystem.cpp:911: Size of sprites outside of atlas: 0.13MB
   3.984 Factorio initialised
...
factorio-current.log
(5.62 KiB) Downloaded 130 times

Guenni7
Fast Inserter
Fast Inserter
Posts: 147
Joined: Thu May 18, 2017 5:53 am
Contact:

Re: Run from ramdisk

Post by Guenni7 »

You might want to test the cache settings first, since I activated Atlas-cache and the other cache my game starts in 3 to 4 seconds (from a normal notebook harddrive).
You can access the Cache-settings with CTRL-ALT-LEFTCLICK on the settings-button.

User avatar
hhrhhr
Inserter
Inserter
Posts: 36
Joined: Mon Feb 18, 2019 11:02 pm
Contact:

Re: Run from ramdisk

Post by hhrhhr »

config:
cache-sprite-atlas-count=1
cache-sprite-atlas=true
compress-sprite-atlas-cache=true
graphics-quality=normal
high-quality-animations=false
high-quality-shadows=false
high-quality-terrain=false
video-memory-usage=low
texture-compression-level=low-quality
run from SSD, cold start:
...
14.977 Data stage cache loaded in: 14.573237 seconds.
...
16.319 Loading sounds...
...
26.620 Initial atlas bitmap size is 8192
...
27.540 Atlases loaded from disk cache.
...
46.164 Parallel Sprite Loader initialized (threads: 7)
48.464 Sprites loaded
...
48.629 Custom mipmaps uploaded.
48.821 Generated mipmaps for virtual atlas of size 16384x12288
49.342 Factorio initialised
not really helped...

Rseding91
Factorio Staff
Factorio Staff
Posts: 13372
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Run from ramdisk

Post by Rseding91 »

hhrhhr wrote:
Thu May 28, 2020 9:29 am
run from SSD, cold start:
not really helped...
That's basically useless. Cold-start anything isn't how you could ever launch from a ramdisk since you have to copy the thing into the ramdisk first anyway - then you could launch it.
If you want to get ahold of me I'm almost always on Discord.

posila
Factorio Staff
Factorio Staff
Posts: 5203
Joined: Thu Jun 11, 2015 1:35 pm
Contact:

Re: Run from ramdisk

Post by posila »

cache-sprite-atlas=true with video-memory-usage=low won't have much of an effect since most sprites won't be included in the sprite atlases.

User avatar
hhrhhr
Inserter
Inserter
Posts: 36
Joined: Mon Feb 18, 2019 11:02 pm
Contact:

Re: Run from ramdisk

Post by hhrhhr »

Rseding91 wrote:
Thu May 28, 2020 4:01 pm
Cold-start anything isn't how you could ever launch from a ramdisk since you have to copy the thing into the ramdisk first anyway - then you could launch it.
I need to copy the game to ramdisk only once, and a cold start, at least in Windows, does not occur only if the game is launched literally immediately after its release. After a few minutes, the file cache of the system is cleared or replaced and the game starts again by reading files from the disk, and not from the memory.

However, if the game cannot start from certain types of media, then I would like to know in advance which ones. Or this is "not-a-bug"?

Rseding91
Factorio Staff
Factorio Staff
Posts: 13372
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Run from ramdisk

Post by Rseding91 »

hhrhhr wrote:
Thu May 28, 2020 5:22 pm
However, if the game cannot start from certain types of media, then I would like to know in advance which ones. Or this is "not-a-bug"?
We don't impose any limits on what storage type files can be on. If you're getting errors they're coming from the C++ filesystem library and are outside of our control (most likely).
If you want to get ahold of me I'm almost always on Discord.

User avatar
hhrhhr
Inserter
Inserter
Posts: 36
Joined: Mon Feb 18, 2019 11:02 pm
Contact:

Re: Run from ramdisk

Post by hhrhhr »

Could you provide an example of your code that opens a file using the standard C++ library?
No matter how I tried, but I could not find an example that could not access the file on any types of virtual disks, including file systems with utf8 and utf16 encodings.

Rseding91
Factorio Staff
Factorio Staff
Posts: 13372
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Run from ramdisk

Post by Rseding91 »

hhrhhr wrote:
Thu May 28, 2020 10:18 pm
Could you provide an example of your code that opens a file using the standard C++ library?
No matter how I tried, but I could not find an example that could not access the file on any types of virtual disks, including file systems with utf8 and utf16 encodings.
The path is created, and then normalized by calling https://en.cppreference.com/w/cpp/filesystem/canonical - specifically 'weakly_canonical'. It's failing and reports the error you're getting.
If you want to get ahold of me I'm almost always on Discord.

User avatar
hhrhhr
Inserter
Inserter
Posts: 36
Joined: Mon Feb 18, 2019 11:02 pm
Contact:

Re: Run from ramdisk

Post by hhrhhr »

With this example, everything is fine:

Code: Select all

#include <iostream>
#include <filesystem>
namespace fs = std::filesystem;
int main()
{
    fs::path p = fs::path("..") / ".." / "config-path.cfg";
    std::cout << "Current path is " << fs::current_path() << '\n'
              << "Canonical path for " << p << " is " << fs::canonical(p) << '\n';
}
/*
g++ -std=c++17 -o test.exe test.cpp 
g++ -v
...
Target: x86_64-w64-mingw32
Thread model: posix
gcc version 9.3.0 (MinGW-W64 x86_64-posix-seh)
*/
H: - my ramdrive (created with "Logical emulation")

Code: Select all

h:\Factorio\bin\x64>test.exe
Current path is "h:\\Factorio\\bin\\x64"
Canonical path for "..\\..\\config-path.cfg" is "h:\\Factorio\\config-path.cfg"

h:\Factorio\bin\x64>factorio.exe

h:\Factorio\bin\x64>   0.000 Error Util.cpp:83: weakly_canonical:  .
: "h:\Factorio\bin\x64/../../config\config.ini"
path1: h:\Factorio\bin\x64/../../config\config.ini
   0.000 Error MessageDialog.cpp:140: Failed to widen string: bad conversion
   0.000 Goodbye

Rseding91
Factorio Staff
Factorio Staff
Posts: 13372
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Run from ramdisk

Post by Rseding91 »

Rseding91 wrote:
Thu May 28, 2020 11:16 pm
... specifically 'weakly_canonical'.
If you want to get ahold of me I'm almost always on Discord.

User avatar
hhrhhr
Inserter
Inserter
Posts: 36
Joined: Mon Feb 18, 2019 11:02 pm
Contact:

Re: Run from ramdisk

Post by hhrhhr »

absolutely no difference:

Code: Select all

#include <iostream>
#include <filesystem>
namespace fs = std::filesystem;
int main()
{
    fs::path p = fs::path("..") / ".." / "config-path.cfg";
    std::cout << "Current path is " << fs::current_path() << '\n'
              << "Canonical path for " << p << " is " << fs::weakly_canonical(p) << '\n';

    std::cout << "file size: " << fs::file_size(fs::weakly_canonical(p)) << " bytes\n";

    p = fs::current_path() / "../../config/config.ini";
    std::cout << "Canonical path for " << p << " is " << fs::weakly_canonical(p) << '\n';

    std::cout << "file size: " << fs::file_size(fs::weakly_canonical(p)) << " bytes\n";
}
reslit:

Code: Select all

h:\Factorio\bin\x64>g++ -std=c++17 -o test.exe -static -s test.c && test.exe
Current path is "h:\\Factorio\\bin\\x64"
Canonical path for "..\\..\\config-path.cfg" is "h:\\Factorio\\config-path.cfg"
file size: 998 bytes
Canonical path for "h:\\Factorio\\bin\\x64\\../../config/config.ini" is "h:\\Factorio\\config\\config.ini"
file size: 26353 bytes

h:\Factorio\bin\x64>factorio.exe -v

h:\Factorio\bin\x64>   0.000 Error Util.cpp:83: weakly_canonical:  .
: "h:\Factorio\bin\x64/../../config\config.ini"
path1: h:\Factorio\bin\x64/../../config\config.ini
   0.000 Error MessageDialog.cpp:140: Failed to widen string: bad conversion
   0.000 Goodbye

User avatar
willis936
Manual Inserter
Manual Inserter
Posts: 3
Joined: Tue Nov 24, 2020 4:14 pm
Contact:

Re: Run from ramdisk

Post by willis936 »

I'm having the same issue. I originally tried just symlinking the saves directory but found posts saying that it was unsupported and a mess.

Q: imdisk 2.0.10 NTFS drive
Factorio 1.0 zip version.
Unzipped Factorio, launched once, copied directory to RAM drive, then attempted to launch from RAM drive.

imdisk creation:

Code: Select all

C:\Windows\System32\imdisk.exe -a -m Q: -s 4G -p "/fs:ntfs /q /y"
Factorio launch:

Code: Select all

C:\WINDOWS\system32>ver

Microsoft Windows [Version 10.0.18363.1198

C:\WINDOWS\system32>factorio -v

C:\WINDOWS\system32>   0.000 Error Util.cpp:83: weakly_canonical: Incorrect function.
: "Q:\Factorio\bin\x64/../../config\config.ini"
path1: Q:\Factorio\bin\x64/../../config\config.ini
   0.000 Goodbye
Why I want to run from a RAM disk:
I'm hosting a dedicated server so I can play with a friend across the country. I recently discovered that manual artillery shots can go much further than autofiring. I sounded a huge area and now the save file is nearly 100 MB. Autosaving takes several seconds and downloading takes a minute. I've hosted game servers (modded minecraft) on RAM drives in the past with great success. Scheduled tasks sync the files to disk at a regular interval.


Edit: reading above, it looks like I need software that emulates a physical drive. There are no FOSS solutions for this on Windows afaik. OSFMount is free and does work, but it does not have an automation interface (GUI only) and does not support persistent RAM drives. I have a linux VM, but I don't think Factorio performance would take kindly to the memory latency that introduces.

I can launch the server with a save file that is on a logical volume RAM drive, however the autosaving is still done to the game launch directory. The download (over LAN) and autosave is faster, even with the autosaves going to NVMe SSD. I believe this is because the disk only has to write and not read/write.

I tried modifying the [path]write-data option in the game config to be on a logical volume RAM drive, but this gives the same error (Util.cpp:83: weakly_canonical: Incorrect function). Is there another way to manually specify the autosave directory?

I do not think that supporting symlinks or logical volumes is a niche ask. What I’m attempting seems like standard best practice for people hosting game servers (even if they don't talk about it much). I suppose the non-standard thing I'm doing is hosting from Windows. This appears to be out of your hands (likely an OS API bug). If anyone has experience with this, let me know.
Last edited by willis936 on Tue Nov 24, 2020 4:37 pm, edited 1 time in total.

Post Reply

Return to “Technical Help”