Load modules from directories without version

Post your ideas and suggestions how to improve the game.
mrvn
Smart Inserter
Smart Inserter
Posts: 3122
Joined: Mon Sep 05, 2016 9:10 am
Contact:

Load modules from directories without version

Post by mrvn » Mon Dec 10, 2018 10:19 am

TL;DR
Load modules from directories without version

What ?
Factorio requires the the directory name for each mod include the version of the mod. Factorio should accept mods in directories without version.
Why ?
I keep mod sources in git in their unpacked form. Factorio requires the the directory name for each mod include the version of the mod. That means on every change the directory has to be renamed and all the content has to be renamed in git. The change in name also means that patches, rebase or merging has to guess at the file rename and frequently fails. Overall just a big pain in the but.

Requiring the version to appear in two places has no benefit and only invites the two to disagree.

Rseding91
Factorio Staff
Factorio Staff
Posts: 9029
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Load modules from directories without version

Post by Rseding91 » Tue Dec 11, 2018 6:51 pm

mrvn wrote:
Mon Dec 10, 2018 10:19 am
Requiring the version to appear in two places has no benefit and only invites the two to disagree.
It has an amazing benefit: seeing the folder of mods tells you what version the mod is. It also can't ever disagree because the game won't load if they disagree.

The solution is simple: part of your release-build process should be taking the file(s) and copying them into a correctly named folder.
If you want to get ahold of me I'm almost always on Discord.

User avatar
bobingabout
Smart Inserter
Smart Inserter
Posts: 6593
Joined: Fri May 09, 2014 1:01 pm
Contact:

Re: Load modules from directories without version

Post by bobingabout » Wed Dec 12, 2018 9:06 am

I still think the fact that it is mandatory for the folder to have the version number in it's name is a huge pain in the arse.

zip files, as downloaded from the mod portal, I can understand. but when you're working on a mod, in the directory of the game, constantly having to change the folder name is annoying (forces me to reload all my open files in the editor every time I change it)
Creator of Bob's mods. Expanding your gameplay since version 0.9.8.

pleegwat
Fast Inserter
Fast Inserter
Posts: 152
Joined: Fri May 19, 2017 7:31 pm
Contact:

Re: Load modules from directories without version

Post by pleegwat » Wed Dec 12, 2018 5:38 pm

As rseding suggests it's easily worked around by not having the suffix in the repository, always editing in your working copy, and copying to a version-marked directory in the 'install' step.

However I suspect since factorio modding is all lua-based, most mods don't use a build/install step in dev environments, and check out directly into the factorio mods folder.

On a linux/mac platform you could set a symlilnk instead (which you'd only have to recreate when the version number changes). But I don't think that works on windows.

Since factorio has to restart for code changes anyway, you could launch factorio with a batch script which also copies the files into the correct directory?

mrvn
Smart Inserter
Smart Inserter
Posts: 3122
Joined: Mon Sep 05, 2016 9:10 am
Contact:

Re: Load modules from directories without version

Post by mrvn » Thu Dec 13, 2018 1:34 pm

As bobingabout says it's anoying for the editor. I mentioned git.

Having a working copy and a play copy is a possibility as are (I hope, never tried) are symlinks. But that requires an extra step every time something is changed. I guess starting factorio with a batch file that updates the mod foldes from the working dirs of mods is a possibility. But still, extra work for no benefit.
The solution is simple: part of your release-build process should be taking the file(s) and copying them into a correctly named folder.
A release would build zip files because that's what you need to release. But zip files take extra time to create and extra time to unpack when starting factorio. That takes away from time spend developing. It's annoying enough to wait for the restart on every change already.

Darinth
Filter Inserter
Filter Inserter
Posts: 266
Joined: Wed Oct 17, 2018 12:17 pm
Contact:

Re: Load modules from directories without version

Post by Darinth » Thu Dec 13, 2018 2:11 pm

As an FYI, symlinks work perfectly fine in windows. They've been available since windows 7 (maybe vista, but I know fully functional symlinks are available in 7). There's not even a need to remake them when changing version numbers, windows allows for renaming of symlinks in file explorer.

With that said... I've not done modding for factorio, but this seems like kind of a pain when doing development. I think the feature is there for good reason, but honestly I feel like it's a feature developers should be able to turn off to stream-line the development process.

User avatar
bobingabout
Smart Inserter
Smart Inserter
Posts: 6593
Joined: Fri May 09, 2014 1:01 pm
Contact:

Re: Load modules from directories without version

Post by bobingabout » Fri Dec 14, 2018 9:37 am

I dunno, maybe I'm doing it wrong, but when I write my mods... they're in the final directory structure on my hard drive, in the mods folder of the factorio install. I edit them directly. I don't use git, and my backups are simply to compress either the entire mods folder, or the specific mod in a rar file.

I mean, for the most part, I mod in wordpad, because I'm just that hardcore! Renaming the folder while a mod file is open means I can no longer save it, so I have to close and re-open.
I also use Sublime Text, which keeps the files open even after closing and re-opening the program, and provides a much more programmer friendly interface. However, due to that initial problem of the working directory being the game install's mod folder, you change the filename to update the version number, EVERYTHING BREAKS, and you have to close all your open files and re-open them again.

Now, how else would I do it? work on it outside of the game's mod folder? Why? the benefit of working with it in the game's mod folder is that with a single (double) click of an icon, the game loads and says shit like "Error, you forgot a comma you dumbass!"

Forcing the mod's folder to contain the version number is an update hell...

To someone like Rseding, how do you program the game itself? you make edits to code, then press the "compile and run" button, right? you don't move files from one folder to another, you just press a button, and it launches the game to test. By enforcing mod folders to include the version number, you force modders to not be able to do the same thing. Try doing all this programming where the game's directory name itself has to include the version number, then tell me how beneficial it is, and completely no inconvenience at all!


Okay, yes, your points do stand, by including the version number as part of the mod directory, it does allow you to tell in an instant what version you have installed, and I'm not suggesting that a final release build in zip format should have this requirement removed, only if the mod is installed in a directory. Throw a warning message in game if you want! (similar to what happens when you have a folder, and zip file for the same mod installed), but honestly, it is an annoying requirement that I've been quietly complaining about for years.
Creator of Bob's mods. Expanding your gameplay since version 0.9.8.

Bilka
Factorio Staff
Factorio Staff
Posts: 1740
Joined: Sat Aug 13, 2016 9:20 am
Contact:

Re: Load modules from directories without version

Post by Bilka » Fri Dec 14, 2018 9:50 am

bobingabout wrote:
Fri Dec 14, 2018 9:37 am
To someone like Rseding, how do you program the game itself? you make edits to code, then press the "compile and run" button, right?
I can't speak for Rseding, but I have scripts that do some stuff for me after compilation. Things like copying around the mods folder and launching a second instance with it for multiplayer testing. Those bat files are linked in my start menu, so multiplayer testing becomes as easy as clicking on a tile in the start menu. Renaming mod folders would be something I'd have a script for if I had any mod developing scripts at all. Moving the files from my git folder to the mods folder based on the version number in the info.json seems rather trivial.
I'm an admin over at https://wiki.factorio.com. Feel free to contact me if there's anything wrong (or right) with it.

User avatar
bobingabout
Smart Inserter
Smart Inserter
Posts: 6593
Joined: Fri May 09, 2014 1:01 pm
Contact:

Re: Load modules from directories without version

Post by bobingabout » Fri Dec 14, 2018 9:54 am

I guess I AM doing it wrong then.

Also, I don't use Git. Git is a complicated foreign language to me that I just don't understand. Which you know already, you've seen some of my commits. you've seen me begging for help in source access and go completely ignored, or told to figure it out myself. (though to be fair that was between when people actually did help me figure it out.) You've seen people in source access laughing at me for completely fucking up my entire git library, trying to fix a problem caused by merging branch into master, instead of master into branch. Why do you think I stopped trying to add to source? It just wasn't worth all the trouble trying to use GIT.
Creator of Bob's mods. Expanding your gameplay since version 0.9.8.

Darinth
Filter Inserter
Filter Inserter
Posts: 266
Joined: Wed Oct 17, 2018 12:17 pm
Contact:

Re: Load modules from directories without version

Post by Darinth » Fri Dec 14, 2018 4:29 pm

So... while I still like the idea of providing a method to override the requirement for modders, I do at least want to do what I can to resolve your immediate issues bob. I think you'd be well served by the use of symlinks and a build process of some kind. A symlink is a folder that, for most intents and purposes, appears to be at one location but is actually located somewhere else. So, you make could make a symlink at "C:\my mod C" to "C:\users\bob\documents\mods\development folder for mod c". In doing so, there would appear to be a folder located at "C:\my mod C" which contains all of the files located at "C:\users\bob\documents\mods\development folder for mod c", but in reality they're both the same folder. Most applications won't be able to tell the difference. So you create a symlink in the factorio mods folder with the proper name of the mod and it points to your folder that actually has the source. When the version changes, you just rename the symlink folder and factorio will see the new name and be none-the-wiser. Because you open the files through your development folder, the past doesn't change from the perspective of sublime or any other editor and you shouldn't have to go through the pain of closing your development environment. If you really want to get fancy (which I'll encourage) you should be able to setup a batch file (or something similar, whatever your particular fancy is) to recreate symlinks and run factorio at the click of a button to ensure that all of you symlinks for your various mods are named properly at all times.

P.S. Completely unrelated to any of this: I'm going to highly encourage you to use a source repository. They're a bit complicated, I understand, but the security of knowing your code is all there in it's many and varied forms and you can always go back if you screw something up to see what changed.... I've saved myself a lot of hours with that functionality alone, not to mention the piece of mind knowing my projects are backed up to remote repositories. </rant about the value of source control repositories>

quyxkh
Filter Inserter
Filter Inserter
Posts: 710
Joined: Sun May 08, 2016 9:01 am
Contact:

Re: Load modules from directories without version

Post by quyxkh » Fri Dec 14, 2018 9:00 pm

A makefile to set the version number and do the install is hard?

Code: Select all

FACTORIO := $(HOME)/.factorio

# because  `bump` updates info.json:
MOD = $(shell jq -j .name info.json)
VER = $(shell jq -j .version info.json)

install:
	MOD=$(MOD) VER=$(VER); \
	rm -f $(FACTORIO)/mods/$${MOD}_[0-9]*; \
	ln -s `readlink -f .` $(FACTORIO)/mods/$${MOD}_$${VER}

install-archive: archive
	MOD=$(MOD) VER=$(VER); \
	rm -f $(FACTORIO)/mods/$${MOD}_[0-9]*; \
	ln $${MOD}_$${VER}.zip $(FACTORIO)/mods

archive:
	MOD=$(MOD) VER=$(VER); \
	git archive @ --prefix=$${MOD}/ -o $${MOD}_$${VER}.zip

bump:
	IFS=. read major minor patch <<<`jq -r .version info.json`; \
		jq .version=\"$$major.$$minor.$$((++patch))\" info.json > newinfo.json
	mv newinfo.json info.json

User avatar
bobingabout
Smart Inserter
Smart Inserter
Posts: 6593
Joined: Fri May 09, 2014 1:01 pm
Contact:

Re: Load modules from directories without version

Post by bobingabout » Tue Dec 18, 2018 9:38 am

quyxkh wrote:
Fri Dec 14, 2018 9:00 pm
A makefile to set the version number and do the install is hard?

Code: Select all

FACTORIO := $(HOME)/.factorio

# because  `bump` updates info.json:
MOD = $(shell jq -j .name info.json)
VER = $(shell jq -j .version info.json)

install:
	MOD=$(MOD) VER=$(VER); \
	rm -f $(FACTORIO)/mods/$${MOD}_[0-9]*; \
	ln -s `readlink -f .` $(FACTORIO)/mods/$${MOD}_$${VER}

install-archive: archive
	MOD=$(MOD) VER=$(VER); \
	rm -f $(FACTORIO)/mods/$${MOD}_[0-9]*; \
	ln $${MOD}_$${VER}.zip $(FACTORIO)/mods

archive:
	MOD=$(MOD) VER=$(VER); \
	git archive @ --prefix=$${MOD}/ -o $${MOD}_$${VER}.zip

bump:
	IFS=. read major minor patch <<<`jq -r .version info.json`; \
		jq .version=\"$$major.$$minor.$$((++patch))\" info.json > newinfo.json
	mv newinfo.json info.json
that's like a completely foreign language to me. the kind of language I can't just intuit.

also, what do you mean by "A makefile"?
Creator of Bob's mods. Expanding your gameplay since version 0.9.8.

Darinth
Filter Inserter
Filter Inserter
Posts: 266
Joined: Wed Oct 17, 2018 12:17 pm
Contact:

Re: Load modules from directories without version

Post by Darinth » Tue Dec 18, 2018 1:49 pm

https://en.wikipedia.org/wiki/Makefile

I'll acknowledge I've never had to directly deal with makefiles myself. They're mostly used on *nix platforms, and most of my development work is on windows. Given that you've indicated you're the same, there's no reason for you to have ever encountered them. They're basically a specialized version of batch files that can perform different tasks based on circumstances and have some extra logic available for doing things like conditionally compiling resources as they're changed (so that C/C++ object files are only recompiled when their source files have changed, for example).

The idea of that makefile is basically to pull the mod name and version, remove the linux symlink, and recreate it. It's the makefile equivalent of the batch file that I'd recommended. For your purposes, I'd recommend a batch file over a makefile. It'll take some learning of what all batch files can do, but using utilities like findstr (a windows utility for extracting text from a file that matches a pattern) I honestly doubt it'll take you long to create a batch file that can pull the mod name and versions from your mods, delete the old symlinks, and then create new symlinks that are always up-to-date. In the long run, I'm pretty sure it'll save you a lot of time not needing to close down your dev environment every time a version number changes.

quyxkh
Filter Inserter
Filter Inserter
Posts: 710
Joined: Sun May 08, 2016 9:01 am
Contact:

Re: Load modules from directories without version

Post by quyxkh » Tue Dec 18, 2018 4:10 pm

Yeah, sorry about that, when it's time to bump the version I say `make bump`, to install it I say `make install`, it constructs and runs the necessary commands to actually make that happen from those recipes. To when I say `make bump` it constructs and runs

Code: Select all

IFS=. read major minor patch <<<`jq -r .version info.json`; \
        jq .version=\"$major.$minor.$((++patch))\" info.json > newinfo.json
mv newinfo.json info.json
`make install` constructs and runs (right now, the mod name and 0.0.9 it pulled from info.json)

Code: Select all

MOD=BenchmarkProduction VER=0.0.9; \
rm -f /home/quyxkh/.factorio/mods/${MOD}_[0-9]*; \
ln -s `readlink -f .` /home/quyxkh/.factorio/mods/${MOD}_${VER}
It checks for out-of-date dependencies and only runs what's necessary, here the targets are all fake and don't exist at all, there's no file named "install" so it always runs the recipes. `install-archive` depends on `archive`, so when I say `make install-archive` it rebuilds the archive first. The same makefile works for any mod, it's instructions for how to find and construct the necessary commands.

`jq`'s a handy json-munger, it's basically xslt for json.

User avatar
bobingabout
Smart Inserter
Smart Inserter
Posts: 6593
Joined: Fri May 09, 2014 1:01 pm
Contact:

Re: Load modules from directories without version

Post by bobingabout » Tue Dec 18, 2018 4:14 pm

quyxkh wrote:
Tue Dec 18, 2018 4:10 pm
Yeah, sorry about that, when it's time to bump the version I say `make bump`, to install it I say `make install`, it constructs and runs the necessary commands to actually make that happen from those recipes. To when I say `make bump` it constructs and runs

Code: Select all

IFS=. read major minor patch <<<`jq -r .version info.json`; \
        jq .version=\"$major.$minor.$((++patch))\" info.json > newinfo.json
mv newinfo.json info.json
`make install` constructs and runs (right now, the mod name and 0.0.9 it pulled from info.json)

Code: Select all

MOD=BenchmarkProduction VER=0.0.9; \
rm -f /home/quyxkh/.factorio/mods/${MOD}_[0-9]*; \
ln -s `readlink -f .` /home/quyxkh/.factorio/mods/${MOD}_${VER}
It checks for out-of-date dependencies and only runs what's necessary, here the targets are all fake and don't exist at all, there's no file named "install" so it always runs the recipes. `install-archive` depends on `archive`, so when I say `make install-archive` it rebuilds the archive first. The same makefile works for any mod, it's instructions for how to find and construct the necessary commands.

`jq`'s a handy json-munger, it's basically xslt for json.
I still have no idea what you're talking about.
Creator of Bob's mods. Expanding your gameplay since version 0.9.8.

User avatar
steinio
Smart Inserter
Smart Inserter
Posts: 2161
Joined: Sat Mar 12, 2016 4:19 pm
Contact:

Re: Load modules from directories without version

Post by steinio » Tue Dec 18, 2018 4:23 pm

+1 for the topic
-1000 for all enforcing of Git or scripts.

The mod version is written in the info.json so why is it also required in the folder name?
Image
Transport Belt Repair Man
My little mods: Link | My favourite mods: Bob's Mods | Angel's Mods | Yuoki Railway Core | EvoGUI | Logistic Train Network
Factorio Cheat Sheet by Denis Zholob

View unread Posts

Koub
Global Moderator
Global Moderator
Posts: 4613
Joined: Fri May 30, 2014 8:54 am
Contact:

Re: Load modules from directories without version

Post by Koub » Tue Dec 18, 2018 5:12 pm

bobingabout wrote:
Tue Dec 18, 2018 4:14 pm
I still have no idea what you're talking about.
Unix/Linux developper stuff. It's a script that allows to automate compilation even of complex programs, managing libraries, dependancies, and whatever fancy stuff programmers do to compile.
Koub - Please consider English is not my native language.

pleegwat
Fast Inserter
Fast Inserter
Posts: 152
Joined: Fri May 19, 2017 7:31 pm
Contact:

Re: Load modules from directories without version

Post by pleegwat » Tue Dec 18, 2018 9:07 pm

bobingabout wrote:
Tue Dec 18, 2018 9:38 am
quyxkh wrote:
Fri Dec 14, 2018 9:00 pm
A makefile to set the version number and do the install is hard?

Code: Select all

FACTORIO := $(HOME)/.factorio

# because  `bump` updates info.json:
MOD = $(shell jq -j .name info.json)
VER = $(shell jq -j .version info.json)

install:
	MOD=$(MOD) VER=$(VER); \
	rm -f $(FACTORIO)/mods/$${MOD}_[0-9]*; \
	ln -s `readlink -f .` $(FACTORIO)/mods/$${MOD}_$${VER}

install-archive: archive
	MOD=$(MOD) VER=$(VER); \
	rm -f $(FACTORIO)/mods/$${MOD}_[0-9]*; \
	ln $${MOD}_$${VER}.zip $(FACTORIO)/mods

archive:
	MOD=$(MOD) VER=$(VER); \
	git archive @ --prefix=$${MOD}/ -o $${MOD}_$${VER}.zip

bump:
	IFS=. read major minor patch <<<`jq -r .version info.json`; \
		jq .version=\"$$major.$$minor.$$((++patch))\" info.json > newinfo.json
	mv newinfo.json info.json
that's like a completely foreign language to me. the kind of language I can't just intuit.

also, what do you mean by "A makefile"?
Don't worry. Makefiles are the kind of language nobody can just intuit.

Trebor
Fast Inserter
Fast Inserter
Posts: 179
Joined: Sun Apr 30, 2017 1:39 pm
Contact:

Re: Load modules from directories without version

Post by Trebor » Tue Dec 18, 2018 10:15 pm

A makefile is the configuration file for the make command. make is an old school command line command used to turn source files into programs and documentation. It can also be used to install files in the correct directories.

User avatar
bobingabout
Smart Inserter
Smart Inserter
Posts: 6593
Joined: Fri May 09, 2014 1:01 pm
Contact:

Re: Load modules from directories without version

Post by bobingabout » Wed Dec 19, 2018 9:14 am

Koub wrote:
bobingabout wrote:
Tue Dec 18, 2018 4:14 pm
I still have no idea what you're talking about.
Unix/Linux developper stuff. It's a script that allows to automate compilation even of complex programs, managing libraries, dependancies, and whatever fancy stuff programmers do to compile.
I'm using windows 7, I thought I made it clear I was using windows, why would unix/linux stuff be useful to me?

Don't worry about me, I'd rather just quit modding. not yet though.
Creator of Bob's mods. Expanding your gameplay since version 0.9.8.

Post Reply

Return to “Ideas and Suggestions”

Who is online

Users browsing this forum: No registered users