[Mod 0.11.x] Foreman - Blueprint Manager

[JamesOFarrell]

Updated for 0.12. All thanks go to Choumiko.

Foreman is a mod for managing blueprints in game. You can manage a list of blueprints in your save game, export them to text and import blueprints others have shared. Multiplayer support has been removed as it was too buggy.
If you are updating to 0.11.3 you might see "Unknown key..." on the buttons (like so). If this happens please unzip the mod and use the folder instead. I am not sure what is causing this issue as it does not happen for me.


Features

• Save blueprints from toolbar (+ button)
  Load blueprints to toolbar (l button)
  Export blueprints to file (e button)
  Import blueprint from text (+ button)
  Import Blueprint Strings (Thanks Dave!)

Foreman_0.0.4.zip

v0.0.4

(24.81 KiB) Downloaded 6410 times

Foreman Bugs

• Button labels are crap, probably need some sprites
  Main button can take a bit of time to display first time

Known bugs with blueprints API. These issues are issues with factorio that hopefully will be resolved soon.

• Rail signals on curved track
  belt-to-ground do not connect properly

If your blueprint contains these items it will not work with Foreman. If you find any more please let me know ASAP and I'll report them. The error should be printed to your console when it happens.

How to Use

Once you have unlocked Automated Construction the Foreman button should appear in the top of your factorio window. Clicking on this button should bring up the main window.

This window will be empty the first time you bring it up. To add a blueprint click the "+" button. This will bring up the import window. You can import either from your toolbar or some import text.
To import from the toolbar (the first one it finds) leave the "Text" field blank. Don't forget to give your blueprint a name!

If you have a blueprint file you want to import, open it in notepad, copy the text, paste it into the text field and click import.

Once you click import you should get a nice friendly message. If the blueprint file has a name saved in it that will be used instead of the one entered.
If the import API considers the blueprint invalid if will let you know and post an error to the console. There are3 known errors at the moment, rail signals collide with the track if they are placed on a bend, belt-to-ground don't connect and assembly machines that have no recipe will not import. These are bugs in the API and I have posted them to the bug forum, hopefully they will be fixed soon

To remove a blueprint from the list click the "x" button
To load a blueprint to your toolbar click the "l" button. The blueprint data will be loaded into the first blueprint in your toolbar.
To export a blueprint to a file click the "e" button. This will export the blueprint into "<factorio-configpath>/script-output/blueprints/name.blueprint". These files are a raw dump of a blueprint table + a name and the icons displayed.

These files can be zipped and uploaded to the forum as they should be under a meg. The other option is to use Dave's Blueprint String Compressor

Blueprint format discussion

At the moment I am using a very simple blueprint file format. Here is the code to create and import the blueprint files so other mods can do the same thing:

Code: Select all

function getBlueprintData(blueprintStack)
	if blueprintStack ~= nil and blueprintStack.isblueprintsetup() then
		local data = {}
		data.icons = blueprintStack.blueprinticons
		data.entities = blueprintStack.getblueprintentities()
		convertToRecipeName(data.entities)
		return data
	end
	return nil
end

function setBlueprintData(blueprintStack, blueprintData)
	if blueprintStack ~= nil then
		blueprintStack.setblueprintentities(blueprintData.entities)
		local newTable = {}
		for i,d in ipairs(blueprintData.icons) do
			table.insert(newTable, {index = i, name=d})
		end
		blueprintStack.blueprinticons = newTable
		return true
	end
	return false
end

function deserializeBlueprintData(dataString)
	if dataString ~= nil then
		local fun, err = loadstring(dataString)
		if not err then 
			return fun()
		end
	end
	return nil
end

function serializeBlueprintData(blueprintData)
	if blueprintData ~= nil and blueprintData.icons ~= nil and blueprintData.entities ~= nil then
		return serpent.dump(blueprintData, {name="blueprintData"})
	end
	return nil
end

I did spend some time working on a compressed blueprint format and managed to get most blueprints down to about 40% of the original size but it occurred to me that if I put it in the first version without thinking it though I would have to support it for ever so i removed it. I would like to eventually move to a smaller format so factorio doesn't have to handle the massive copy paste job but I am interested to see if anyone has any ideas about it. If people are interested let me know and I'll post what I have. The only limitation of the future blueprint format is it must be able to be pasted into a factorio GUI text box as this is the only way to get the data into a game at the moment.

Feel free to post blueprint files here or anywhere else. Here is a test one for people to import. This is 2 blueprints, one is a accmulator level detector and the other is a coal power station that will fire-up when your batteries get low, this is based on the wiki design. Place them both, connect them with a red wire and load up the chest with 26-27 logs.

PowerFailover.zip

Coal power fail-over blueprints

(2.83 KiB) Downloaded 837 times

Foreman license: MIT

[Boogieman14]

Very cool stuff, I can't wait to give it a try.

As for the blueprint format, I'm not sure if Factorio actually exposes a zip method to lua, but if it does (or the devs are open to the idea), perhaps simply using that followed by a base64 encoding would be most flexible.

[JamesOFarrell]

As for the blueprint format, I'm not sure if Factorio actually exposes a zip method to lua, but if it does (or the devs are open to the idea), perhaps simply using that followed by a base64 encoding would be most flexible.

I had a look for a native implementation of zip but had no luck. What I did find was a native lua implementation of lzw (here), which is a text compression algorithm, it has an mit license so i pinched it. If you give it a dictionary that matches your input you can compress data pretty well. Zip would be nice though. Base64 bloats out data quite a bit, i played around with it for a while but limiting the text to 64 characters when the text format and clipboard can handle so much more feels like going in the wrong direction.

[Boogieman14]

Yeah, base64 bloats it out, but given that your example reduced from 31kb to 3kb zipped, an increase back to about 4kb isn't too bad. Could also use Ascii85 for a slightly more efficient encoding. I'm still unsure about using ascii values >127 for any form of electronic communication, that just seems too much of a risk with all these different platforms and all.

[JamesOFarrell]

Ascii85 looks good, sort of what i what looking for. Now to find a free zip implementation in lua.

[Boogieman14]

Or try and get the devs to expose their zip functions to lua? Their (likely native C++) implementation is very likely faster than anything that could be built in lua.

[JamesOFarrell]

That would be much easier. I'll submit a request in the next few days..

[laige]

This is a very good mod. Ty

[Boogieman14]

So far it works well. I noticed it's possible to have the same name in the library multiple times and when you export both, the first one gets overwritten silently. Not sure if this should have some kind of safeguard or confirmation before overwriting (or if confirmation isn't very easy, perhaps just add a sequence number or so).

I also exported some of my regular builds.

First is a defensive line that can be placed in a straight or at a corner by using the lamps as key points:

Walls.zip

(1 KiB) Downloaded 538 times

Next is my standard outpost blueprint (I only ever use this with the train stop in horizontal orientation, not sure if everything lines up correctly in vertical):

Outpost.zip

(5.88 KiB) Downloaded 511 times

And just for shits and giggles, I've also made a blueprint of a massive ore sorter I built for use with Bob's Ores. This beast clocks in at a 1.4MB blueprint file, it contains about 7.5k entities

Ore-sorter.zip

(73.58 KiB) Downloaded 626 times

And .blueprint is not an accepted attachment extension for this board. If this ends up becoming the final format, perhaps the devs would be open to the idea of allowing this extension for easier sharing (but if you do end up with the Ascii85 encoding, a code block would probably do just fine for most blueprints)

[ssilk]

This is a fantastic idea.

For the pasting in the forum: well zip is ok, an own extension might be useful. I fear about blueprints, which contain bad code, do something nasty... But only a bit.

I think for not so big BP it can work to use a code-block and spoiler?

What I was wondering: as it looks for me, the BPs are lua-code. Well, this is the most future compatible way and good enough for now, but I won't call it then blueprint anymore. Isn't that not much more like a function?
I mean everything can be shared with this, not only BPs but also other useful Lua code, which is to small as mod or give do fix bugs during a game. This is cool.

But in my eyes the BPs are a special file format, or two (compressed and human readable; maybe a mix of pure ascii graphics would and some description part might be also an idea?), and the format should be part of factorio core, but is some interpreted (not directly executed) format.

But it is the right way to implement and discuss this first with blueprints.

[JamesOFarrell]

What I was wondering: as it looks for me, the BPs are lua-code. Well, this is the most future compatible way and good enough for now, but I won't call it then blueprint anymore. Isn't that not much more like a function?

The way it work is Factorio uses a serialization library called serpent, you can see this if you type

Code: Select all

/c game.players[1].print(serpent.dump(game.players[1].getinventory(2)))

Should print the contents of your main inventory. This is what I use to export the blueprint data from factorio. To import the data you run the output though a function called loadstring. This is where the trouble comes in, loadstring will load a string as lua code and the allow you to execute. On the main lua page about loadstring it has "Before you use it, make sure that there is no simpler way to solve the problem at hand.". Obviously I ignored that advise.

I mean everything can be shared with this, not only BPs but also other useful Lua code, which is to small as mod or give do fix bugs during a game. This is cool.

It is but it means you can never use this mod in multiplayer with people you don't trust as they will always have access to a console. I would like to work around this issue at some point as it really doesn't sit well we me.

But in my eyes the BPs are a special file format, or two (compressed and human readable; maybe a mix of pure ascii graphics would and some description part might be also an idea?), and the format should be part of factorio core, but is some interpreted (not directly executed) format.

But it is the right way to implement and discuss this first with blueprints.

It would be nice if there was a official format but it is still early days. This is only an issue because of the text input method we have to use to get data into factorio. If we could load a file from disk it wouldn't matter how big the files were but sadly loading data from disk would cause desync as it can be different in 2 locations.

Ultimately going forward I will need to write an import function that doesn't use loadstring, I avoided doing this for the first release as I wanted to get something out and see what people thought. Once Foreman has a import function that is not executing lua code directly I'll be a lot happier.

But only a bit.

Your post pretty much sums up my though process leading up to release and why I stripped out the crazy format I had invented. Any format needs to be easy to use for other mods so we are not locked in to using what ever drunken crazy idea I came up with even if that means I'm running lua code from untrusted sources in the first release. Glad there is a discussion going on.

I'm updating pocket bots for multiplayer use at the moment but after that Foreman will probably get some more features added. I'm thinking a mirror blueprint feature would be a nice addition as it is something missing in vanilla factorio which shouldn't be to hard to implement. Keep posting blueprints, was great seeing the ones that Boogieman14 posted, even if his defenses are slightly lacking (More turrets!)

[ssilk]

Ok, I have two thoughts.

The first is: at some time before 1.0 there will be something like an automatic mod-loader in factorio. Some inbuilt browser, where you can choose the wanted mods. And of course for multiplayer a loader, which automatically downloads needed mods.

And now you come with this serialized blueprints, which can be used to execute LUA . So my thought is: where is then the difference between a blueprint and a mod?

I really like the idea, that there is NO difference! In other words: If you download a mod or a blueprint, there is no difference! this can be extended: building plans are blueprints in the right order/at the right time. A mixture of mod and blueprint!

The second is about the security: there are other serialization methods, I use in my factorio-loader: https://github.com/alexaulbach/Factorio ... er/library
Note, that you should use the originals (they are free, so I included them)!

The JSON.lua produces and reads json. Inspect produces lua-code. There is eventually also a XML lib out there.

I think json would be a good start.

And then there is the need to check the integrity. I recommend to add some kind of checksum. First just a simple md5 or sha256 checksum to see, if the blueprint has been modified (checking by hand). And as the far, far away target I see a checksum, which can only be generated, when you log into your account to make an automated testing possible.

[JamesOFarrell]

There is at least once server browsers being built at the moment which is intending on auto loading mods. With the factorio command line connection options they shouldn't be far away.

The difference between blueprints and a mod is that the mod has to be installed everywhere but the blueprint files can be pasted in. So as long as Foreman is loaded you could load other mods (that contain no prototypes) without anyone installing them. Interesting idea and probably worth playing with. Along the same line, I also noticed the other day that you can register for events in the console, so you can put together one liners that run in the background. These aren't saved sadly but I am sure they could be. If you want to see it in action put this in your console, it should also work in multiplier games with command enabled.

Code: Select all

/c game.onevent(defines.events.ontick, function (event) if event.tick % 23 == 10 then for i,player in ipairs(game.players) do player.color = {r=math.sin(event.tick/120 + i), g=math.cos(event.tick/120 + i), b=math.sin(event.tick/110 + i)} end end end)

JSON is a good option as it is already used in factorio and there are many libraries available. I might start doing some tests to see what size files it produces. Check-sums are always a good idea but it wont help the security issue just the "my data is corrupted" issue. JSON fixes the security issue completely though as Foreman wouldn't be executing untrusted strings as code, just reading it as data.

[ssilk]

Well, with the checksums... It's possible to integrate the Factorio/Foreman version in it, so that it is possible to see, if this blueprint is still working with this version. I think this is really needed; the map seed for example has also this, which brings me to the idea, that the map seed algorithm can eventually be used to output the blueprints.

[Boogieman14]

was great seeing the ones that Boogieman14 posted, even if his defenses are slightly lacking (More turrets!)

I'll try to do better next time

[JamesOFarrell]

I'll try to do better next time

Don't worry, it has more turrets than any other posted blueprint.

[Degraine]

Seems like there's some problems with the mod - I tried using Boogieman's ore sorter blueprint and it came out with all of the underground belts facing in one direction. They're running in the right direction but half of them have the wrong orientation.

[JamesOFarrell]

Seems like there's some problems with the mod - I tried using Boogieman's ore sorter blueprint and it came out with all of the underground belts facing in one direction. They're running in the right direction but half of them have the wrong orientation.

Looks like the blueprint API is ignoring the connections between the belt to ground. I tried to add a work around but it doesn't look like that information can even be obtained by lua at all. I've logged it here. Hopefully it will be fixed soon. Thanks for the bug report

[Boogieman14]

Despite the belt problems, I'm still going to post a few more of my designs. They might still help uncover some more funkiness

First off, my train unloader, matches the outpost layout I posted earlier. Unloads two wagons (<10 seconds with vanilla ores, quite a bit more with ore mods at 4k per wagon ), loads the third wagon with supplies:

Train Unloader - West exit.zip

(3.31 KiB) Downloaded 381 times

Train Unloader - East exit.zip

(3.52 KiB) Downloaded 387 times

And a mass-storage setup. Most of the stuff passes straight through, some is redirected to the buffer areas to the side. On each side, there's two smart chests with both red and green wire connected (connect whichever you need elsewhere to the rest of your network). Chests fill up center to edge and empty edge to center, so those smart chests can be used as a kind of level indicator. On the outpost side, I have smart inserters loading the train wagons, these have circuit conditions to only work when there's less than a certain amount in the network. (I use 20k when I'm playing with any of the custom ore mods, as with those the chests hold 9.6k each, with vanilla 9k is probably a sensible setting (should be less than 4x what one chest holds, obviously). The extra chests toward the edge will hold anything that happens to still be in transit when the highwater mark is reached, while the chests toward the center supply downstream while new supply is being loaded and transported in. These storage setups are designed to be modular, and live below that monstrosity of an ore sorter I posted earlier

Mass storage.zip

(2.68 KiB) Downloaded 357 times

[Boogieman14]

You've probably already seen this one, it pops up as soon as a saved game finishes loading (single player)