Mod-Header Discussion (PLEASE READ)

[Narc]

A bot will also have no problems without the hyphens, and the hyphens look bad. That is what I said.

Ok, I removed it. How is that looking now: https://forums.factorio.com/forum/vie ... 89&t=13995

I'm not sure that link goes where you intended?

And the "super flexible" thing is exactly what you're aiming to remove so that the entries are machine-readable.

No, what I want to do is something, which helps to create the first database.

Okay, I like that idea, but I've too often seen makeshift "temporary" solutions be left permanent just because they worked "well enough". A database of Factorio mods right now (for v0.12+) would be pretty quick to collate, simply because there aren't that many of them. Now is the perfect time to do it -- and we can add the older mods slowly, over time, especially if their authors pitch in.

After the first breakthroughs we will have a modding platform, where modders can add their mods, download, binary, sourcecode, links and all other essential information needed to handle a mod and also a link to the (essentially needed) discussions. Which is then this forum. Or at some point also somewhere else.

Okay. So let's do that, then. I'm not, however, convinced of the utility of making a bot to read the mod threads once to create the initial database.

The target is to enable, that this forum-threads are eventually not longer needed, but also not needed to be removed, cause why removing a working system?

I never said anything about removing forum threads. In fact, I rather assumed from the start we'd keep the discussions basically where they are already.

But part of discussion is also, how to make both: machine- and human readable.

My experience has been that machine-readable parts tend to be no more than marginally human-readable, unless the humans in question happen to be programmers. Not really a ringing endorsement.

Yeah... except forums are lousy to begin with.
... A "modding platform with a mini-forum per mod" means "a site to upload files to, showing some meta-data about the files (some of which can be automatically derived from the files themselves), and linking to a forum thread". None of that is difficult, and it's a one-time setup instead of requiring posters and moderators to be careful about how the magical first post in a mod thread gets formatted.
... To the extent that forums are good for anything, they're good for discussions within large groups of people. Trying to bash them into a form that makes them good for something else is likely to take longer than doing the right thing in the first place, as well as being more fragile and unfriendly.

I would count your critics as o. k. if you made a usable alternative suggestion.

Alternative? Write a website! It can be really simple, to start. You need:
- a way for people to register and log in (see about reusing forum logins; it should be possible)
- a place for logged in people to upload their mods and fill in some basic metadata (Factorio version, etc. -- we have the requirements in the example template)
- a way for anyone (including guests) to search or browse the mods, and download them (preferably with support for automated download tools like wget).

This is enough to get us started, and I'd estimate it at maybe two days of work for me, if I could dedicate that kind of time to doing it. I could be under-estimating (like most software engineers, I'm prone to that), but it shouldn't take more than a week.

Then, once we have this, the next iteration could maybe add alternative downloads, or version history support (keep older uploads). The next iteration after that might have private sharing codes (for closed testers). A way to download more than one mod at a time (for convenience). Linking mods together when they're subsets of the same Big Mod (for discovery). All of that can come in step by step.

The question is: What can be improved NOW to make things better than NOW?
Which means: This is of course a temporary state. We don't know, how long the current state will last. It can take a year. I don't calculate for that this year. Maybe after 0.13 release?
You are really invited to make it better.

I think NOW is pretty okay as it is. I think we could change it in no more than a couple of weeks, if we wanted to.

The only regret I have in having entered this discussion in the first place is not being able to say "don't worry, I'll do it myself". It would be possible for me to do it (or, at least, start it), but I can't spare the time. But please, let's not make "temporary" solutions that we keep paying for years and years from now.

[ssilk]

I'm not sure that link goes where you intended?

Oh. Fixed. This was meant: https://forums.factorio.com/forum/vie ... 91&t=13835

Okay, I like that idea, but I've too often seen makeshift "temporary" solutions be left permanent just because they worked "well enough". A database of Factorio mods right now (for v0.12+) would be pretty quick to collate, simply because there aren't that many of them. Now is the perfect time to do it -- and we can add the older mods slowly, over time, especially if their authors pitch in.

Well said. Hmmm. But I don't see someone doing that.
Try to understand me and my sight as moderator/admin: I would still prefer to have that information in a defined form, until some reliable and long-term database or other solution is available.

But part of discussion is also, how to make both: machine- and human readable.

My experience has been that machine-readable parts tend to be no more than marginally human-readable, unless the humans in question happen to be programmers. Not really a ringing endorsement.

You're right. On the other hand: It is possible. For example, I have a long experience in reading unformated stuff from advertises.
And still the above argument to have mod info in a some defined form is a useful thing, because even as human my eyes learn, how to read stuff very fast.

Alternative? Write a website! It can be really simple, to start. You need:
- a way for people to register and log in (see about reusing forum logins; it should be possible)
- a place for logged in people to upload their mods and fill in some basic metadata (Factorio version, etc. -- we have the requirements in the example template)
- a way for anyone (including guests) to search or browse the mods, and download them (preferably with support for automated download tools like wget).

O. k., this is of course the obviously right way.
But why should I do that? I mean it's my job, my living, I do this kind of stuff since >20 years, but in the evening I like to do other things.

This is enough to get us started, and I'd estimate it at maybe two days of work for me, if I could dedicate that kind of time to doing it. I could be under-estimating (like most software engineers, I'm prone to that), but it shouldn't take more than a week.

And I prefer to make a plan, that enables to hold the current state for more than half a year or so, cause I don't see any serious, reliable afford doing that.
Maybe (Maybe!) I find time to do something with it, when I have my slack-time. But, in the evenings like these I really like more to sit on my balcony, drink beer and make plans, that work guaranteed.

I think NOW is pretty okay as it is. I think we could change it in no more than a couple of weeks, if we wanted to.

And who is "we"?

To make it glass-clear: I really would like to have that kind of solution. But before anybody will put too much work into it: The devs plan this! It's definitely on their TODO-list. I mean they would be glad, if someone makes something. If someone wants to do that: Please talk with the devs first!! There are plans, there are needed things and so on.

The only regret I have in having entered this discussion in the first place is not being able to say "don't worry, I'll do it myself". It would be possible for me to do it (or, at least, start it), but I can't spare the time. But please, let's not make "temporary" solutions that we keep paying for years and years from now.

I made in 1992 some temporary solution, that still works. Everybody likes it.

[Narc]

I'm not sure that link goes where you intended?

Oh. Fixed. This was meant: https://forums.factorio.com/forum/vie ... 91&t=13835

Um. I'll be honest, I really can't tell what you changed. But, if I may make a suggestion, this is what I'd like to see:

• Latest Release: v1.2.3, 2038-04-01
• Factorio Version: 3.42.1
• Download: Preferred, Alternate

They look nicer as bullets (they scan better visually), but the hyphens (as I've been saying) in, e.g., "Latest-Release", really aren't necessary. Compare the regexes: /latest release: v([0-9.]+), ([0-9]{4}-[0-9]{2}-[0-9]{2})/i and /latest-release: v([0-9.]+), ([0-9]{4}-[0-9]{2}-[0-9]{2})/i.

I think I'd also like to see (both optional) a Website entry and a License entry... or maybe just declare that additional informational lines may be added if the poster wishes.

Okay, I like that idea, but I've too often seen makeshift "temporary" solutions be left permanent just because they worked "well enough". A database of Factorio mods right now (for v0.12+) would be pretty quick to collate, simply because there aren't that many of them. Now is the perfect time to do it -- and we can add the older mods slowly, over time, especially if their authors pitch in.

Well said. Hmmm. But I don't see someone doing that.
...And still the above argument to have mod info in a some defined form is a useful thing, because even as human my eyes learn, how to read stuff very fast.
...But why should I do that? I mean it's my job, my living, I do this kind of stuff since >20 years, but in the evening I like to do other things.
...in the evenings like these I really like more to sit on my balcony, drink beer and make plans, that work guaranteed.

I have to agree -- I have the same problem myself, and I've only been doing it for 11 years. I might be willing and able to support a small mod like the one I published; I'm not willing to support and develop a centralized mod store (even if it would be really easy to get started).

The only regret I have in having entered this discussion in the first place is not being able to say "don't worry, I'll do it myself". It would be possible for me to do it (or, at least, start it), but I can't spare the time. But please, let's not make "temporary" solutions that we keep paying for years and years from now.

I made in 1992 some temporary solution, that still works. Everybody likes it.

Good anecdote.

Honestly, the proposed template looks fine to me, aside from the minor quibbles above. Long descriptions, known bugs, etc., can all be free-form -- really, the only absolutely key piece of information that must be present is the official preferred (direct? can we require direct?) download link, isn't it? Most of the things a mod database would care about are part of the info.json, so the mod db bot will need to know where to get the mod from.

[ssilk]

I've changed it again, it includes again license and dependencies.

[Narc]

I like it.

A note on dependencies: since we're including them for human reference (the machine-readable listing is in info.json), can we suggest/require that they be listed as links to the specific mod threads?

Relatedly, do we want to mention optional dependencies? I'm thinking of something like RSO that can integrate with Dytech, Bob's mods, etc., but does not require them.

[ssilk]

I like it.

A note on dependencies: since we're including them for human reference (the machine-readable listing is in info.json), can we suggest/require that they be listed as links to the specific mod threads?

Hm.
The modder can just copy the line from the info.json. If he writes it correctly, later a robot can make links out of it.
I would say for mod-packs it is a must (but not in the header), the current practice is just o. k.

Relatedly, do we want to mention optional dependencies? I'm thinking of something like RSO that can integrate with Dytech, Bob's mods, etc., but does not require them.

Incompatibilities? Yes, but that doesn't belong to the header, because basically every mod is compatible to each other. The devs had put very big effort into that feature.
Hmmm. What you mean here is, that one mod is recommended to be used with another. Also yes, but also not in the header, that is part of the description.

[Narc]

Sounds good to me!

I decided to give it a try, and see what the template looks like when applied to a simple mod like my EvoGUI -- check it out over here. I think I rather like it -- pretty much the entirety of my first post was just writing the exact same condensed information in long hand.

@ssilk, what do you think? Looks good, right?

[ssilk]

Yeah. I really like this. The eye sees immediately the important informations. And it is not so difficult to create a simple parser, that can read the info - basically the download-link.

I re-added some optional fields. I really think the current info-header should be basically more or less exactly the same, as the info.json (where the info.json is missing some of those fields). https://forums.factorio.com/forum/vie ... 91&t=13835

And it seems, that this state doesn't last so long, see this fine article: https://forums.factorio.com/forum/vie ... 37&t=13563

Maybe someone can write a mod, that generates info.json AND the forum-info-header?

[Narc]

I re-added some optional fields.

Thanks for the notice, I've updated the EvoGUI thread to match. I'm really liking the look of this template -- I wasn't enthused with it to begin with, but it's actually quite pretty; and I've always approved of the consistency a common template creates.

[ssilk]

For me this feels now right. I wait some days, also asking the devs for comments this time before I make it "official".

[Talguy]

May I suggest adding a template version, the same way the HTTP protocol starts with HTTP/1.0 or HTTP/1.1 or HTTP/2.0?

Something like MODHEADER/1.0. Also see https://forums.factorio.com/forum/vie ... 519#p95519

[ssilk]

That's why there is a rule to name the subject "[MOD ...]....". Or do I miss something here?

[markheloking]

He's talking about the Mod-Header version used, so that when parsing it with machine code, it knows what version of the template to look for/parse it with (in case the template changes again in the future). I think all things considered that's a good and necessary thing anyway, otherwise all other code based on the template will break every time a tiny bit changes in it.
Version revise ALL THE THINGS

[obstinate]

Is it acceptable to scrape this board? I assume that was the point of having machine readable mods, but just want to confirm.

(If I were to scrape, I'd probably scrape once per hour. I could limit myself to 1QPS, or whatever seems reasonable. I would not scrape images, only the forum topics themselves. There would need to be one pass where I read everything, but after that I would only examine changed forum topics. I could use a robot name of your choosing to ensure that y'all could rate limit me easily if I screw up.)

[ssilk]

I assume you speak of some kind of spider (a scanner, that crawls the forums). There are already some (google, bing etc.). I can put you on the lists of known spider bots and you can follow the rules for spiders in PHPBB and follow the robots.txt rules.

Which means in practice: Use already existing spiders only. The reason is simple: If a spider runs crazy, you also cannot read the forum.

Interesting links I found:
https://www.phpbb.com/community/viewtop ... &t=1761395
https://www.phpbb.com/community/viewtopic.php?t=935605
http://www.phpbb-seo.com/en/seo-techniq ... e2763.html (very old)

[obstinate]

Looks like this site's robots.txt is open, and specifies the maximum crawl rates. If there is already a fetcher written for appengine that obeys robots.txt, I'll reuse it. Otherwise I'll write mine carefully and make sure to check robots.txt for disallow.

[daniel34]

For me this feels now right. I wait some days, also asking the devs for comments this time before I make it "official".

What is the current state of the mod header?
The stickied Mod-Description-Template is still spoilered out and is linking to this topic, and the Mod forum rules also link to the unfinished template.

Modding portal
For a while now there has been a community modding database called FactorioMods in development. Actually we have had plans to do our own database for quite some time. Hence this was almost like a sign to us=)) We got in touch with the project owner and started talking. The result is that we will work together on the official Factorio modding portal. The basic functionality will be very similar to what is there already at Factoriomods. However the whole website will have tighter integration with the Factorio infrastructure - namely using Factorio authentication and user management as well as our CDN for mod delivery. There are plenty of other small features that are planned. We will bring you the details once the progress is more apparent. The current plan (plan is a plan and hence it might change =)) is to bring basic modding portal and its game integration (listing, viewing and downloading mods) in the 0.13.

As the devs are thinking about an official mod portal and there already is a fan-made one, I don't think it should be a requirement for the mod-header to allow automated parsing by bots, as most current mod threads would have to be changed and new modders might not be aware of such a requirement. As DaveMcW puts it:

A better goal for machine reading is to download the mod.zip and extract info.json. This can be automated without bothering mod authors and users.

I think it should just be a list of requirements that should be in the mod post and maybe give a nice formatted example of a mod template, but also allow the modders to make their own mod post layouts as long as all the required information is there.

[Zeblote]

Well, my thoughts about this are still the same.

Should I upload a mod at some point, I'll try to make the topic look as good as possible to a human reader. I don't give two damns whether a bot can parse the topic or not, and your suggested header is ugly.

Once there actually is a significant mod portal, everyone will upload new mods to it instead. There's no need to automatically add all the old ones.

[Klonan]

Well, my thoughts about this are still the same.

Should I upload a mod at some point, I'll try to make the topic look as good as possible to a human reader. I don't give two damns whether a bot can parse the topic or not, and your suggested header is ugly.

Once there actually is a significant mod portal, everyone will upload new mods to it instead. There's no need to automatically add all the old ones.

I don't think this is a good mindset, the mod portal will need somewhere to start off.

Personally i use a mod header, filled with the essential information, and then make any sort've elaborations underneath that, for example: https://forums.factorio.com/forum/viewtop ... 93&t=18879

I think this is a good way, its clear for the reader, and also for any bots it is easy to look through. The header isn't just for bots but people too need the quick facts at the top

[Zeblote]

Personally i use a mod header, filled with the essential information, and then make any sort've elaborations underneath that, for example: https://forums.factorio.com/forum/viewtop ... 93&t=18879

I don't think any of that is essential information.



Your topic would look a lot nicer with just a headline, the two images, a description and download link.