Factorio Forums

TL;DR

Please add RSS data to Mod portal entries, so various bots, et cetera, will actually work.

What ?

I know of several people who are running bots (discord, others) that are used to announce mod updates to mods by either community members, or other authors of interes.

Currently, in order to do this successfully, we have to scrape fields out of the mod recently updated list by regex pattern matching to the code. It would be really nifty if you guys could add in RSS fields for mods.factorio.com:

Channel Fields:
- Title (Factorio Mod Portal? Page title?)
- Description (Optionally, what version is searched?)
- Link (https://mods.factorio.com)

Then for each item on the Mod portal (or at least the recent ones) it would be helpful to have:
- Title (Mod name?)
- Description (Mod description)
- Link (Link to Mod)
- pubDate (Date mod last updated)
- Author (Mod Author)
-

This could be a separate feed file, or just embedded in the current page, in the appropriate places.

Why ?

This would go a long way toward letting others integrate tools that use data on the mod portal, it also solves us having to update RSS scrapers every time Wube changes the Mod portal (which has been happening a lot lately in preparation for 1.0)

+1 We have a automated bot that promotes any new mods by selected mod authors on our discord, and i have to say getting the system working was a nightmare, but then each time it IS working its broken within a few weeks due to changes in the mod portal. a RSS feed would be so much easier for us!

Is there something missing in the mod portal's api? https://mods.factorio.com/api/mods?page ... ersion=1.0

boskid wrote: ↑

Tue Aug 11, 2020 4:57 am

Is there something missing in the mod portal's api? https://mods.factorio.com/api/mods?page ... ersion=1.0

Yes, RSS data.

The api may be great, but it means that instead of using any of a dozen or more existing systems that use RSS for what RSS is really good at, I have to write a custom application that interfaces between the mod portal API and any of a dozen or more existing systems that use RSS for what RSS is really good at.

Or you know, an entire bot with everything it needs from scratch just to start with json and end in discord, or whatever other destination it's supposed to go to.

If it's a matter of manpower, give me access and I'll submit a pull request with the code in question.

RSS is a better way of supporting BOTs. As a fairly frequent mod updater, I note that the existing bot application built off the api 'struggles' to find updates when they are made. When it works, it is fine. You get all the relevant information from the update. But, I find the bot is only working about 20% of the time. The rest of the time it is simply missing my updates.

So is it just the bot? - I don't know. I didn't build the bot. I do know that an RSS feed is a much better way of accessing this sort of 'alert' data.

boskid wrote: ↑

Tue Aug 11, 2020 4:57 am

Is there something missing in the mod portal's api? https://mods.factorio.com/api/mods?page ... ersion=1.0

I looked at the wiki for the documentation on this API. https://wiki.factorio.com/Mod_portal_API

The documentation only shows that "page / page_size / namelist" are fields that can be added to the GET url. In the example you gave above "version" is an undocumented field that can be used. What other fields are available to filter on?


Lastly is it possible to change the sort field that the results are indexed on. Currently the API returns everything in mod nameorder. If we want to use the API to create a RSS feed, we would ideally want the results to list just the most recent updates since we last checked the API.

boskid wrote: ↑

Tue Aug 11, 2020 4:57 am

Is there something missing in the mod portal's api? https://mods.factorio.com/api/mods?page ... ersion=1.0

It also appears to be missing a way of retrieving most recently updated mods.

When you say rss all I think of is the real solar system mod for KSP

I am in the same boat. RSS would be my preferred way to handle this without having to run some crappy middleware to extract data, please consider the improvement.

Other than @boskid giving non-functioning alternatives, I have heard nothing in response to this.

Anyone?

RSS is a great standard. Custom API's are rich but may change one day and break all the things!

So Boskid i assume from Wubes end it would be fairly quick add a 'latest mods' rss feed, any thoughts on this? just the latest mod feed would be great! last 50 mods updated i think would cover anything we need

To avoid misunderstandings and flames: I would appreciate having an RSS stream for the mods.

OzGoober wrote: ↑

Tue Aug 18, 2020 3:25 am

RSS is a great standard. Custom API's are rich but may change one day and break all the things!

Sorry, but that is a kind of stupid argument. That’s a Json API. They are kind of custom, but are not just changing, they are extended (normally). There are literally dozens of libs In nearly any programming language, that can handle reading that without breaking, if new fields are added. It’s the “short form” of XML.

And RSS is a format, which has no far future. My opinion, but all big browsers need nowadays an extension to read it. That’s a big hurdle. Also - even if the standard is very clear, the implementation is in 50% buggy (to my experience). In my opinion it exists only, because between 2000 and 2010 everyone needs to have RSS.

And what disturbs me a very little bit is the “weekly request” (You, JD, Uthrom). Seems that you really need this.
And the simplest way to use it for you is RSS.

It depends on for what the data is used, but I nearly always would prefer the json over the RSS. Faster, less implementation errors possible, less overhead, more flexible. I would prefer RSS only, if I also need to generate RSS. Is that so?

ssilk wrote: ↑

Wed Aug 19, 2020 6:23 am

And RSS is a format, which has no far future. My opinion, but all big browsers need nowadays an extension to read it. That’s a big hurdle. Also - even if the standard is very clear, the implementation is in 50% buggy (to my experience). In my opinion it exists only, because between 2000 and 2010 everyone needs to have RSS.

RSS (and its cousin Atom) is alive and well. Not so much as a (news) feed for end users anymore (mainly because monetizing RSS feeds through ads is kinda hard...), but more for machine to machine communication as a standardized format to integrate feeds of (semi-)regular updates from one site into other sites, things like that.

ssilk wrote: ↑

Wed Aug 19, 2020 6:23 am

It depends on for what the data is used, but I nearly always would prefer the json over the RSS. Faster, less implementation errors possible, less overhead, more flexible. I would prefer RSS only, if I also need to generate RSS. Is that so?

For writing some custom client I would prefer a JSON API nowadays, I agree. But for RSS there are for example ready made discord bots available where you only have to set it up with a RSS feed URL to get automatic posts on discord whenever a new update appears in the feed. There are no ready made solutions to provide similar capabilities based on custom JSON APIs for someone who doesn't necessarily have any programming skills.

blahfasel2000 wrote: ↑

Wed Aug 19, 2020 8:27 pm

for RSS there are for example ready made discord bots available where you only have to set it up with a RSS feed URL to get automatic posts on discord whenever a new update appears in the feed.

Ok. That’s what I needed to know.

ssilk wrote: ↑

Wed Aug 19, 2020 6:23 am

And what disturbs me a very little bit is the “weekly request” (You, JD, Uthrom). Seems that you really need this.
And the simplest way to use it for you is RSS.

This was detailed by me earlier in the thread -- one can use off-the-shelf things with RSS scrapers that broke 3 times in 3 weeks while you changed the website, feed into a bot already written, it automatically has sorting, searching, you can pick the version, etc -- just by adding the requisite RSS code to the existing mod portal, OR

.. write a custom bot that pulls the entire mod list (remember: json API doesn't allow for sorting), sort and filter "inhouse" and then posts to discord (in this case).

The fact that boskid brushed it aside with "Use this other API" - which then doesn't actually do what's needed, or in a manner that easily interacts with existing software is why I keep pinging this post.

Rule #1 in making things work: Find existing stuff that does it for you
Rule #2: Make the least amount of changes to get #1 to work.

Writing custom json-to-RSS or even worse, json-to-discord code and getting it to work on some cloud service is certainly not either of those.

Uthrom wrote: ↑

Wed Aug 19, 2020 10:31 pm

- one can use off-the-shelf things with RSS scrapers that broke 3 times in 3 weeks while you changed the website, feed into a bot already written, it automatically has sorting, searching, you can pick the version, etc -- just by adding the requisite RSS code to the existing mod portal, OR

.. write a custom bot that pulls the entire mod list (remember: json API doesn't allow for sorting), sort and filter "inhouse" and then posts to discord (in this case).

Well, I don’t want to blame. Just from the logical/social aspect:
- yes, parsing the website is a bad idea, but that is kind of what you have chosen in the past to whatever reason. You cannot blame anybody for that.
- So the remaining good ideas is using the api, or using an non-existent RSS feed
- you choose to ask for the RSS, because it integrates much better into discord. Understandable, and makes some kind of sense, if there is no time pressure or so to have that, because you have nothing you can do to influence that development.

It is of course useful and it makes sense (I also said, I support this), but - I can’t help - it’s kind of thoughtless to ask for a feature which needs about a week to plan, to implement, test, put it on the website, write docs, announce etc. after 1 week it is first requested. In a time, where the fokus is shifted away from mods.

Can we have a rss for mods portal, please? With recently updated mods.

+1

It would help to check on all the changelog from my prefered mods.

More precisely:

(Example)
https://mods.factorio.com/mod/Industria ... /changelog
with RSS Link
https://mods.factorio.com/mod/Industria ... g/feed.xml
or
https://mods.factorio.com/mod/Industria ... ngelog.rss