Feature Request - RSS Data on Mod Portal

Post your ideas and suggestions how to improve the game.

Moderator: ickputzdirwech

Uthrom
Inserter
Inserter
Posts: 33
Joined: Fri Jan 11, 2019 4:01 pm
Contact:

Feature Request - RSS Data on Mod Portal

Post by Uthrom »

TL;DR
Please add RSS data to Mod portal entries, so various bots, et cetera, will actually work.
What ?
I know of several people who are running bots (discord, others) that are used to announce mod updates to mods by either community members, or other authors of interes.

Currently, in order to do this successfully, we have to scrape fields out of the mod recently updated list by regex pattern matching to the code. It would be really nifty if you guys could add in RSS fields for mods.factorio.com:

Channel Fields:
- Title (Factorio Mod Portal? Page title?)
- Description (Optionally, what version is searched?)
- Link (https://mods.factorio.com)

Then for each item on the Mod portal (or at least the recent ones) it would be helpful to have:
- Title (Mod name?)
- Description (Mod description)
- Link (Link to Mod)
- pubDate (Date mod last updated)
- Author (Mod Author)
-

This could be a separate feed file, or just embedded in the current page, in the appropriate places.
Why ?
This would go a long way toward letting others integrate tools that use data on the mod portal, it also solves us having to update RSS scrapers every time Wube changes the Mod portal (which has been happening a lot lately in preparation for 1.0)

JD-Plays
Burner Inserter
Burner Inserter
Posts: 19
Joined: Thu Mar 16, 2017 10:13 pm
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by JD-Plays »

+1 We have a automated bot that promotes any new mods by selected mod authors on our discord, and i have to say getting the system working was a nightmare, but then each time it IS working its broken within a few weeks due to changes in the mod portal. a RSS feed would be so much easier for us!

User avatar
boskid
Factorio Staff
Factorio Staff
Posts: 2227
Joined: Thu Dec 14, 2017 6:56 pm
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by boskid »

Is there something missing in the mod portal's api? https://mods.factorio.com/api/mods?page ... ersion=1.0

Uthrom
Inserter
Inserter
Posts: 33
Joined: Fri Jan 11, 2019 4:01 pm
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by Uthrom »

boskid wrote:
Tue Aug 11, 2020 4:57 am
Is there something missing in the mod portal's api? https://mods.factorio.com/api/mods?page ... ersion=1.0
Yes, RSS data.

The api may be great, but it means that instead of using any of a dozen or more existing systems that use RSS for what RSS is really good at, I have to write a custom application that interfaces between the mod portal API and any of a dozen or more existing systems that use RSS for what RSS is really good at.

Or you know, an entire bot with everything it needs from scratch just to start with json and end in discord, or whatever other destination it's supposed to go to.

If it's a matter of manpower, give me access and I'll submit a pull request with the code in question.

User avatar
Sandman2003
Long Handed Inserter
Long Handed Inserter
Posts: 67
Joined: Sun Nov 30, 2014 5:02 am
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by Sandman2003 »

RSS is a better way of supporting BOTs. As a fairly frequent mod updater, I note that the existing bot application built off the api 'struggles' to find updates when they are made. When it works, it is fine. You get all the relevant information from the update. But, I find the bot is only working about 20% of the time. The rest of the time it is simply missing my updates.

So is it just the bot? - I don't know. I didn't build the bot. I do know that an RSS feed is a much better way of accessing this sort of 'alert' data.

billbo99
Fast Inserter
Fast Inserter
Posts: 131
Joined: Fri Nov 02, 2018 9:19 am
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by billbo99 »

boskid wrote:
Tue Aug 11, 2020 4:57 am
Is there something missing in the mod portal's api? https://mods.factorio.com/api/mods?page ... ersion=1.0
I looked at the wiki for the documentation on this API. https://wiki.factorio.com/Mod_portal_API

The documentation only shows that "page / page_size / namelist" are fields that can be added to the GET url. In the example you gave above "version" is an undocumented field that can be used. What other fields are available to filter on?


Lastly is it possible to change the sort field that the results are indexed on. Currently the API returns everything in mod nameorder. If we want to use the API to create a RSS feed, we would ideally want the results to list just the most recent updates since we last checked the API.

Uthrom
Inserter
Inserter
Posts: 33
Joined: Fri Jan 11, 2019 4:01 pm
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by Uthrom »

boskid wrote:
Tue Aug 11, 2020 4:57 am
Is there something missing in the mod portal's api? https://mods.factorio.com/api/mods?page ... ersion=1.0
It also appears to be missing a way of retrieving most recently updated mods.

User avatar
NotRexButCaesar
Smart Inserter
Smart Inserter
Posts: 1120
Joined: Sun Feb 16, 2020 12:47 am
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by NotRexButCaesar »

When you say rss all I think of is the real solar system mod for KSP
—Crevez, chiens, si vous n'étes pas contents!

zombieland4942
Manual Inserter
Manual Inserter
Posts: 1
Joined: Wed May 27, 2020 2:02 am
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by zombieland4942 »

I am in the same boat. RSS would be my preferred way to handle this without having to run some crappy middleware to extract data, please consider the improvement.

Uthrom
Inserter
Inserter
Posts: 33
Joined: Fri Jan 11, 2019 4:01 pm
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by Uthrom »

Other than @boskid giving non-functioning alternatives, I have heard nothing in response to this.

Anyone?

OzGoober
Manual Inserter
Manual Inserter
Posts: 1
Joined: Fri Aug 14, 2020 8:07 am
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by OzGoober »

RSS is a great standard. Custom API's are rich but may change one day and break all the things!

JD-Plays
Burner Inserter
Burner Inserter
Posts: 19
Joined: Thu Mar 16, 2017 10:13 pm
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by JD-Plays »

So Boskid i assume from Wubes end it would be fairly quick add a 'latest mods' rss feed, any thoughts on this? just the latest mod feed would be great! last 50 mods updated i think would cover anything we need

User avatar
ssilk
Global Moderator
Global Moderator
Posts: 12888
Joined: Tue Apr 16, 2013 10:35 pm
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by ssilk »

To avoid misunderstandings and flames: I would appreciate having an RSS stream for the mods.
OzGoober wrote:
Tue Aug 18, 2020 3:25 am
RSS is a great standard. Custom API's are rich but may change one day and break all the things!
Sorry, but that is a kind of stupid argument. That’s a Json API. They are kind of custom, but are not just changing, they are extended (normally). There are literally dozens of libs In nearly any programming language, that can handle reading that without breaking, if new fields are added. It’s the “short form” of XML.

And RSS is a format, which has no far future. My opinion, but all big browsers need nowadays an extension to read it. That’s a big hurdle. Also - even if the standard is very clear, the implementation is in 50% buggy (to my experience). In my opinion it exists only, because between 2000 and 2010 everyone needs to have RSS. :)

And what disturbs me a very little bit is the “weekly request” (You, JD, Uthrom). Seems that you really need this.
And the simplest way to use it for you is RSS. ;)

It depends on for what the data is used, but I nearly always would prefer the json over the RSS. Faster, less implementation errors possible, less overhead, more flexible. I would prefer RSS only, if I also need to generate RSS. Is that so? :)
Cool suggestion: Eatable MOUSE-pointers.
Have you used the Advanced Search today?
Need help, question? FAQ - Wiki - Forum help
I still like small signatures...

blahfasel2000
Inserter
Inserter
Posts: 49
Joined: Sat Mar 28, 2020 2:10 pm
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by blahfasel2000 »

ssilk wrote:
Wed Aug 19, 2020 6:23 am
And RSS is a format, which has no far future. My opinion, but all big browsers need nowadays an extension to read it. That’s a big hurdle. Also - even if the standard is very clear, the implementation is in 50% buggy (to my experience). In my opinion it exists only, because between 2000 and 2010 everyone needs to have RSS. :)
RSS (and its cousin Atom) is alive and well. Not so much as a (news) feed for end users anymore (mainly because monetizing RSS feeds through ads is kinda hard...), but more for machine to machine communication as a standardized format to integrate feeds of (semi-)regular updates from one site into other sites, things like that.
ssilk wrote:
Wed Aug 19, 2020 6:23 am
It depends on for what the data is used, but I nearly always would prefer the json over the RSS. Faster, less implementation errors possible, less overhead, more flexible. I would prefer RSS only, if I also need to generate RSS. Is that so? :)
For writing some custom client I would prefer a JSON API nowadays, I agree. But for RSS there are for example ready made discord bots available where you only have to set it up with a RSS feed URL to get automatic posts on discord whenever a new update appears in the feed. There are no ready made solutions to provide similar capabilities based on custom JSON APIs for someone who doesn't necessarily have any programming skills.

User avatar
ssilk
Global Moderator
Global Moderator
Posts: 12888
Joined: Tue Apr 16, 2013 10:35 pm
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by ssilk »

blahfasel2000 wrote:
Wed Aug 19, 2020 8:27 pm
for RSS there are for example ready made discord bots available where you only have to set it up with a RSS feed URL to get automatic posts on discord whenever a new update appears in the feed.
Ok. That’s what I needed to know.
Cool suggestion: Eatable MOUSE-pointers.
Have you used the Advanced Search today?
Need help, question? FAQ - Wiki - Forum help
I still like small signatures...

Uthrom
Inserter
Inserter
Posts: 33
Joined: Fri Jan 11, 2019 4:01 pm
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by Uthrom »

ssilk wrote:
Wed Aug 19, 2020 6:23 am

And what disturbs me a very little bit is the “weekly request” (You, JD, Uthrom). Seems that you really need this.
And the simplest way to use it for you is RSS. ;)
This was detailed by me earlier in the thread -- one can use off-the-shelf things with RSS scrapers that broke 3 times in 3 weeks while you changed the website, feed into a bot already written, it automatically has sorting, searching, you can pick the version, etc -- just by adding the requisite RSS code to the existing mod portal, OR

.. write a custom bot that pulls the entire mod list (remember: json API doesn't allow for sorting), sort and filter "inhouse" and then posts to discord (in this case).

The fact that boskid brushed it aside with "Use this other API" - which then doesn't actually do what's needed, or in a manner that easily interacts with existing software is why I keep pinging this post.

Rule #1 in making things work: Find existing stuff that does it for you
Rule #2: Make the least amount of changes to get #1 to work.

Writing custom json-to-RSS or even worse, json-to-discord code and getting it to work on some cloud service is certainly not either of those.

User avatar
ssilk
Global Moderator
Global Moderator
Posts: 12888
Joined: Tue Apr 16, 2013 10:35 pm
Contact:

Re: Feature Request - RSS Data on Mod Portal

Post by ssilk »

Uthrom wrote:
Wed Aug 19, 2020 10:31 pm
- one can use off-the-shelf things with RSS scrapers that broke 3 times in 3 weeks while you changed the website, feed into a bot already written, it automatically has sorting, searching, you can pick the version, etc -- just by adding the requisite RSS code to the existing mod portal, OR

.. write a custom bot that pulls the entire mod list (remember: json API doesn't allow for sorting), sort and filter "inhouse" and then posts to discord (in this case).
Well, I don’t want to blame. Just from the logical/social aspect:
- yes, parsing the website is a bad idea, but that is kind of what you have chosen in the past to whatever reason. You cannot blame anybody for that.
- So the remaining good ideas is using the api, or using an non-existent RSS feed
- you choose to ask for the RSS, because it integrates much better into discord. Understandable, and makes some kind of sense, if there is no time pressure or so to have that, because you have nothing you can do to influence that development.

It is of course useful and it makes sense (I also said, I support this), but - I can’t help - it’s kind of thoughtless to ask for a feature which needs about a week to plan, to implement, test, put it on the website, write docs, announce etc. after 1 week it is first requested. In a time, where the fokus is shifted away from mods. :)
Cool suggestion: Eatable MOUSE-pointers.
Have you used the Advanced Search today?
Need help, question? FAQ - Wiki - Forum help
I still like small signatures...

User avatar
dzbanek
Inserter
Inserter
Posts: 28
Joined: Sun Jul 03, 2016 8:26 am
Contact:

RSS

Post by dzbanek »

Can we have a rss for mods portal, please? With recently updated mods.

Thunder2k
Burner Inserter
Burner Inserter
Posts: 5
Joined: Thu Jan 28, 2021 3:14 pm
Contact:

Re: RSS

Post by Thunder2k »

+1

It would help to check on all the changelog from my prefered mods.


Post Reply

Return to “Ideas and Suggestions”