Koub wrote: ↑Thu Jul 25, 2019 12:34 pm
The only answer I have found to the "too much data to find things" is "need a better search engine" and/or "have a tag system", knowing that a tag system needs real investment in time to tag appropriately every single suggestion or contribution to be "findable" with tag searches.
Is there a way to aggregate the new posts automatically and receive them as a random person like you can do with RSS ?
I have done a thing at school quite some time ago now, to spare me some time reading press.It was 3 different pieces of scripts i hacked together.First using RSS parser like Magpie(gpl license), to gather the news article/new posts text in my computer.
Then I used a part of a javascript library
https://d3js.org/ (bsd license) to make a bubble chart based on string occurences. It would show many bubble each being a word and bigger the more the word was used. I could click the bubble with a word and it would link me to the list of article containing the word in an RSS reader from where i could launch the webbrowser and deal with the ads and cookies if it was really needed. I could also swich it to make each bubble a particular RSS feed with color for the title of press, and the size, the number of article.
You could imagine it as a mix of those example of use case for the javascript library DDD ( data driven document).
https://observablehq.com/@d3/bubble-chart http://codementum.org/exoplanets/
I had to use a lexicon, (3rd piece of code i got for free and open but it wasn't english
) to remind me of the list of word that are not important ( the of at do it .... ) so that it would show at some point very useful information easily , name of countries, region , cities, companies , politician, disasters, , also stuff like elections, date and time and words that are only used a lot in the press and very few in real life, that makes you click on it to know why would they talk about that; it surprinsigly easy to cut down the "useless string" number, since it was the biggest bubble at first.
I used php, javascript, and html/ css, while not really knowing what i was doing, there are a lot ef example too make it easier to use the DDD library. It was very badly optimised, i requested the whole process to be done once of twice a day and it took like 2min30sec to display the chart for the first time, making the calculations for the bubbles and writing the file. Navigating after the file were created was decently fast for more text data than what is sent daily on the forum i think. I had started looking at AJAX things, but that was too much time investement to learn for the use. ( I wanted it web based)
The search engine helps you find when you know what you are looking for already in a way, and the tag system doesn't really have to be smart or tedious or precise, since with enough data/post, you could identify the main meaningful words automatically ! it was a very useful tool for me to answer question on the news, knowing what would be the topics in advance i only had to read one or two long and non-biaised article on each topic, rather than dealing with all the junk to figure out what was the new thing at the moment.
Context, like "this is a question/stuff i show" , "this is adressed to everyone/mainly to the devs/modders", "this is a suggestion" are the things that are difficult to get with only string occurences but that is also how the forum is atm organised so that posts are already organised to spare the smart work hard to teach the machine
! At least for the second point of OP, between belts or bots, belt and bots, train, combat, blueprint and train and so on.
For the implemented/outdated , 53.5% implemented, that kind of thing is different. For very long discussion and if you were to be VERY precise about it , you could watch a bubble chart in motion, where bubble would represent the occurences of string in the last 5/ 10 / 20 post, and would grow or reduce so that you would have a representation of the topic discussed over time, and you would notice the change of topic/ new elements. But that still wouldn't fix the problem to know if it's relevant or not for anyone