[jiri][0.16.12] Unintuitive search term parsing

This subforum contains all the issues which we already resolved.
User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 3773
Joined: Tue Jul 12, 2016 9:03 am
Contact:

[jiri][0.16.12] Unintuitive search term parsing

Post by eradicator » Sun Jan 07, 2018 9:26 am

What?
When using the search function in the crafting or technology menu the parser seems to consider every letter of a typed word as a seperate search term, increasing the amount of false positive results.
Examples (pointed out by @DaveMcW in this thread):
DaveMcW wrote: The spellchecker is very aggressive, results for "Heat" give:
Logistics -> Arithmetic combinator
Production -> Chemical plant
Intermediate products -> Heavy oil cracking to light oil
Also " " (space) is inconsisently considered either part of the search term if not followed by anything else:
e.g "exchanger " (space at the end) -> no results
or used as a delimiter if between two words:
e.g. "exchanger heat" -> heat exchanger (one result)
Some funny examples of current behavior
Expected behavior:
Most search features i know that are target at "the average end user" use " " (space) as the sole delimiter and treat it as a logic AND. Everything not delimited by spaces is considered a single word and must therefore be present in each search result as a whole word.
Author of: Hand Crank Generator, Screenshot Hotkey 2.0
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

posila
Factorio Staff
Factorio Staff
Posts: 4017
Joined: Thu Jun 11, 2015 1:35 pm
Contact:

Re: [0.16.12] Unintuitive search term parsing

Post by posila » Sun Jan 07, 2018 9:52 am

I am not going to sort this report out, I am going to throw my two cents here. The change was intentional - for consequtive characters if searches those characters in that order. It is quite powerfull, for example you can filter out everything but heat exchanger by just typing "hx" whearas "h x" would find also kovarex enrichment and explosive shells. I agree it is confusing if you don't know how it works.

kovarex
Factorio Staff
Factorio Staff
Posts: 7424
Joined: Wed Feb 06, 2013 12:00 am
Contact:

Re: [0.16.12] Unintuitive search term parsing

Post by kovarex » Sun Jan 07, 2018 12:12 pm

I also thought it was a bug until it was explained to me by the guys who implemented it, because this is something they are used to in their software.

I would say, that we should have an option for this kind of search that should be off by default.

User avatar
Aprillion
Inserter
Inserter
Posts: 34
Joined: Sun Apr 16, 2017 10:43 am
Contact:

Re: [0.16.12] Unintuitive search term parsing

Post by Aprillion » Sun Jan 07, 2018 3:18 pm

now that I read multiple times about the explanation, I still think it's a bug - sounds like 99% of times non-dev users search something, there are too many false negative results without knowing enough about the feature to use it with benefits...

BTW which software works this way? if I imagine some tool (IntelliJ, VS Code, Excel, browser's find on page, spotlight, glob file name expressions, ...) changed their search behaviour into this, that would be the worst nightmare

what I usually expect is:
hx => match(/hx/i)
heat ex == ex heat => match(/heat/i) and match(/ex/i)
heat*ex => match(/heat.*?ex/i)
h*x => match(/h.*?x/i)

and sometimes advanced parsing is supported that looks intuitive to users but very tough to specify correctly:
3 copper => amount:3 and ingredients_or_products_match(/copper/)

but hx => match(/h.*?x/i) was very unexpected indeed

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 3773
Joined: Tue Jul 12, 2016 9:03 am
Contact:

Re: [0.16.12] Unintuitive search term parsing

Post by eradicator » Mon Jan 08, 2018 4:35 am

kovarex wrote:I also thought it was a bug until it was explained to me by the guys who implemented it, because this is something they are used to in their software.
Factorio does not do any such explaining though. And personally i never understood why there were all those "odd" search results until i read the post by @DaveMcW because i've never encountered this particular search term parsing pattern.
posila wrote:It is quite powerfull, for example you can filter out everything but heat exchanger by just typing "hx" whearas "h x" would find also kovarex enrichment and explosive shells. I agree it is confusing if you don't know how it works.
Even though i bet there are some situations (large code bases maybe?) in which this sort of search behavior can be benefitial i don't think it is benefitial to most of factorio players.
The "powerful" behavior you describe honestly sounds to me like a sort of "dynamic shortcut" that works by remembering specific search words. I.e. if $some_person wants to find a "heat exchanger" for the first time it is neither intuitive nor faster for that person to come up with a sequence of letters that happens to uniquely identify the result they want in the given set of item names (e.g. "hx"), and much faster to just type four letters "heat". So the system is only benefitial to the minority of users who take the time to remeber the optimal character sequence for items that they search for very often.
Another drawback (imho) of the current system is that the false positives are mostly completely unrelated items. Examlple:
With "normal" search "heat" gets you "heat exchanger" and "heat pipe", two items directly related to each other.
With "sequential letters" (is there a proper name for this type of search?) if you were in a modded environment searching for your preferred term "hx" will find you "heat exchanger" but possibly also "sulfur hexaflouride" and "short logistic extender", both of which are probably completely unrelated to what you want to do at the time.
kovarex wrote: I would say, that we should have an option for this kind of search that should be off by default.
That would be the obvious "best of both" solution :).
Author of: Hand Crank Generator, Screenshot Hotkey 2.0
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

User avatar
DDDGamer
Inserter
Inserter
Posts: 30
Joined: Mon Oct 17, 2016 2:16 pm
Contact:

Re: [0.16.12] Unintuitive search term parsing

Post by DDDGamer » Mon Jan 08, 2018 7:38 am

Perhaps this is related, but i would agree, im searching for iron trains with keyword "iron" and i get extraneous results with stone and circuits.
If i search for "iron" i expect only results with "iron" showing.
train-search-bug.png
train-search-bug.png (934.14 KiB) Viewed 2093 times
I i think it would be nice to be able to search train station by *tags*:
for example if you name a station [Smelting][Export][Iron] that would parce into tags "Smelting", "Export" and "Iron" that you can just use to quickly filter results by.
(Think newegg.com where i can narrow down my shopping search results based on specific tags like RAM, CPU, Motherboard, etc..)

User avatar
DDDGamer
Inserter
Inserter
Posts: 30
Joined: Mon Oct 17, 2016 2:16 pm
Contact:

Re: [0.16.12] Unintuitive search term parsing

Post by DDDGamer » Mon Jan 08, 2018 7:48 am

On the other hand this search performs as expected
train-search-station-ok.png
train-search-station-ok.png (51.79 KiB) Viewed 2091 times

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 3773
Joined: Tue Jul 12, 2016 9:03 am
Contact:

Re: [0.16.12] Unintuitive search term parsing

Post by eradicator » Mon Jan 08, 2018 10:25 am

DDDGamer wrote: [CIRcuit][ExpOrt-GreeN]
[MEF] ArtIlleRy LOadiNg
[MEF][ImpoRt][StONe]
I have to say that even though i know how it works now i find it quite difficult to locate the matching sequence. Interesting that the station name search seems to use normal search, nice find ;).
Author of: Hand Crank Generator, Screenshot Hotkey 2.0
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

User avatar
DDDGamer
Inserter
Inserter
Posts: 30
Joined: Mon Oct 17, 2016 2:16 pm
Contact:

Re: [0.16.12] Unintuitive search term parsing

Post by DDDGamer » Wed Jan 10, 2018 10:21 am

Why would anyone even want this. I am a programmer myself, and i see no use for that type of search.

Bilka
Factorio Staff
Factorio Staff
Posts: 2330
Joined: Sat Aug 13, 2016 9:20 am
Contact:

Re: [jiri][0.16.12] Unintuitive search term parsing

Post by Bilka » Wed Jan 10, 2018 3:59 pm

DDDGamer wrote:Why would anyone even want this. I am a programmer myself, and i see no use for that type of search.
I'm sometimes still playing 0.15 and dearly miss the fixed search there, so I noticed what it makes better. Here's an example that I personally use:
Image

It also still finds things even if you typo the search word:
Image
I'm an admin over at https://wiki.factorio.com. Feel free to contact me if there's anything wrong (or right) with it.

User avatar
DDDGamer
Inserter
Inserter
Posts: 30
Joined: Mon Oct 17, 2016 2:16 pm
Contact:

Re: [jiri][0.16.12] Unintuitive search term parsing

Post by DDDGamer » Wed Jan 10, 2018 9:38 pm

I think the benefit of having cool shortcut tricks /misspelled words does not outweigh not having standard search functions.
If there is both i see no problem, use whatever search terms you want sort of thing, but if the expectation is to display only iron trains when you search for iron and get superfulous/irrelevant information, thats is a a failure for the search algorithm.
Basically it becomes a "Norman Door" where you have to explain to the user how a door works, instead of the user just using the door. (Think those doors that have a label "push" or "pull", those are frustrating because that design is very bad and unintuitive, that to compensate for the bad design has to instruct the user on how to use it.)

Jap2.0
Smart Inserter
Smart Inserter
Posts: 2053
Joined: Tue Jun 20, 2017 12:02 am
Contact:

Re: [jiri][0.16.12] Unintuitive search term parsing

Post by Jap2.0 » Wed Jan 10, 2018 10:22 pm

I agree that it should be an option. I personally rarely use the search, but from reading this I've found that it's a feature that is cool and useful but also very unintuitive.
There are 10 types of people: those who get this joke and those who don't.

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 3773
Joined: Tue Jul 12, 2016 9:03 am
Contact:

Re: [jiri][0.16.12] Unintuitive search term parsing

Post by eradicator » Thu Jan 11, 2018 5:29 am

@Bilka:
Your examples demonstrate well what i criticised about the unintuitiveness of the current algorithm:

"LDS": This is a typical memorized shortcut. No new player will use it. It produces false positives (you have 3 tabs of results). And with a "normal" search algorithm you could reach the same result precision using "low" or "den".

"SCINC": So you found a typo that happens to have the same result as your intended search term. Had you typoed "since", "sciene", "scienv", etc this would not be the case. So it's more of a random side effect than an intended feature. There are methods to match "similar" strings when searching, which would then work for all other typos as well. I'm not sure i'd want that in factorio, but it would still be a better feature than coincidentally working or not depending on how one typoes.

Conclusion: I'm still in favor of defaulting to a normal algorythm. And no objections to obtionally include the current one ofc.
Author of: Hand Crank Generator, Screenshot Hotkey 2.0
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

Jap2.0
Smart Inserter
Smart Inserter
Posts: 2053
Joined: Tue Jun 20, 2017 12:02 am
Contact:

Re: [jiri][0.16.12] Unintuitive search term parsing

Post by Jap2.0 » Thu Jan 11, 2018 11:26 am

I believe that "since" and "sciene" would work.
There are 10 types of people: those who get this joke and those who don't.

Caine
Fast Inserter
Fast Inserter
Posts: 213
Joined: Sun Dec 17, 2017 1:46 pm
Contact:

Re: [0.16.12] Unintuitive search term parsing

Post by Caine » Thu Jan 11, 2018 11:39 am

DDDGamer wrote:Why would anyone even want this. I am a programmer myself, and i see no use for that type of search.
For example, it is pretty great for matching camelcased identifiers (by just typing the capital letters).

jiri
Former Staff
Former Staff
Posts: 8
Joined: Sun Jul 30, 2017 9:48 pm
Contact:

Re: [jiri][0.16.12] Unintuitive search term parsing

Post by jiri » Thu Jan 11, 2018 1:43 pm

Hi,

I'm the guy who wrote the new search function. The way it works is actually really simple, it just checks that the search pattern and the searched terms have the same letters in the same order and treats each word of the search pattern separately. What that means is that if you get a lot of results at first, you can just hit space and refine the search.
Why would anyone even want this. I am a programmer myself, and i see no use for that type of search.
I find this very useful when coding, because most of the time, I don't know what I'm looking for and this enables me to refine the search very fast. With the larger number of false positives, you get a better overview of the searched space and you're less likely to miss something. Come to think of it, that's probably not the best approximation of how people use the search function in Factorio, as the "exploration" phase happens through the GUI and people mostly use the search function to get something they already know they want.

The main reason I did this was to make the search function a bit more forgiving for typos. After reading this thread, I do agree that it can be significantly improved, so I'll look into making it work more like the original one, while still trying to make it more forgiving for clumsy people like me :)

User avatar
DaveMcW
Smart Inserter
Smart Inserter
Posts: 3014
Joined: Tue May 13, 2014 11:06 am
Contact:

Re: [jiri][0.16.12] Unintuitive search term parsing

Post by DaveMcW » Thu Jan 11, 2018 2:19 pm

jiri wrote:make it more forgiving for clumsy people like me :)
https://en.wikipedia.org/wiki/Damerau%E ... n_distance

P.S. I made this suggestion in a different topic, but it would be really nice to remove item groups in the search results so everything is in one tab.

Zavian
Smart Inserter
Smart Inserter
Posts: 1460
Joined: Thu Mar 02, 2017 2:57 am
Contact:

Re: [jiri][0.16.12] Unintuitive search term parsing

Post by Zavian » Thu Jan 11, 2018 2:55 pm

DaveMcW wrote:
jiri wrote:make it more forgiving for clumsy people like me :)
https://en.wikipedia.org/wiki/Damerau%E ... n_distance

P.S. I made this suggestion in a different topic, but it would be really nice to remove item groups in the search results so everything is in one tab.
I'm not sure I agree. Showing the tabs helps players learn where to find things, so that in future, they don't need to search, and can just go to the correct tab.

Caine
Fast Inserter
Fast Inserter
Posts: 213
Joined: Sun Dec 17, 2017 1:46 pm
Contact:

Re: [jiri][0.16.12] Unintuitive search term parsing

Post by Caine » Thu Jan 11, 2018 3:12 pm

I think the false positives of using the edit distance may be even less intuitive than the current search pattern, although it will strongly depend on the number of edit operations you allow.

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 3773
Joined: Tue Jul 12, 2016 9:03 am
Contact:

Re: [jiri][0.16.12] Unintuitive search term parsing

Post by eradicator » Thu Jan 11, 2018 3:32 pm

jiri wrote:I find this very useful when coding, because most of the time, I don't know what I'm looking for and this enables me to refine the search very fast. With the larger number of false positives, you get a better overview of the searched space and you're less likely to miss something.
That pretty much what i suspected this was used for. But as you already noticed yourself the inventory search is a quite different usecase :D. I wonder if it would be feasible to actually highlight the matched pattern in the search result, i.e. searching for heat resulting in a tooltip of "Heat pipe" when hovering heat pipe in the search results. But i have a feeling that that would require a largish rewrite of that part of the GUI.
jiri wrote:The main reason I did this was to make the search function a bit more forgiving for typos. After reading this thread, I do agree that it can be significantly improved, so I'll look into making it work more like the original one, while still trying to make it more forgiving for clumsy people like me :)
Wikipedia apparently has a whole list page for different edit-distance alrotythms: Category:String_similarity_measures. Personally for that kind of "typo aware" search function i'd prefer a two-staged results page. On the top show only exact results, and then below have a clearly marked list of "similar results"/"did you mean?" results that include possible typos. Because not everything that looks like a typo nessecarily is a typo.
Zavian wrote:
DaveMcW wrote: P.S. I made this suggestion in a different topic, but it would be really nice to remove item groups in the search results so everything is in one tab.
I'm not sure I agree. Showing the tabs helps players learn where to find things, so that in future, they don't need to search, and can just go to the correct tab.
Personally i've played enough factorio to know all the items. So i use the search functionality exclusively to get where i want faster. Because for me it's faster to just ctrl-f -> "heat" -> click heat pipe than to remember which category heatpipe belongs to and then find the symbol in a heap of other symbols. (My "finding symbols in heaps of other symbols"-skill might be below average. Which is not helped by the fact that most building icons are tiny "kinda-grey-blackish-blobs". :P) The problem with category tabs for the results in my usecase is that they would (often) require me to click several buttons before i can see the result i want. I.e. instead of one click i'd need to do 4-5 clicks again, which nullifies all the speed benefits from using the search function in the first place (for me). Which is a major reason why i have completely turned of categories (and wasn't too happy that the options menu entry got removed :x, for the rare cases where i did enable categories temporarily. For example when explaining things to someone who has them enabled. Or debugging my mods.). Though i don't play with any mods that add large amounts of recipes, otherwise turning of categories completely would probably not be feasible anymore.

PS: Also just because people sometimes misinterpret my posts. I'm not angry or something. I'm just trying to explain how i play the game ;). And thanks @jiri for the effort to help find a more universal solution.
Author of: Hand Crank Generator, Screenshot Hotkey 2.0
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

Post Reply

Return to “Resolved Problems and Bugs”

Who is online

Users browsing this forum: No registered users