Solved: Regular expression help

[Pi-C]

I need some help with regular expressions. I want to get the base name of an entity that "AAI Programmable Vehicles" has appended a suffix to. These suffixes can differ, like this:

Code: Select all

item-_-ghost
item-0-_-driver

I want to get "item" in both cases. This works, but I have to use a different regex for each name pattern:

Code: Select all

item_name = string.match("item-_-ghost", "^(.+)%-_%-.+$")
log(tostring(item_name))

item_name = string.match("item-0-_-driver", "^(.+)%-[0-9]+%-_%-.+$")
log(tostring(item_name))

I tried to cover both cases with one regex, but this returns just nil:

Code: Select all

item_name = string.match("item-_-ghost", "^(.+)(%-[0-9]+)*%-_%-.+$")
log(tostring(item_name))

item_name = string.match("item-0-_-driver", "^(.+)(%-[0-9]+)*%-_%-.+$")
log(tostring(item_name))

How could I get this to work?

[posila]

Split from gameplay questions thread

[asdff45]

If you want to ignore the rest after the `-`, then `^[^-]+` is a way to do it.

Code: Select all

item_name = string.match("item-_-ghost", "^[^-]+")
log(tostring(item_name))

item_name = string.match("item-0-_-driver", "^[^-]+")
log(tostring(item_name))

This will print out `item` in both cases.

To clarify how it works:
`^` to specify that this has to be at the beginning of the string.
`[^-]` matches everything, that is not an `-` character
`+` The previous token, has to be there one or more times.

When it finds a `-` the only token we have specified is not given anymore, so it will move onwords. Since the regex is finished at that point, it just returns `item`.

Here is a really helpful site, to develop a regex: https://regexr.com/

[Pi-C]

If you want to ignore the rest after the `-`, then `^[^-]+` is a way to do it.

Code: Select all

item_name = string.match("item-_-ghost", "^[^-]+")
log(tostring(item_name))

item_name = string.match("item-0-_-driver", "^[^-]+")
log(tostring(item_name))

This will print out `item` in both cases.

Sorry, that wouldn't work! However, it's my fault because I didn't make it clear that "item" was meant to represent a string that could include '-'. It was an over-simplification on my part. Anyway, I've tested my mod with different vehicle mods; here are some of the item names I want to get:

Code: Select all

better-cargo-plane
crawler
dirigible-blimp
Dodge-Challenger
Hauling-Truck
hcraft-entity

I've even found a case that is worse because the name actually contains a number:

Code: Select all

raven-1

When it finds a `-` the only token we have specified is not given anymore, so it will move onwords. Since the regex is finished at that point, it just returns `item`.

Sorry, but for the names from the list above, that would only match "crawler" (part of the item name I want to get would be cut off for the other items). AAI always appends "-_-STRING" to the names, and in some cases there even is "-[0-9]+" before that, that's why I included that in my regex.

I already have a fall-back, so if the regex returns an item name that doesn't match any item it's not really a problem. But of course I want the regex to be general enough that it finds a correct result in most cases.

An explanation what I wanted to do:

Code: Select all

"^(.+)(%-[0-9]+)*%-_%-.+$"

-- Use everything from the start of the string
^(.+)			

-- Look for one or more numbers preceded by '-'.  It's not guaranteed to exist (indicated by '*')
(%-[0-9]+)*		

-- Look for AAI's separator
%-_%-

-- Match everything from that position to the end of the string because AAI uses different suffixes
.+$

Here is a really helpful site, to develop a regex: https://regexr.com/

Thanks, I'll check that out!

[Tynach]

Your examples include the names you want to pull, but not the sorts of things that those names might be embedded in. Will the parts you want to cut off at the end always include either an underscore or a number, and always be preceded by a dash?

[Pi-C]

Your examples include the names you want to pull, but not the sorts of things that those names might be embedded in. Will the parts you want to cut off at the end always include either an underscore or a number, and always be preceded by a dash?

These are actual names I've seen in the logs:

Code: Select all

hover-car-0-_-driver
Dodge-Challenger-0-_-driver
Hauling-Truck-0-_-drive
hcraft-entity-0-_-driver
car-vehicle-machine-gun-_-driver
tank-tank-cannon-_-driver

So all of them will contain "-_-STRING", and some of them may contain "-NUMBER". Looking at AAI's contol file, STRING may be any of these:

Code: Select all

"solid"
"ghost"
"navigator"
"driver"
"buffer"

So far, "0" is the only value for NUMBER I've seen yet -- but it may be that other numbers could also occur. That's why I've used "[0-9]+" in the regex.

[asdff45]

So, `.+?(?=-_)` would match everything until `-_`, so it will return

Code: Select all

hover-car-0-_-driver			=> hover-car-0
Dodge-Challenger-0-_-driver		=> Dodge-Challenger-0
Hauling-Truck-0-_-drive			=> Hauling-Truck-0
hcraft-entity-0-_-driver		=> hcraft-entity-0
car-vehicle-machine-gun-_-driver	=> car-vehicle-machine-gun
tank-tank-cannon-_-driver		=> tank-tank-cannon

And `^.+?(?=-[0-9]?[-_])` would match everything until `-_` or `-0`, so it will return

Code: Select all

hover-car-0-_-driver			=> hover-car
Dodge-Challenger-0-_-driver		=> Dodge-Challenger
Hauling-Truck-0-_-drive			=> Hauling-Truck
hcraft-entity-0-_-driver		=> hcraft-entity
car-vehicle-machine-gun-_-driver	=> car-vehicle-machine-gun
tank-tank-cannon-_-driver		=> tank-tank-cannon

[Oktokolo]

These are actual names I've seen in the logs:

Code: Select all

hover-car-0-_-driver
Dodge-Challenger-0-_-driver
Hauling-Truck-0-_-drive
hcraft-entity-0-_-driver
car-vehicle-machine-gun-_-driver
tank-tank-cannon-_-driver

So all of them will contain "-_-STRING", and some of them may contain "-NUMBER". Looking at AAI's contol file, STRING may be any of these:

Code: Select all

solid|ghost|navigator|driver|buffer"

So far, "0" is the only value for NUMBER I've seen yet -- but it may be that other numbers could also occur. That's why I've used "[0-9]+" in the regex.

The following should match the desired item names whether they contain an unsigned int or not:
prefix, n, type = string.match(name, "^(.-)(%-[0-9]+)?%-_%-([a-z]+)$")
You should check type against the list of allowed type strings. LUA decided to do yet another homebrew regex dialect more limited than already existing dialects and therefore does not support (solid|ghost|navigator|driver|buffer).
If at least one int is preceding the dash-underscore-dash separator, the last such int is captured as n.
You should check for item names consisting of only prefix or prefix .. n.

I did not actually test this, so it might match shit, fail with an error, or create a singularity destroying the known universe.

[asdff45]

I just noticed, that LUA has no full RegEx implementation and both mine and Oktokolo's are not working.

So i created one within the LUA playground: `^(.+)%-_%-.+`, this returns the same, as the first one in my last post:

Code: Select all

prefix = string.match(name, "^(.+)%-_%-.+")

hover-car-0-_-driver			=> hover-car-0
Dodge-Challenger-0-_-driver		=> Dodge-Challenger-0
Hauling-Truck-0-_-drive			=> Hauling-Truck-0
hcraft-entity-0-_-driver		=> hcraft-entity-0
car-vehicle-machine-gun-_-driver	=> car-vehicle-machine-gun
tank-tank-cannon-_-driver		=> tank-tank-cannon

Getting the number away with only one call, seems not possible.
As soon as im adding the number into that pattern, it will break or nothing changes

[Pi-C]

The following should match the desired item names whether they contain an unsigned int or not:
prefix, n, type = string.match(name, "^(.-)(%-[0-9]+)?%-_%-([a-z]+)$")

Typo? Guess it should be "^(.+)" instead of "^(.-)".

You should check type against the list of allowed type strings. LUA decided to do yet another homebrew regex dialect more limited than already existing dialects and therefore does not support (solid|ghost|navigator|driver|buffer).

I don't think that such a hard-coded list is a good idea! It's well possible that I missed some suffixes, it also could be that Earendel decides to add new ones or change existing ones in the future. So I thought the more general "%-(.+)$" would work better.

I did not actually test this, so it might match shit, fail with an error, or create a singularity destroying the known universe.

I got that to work in my editor (also with sed, after removing the percent signs and adding escape characters for bash). However, it doesn't work in the game. As asdff45 pointed out, that's because of Lua's limited RegEx implementation, so that really isn't your fault.

Anyway, thanks for your help!

[Pi-C]

I just noticed, that LUA has no full RegEx implementation and both mine and Oktokolo's are not working.

Yep, I've got the same result.

So i created one within the LUA playground: `^(.+)%-_%-.+`, this returns the same, as the first one in my last post:
-- snip --
Getting the number away with only one call, seems not possible.
As soon as im adding the number into that pattern, it will break or nothing changes

That's exactly what I've experienced! I was sure it must work, but still got the wrong results. In the end, I've worked around it with this definition:

Code: Select all

item_name = string.match(state.car.name, "(.+)%-[0-9]+%-_%-.+$") or
                         string.match(state.car.name, "^(.+)%-_%-.+$")

For the first match, I used "(.+)%-[0-9]+" so numbers that actually belong to the entity name (like "raven-1" mentioned above) would be preserved. Needless to say, a one-catches-everything expression would have been so much nicer, and it bothered me quite a lot that I couldn't get it to work. (I always thought I knew a bit or two about regexes -- of course, getting the syntax right in different environments like sed, grep, awk etc., possibly with having to escape special characters in a shell, still is something else!)

So, thanks a lot for taking the time to test things -- and for confirming that the problem is indeed caused by the limitations of Lua!

[Oktokolo]

The following should match the desired item names whether they contain an unsigned int or not:
prefix, n, type = string.match(name, "^(.-)(%-[0-9]+)?%-_%-([a-z]+)$")

Typo? Guess it should be "^(.+)" instead of "^(.-)".

No, that should be the non-greedy quantifier (manual confirms support for this). If the first group is greedy, there never is anything left for the optional second group to match.
But LUA doesn't support quantifiers at closing paranthesis, so i have to get the question mark inside the second group:
prefix, n, type = string.match(name, "^(.-)(%-?[0-9]*)%-_%-([a-z]+)$")
This pattern is slightly less strict but still matches the number separately.
LUA Playground

[Pi-C]

Sorry, I must have missed this post!

The following should match the desired item names whether they contain an unsigned int or not:
prefix, n, type = string.match(name, "^(.-)(%-[0-9]+)?%-_%-([a-z]+)$")

Typo? Guess it should be "^(.+)" instead of "^(.-)".

No, that should be the non-greedy quantifier (manual confirms support for this). If the first group is greedy, there never is anything left for the optional second group to match.

Ah, yes. That's useful!

But LUA doesn't support quantifiers at closing paranthesis, so i have to get the question mark inside the second group:
prefix, n, type = string.match(name, "^(.-)(%-?[0-9]*)%-_%-([a-z]+)$")
This pattern is slightly less strict but still matches the number separately.

Great, this is just what I needed! Thanks a lot for your help.