Signal Compression or Encapsulation

Post your ideas and suggestions how to improve the game.

Moderator: ickputzdirwech

Aricitic
Long Handed Inserter
Long Handed Inserter
Posts: 57
Joined: Sat Nov 16, 2024 5:30 am
Contact:

Signal Compression or Encapsulation

Post by Aricitic »

TL;DR
Enable some way to "compress" or "encapsulate" multiple signals into one... with the ability to "extract" them again later.
What?
Add the ability to take multiple signals, such as a list of ingredients, and compress them into one signal --> but specifically one that can allow us to extract these signals again at a later time/operation.
Why?
Currently, to enumerate the ingredients of a recipe, a player must have the building that makes that recipe be able to receive a signal for the result of the recipe AND transmit the ingredients.
Ok, fair enough, but what if I want to enumerate the ingredients for each ingredient or at different quality levels? How do I "capture" one while requesting another? If it's the levels of quality, this is simple, you send the results through a Selector Combinator with "Quality transfer" "desired quality" and "Each" as the 'target signal'... and you do this for each quality you want (up to five times)...
But how to enumerate the sub-ingredients? I've tried using an S-R (or R-S) Latch, but it catches ALL signals. I've tried this many different ways and many different times.

So. Instead if we could take a set of signals and encapsulate them into "one" signal (similar to the "each" or "anything" or "everything" signal); then transmit that, to be broken down into its components again later, we wouldn't have this issue.

And that isn't the only use case. Any time a person needs to transmit anything, everything, or each... or any set of signals, without interfering with other signals on the same circuit, they either have to create a "MASSIVE" external circuit or simply "not" do it. Needless to say, trying to catch an "each" in a latch is extremely difficult.
Of note:
This will reduce complexity. I do not ask that it be provided freely. As with my last suggestion (in which I forgot to mention this key point) I am perfectly happy for this to be locked behind a technology. That isn't a problem.

What may be a problem is people claiming that *because* this reduces complexity it *shouldn't* be added, *period*. To which I respond: Unlill Wube takes away logistics (bots) outright, this is a non-argument. Heck, Korvax Enrichment simplified things for players by providing a source of otherwise rare U235.

Final thoughts: Use case? Sure: Space Platforms.
Need I say more? Yes? Ok: Space Platforms are expensive to produce and ship into space so the room to build things in space is at a premium until you are fairly efficient and technologically sufficient. Until then the ability to tell one machine to produce anything you need depending on what you have available, and to do this with as few combinators as possible, is EXTREMELY important.
Need I say more?
Aricitic
Long Handed Inserter
Long Handed Inserter
Posts: 57
Joined: Sat Nov 16, 2024 5:30 am
Contact:

Re: Signal Compression or Encapsulation

Post by Aricitic »

A somewhat more simple solution for the issue with recipes would be to simply add a signal for each "thing" that can be crafted. Just... some way to distinguish between an "ingredient" vs a "recipe" within the signal(s) it/them self/ves...
Tertius
Smart Inserter
Smart Inserter
Posts: 1082
Joined: Fri Mar 19, 2021 5:58 pm
Contact:

Re: Signal Compression or Encapsulation

Post by Tertius »

As far as I see this request, you want an assembler in read recipe mode output a value, and this value contains an encoded list of all ingredients, including the ingredients required to make the ingredients?
This is not feasible, or not a complete solution, because there exist multiple recipes for several items, and you're unable to give the assembler the info which of these recipes to pick. There are also recipes an assembler cannot process but some other production building. How far do you want to descend the dependence tree? Down to the ore, water and crude oil? Such a result isn't practical. Or down to the plates? What if you have extra production lines for the green, red and blue chips?

And if you "extract" the encoded value with some new functionality, you need the same circuit complexity to process the result you need if you get the info in the first place.

As far as I see it, it's only practical to handle the recipe at hand. Not any dependent recipe.
You start with a list of items to craft.
You process the items one after the other.
To process the item at hand, you load it as recipe into an assembler and read the ingredients.
Then you process the ingredients, filter your individual factory has already in stock, or you want to acquire from elsewhere, and get the ingredients you need to craft.
You add these ingredients to the items to craft, in case they aren't there already.
Then you process the list of items again.
If you added no new item to the list of items to craft, you finished descended the dependency tree.
Now you process the list of items to craft, ordered by "most recently added".
So you feed the most recently added item as recipe to the assembler and let it craft the thing. It can be crafted, because it had no unavailable ingredients.
Then the next. It can also be crafted, because any dependent ingredient was more recently added, so it was crafted before.
And so on up to the first item added.

It seems you want the game engine provide some magical device that does this for you. I guess that will not happen, since there are many external dependencies: the factory-specific info which item is available and by what means, and which recipes to use, and what to do if there is simply no means to currently craft some dependent ingredient.

The algorithm I outlined can be implemented with not too many combinators. The central combinator is a memory cell that is a counter that counts the ticks since each item was added. Each item enters with value 1 and is increased by 1 every tick. The item with the lowest value is the most recently added item. The list of items is ordered, and the ordering is stable, since every entry is increased every tick.
A second memory cell is required. It also contains all items to craft, but as value it counts the amount to create, so you know when to change the recipe for the item currently being crafted and the next recipe can be crafted.

However, what do you want to do if the stock changes unexpectedly? What if some dependent item you just crafted is taken away by some random means?

All this is so complex, the player factories so different, you cannot build it into one magical device given by the game engine. You need to handle all this with your own circuit setup. Keep in mind game engine mechanics need to be universal. Not specific to your imagined factory setup.
Aricitic
Long Handed Inserter
Long Handed Inserter
Posts: 57
Joined: Sat Nov 16, 2024 5:30 am
Contact:

Re: Signal Compression or Encapsulation

Post by Aricitic »

Tertius wrote: Tue Dec 17, 2024 11:04 am As far as I see this request, you want an assembler in read recipe mode output a value, and this value contains an encoded list of all ingredients, including the ingredients required to make the ingredients?
This is not feasible, or not a complete solution, because there exist multiple recipes for several items, and you're unable to give the assembler the info which of these recipes to pick. There are also recipes an assembler cannot process but some other production building. How far do you want to descend the dependence tree? Down to the ore, water and crude oil? Such a result isn't practical. Or down to the plates? What if you have extra production lines for the green, red and blue chips?

And if you "extract" the encoded value with some new functionality, you need the same circuit complexity to process the result you need if you get the info in the first place.

As far as I see it, it's only practical to handle the recipe at hand. Not any dependent recipe.
You start with a list of items to craft.
You process the items one after the other.
To process the item at hand, you load it as recipe into an assembler and read the ingredients.
Then you process the ingredients, filter your individual factory has already in stock, or you want to acquire from elsewhere, and get the ingredients you need to craft.
You add these ingredients to the items to craft, in case they aren't there already.
Then you process the list of items again.
If you added no new item to the list of items to craft, you finished descended the dependency tree.
Now you process the list of items to craft, ordered by "most recently added".
So you feed the most recently added item as recipe to the assembler and let it craft the thing. It can be crafted, because it had no unavailable ingredients.
Then the next. It can also be crafted, because any dependent ingredient was more recently added, so it was crafted before.
And so on up to the first item added.

It seems you want the game engine provide some magical device that does this for you. I guess that will not happen, since there are many external dependencies: the factory-specific info which item is available and by what means, and which recipes to use, and what to do if there is simply no means to currently craft some dependent ingredient.

The algorithm I outlined can be implemented with not too many combinators. The central combinator is a memory cell that is a counter that counts the ticks since each item was added. Each item enters with value 1 and is increased by 1 every tick. The item with the lowest value is the most recently added item. The list of items is ordered, and the ordering is stable, since every entry is increased every tick.
A second memory cell is required. It also contains all items to craft, but as value it counts the amount to create, so you know when to change the recipe for the item currently being crafted and the next recipe can be crafted.

However, what do you want to do if the stock changes unexpectedly? What if some dependent item you just crafted is taken away by some random means?

All this is so complex, the player factories so different, you cannot build it into one magical device given by the game engine. You need to handle all this with your own circuit setup. Keep in mind game engine mechanics need to be universal. Not specific to your imagined factory setup.
First, I must apologize, I'm not feeling too well and am therefore not thinking too well. (yes, I use "big words" when feeling unwell... it's a curse -- and that's not a joke)
Second, you misunderstood the premise of my request so I'm going to respond under that assumption -- which is actually good because it forces me to reevaluate how I initially phrased my request and its explination.

I don't want a recursive encapsulation, which is what seems to be your interpretation. Heck, if I could use logistic groups or a constant combinator to predefine these myself, that would be perfectly acceptable. Or, if I could find a sufficient way to store captured signals - and those alone - that would be sufficient, and would allow me to utilize what you are suggesting for what I want.

I also don't want to have to recursively extract the sub-ingredients *each time* I need to get them. I want a simple, one-time, request to get the ingredients. Again, my initial response to my request would be sufficient to do this -- i.e. circuit signals that indicate a recipe itself for each craftable item/building('s item). These exist for some things like coal, but not others like circuits.

Additionally, if I could do a simple, one-time set of signals into, say, signal "A", and then convert it back into those signals, without using more than three Combinators, that would ALSO be acceptable, but I cannot. I've tried -- so, either I'm too stupid to figure it out, or it's just not possible for what I want: *EACH* -> *A* -> *(original) EACH* == not possible.
TL;DR
TL;DR: What I don't want: Factorio to automatically make any sub-ingredients for me without any input. That's unnecessary for me. What I *DO* want: to be able to store and utilize a list of signals dynamically without interfering with other signals: i.e. *iron* *electric circuit* -> *A* without interfering with *Iron* or *electric circuit*'s original signal(s).

-- Use-case --
SImilar to what you thought, I DO want to use this to do automated crafting, even *WITH* subcomponents, but I'm more interested in seeing if the Logistics Network *HAS* those subcomponents than initially crafting them. I.E. I have a chest with Common, Uncommon, Rare (etc) versions of those components and want to see if I have *enough* of each to start the crafting. Currently, this is N+ comparisons for A+ Comparators where "N" is the number of ingredients and "A" is the number of qualities I want to compare.
Additionally, I don't want to have a literal "bank" of Comparators for literally every list of ingredients of any item in the game that can be crafted. I would like to capture, once and only once, the ingredients I need to simply pass a simple check of whether I have them or not. I'm absolutely fine with the requirement of having a Selector Comparator for each quality I want to test, that's not a problem and is understandable. It outputs only one quality for any input. But there is currently no way to create a latch that catches, and keeps, the exact number of required items without changing the number, or list of items... (that I know of).


Ah, ok, I'm going to stop now. As I said, I don't feel well, which is making me not think well, and, apparently, not see well. So... sorry if I misunderstood (or misread) your post... or was offensive in any way. And, thank you for your help.
Tertius
Smart Inserter
Smart Inserter
Posts: 1082
Joined: Fri Mar 19, 2021 5:58 pm
Contact:

Re: Signal Compression or Encapsulation

Post by Tertius »

Aricitic wrote: Wed Dec 18, 2024 6:26 pm I would like to capture, once and only once, the ingredients I need to simply pass a simple check of whether I have them or not.
With all qualities. Something like this? It inputs one item (here: green circuit) and outputs all qualities of this item with enough quality ingredients in stock.
12-19-2024, 00-26-08.png
12-19-2024, 00-26-08.png (253.68 KiB) Viewed 747 times


On the left, the flow starts. It first iterates over all qualities with the counter C and applies the qualities to the item to inspect one after the other.
The qualified item is given as recipe and the ingredients read back. The ingredients are subtracted from the stock (the constant combinator on the top), and if everything is still positive, there are enough ingredients available.
The combinator on the next to last column is collecting the items that pass the test during one full clock iteration from 0 to 4.
And the 2 combinators on the last column is a memory cell that stores the collected items when C=4, which is the moment all qualities were iterated over.

The 3 +0 combinators behind the quality transfer combinator are for latency compensation, so the correct quality recipe item is available at the same tick as ingredients are checked for availability.

If you transfer the quality to the ingredients instead of transferring it to the recipe item, you can get rid of these 3 combinators, but it's technically not a correct solution, because it gives wrong results if balancing changes take place that require a different amount of ingredients for different qualities. Unlikely as game update, but could happen.

If you want to process a list of recipes instead of just one, you need another counting loop that iterates over this list, switching from item to item every 5 ticks instead of every tick. You need to change the "set" condition for the final memory cell to the additional counter variable instead of the current C, because you want to set the memory cell the moment all items with all qualities were processed completely. This outer loop is probably 2 additional combinators, one for the counter itself and one for a selector combinator to pick the corresponding item.
Aricitic
Long Handed Inserter
Long Handed Inserter
Posts: 57
Joined: Sat Nov 16, 2024 5:30 am
Contact:

Re: Signal Compression or Encapsulation

Post by Aricitic »

Tertius wrote: Wed Dec 18, 2024 11:44 pm
Aricitic wrote: Wed Dec 18, 2024 6:26 pm I would like to capture, once and only once, the ingredients I need to simply pass a simple check of whether I have them or not.
With all qualities. Something like this? It inputs one item (here: green circuit) and outputs all qualities of this item with enough quality ingredients in stock.

12-19-2024, 00-26-08.png



On the left, the flow starts. It first iterates over all qualities with the counter C and applies the qualities to the item to inspect one after the other.
The qualified item is given as recipe and the ingredients read back. The ingredients are subtracted from the stock (the constant combinator on the top), and if everything is still positive, there are enough ingredients available.
The combinator on the next to last column is collecting the items that pass the test during one full clock iteration from 0 to 4.
And the 2 combinators on the last column is a memory cell that stores the collected items when C=4, which is the moment all qualities were iterated over.

The 3 +0 combinators behind the quality transfer combinator are for latency compensation, so the correct quality recipe item is available at the same tick as ingredients are checked for availability.

If you transfer the quality to the ingredients instead of transferring it to the recipe item, you can get rid of these 3 combinators, but it's technically not a correct solution, because it gives wrong results if balancing changes take place that require a different amount of ingredients for different qualities. Unlikely as game update, but could happen.

If you want to process a list of recipes instead of just one, you need another counting loop that iterates over this list, switching from item to item every 5 ticks instead of every tick. You need to change the "set" condition for the final memory cell to the additional counter variable instead of the current C, because you want to set the memory cell the moment all items with all qualities were processed completely. This outer loop is probably 2 additional combinators, one for the counter itself and one for a selector combinator to pick the corresponding item.
Well, that's a heck of a lot smaller than what I had setup... Can't say I'm surprised though...

I'm not able to input the blueprint at the moment, so let me ask what I hope is a simple question: how on earth did you get it to "store" the information without endlessly increasing the count? That's the main problem I always had. A latch would always endlessly count up, or wouldn't store it...
Tertius
Smart Inserter
Smart Inserter
Posts: 1082
Joined: Fri Mar 19, 2021 5:58 pm
Contact:

Re: Signal Compression or Encapsulation

Post by Tertius »

This is the memory cell:
combinator_16229000_416x224.png
combinator_16229000_416x224.png (46.96 KiB) Viewed 680 times


The top combinator passes the incoming signal while S<>0 (or while whatever set condition you define is true) to the output.
The bottom combinator has the output looped back to the input and passes the signal while S=0 (or while whatever set condition you define is false).
So there is always exactly one combinator outputting the signal: either the top or the bottom but never none and never both. The bottom actually stores the signals, while the top keeps it for the set operation.
This 2 combinator memory cell variant became possible with 2.0 because of the distinct handling of red and green wire.
12-20-2024, 10-22-10.png
12-20-2024, 10-22-10.png (279.91 KiB) Viewed 677 times
Aricitic
Long Handed Inserter
Long Handed Inserter
Posts: 57
Joined: Sat Nov 16, 2024 5:30 am
Contact:

Re: Signal Compression or Encapsulation

Post by Aricitic »

Ok, I think I see what I was doing wrong with my setup(s). I always did "If 'each' - blah, blah - -> return 'each'". It never occurred to me that I could simply return everything. I "believed" I had tried to do something similar, but it wouldn't allow me to pass out an argument I hadn't taken in, i.e. each -> each, anything -> anything, etc -> etc.

Thank you for the information.


(however, I still think some form of encapsulation would be useful, and I had other examples, which a "smart" person would have written down... I, uh... did not... I'll reply with them if/when I think of them again.)
eugenekay
Fast Inserter
Fast Inserter
Posts: 165
Joined: Tue May 15, 2018 2:14 am
Contact:

Re: Signal Compression or Encapsulation

Post by eugenekay »

You could implement Code Division Multiple Access over a set of Combinators (implementing Walsh codes and Matrix multiplication would be fuuuun!) if you need to have Simultaneous overlapping signals on one Wire; or possibly Time Division Multiple Access using several Decider Combinators and a “Clock signal” to indicate which channel is currently active.
Aricitic
Long Handed Inserter
Long Handed Inserter
Posts: 57
Joined: Sat Nov 16, 2024 5:30 am
Contact:

Re: Signal Compression or Encapsulation

Post by Aricitic »

eugenekay wrote: Sat Dec 21, 2024 5:04 am You could implement Code Division Multiple Access over a set of Combinators (implementing Walsh codes and Matrix multiplication would be fuuuun!) if you need to have Simultaneous overlapping signals on one Wire; or possibly Time Division Multiple Access using several Decider Combinators and a “Clock signal” to indicate which channel is currently active.
Oh. I'm aware of Bit Manipulation. I vaguely recall that the Arithmetic Combinator has this already, I'd check but Factorio is updating (and I restarted recently and haven't gotten that technology yet... /cry).

Honestly, for almost any problem, Factorio already has a solution. The request is less about "Here is a problem with no solution..." and more about "Here is a solution that is easier and more compact than already exists."

For example: Say you want to count how many asteroid chunks are on a belt or in the Space Platform inventory. Currently, you require 2 to 4 arithmetic combinators to count the combinations - A+B = C and C+D = E (plus one for each other thing needed to count, say ammo, etc), where A, B, and D are asteroid chunks and C and E are alternative signals. But this ONLY gives you the grand total, not the sum of each asteroid, so if you need that AS WELL, you have to have a new signal for that (which, honestly, is easy, so it is a "bad example" but it 'provides' my point).

Ok, how about this instead, you need to know how many asteroid chunks you have in total, but for each quality (since these react and process differently to one another and per quality level). Same as above, but much more than 4 combinators and the signals get mixed much more easily.
Yes, you could certainly shove them all into "A", "B", "C" (etc) and be done with it... but then you have to remember what these signals are... <- THIS could be resolved by allowing signals: "Greater than," "Less than" (etc) as separate signals. And that WOULD BE a solution to what I'm asking for... Point in fact, actually, this already exists for many things, including inserters AND splitters. You can actually just state "quality 'is greater than common'" for a splitter.

EDIT: Ok, about 10+ days later I realized you weren't talking about "Bit Manipulation" and now I feel stupid (again)... I'll check into those, but I still feel that there is a need to have tunnels for this kind of stuff. Like: If a processor can do it IRL why can't a thing the size of a person do it in game?
Last edited by Aricitic on Sat Jan 04, 2025 5:10 am, edited 1 time in total.
Muche
Filter Inserter
Filter Inserter
Posts: 714
Joined: Fri Jun 02, 2017 6:20 pm
Contact:

Re: Signal Compression or Encapsulation

Post by Muche »

My usual solution for summing arbitrary number of signals is one constant combinator listing all signals of interest with value 1, then one arithmetic combinator with Each[R-storage] * Each[G-constantcombinator] -> A.
Outputting Each instead of A works as a signal filter.
Yes, setting all quality levels of interested signals is tedious, but can utilize logistic sections/groups now, so needs to be done only once.
Aricitic
Long Handed Inserter
Long Handed Inserter
Posts: 57
Joined: Sat Nov 16, 2024 5:30 am
Contact:

Re: Signal Compression or Encapsulation

Post by Aricitic »

Muche wrote: Sat Dec 21, 2024 11:53 pm My usual solution for summing arbitrary number of signals is one constant combinator listing all signals of interest with value 1, then one arithmetic combinator with Each[R-storage] * Each[G-constantcombinator] -> A.
Outputting Each instead of A works as a signal filter.
Yes, setting all quality levels of interested signals is tedious, but can utilize logistic sections/groups now, so needs to be done only once.
I need to better understand this, if you'd be so kind.

My current goal with signal compression would, as previously stated, be easily resolved with a signal for each type of recipe. If we had some way of distinguishing between recipe and ingredient then I would have to find a new reason for this to be relevant.

What you are suggesting, if I understand it correctly -- which I may not be since I stayed home sick and it's late -- is unambiguous. What I need is ambiguous. I need something that could yield a result for anything, not just for what I specifically list.

No. I'm not trying to make a machine that crafts anything for me with no work -- I could ask for a mod for that... but a machine that I can place and specify, once, what I want it to craft, then it tells Requester or Buffer chests to collect the ingredients for the unlocked qualities.

-- Again, restarted and am still not to any other planet yet, so focusing on that for now, therefore I don't have any examples of what I *had* been doing as of now.
Muche
Filter Inserter
Filter Inserter
Posts: 714
Joined: Fri Jun 02, 2017 6:20 pm
Contact:

Re: Signal Compression or Encapsulation

Post by Muche »

Here is a blueprint with assortment of circuits. Maybe some of them will be of use for you
Aricitic
Long Handed Inserter
Long Handed Inserter
Posts: 57
Joined: Sat Nov 16, 2024 5:30 am
Contact:

Re: Signal Compression or Encapsulation

Post by Aricitic »

Muche wrote: Sat Jan 04, 2025 6:47 am Here is a blueprint with assortment of circuits. Maybe some of them will be of use for you
Sorry for the extremely late reply, but I finally got around to importing that blueprint. Um, I can vaguely see what it's supposed to do, but would you be so kind as to explain?
User avatar
Khazul
Fast Inserter
Fast Inserter
Posts: 199
Joined: Fri Sep 03, 2021 4:47 am
Contact:

Re: Signal Compression or Encapsulation

Post by Khazul »

I have long thought that ingredients and results of a recipe should be handled by a combinator. It would actually be a good extra function on the selector combinator along with an ingredient ordered means of indexing (ingredient by index and result by index) as the current indexing is based on the amount which makes it impossible to match fluids to machine inputs/outputs automatically. Also there is no current way to read additional results.

Would such a combinator solve your problem?
Muche
Filter Inserter
Filter Inserter
Posts: 714
Joined: Fri Jun 02, 2017 6:20 pm
Contact:

Re: Signal Compression or Encapsulation

Post by Muche »

Aricitic wrote: Sat Jan 18, 2025 7:03 pm
Muche wrote: Sat Jan 04, 2025 6:47 am Here is a blueprint with assortment of circuits. Maybe some of them will be of use for you
Sorry for the extremely late reply, but I finally got around to importing that blueprint. Um, I can vaguely see what it's supposed to do, but would you be so kind as to explain?
Under the AM3 there are two ways how to isolate a signal - indexed by the value, and indexed by the signalID.
To the right is similar indexing, this time the index is not static, but dynamic based on signal I (+timer to cycle its value).
To the left is isolating a subset of signals, based on constant combinator.
On the top left it's getting a sum of values of the same subset of signals.
And on the very left it's just display of signal values, for debugging/demo purposes.
Aricitic
Long Handed Inserter
Long Handed Inserter
Posts: 57
Joined: Sat Nov 16, 2024 5:30 am
Contact:

Re: Signal Compression or Encapsulation

Post by Aricitic »

Muche wrote: Sat Jan 18, 2025 10:04 pm Under the AM3 there are two ways how to isolate a signal - indexed by the value, and indexed by the signalID.
To the right is similar indexing, this time the index is not static, but dynamic based on signal I (+timer to cycle its value).
To the left is isolating a subset of signals, based on constant combinator.
On the top left it's getting a sum of values of the same subset of signals.
And on the very left it's just display of signal values, for debugging/demo purposes.
Ok, thank you for the explanation. I'll take a closer look into it, should be useful for what I'm trying to do.
Post Reply

Return to “Ideas and Suggestions”