PID controller

raidho36 · Post by **raidho36** » Tue Oct 10, 2023 12:23 am

I made this to control nuclear reactors but in principle it could be used for pretty much anything, though there isn't a high demand for that outside of nuclear.

What it does is applies additional power so to say in order to make input values match with the ones specified in the controller. For example you want a steady 20k steam in a nuclear reactor buffer tank. It'll keep getting exactly to it no matter if reactors are idling or running full power or swinging or spiking. The P part makes it hard to get too far away from the desired value. The I part makes it always give it more gas as long as it's still off target. And the D part makes it respond rapidly to fluctuations and especially to sharp spikes. For simplicity's sake a lot of math combinators were omitted at expense of having to work with extremely tiny I values and extremely huge D values, but that's OK, it works all the same in the end.

The values need tweaking though. It might be a good idea to install a fast-forward mod to tweak nuclear reactor control loops because it takes forever for any change to take effect. You can start by setting everything to zero except one of the values.

P value: if affects how strongly it reacts to just being away from desired values. Basically, it's rubberbanding. The further away you are from your target, the harder it pulls. But as soon as you start getting closer, it gets weaker. To tweak it, just make it pretty big, just see that small changes don't make it overshoot. With other values disabled, a good value normally makes it hover at 30-40% better than "nothing" where it's actually just empty.

I value: this makes it creep towards the desired value, the longer it was off target the stronger is the effect. It's like if you run but can't catch up to someone, you run even faster. It's usually a very small value and tweaking it is just a tradeoff between the speed (bigger values) and avoiding hard overshooting (smaller values).

D value: it momentarily gives it a hard kick if it sharply slows down for whatever reason. Like when someone shoves you, you instantly resist most of it and maintain balance, but not beyond that. In a perfect world, simply instantly resisting 100% of the external force would be all you ever need. But in reality there are all sorts of delays and momentum, so you need to pick a good optimal point between not enough reaction and losing balance, and too hard reaction and losing balance in the other direction. So just tweak it until it more or less instantly compensates for sharp changes, but it doesn't overdo it.

Anyway here it is.

: Untitled.png (1.29 MiB) Viewed 3583 times

The amount of combinators is probably suboptimal but on the other hand it can simultaneously control all signals except the 6 used internally (dot, white, P, I, D, X), they all share the same PID values though.

Lite version without clock and filtering.

: PID lite.jpg (107.62 KiB) Viewed 3504 times

Here's an inserter speed controller to go along with it, since it just outputs arbitrary numbers that don't actually mean anything other than "bigger number = faster speed".

mmmPI · Post by **mmmPI** » Tue Oct 10, 2023 5:19 am

raidho36 wrote: ↑Tue Oct 10, 2023 12:23 am For simplicity's sake a lot of math combinators were omitted

Although that's not my favourite math, i think some of it is necessary to understand or to see why it's beautiful because this is quite an advanced contraption

I'll try to fill some gaps with pieces from my recent research all in there, please correct me if i'm wrong :

PID stands for Proportionnal, Integral , Derivative, don't fear the words like i did that's just 3 ways of understanding what is happening in a factory. In a steam buffer for the current example. One can imagine the quantity of steam being measured regularly twice per second all the time. ( in game units this interval is every 30 ticks, and stored in the combinator in the red square "input clock" as the white signal ).

The PID controller require in addition to the cycle lengh a target/objective that is manually choosen by the player. Like 100K steam.
The PID controller is a system that output 1 number , from those 3 ways of looking at a steam buffer quantities.
The PID controller require player to give 3 different weight to those 3 ways. ( the number P I D ).
The higher the PID's controller output number the stronger the indication that you need more , the lower ( negative) the number, the stronger the indication that you have surplus.
Such number is updated every cycle.

The first way to understand is to compare with a target value and just do a subtraction. Say you decide you want 100K steam as a target objective, if the value measured is 200K you have too much ( 100K) , if the value is 75K , not enough ( 25K ) . Pretty simple so far. The result of the subtraction is the error. This operation is done in the "input control" block. ( repeatdly )

Why Proportionnal ? Because then the error is compared with the target, to know if the error is big or not in regard to the target. 100K / 100K = 1 and 25K / 100K = 1/4. That is getting tricky already because 1/4 is not a number one can use with combinators. The idea is to multiply by a big number X but we'll come back to this later.

The second way to understand is wether or not the steam is currently increasing or decreasing. That's the D for Derivative, i put it second because it's the second most logical question to ask. If you have not enough steam, you want to know if it's increasing and it's going to solve itself, or if it's decreasing still and the problem is getting worse.

For this with combinators the clock is used with the 2 frame memory. Here the previous measure of steam is used for a subtraction with the current steam level measured. If the result is positive, it is increasing, if the result is negative. Logic, easy way to make derivative. We'll see how to link with the proportionnal later.

The third way to understand is the Integral, which is obviously the worst when it comes to math. But also because it's the one that takes to the longest time for the machine make use of and for the human to configure since as the name suggest, one has to look at the whole collection of every measure the machine has ever done. That's why there is the accumulator part on the bottom left. This one add the error from the first measure and forever remember them. So if steam stays above 100K for several measurement. the positive error will acumulate. If steam stays under 100K for several measurement, it will remove some of the accumulated error. If it stays longer, it will turn the accumulated error into negative. Of course with combinator positive and negative can be changed easily. With the blueprint the accumulator store a bigger number if there is copper or iron missing, and the number decrease when there is too much copper and iron, so looking at this 3rd way, the machine knows if copper or iron has been a bottleneck or a surplus historically. Reading the explanation for the I value, i suppose that can be interpreted as if you know you were under the target for a long time, you will want to counteract this really strongly, whereas if you went under the target not even 30 sec earlier, you will want to counteract this, lightly for steam.

How do we counteract ? That's related to that huge number X, we're almost there but there is the "rate of change calculator" in the middle of the blueprint, I think that is an "edge detector" but i'm never sure when i see one that it doesn't serve other purpose. What it does is that it detect when the clock finishes a cycle and that the measured value stored in the 2 frame buffer are changed. Because then during 1 tick, the output of those two combinator do not cancel each other. At this point it could also be part of the derivative and i'm confused.

Anyway, we have our 3 numbers, how do we mix them into just 1 ?

wikipedia page gives :
output := Kp × proportional + Ki × integral + Kd × derivative

Super easy ! Just pick Kp and Ki and Kd, thanks to the instructions and play with it in game to see the consequences of the tweakings

That is good enough i think for the PID controller logic.

But that's not exactly what the blueprint is doing as the quote i took suggest i suppose. It is more complicated than that to implement because of the big X number and the signals contamination risk:)

The clock compensator i think is there to avoid loosing value between 0 and 1 like the big X signal. But this time it applies to all signals.

One can understand the "signal filters" as being there because you dont want the time signal to contaminate the end result. That require some filter combinator. You don't want the signal P I or D in the end result either, here another need for filter combinator. Those are not really part of the PID conceptually, more implementation necessity. Unlike the last part :

"PID control" being where player select its PID value and the operations yielding the final results, ~~adding and removing~~ dividing the big X to avoid loosing value when the result of a division could yield between 0 and 1. But it's hard to tell precisely due to the compact implementation where the separation is precisely between what is filter and what is part of the PID operation. The combinator dividing each by X seems like a filter to me or rather outside the strict PID logic, and more implementation related but that's not very important

Hopefully there not too much wrong last time i had to dealt with those math it was for a trading algorithm in the game Bitburner , it has many similarities where you could try to analyze the trend of a stock based on the derivative D to see change of trend, looking at its volatility is the variation compared to the price similar to the P . And the integral could play a role in the model too but one could also be creative to model the time/history factor for trading algorithm : the mean error in a timeframe, the difference between the mean error in the last 10% of that time frame vs the average of the other 10%, the second order derivative in a time frame and so on, i suppose there exist other control mechanism based on slightly different use of math formula.

I learned in real life those kind of math/algo help create models for pricing options, or evaluating one's position and not everyone uses the exact same but their similarities i see more in the control mechanism facing the unknown/stochastic variation like the electric consumption goes up and down, and you want to keep energy buffer by adding or not adding fuel, the prices of stocks and options goes up and down, and you want to keep the net worth by selling or buying.

Again , thanks for sharing ! I hope you don't mind me writing part of a book there, to my defense it's about your creation !

raidho36 · Post by **raidho36** » Tue Oct 10, 2023 3:25 pm

mmmPI wrote: ↑Tue Oct 10, 2023 5:19 am

I'll explain exact purpose of each component. You got most of it right.

Input control just computes the error, by subtracting current value from setpoints in the constant combinator. It does it in real time because it makes no difference and it's a simpler design that way.

Input clock controls memory, but also by extension the entire apparatus. The 3 control paths are not the same length so you don't want it to run every tick, though you probably could since they all run in parallel and the starting points -memory cells- are synchronized.

2 frame memory serves double duty. It holds current value for the P multiplier and is used for the derivative calculation. It feeds directly into rate of change calculator aka derivator, and into P multiplier.

Rate of change calculator, as the name suggests, computes rate of change. It feeds into D multiplier.

Accumulator feeds from 1st frame memory, i.e. present value. It feeds into I multiplier.

PID control blocks multiply respective signals by P, I, D values, add them up, and divide by X. The output is the control value, which is assumed to drive the production of corresponding signal.

Clock compensators are attached to I and D logic lines because otherwise longer clock cycles produce smaller integrals and larger derivatives, which affects PID behavior. They basically just exist so that you don't need to tweak PID values if you decided to tweak clock rate.

PID control values are selected to be large, and then divided by constant X to employ programming trick to use plain integer math as fixed point math, which in practice is close to "real" floating point math we all love (and hate). By selecting values with 5 zeroes and then dividing them by 100 000, we get 5 decimal places of internal accuracy. Derivative's clock compensator had to be staggered, multiplied by D first and divided by clock rate second, because the unit normally produces very small values and they completely vanish if that division is performed first. Accumulator's clock compensator sits after it because putting it before the accumulator leaks magnified clock signals into it, preventing operation. The practical upshot is that if you change clock rate the system loses setpoint and needs to readjust, but it works all the same.

Signal filters just remove the 6 internal signals. They only exist to not clog up the control wire with unrelated garbage. They do "signal * -1" for each of 6 internal signals and mix that into the output, canceling them out.

You don't necessarily need the clock compensators and the signal filters if you're willing to put up with the problems they remove. You also don't necessarily need the clock, so with that in mind here's a simpler smaller version.

: PID lite.jpg (107.62 KiB) Viewed 3505 times

On an unrelated note, stock market technical analysis is wall street version of tarot cards reading. As a competent programmer I explored that thoroughly by the means of compiling all sorts of statistics, all of which suggests there is not even a slim correlation of anything with anything of any kind whatsoever. Maybe a powerful enough AI looking for hidden patterns can do it semi-reliably (just 60% success rate would be enough), but I'm not holding my breath.

mmmPI · Post by **mmmPI** » Wed Oct 11, 2023 1:56 pm

raidho36 wrote: ↑Tue Oct 10, 2023 3:25 pm Clock compensators are attached to I and D logic lines because otherwise longer clock cycles produce smaller integrals and larger derivatives, which affects PID behavior. They basically just exist so that you don't need to tweak PID values if you decided to tweak clock rate.

Accumulator's clock compensator sits after it because putting it before the accumulator leaks magnified clock signals into it, preventing operation. The practical upshot is that if you change clock rate the system loses setpoint and needs to readjust, but it works all the same.

I had not realized that in the original build (amongst other things). I think getting rid of the clock and tweaking the sensibility of the correction by adjusting PID constant make sense to reduce the complexity of the build for the lite version which looks very nice !

I found both difficult to configure anyway and not more difficult if the clock is fixed as in the smaller one, as there are still 3 other parameters to tweak.

I read that manual tuning method could be tweaking P first, so that the initial overshoot is between 10 to 15%. Then tweaking D to reduce that overshoot during the next series of test. And finally tweaking I in other series of tests if necessary so that the initial overshoot is reduced to something between 5 and 10%. I found it helpful.

Another difficulty i have trying to configure PID value is that in order to measure this overshoot one has to wait for the real system to act or make a copy in lab environment and run game at fast speed.

And another one that's troubling me from a math stand point, is that without clock the accumulator increases really fast in real time to the point where overflow is a risk for high target value ; as the static error is added 60 times per second when the system is stable. That translate to an allowance for maximum static error of around 10K for overflow every hour ((2^31) - (60^3) x = 1) Which would cause a system-breaking sign inversion i fear. Which makes me say that the small version is more suited to small quantity not millions of steam in buffer. But also for small quantities where some static error is allowed so which may be rare in vanilla factorio.

Nonetheless

Thanks again for sharing ! Seeing the build and the blueprint is very helpful to understand the mechanism compared to the formula and tables.

On an unrelated note, stock market technical analysis is wall street version of tarot cards reading. As a competent programmer I explored that thoroughly by the means of compiling all sorts of statistics, all of which suggests there is not even a slim correlation of anything with anything of any kind whatsoever. Maybe a powerful enough AI looking for hidden patterns can do it semi-reliably (just 60% success rate would be enough), but I'm not holding my breath.

That's the BlackRock's Alladin "king of thieves" AI you are describing there

, i was refering to specific models that do not try to correlate real life data for prediction, but are more akin to statistical martingale ( as in bettiing strategy ) assuming that day-to-day stock variation are random, stochastic, or analog to brownien geometric motion on a 1 dimension axis, price go up or price go down seemingly randomly. I read the Black-Scholes is one such model used in real life where amongst the fundamental hypothesis there is the "random walk" assumption.

You would probably get it faster if you read from it directly rather than my hobbyist-programmer attempt at explaining a level of the (free open-source & on steam programmation-oriented) game Bitburner

The last model here is one (not-mine) elaborate algo that would be a "top bracket"/google answer for that game i suppose.

In case to put that in context and with simplification there is puzzle in the game where prices of stocks are updated randomly each tick, with a range (40-60%) chance of increase/decrease. This being influenced by another number the "trend" that goes up and down in cycle of unknown period but can also be randomly switch each tick with a lower probability. ( and some players action ) The challenge is to write a script so that you can predict if the "trend" is upward or downward despite the random variation ( like noise against which the PID protect thanks to the integral pat ) and the random reversal of trend ( derivative and proportionnal).

Considering the PID, the math i used for that level were similar, analyzing a repeadly measured value, its derivative, and also the integral , plus some other things i tried. With the aim of providing an output, ( -- / -/ =/ +/ ++ ) supposed to indicate the trend, and some confidence index to take purchase and sell decision. That's where i identify them both as feedback loop control mechanism for "random" process. The PID aim at influencing the result by providing a signal that affect the following measurements.( inserter add fuel) Whereas in the betting algo, you purchase or sell based on those measurement but "wait for a reversal" or do other game actions to influence the trend.

Post by **Koub** » Wed Oct 18, 2023 6:55 am

[Koub] I split brutally the thread in half, at the point where the off topicness started crawling. You'll find the rest of the thread here :
viewtopic.php?f=27&t=109245

Factorio Forums

PID controller

PID controller

Re: PID controller

Re: PID controller

Re: PID controller

Re: PID controller