The Cloud DNNC is a project to replace the PID controller from the most common control systems: simple in simple out (SISO) systems with first order plus dead time FOPDT and second order plus dead time (SOPDT) system behavior. Since the DNNC does not require any parameterization, it is very easy to use and no in-depth knowledge of the given process parameters is necessary. By considering the process as a black box, it is even possible to provide a cloud controller as a service. Otherwise a model of the process and controller parameters derived from it would be required.
Of course, the question of what makes sense immediately arises, since process security goes hand in hand with the availability of my server service. Therefore, everyone should be clear: this service is a playground. You are welcome to connect your simulated or real process to the controller and carry out various tests. I do not guarantee service availability. I would then be happy to receive your experience reports so that I can further optimize the controller.
The neural network is permanently optimized in the cloud on various CPU and GPU resources. I will inform you about updates and improvements of the DNNC on this blog.
My final goal is to port the network to a CortexM4 microcontroller. The “deep” in deep neural network is therefore limited by the resources of such a microcontroller.
The following graphic shows the closed control loop:
It is actually as simple as it looks. Basically, the service expects an HTTP GET request with the setpoint (setpoint SP) and the actual process value (y (t)). The controller output (u (t)) that is to be applied to the actuator is returned. In order that everything from level to temperature control can be covered, the input and output variables must be scaled and transmitted without units. The value ranges of the DNNC should always be used from 0 to 100%. The same procedure can be found in many industrial controllers.
Process example: temperature control for soldering furnace
Temperature range: 0 ° C to 260 ° C => 0% = 0 ° C, 50% = 130 ° C
Actuator: Triac with full wave control (0 to 50) => 50% = 25 full waves
The DNNC will suggest a manipulated variable with a lot decimal places, but in many cases this can simply be rounded. However, it should not be pushed so far that the DNNC is used as a two-point controller (> 50% = on, <= 50% off); this can work but gives poor controller results.
Comparison of PID and DNNC:
As can be seen, the DNNC loses out against a well-coordinated PID controller, but it can score if the process changes and most processes do: Filters become clogged with dirt, flow temperatures are different, fill quantities vary, etc. The selected parameters in the graph show the generalization of the controller.
In principle, any programming language can be used with the help of the HTTP API. I provide a Python script with a class and an example to simulate a process system using the scipy package:
If you want to control a real process with Hardware in the Loop (HIL), you are cordially invited. Please note the ping times of approx. 70 ms lead to another dead time. It also occasionally happened that requests received a timeout + retransmit and the response came from the server only after four seconds. I therefore recommend regulating “lazy” processes and sending inquiries every second or fewer.
Function and behavior of the controller
The neural network receives the input and output values of the process system as inputs and tries to „estimate“ the further behavior. It therefore approaches the task and is inferior to a controller with a known process model with regard to control speed. With varying process parameters, the DNNC has a clear advantage. However, the possible process parameters also have their limits.
The design of the DNNC is a compromise between covering additional process parameters or faster setpoint tracking. If the narrow parameters for a given process are known, a “specialist system” could easily be learned. However, this has also been possible for a long time with Matlab tools and some other systems.
As already mentioned, the DNNC starts with no knowledge of the process when sending the first control variable. As a result, it can only try a very small actor output, there is a risk that the process gain is very high and the dead time is long. As result a large overshoot would be likely. After the DNNC has got a „feeling“ for the dead time of the system after 4 to 5 values, the control variable can be managed more aggressively.
The neural network is trained with the help of reinforcement learning in the Amazon (AWS) cloud. In regular stages, I have several hundred systems evaluated and use them to calculate an average. I plot the systems that got an (abstract) rating below 0 and compare the system parameters.
A few numbers to get a feeling for the rating function:
- If the control output is randomly varied between 0 and 100%, the rating is -835.
- If the setpoint 0.0 is specified and the controller remains at 0.0 u (t), a best value of 904 can be achieved.
- Systems that have a large time constant and high damping cannot achieve a rating above 0, in spite of the perfect controller.
The grid score for the currently hosted network from 02.02.2020 ( DNNCv1.0 ) = 319.7 :
Dead times with four time steps and more should therefore be avoided. Optimizations for this are still ongoing. If you look halfway through the graph, you will see: Brown and purple on the way to the 100% setpoint were rated poorly (<0.0), even though the controller was almost fully open. Accordingly, the evaluation scale is certainly in need of improvement, but is sufficient to compare different training processes.
Furthermore, it should be considered that these systems are difficult to control (Tu / Tg <1/5) and that a well tuned PID is already reaching its limit:
Just looking at the worst system is frustrating too. Here are all systems with a GridScore of over 500:
As you can see, no „setpoint 100 %“ system reaches this score, for purely mathematical / physical reasons.
Process controller without parameterization? Only half the truth!
In fact, no tuning parameters need to be set, but scaling is necessary. This has a large influence on the stability of the DNNC. The route reinforcement refers to the scaled area. If, for example, a heating coil is available that could heat a water pipe to 100 ° C and it is only a matter of keeping a pipe free of ice (setpoint ~ 4 ° C), it would be more appropriate to set the control value range of the DNNC (0 to 100%) only to 10 or use 20% heating power to stay in the stable gain section. Otherwise the process path can also be scaled from -10 to 100 ° C, but this leads to somewhat poorer setpoint tracking.
In addition to the scaling that has an impact on the route gain, the time behavior must also be considered. If the trace heating mentioned above (which certainly does not become a water heater, as I exaggerated) shows, only after two hours in the work area, no controller values should be requested from the server every minute. The dead time until there is a temperature change at the sensor would be much longer than four time steps in this example.
So the user needs a rough clue about his process and maybe he even has to record a step response. Is the DNNC just a bluff? Ultimately, this is decided by practical use! The DNNC starts to regulate a very wide range of process parameters in order to be able to cover a great deal for everything from seconds to days, from oversized to undersized actuators. This requires scaling with some background knowledge of the process.
The DNNC will continue to be optimized by me and will be updated occasionally. I will see in the next few weeks how stable my web service will run if there are many inquiries. Here I still lack experience.
A Jupyther notebook is already being planned, which will present a few simulations and should still close some user knowledge gaps. In addition, I have not yet shown how robust the DNNC reacts with 1% measurement noise and load jump, etc.
Unfortunately, the DNNC was unable to regulate the cooling water of our fuel rods. Where can I claim the costs caused by the super meltdown?
Please use this service at your own risk and set sensible critical switch-off thresholds beforehand.
My self balancing robot keeps falling over. What am I doing wrong?
- no PTxDT system – could work, but not tested
- I estimate the offline version of the controller with 200 µs computing time on a CortexM4. For this online version, I recommend trying less gravity.
Is the project open source?
No, the controller itself is not open source. However, I will publish some marginal areas of it, for example the simulation of PTxDT systems with Python and later the framework of my flask backend. So it remains interesting for hobbyists.
Can the DNNC also be optimized according to the actuating energy?
There is currently no effort on my part to do so. Possibly. an additional low-pass filter can be inserted between the DNNC output and the actuator, but this will lower the control quality.