Switching off

Why operators turn advanced controls off (and how to prevent them from doing so)

THIS article focuses on multivariable predictive control, which has become the standard algorithm for advanced control in many process industries. You’ll currently find it applied in sectors from refining and petrochemicals (where it is typically used in larger applications to control and optimise a whole unit) to smaller applications in the food industry (where it might control a smaller unit such as a dryer or an evaporator).

Multivariable control has the advantage that it is very flexible and can control a process from small to large and optimise it at the same time. The benefits of advanced control are well accepted by most process industries today. By controlling key variables, and reducing their standard deviation, you can shift the mean of key variables towards their constraint, pushing the unit and ‘riding’ the (changing) constraints for the optimum operation. A computer can perform this every minute, 24 hours a day, outperforming even the best process operator.

A good analogy is cruise control on a car which can hold the speed of a car to almost exactly the desired speed in light of disturbances.

As an advanced process control (APC) engineer myself, I share the frustration that engineers feel when an APC that they have worked so hard to get on control is switched off some time later. Often there are genuine reasons why this happens. To get the maximum utilisation and benefits from advanced control applications, it is a good thing to know why operators turn control schemes off that presumably worked when they were first commissioned.

We’ll go through the main reasons why a multivariable control scheme is switched off, and how these problems can be addressed.

APC doesn’t work

On most sites, operators have the ultimate say, and can switch off an advanced control either because it doesn’t work or at least it doesn’t work how they would like it to work.

Sometimes the reason that is doesn’t work is of their own doing but they do not realise that they have made changes that have caused problems.

If it doesn’t work (for whatever reason), then the APC engineer needs to troubleshoot and fix it. On one occasion, I was called in to look at a large optimisation scheme on a FCC (fluidised catalytic cracker) on an oil refinery with a number of controllers; the reactor, regenerator, debutaniser, and a number of other light ends distillation units, all running under an optimiser. The optimiser had been switched off and left off for quite a while. It turned out that nearly all of the basic controllers were constrained so there were no degrees of freedom left for optimisation. The operators had (gradually over time) clamped most of the low and high limits of the flows, temperatures and pressures (manipulated variables) and also many of the constraints (controlled variables) which might include key product analyses, unit temperature limits, safety limits, valve limits, and delta pressure limits (used to indicate flooding) etc.

This is the most common issue that I’ve seen with multivariable controllers. I’ve seen it replicated in oil refining (as in the above example) and in petrochemicals units.

Figure 2 shows a simple example of a distillation column with two key operator handles (top temperature and bottom reboiler flow).

Figure 3 shows that shrinking the operating window by putting in tight limits on the manipulated variables (MVs) reduces the scope for optimisation.

Good training and discipline is needed for operators to resist the temptation to clamp manipulated variables. By clamping in the manipulated (and controlled) variables they are effectively reverting to a kind of ‘setpoint control’ and preventing  the advanced control from having any manoeuvrability.

We will now examine the main reasons why APC may get switched off.

Key constraint is missing

It may be that a key constraint (from an operations perspective) is missing. It may have been missing from the start or it could be a new variable that has become a constraint. It is important to consult the operations team during the design of advanced controls so that they share the ownership of the APC solution that is developed. It might also be that the APC moves a combination of manipulated variables to relieve a constraint that they do not like. Perhaps the APC is doing the right thing (and it’s more of a training issue) or it might be that the APC needs some ‘know how’ built into it (for instance the priority of moves or the priority of constraints) that might need adjusting based on operator experience or objectives that might have changed over time.

During the functional design of an application, one of the key parts of the design procedure should be to sit down with a number of shift operators and go through how they run the unit (in manual), which manipulated variables they use, which constraints they monitor, and what they do to relieve those constraints.

It can also be the case that a model is wrong or missing from the matrix.

Multivariable controllers are based on a dynamic matrix of manipulated (and disturbance) variables against controlled variables (constraints)

A typical small matrix is shown in Figure 4. Wherever there is a relationship between variables, a model should be present.

Perhaps a model has been missed out? This might give poor controller performance.

Operators don’t understand how it works

Whilst some operator training is often carried out as part of a good APC project, these days, this might not be very comprehensive. In the best situation, operators are taken out of the shift system for a training day where they are properly trained in how to get the most out of APC using a process simulator, usually running in a standalone computer.

It is quite reasonable that if the operators don’t know how the APC scheme works, then they would switch it off. They should have been trained as part of the commissioning process but if a shift is accidentally missed out then they have a legitimate reason to switch off the APC.

I have been to sites where a new operator has come onto shift who has not seen the APC before. They might have been off for a long break and missed the training or they might be new to working on the panel, having spent most of their time on the outside unit.

Either way, the engineer who is commissioning the APC should have a list of all the operators and shifts that they need to train in order to ensure that all shifts are up to speed with the new application. In this case, we have a fairly simple problem to solve. It is a training issue. There are two main options for operator training:

• At-the-panel training. The engineer sits at the panel when the operator is on shift and explains to them how the APC works, how to switch it on and off, change constraint limits, drop non-critical variables etc, working their way through all of the shifts and all of the shift operators. Sometimes a new operator comes on shift who never went through the training when the application was first put on control and so missed the training. Or an outside operator who normally does not work on the panel is called in for some reason to provide cover and has not yet been exposed to the APC. These too need training.
• Classroom training. A (typically) one-day session in a classroom using a combination of presentations and practicals (using process simulations, preferably including some of the operators’ actual units). It is a great way to train operators and a great supplement rather than a substitute for training at the panel.

Wrong Model/Model is out of sync with the process dynamics

Given time, a model may drift. When the model was originally identified, it may have been correct, but over time, the process may have changed. For instance, the unit may be running at a different feed rate and the model dynamics may be sufficiently different to cause some degradation in the model quality to the extent that the model does not work well enough for the operator’s satisfaction. It is important to always do a sanity check on models to check that they make sense from a process point of view and not to just trust the results that are obtained from a computer identifier package. A computer package can statistically derive a model that is ‘incorrect’ for a number of reasons, for example, where gains for mass in and mass out do not perfectly mass balance properly (due to meter errors).

In Figure 5, the multivariable controller’s performance is very much dependent on the quality of the models. The actual measured value needs to coincide reasonably well with the prediction. If there is a large difference in the magnitude of the changes (gain) or in the dynamics (deadtime and lag) then the performance of the controller may be reduced.

Wrong tuning

The tuning of a multivariable controller will affect the performance. This is especially true of the key tuning on the controlled variables (CVs) and the manipulated variables (MVs). In the case of the CVs, multivariable control packages have weightings to set the priority of the CVs so that the most important CVs are controlled in the situation where there are insufficient degrees of freedom to control everything. These weightings have to be set correctly. In the case of the MVs, there are also often weightings to preferentially use some MVs for control and others for optimisation and/or to reduce the movement of some MVs which should not move too much nor too fast. Careful setup of the tuning is therefore very important.

Inferentials

Many multivariable controllers make use of soft sensors or ‘inferentials’ to infer key product qualities. Very often, developing good (and representative) inferentials is very important to developing a good controller if these inferentials are key constraints on the unit.

Nonlinear

If the controller is running a unit which experiences noticeable nonlinearity within the normal range of operation, then it may be that the controller works well some of the time, but when it enters the nonlinear region, it does not perform as well. In this case, it may need careful tuning to cope with the nonlinear region (compromise tuning) or else a more sophisticated nonlinear transform might be needed to enable good performance across the operation range.

Wrong optimisation

The optimisation case for a controller is normally set up as a linear program (LP) or a quadratic program (QP). It is important to set the optimisation up correctly or else the controller will push to the wrong constraints.

Controller too cumbersome

It is well accepted that it is easier to step test, commission and maintain smaller controllers than larger controllers. When a problem occurs there is a tendency for the operators to switch the entire controller off. They will not always take the trouble to isolate a problem and keep the rest of the controller running. A large controller with many variables is more likely to be switched off (unnecessarily) than a smaller controller.

Some multivariable control packages allow ‘sub controllers’ to be set up to break up a large control problem into smaller chunks.

Lack of maintenance

Many of the above factors can be summed up as lack of maintenance of the application. Good maintenance for advanced control is essential for continued good performance. The tuning, performance and particularly the model quality needs to be monitored, with adjustments made over time. There are some automated tools for this such as automatic step testing tools that carry out a partial re-step if a model drifts too far off. Commercial multivariable controllers can tolerate a certain amount of model error but control performance does suffer as the model match with the process drifts further apart.

Doesn’t make their life easier

Any good APC scheme should make the operator’s life easier by taking care of the control of a part or whole of a process unit. If an APC does not really help run the unit better, then it is likely that the operators will switch it off.

Prefer to do it themselves (don’t like cruise control)

I’ve sometimes seen operators switch off an advanced control simply because they prefer to run the unit themselves, changing setpoints and observing how the process responds ‘in manual’. Cruise control on a car is a good analogy. Whilst we might understand this (some people don’t like using cruise control  because they prefer to ‘drive’), it is clear that on a clear motorway with little traffic, it controls the speed more accurately than a human (and reduces the risk of speeding).

By setting the speed close to the maximum (for example 69 mph on a 70 mph motorway) the driver can get close to the constraint limit without actually exceeding it. The analogy is a good one because if there is an abnormal situation (traffic jam or slow moving traffic), then the driver can revert to manual control. Likewise, on a process plant, if there is a major disturbance or shutdown then the operators can revert to controlling the plant manually.

In this case, training the operators, showing them how advanced control can benefit the smooth and efficient running of the process unit, and how it can make their life easier by taking over a manual task, usually helps.

Option of fixing limits

Finally, there is an option of only allowing operators to turn an APC scheme on or off and possibly changing a few key constraint limits.

There are two schools of thought on this. Depending on the site and the operators, one approach might work better than the other.

The first school of thought is to give the operators the ability to change CV and MV limits, to drop and undrop CVs and MVs (that are non-critical). In other words, to give them as much freedom as possible. Most plants fall into this category.

The second school of thought is to lock down some parameters, for example some low and high limits. The operators are limited to switching the APC on or off and changing only essential limits, required for operation. The obvious advantage is that they cannot ‘clamp in’ a controller’s limits detrimentally, defeating its ability to control the unit. Also, if they switch the APC off, it forces the engineers to address the problem properly. The downside is that it may reduce good operators’ ability to be creative and think about more active constraint monitoring. So whilst few sites operate this policy, it does prevent operators from clamping MVs.

Conclusion

Involving operators from the start of a project, training them well and then setting up a good maintenance system is an essential part of retaining your investment in advanced control.