Pmceedings of the 1999 IEEUASME

International Conference on Advanced lntelligent Mechatronics

September 19-23, 1999 Atlanta, USA

Three-way Catalytic Converter Modelling:

A Machine Learning Approach for the Reaction
Kinetics
Luigi Glielmo, Stefania Santini
Dipartimento di Informatica e Sistemistica, Universita di Napoli Federico I1
via Claudio 21, 80125 Napoli, Italy
E-mail: {glielmo,santini}Odisna.dis.unina.it

Michele Milano
Institute of Fluid Dynamics, Swiss Federal Institute of Technology (ETH)
Sonneggstrasse 3, 8092 Ziirich, Switzerland
E-mail: milanoQifd.mavt .ethz.ch

Gabriele Serra
Magneti Marelli Divisione Controllo Motore
via Timavo 3, Bologna, Italy

Abstruct- In this paper we present a novel approach sions, where reaction rates depend on concentrations
to the problem of three-way catalytic converter dy- and temperature, should yield a model sufficiently fiex-
namic modeling; one of the main issues related t o mod-
eling these devices is the reaction kinetics submodel, ible to match the real behavior of a large variety of
that has to be at the same time simple and flexible catalyst formulations and washcoats.
enough t o capture all the significant features of the real Unfortunately, the lack of precise kinetic measurr-
system. We propose the use of machine learning tech-
niques to solve this problem: a neural network struc- ments, especially during transient warm-up phase,
ture for the kinetic submodel and a genetic algorithm makes the choice of suitable reaction kinetic expres-
to tune its parameters. In this way the difficulties aris- sions and the tuning of the parameters probably the
ing from the identification of the resulting overall model
are avoided.
most crucial aspects for model reliability.
Thus, there is a need for a method allowing the es-
timation of the kinetic data from more easily available
I. INTRODUCTION experimental data, like pollutant emission measure-
Stricter legislation on engine exhaust gases makes ments at the inlet and the outlet of the convert,er.
significant the optimization of the three-way catalytic Since the reaction kinetics are not straightforward
converter (TWC) dynamical behavior. From this point to model, it is convenient to use a black-box approach
of view, an important task is to predict the transient which allows also an easier compensation for possible
TWC emissions (converter output) given the engine noise that could be present in the input-output data.
output for any driving cycle of interest. Indeed the Neural network (NN) models are particularly suitable
most important part of the cycles, as regards air pol- for the purpose of dealing with these problems [5].
lution and thermal stress, is the warm-up phase. Taking into account the reduced order dynamical
Numerical simulations of the three-way catalytic model of TWC presented in [4], here we implement
converter could advantageously substitute expensive a neural network submodel for the reaction kinetics
experiments with an engine on a test bench. More- deeply embedded in the TWC model structure.
over, the design of warm-up controllers and real time Since both the lack of data available to directly train
application demand for simpler, reduced order models the NN and the model architecture make difficult to
(see, for example, [3], [4]). Simplified reaction scheme define a parametric gradient, necessary for the stan-
and rate expressions [ll],[14] are used in most models dard backpropagation technique, a genetic algorithm
111, [2], [3], [4], [la]. These empirical kinetic expres- is used here to train the NN submodel without the

0-7803-5038-3/99/$10.000 1999 IEEE 239

 need for a gradient computation. +
C ~ H G 4.502 + 3C02 3Ha0,+ (2b)
CH4+202 -+ C O , + 2 H 2 0 , (2c)
11. A DYNAMIC
MODELOF T W C CO+NO -+ CO2 +0.5Na, (2d)
In this section we present the dynamic TWC model H2+0.502 -+ HsO. (2e)
described in detail in [4].
Considering p chemical species taking part in q cat- For purpose of control design and real-time implemen-
alytic reactions, it is possible to describe the transient tation the detailed model (1) is too heavy from a com-
thermal behaviour of the TWC through the following putational point of view. To overcome this obstacle a
set of partial differential equations (PDEs) [l]: simplified model has been developed.
We remind that the polluting gases flow out of the
engine, they pass through the TWC arid are converted
into relatively harmless gases. Usually the converter
is a ceramic monolith (tubular reactor): gas flows
through the passes and is adsorbed by the substrate
where reacts. Only on this substrate the catalytic reac-
tions, minimizing the pollutant emissions, can happen.
Our numerical simulations .with model (1) have
shown that the adsorption phenomenon is significant,
and X g , i - X,,i -+ 0, for i = 1,. . . , 6 , along the c.011-
verter. Then a possible way to simplify the model is to
(:niphasize this behavior by letting, in the rnotlel (1),
the xlsorption paramet,ers. k:n,i teiid to infinity and,
hence, xg,zz X,,j = xz
for i = 1,. . . , G .
Taking into account the above considerations, and
The above equations describe, respectively, the energy letting x = ( X I , . . .,xG)* ant1 R = ( R I , . . . R , ( ; ) ~ ~ ;
equilibrium in the gas phase, the mass equilibrium in the simplified model follows:
the gas phase for chemical species i, the energy equi-
librium in the solid phase ,and the mass equilibrium
in the solid phase for species i. Moreover, Tg and T,
. are, respectively, the temperature of the gas phase and
the reactive surface (substrate); X,,2 and X,,, are the
concentrations of the i-th species in the gas phase and
along the substrate expressed in [mol/m3] units; k ~ , ,
is the mass transfer coefficient (from the gas phase t o
the substrate) for species i ; U,,, is the mean gas velocity ax =
(1 - E ) - -U??,(~,X)-
ax - R ( X , T s ) .( 3 ~ )
in monolith; R, is the specific reaction rate for the i-th at a2
species and RI is the specific reaction rate for the I-th The boundary and initial conditions are ( t 2 0,u: E
chemical reaction; the independent variables t and x
[O, LI)
are respectively the time and the axial length along
the monolith; the various coefficients are illustrated aT,
in the Appendix. The specific reaction rate for the i- -(t, L) = 0 (Adiabatic constraintiQa)
at
th species R, and the specific reaction rate for the 1-th
chemical reaction RI depend in a nonlinear way by the
temperature and the concentrations of the chemical
species on the reactive surface, i.e. R, = R,(T,, X,,,),
RI = R:(Ts,Xs,z).
The chemical model adopted includes six chemical
species (CO, C ~ H GCHq,
, NO, Ha, 0 2 ; hence p = 6) where T,*(t) and X ; ( t ) = (X,*,l(t),. . . ,X;,,(t))T are
[l];they take part into five chemical reactions (q = 5) respectively the temperature of the exhaust gas and
that include the oxidation of CO, Ha, HC, as well as the concentrations of the chemical species at the in-
the NO reduction: let of the TWC, T,*(x) is the initial temperature of
the substrate, X * ( z )’= ( X ; ( z ) ,. . . , X ; ( X ) ) are
~ the
C0+0.502 + Con, (24 initial concentrations and L is the T W C length.

240
 111. THENEURALNETWORKEXTENSION
OF THE as = ( X C O r X C ~ H s , ~ C H ~ , X N 0 , X H 2 , TzsE, ~ m ) T ;
U
TWC MODEL IR7 is the output vector of the first layer; W , E IR7x7
The neural network extension of the model (3) has and W2 E IR5x7 are the network weight matrices;
been developed using a NN representation of the spe- bl E R7 and b2 E R5 are the bias column vectors;
cific reaction rates vector R’ = ( R i ,. . . ,Rb)T (equa- and the tanh(.) operator is intended component-wise.
tion (3b)); in this way the neural network is embebbed The mean gas velocity in monolith U,,, has been in-
into the model equations (3). cluded in the input vector U because previous identifi-
The kinetic model (see Figure 1) has the following cation experiments of the system have shown that this
further information improves the performance of the
structure:
whole model.

n
where D = diag(Xc.0, X C ~ H XC~ ‘, HXNO,
~ , X H ~ )E
R N NE IR’ is the output vector of the NN;T, RNN,1
is the temperature of the reactive surface (substrate)
and Tt is a prefixed threshold of the temperature. RNN, 2
On the basis of the reaction kinetics, according to
RNN, 3
the chemical scheme (2), it is straightforward to obtain
the vector of the rates of the chemical species R =
RNN, 4
( R I , .. . ,R G )(equations
~ (3c)) since it is related t,o
RI .
RNN, 5

Fig. 2. Two-layer network architecture (biases not shown for

clarity).

dNN I

Fig. 1. Structure of the reaction kinetics submodel.

As shown in (5) and Figure I, R’ has been com-

IV. THEINTEGRATION
ALGORITHM
The integration algorithm, described in detail in [4],
is based partly on a space-discretization, partly on the
‘method of characteristics’; the separation being al-
lowed by a two time scale analysis of the system.
The structure of the system (3) (see Figure 3) can be
interpreted as the coupling of a thermal subsystem and
a chemical subsystem (respectively equations (3a)-
(3b) and equation (3c)).
puted from the outputs of a fully connected multilayer However, noticing that, in practice, the thermal dy-
feedforward neural network. The mathematical model namics described by (3a)-(3b) are much slower than
of such a NN is completely defined by the number of the chemical dynamics of (3c), it is possible to set up
layers, the specification of the output function for the an approximate integration scheme, based on a sin-
neurons in each layer, and the weight matrices for each gular perturbation approach [7], that allowed us to
layer. In our case the two-layers neural network used decouple the integration procedure:
in the kinetic model can be specified as follows (see equations (3a)-(3b) have been solved using a fi-
Figure 2): nite difference scheme (‘method of lines’) [SI. The
distributed parameter model is converted into a
.z = tanh(Wlu+bl), (64 lumped one by a finite difference scheme, thus
R
” = W2x+b2, (Gb) considering a discrete number of spatial elements,
each described by time-varying variables.
where R
” . . ,, R”,5)T is the output vec-
= ( R N N ,. ~ Using the ‘method of characteristics’, the prob-
tor of the lem of solving the ‘quasi linear’ hyperbolic PDEs
NN; U is the input vector of the neural net defined (3c) of the chemical submodel reduces to that of

24 1
 the whole model and the measured experimental data:

where ylk is the measured output; yIz is the simula-

+
tion result; h refers to the output channel of the T W C
model (temperature and concentration of the chemical
species all evaluated at the outlet of the TWC); VI,, are
i scaling factors; ti, and tfi,, are respectively the initial
and final time instant of the simulation.
The trainidg process employs a Genetic Algorithm
(GA) [lo], obtained by a modification of the Con-
trolled Random Search (CRS) algorithm by Price [13].
, ., ~- The CRS algorithm has been demonstrated to be ef-
fective in solving global optimization problems [9], but,
.Fig. 3. Structure of the system. often with too long convergence time due t o the large
population size necessary to avoid local minima. A
considerable improvement of the CRS algorithm for
solving a system of ordinary differential equations both the convergence time and the reliability of' the
(ODES) [6]. solutions is presented in [lo]. The CRS can be seen as
a Genetic Algorithm with zero mutation probability.
V . THEIDENTIFICATION
PROCEDURE Within the framework of the CRS a variable muta-
tion operator was designed. In virtue of this context-
In the TWC model the neural network is merely a dependent mutation operator, this GA needs a much
simple nonlinear mapping between the input and out- smaller population size than that one necessary to
put quantities of the kinetics submodel. The NN con- CRS to properly converge, so that the convergence
tains a certain number of parameters (weights W and time can be much smaller, too. It was also demon-
biases b ) , whose final values are determinated during strated that the cross-over operator best exploits the
the training period. fitness function regularities, much like a gradient based
To train a static multilayer feed-forward neural net- algorithm, even if the gradient itself is never needed.
work the standard backpropagation algorithm could The identification procedure uses as experimental
be used [ 5 ] . However for the backpropagation algo- data the first 800 seconds of an ECE cycle. The follow-
rithm to work properly, it is necessary to have the ing table gives a better insight into the identification
desired output for the NN available. In our case the procedure:
desired outputs are the reaction rates; since these data
are not directly available, the NN model has to be I Two-layer NN
embedded in the TWC equations (3) and the desired
NN structure 7.7.5
outputs become the outputs of the whole model. This
Number of generations
means that one should take also the model structure
Best fitness
into account when writing down the updating rule for
Number of parameter
the network parameters. This is not an easy task with
Number of population elements
our model, and furthermore, from some previous iden-
tification experience, it turned out that the model it-
self is quite difficult t o be identified using a standard RESULTS
VI. SIMULATION
least-squares technique, even without the presence of The simulation results are shown in Figures 4-8. In
the N N extention. all figures a comparison between real d a t a and the
All these reasons suggested to use a stochastic algo- TWC model output is reported; the outputs result-
rithm, since it performs a full exploration of the whole ing from the validation phase, starting after 800 sec,
space of solutions, thus avoiding local minimum points, are plotted together with the results obtained during
without the need for any gradient information. the training.
In this context, a learning rule is used to adjust the Results show a good fit between the model output,
weights and the biases of the NN in order to mini- and the real behavior of the TWC for the temperature
mize the sum-squared error between the outputs of dynamics, also in the validation phase. As regards the
 T i m (3)

Fig. 4. T, time history. Fig. 7. CH4 time history

2500

,, , ,-
- - . - ,,,,,....,,
*...,..,
,.,1,,1,1,,.*..,,
,",,,i 1/1

I I , I I ~ , I
I I 1 I I , , I
,( I I, I , ~ I1

-05' J -500'
0 200 400 600 800 1000 1200 1400 0 200 400 600 800 1000 1200
Tlme 151 Time [r]

Fig. 5. CO time history. Fig. 8. NO time history.

concentrations, we can see that in all cases t,he pre-

lightoff phase is well approximated, even though the
quality of the training data in that region is not partic-
ularly good. Most important, the model always cap-
tures correctly the transition between the pre-lightoff
and the post-lightoff phase. After the lightoff phase
the catalysis effect is reproduced reasonably well, also
with the validation data. Better results could be ob-
tained using more experimental data in the training
phase of the Neural Net.
We point out that:
the algorithm has been developed on Matlab
-1000'
0 200 400 600 800 1000 1200 DO
5.2/Simulink 2.0 environment with the support
Tme ($1 of C compiled S-function to shorten the compu-
Fig. 6. C3H6 time history tational time;
as an input we used the first 1200 seconds of' an

243
 ECE cycle (experimental data); [8] Lin, C. C., and L. A. Segel, Mathematics Applied to Deter-
the simulation of 1200 real seconds of the warm- ministic Problems in the Natural Sciences, Society for Indus-
trial & Applied Mathematics, 1988.
up along an ECE cycle takes 5 sec. on a on a PC, 191 Milano L., Barone F., Milano M., ‘Time domain amplitude
Intel Pentium I1 350MHz Processor, 32 Mb,RAM. and frequency detection of gr@vitational waves from coalesc-
ing binaries,’ Physical Review D, vol. 55, n. 8, pp. 4537-4554.
[lo] Barone, F., M. Milano, ‘An Identification Procedure per-
VII. CONCLUSIONS forming Automatic Sensitivity Analysis based on Evolution-
ary Programming,’ submitted t o IEEE ’IYansactions S’ystems
We presented some preliminary results relative to Man and Cybernetics.
the use of machine learning techniques to include a [ll) Montreuil C. N . , S. C. Williams, and A. A. Atlarnczyk,
Modeling Current Generation Catalytic Converters: Labo-
NN reaction kinetics submodel into a TWC model. ratory Experiments and Kinetic Parameter Optimization-
To overcome the problems arising from the resulting Steady State Kinetics,’ SAE paper 920096, 19%.
overall model structure, we used a GA to train the [12] Oh, S. H. and J. C. Cavendish, ‘Mathematical Modeling
of Catalytic Converter Lightoff. Part 111: Prediction of Ve-
NN submodel without the necessity of gradient infor- hicle Exhaust Emissions and Parametric Analysis,’ AICliE
mation. Journal, vol. 31, No. 6 , June 1985, 943-949.
[13] Price, W. L. , ‘A controlled random search procedure for
Further improvements of this model will be explored global optimization,’ The Computer Journal, vol. 20, 11. 4,
by looking at more sophisticated NN architectures. 1976, pp. 367-370.
[14] Voltz, S. E., C. R. Morgan, D. Liederman and S. M. Jacob,
‘Kinetic Study of Carbon Moiloxide and Propylene Oxidation
V I I I . APPENDIX on Platinum Catalysts,’ lrid. EJIE.CXem. Prod. lies. Dev..
J/kg I< specific heat capacity of .gas Vol. 12, No. 4, 1973, pp. 294-301;
J/kg I< specific heat capacity of substrate
mol/inZs specific reaction rate for species i
mol/rn2 s specific i.eaction rate for the chemical reactioii 1
m2/m3 active area/volume ratio of the monolith
W/m’ K convective heat transfer coefficient
(from g i s ’ t o substrate) .
W/m2 K heat transfer coefficient
m/s mass transfer coefficient for species i
m2/m3 external area/volume ratio
K ambient, temperature
m/s mean gas velocity in monolith
J/mol heat of ith-reaction
void fractioii. , .

kg/m3 pas density

kg/m3 substrate cletisit,y
W / m I< substrate thermal conductivity
(from substrate to ambient)
mol/m3 gas phase concentration of i-th species
mol/m3 solid phase concentration of i-th species

R.EFERENCES
[l] Abate, G., L. Glielmo, P. Rinaldi, S. Santini, G. Serra, ‘Nu-
merical Simulation and Identification of the Dynaqic Be-
haviour of Three Way Catalytic Converters’, ICE97-3rd In-
ternational Conference on Internal Combustion Engines:,-Ex-
periments and Modelling, Capri, Italy, 1997.
[2] Baba, N., K.. Ohsawa, and S. Sugiura, ‘Numerical Approach
for Improving the ConversiowCharacteristics of Exhaust Cat-
alysts under - Warming-Up Condition,’ SAE paper 9620 7G,
1996.
[3] Glielmo, L., P. Rinaldi, S. Santini, %. Serra, ‘Modelling and
Numerical Simulation of the Dynamic Behaviour of Three
Way Catalytic Converters’, Proc. IEEE Conference on Con-
trol Applications, Trieste, Italy, September 1998.
[4] Glielmo, L., S. Santini, G. Serra, ‘A Two-Time-Scale
Infinite-Adsorption Model of Three Way Catalytic Convert-
ers’, t o be presented at American Control Conference, San
Diego, U.S.A., 1999.
[5] Haykin S., Neural Networks, McMillan College, 1997.
[6] .Jeffrey, A., Quasilinear Hyperbolic Systems and Waves, Pit-
man Publishing, London, 1976.
[7] KokotoviC, P. V., H. K. Khalil and J . O’Reilly, Singular
Perturbation Methods in Control: 4naJ,ysis and Design, Aca-
demic Press, London, 1986.

244