\documentclass[a4paper,11pt,twoside]{nlmimodoc}      % Specifies the document
\usepackage[dvips]{epsfig}    

                             % The preamble begins here.

%\pagestyle{empty}
\pagestyle{headings}
\textwidth              16.0cm
\hoffset                -1in
\oddsidemargin          2.5cm
\evensidemargin         2.5cm
\voffset                -1in
\topmargin              1.0cm
\textheight             25.1cm

\title{\huge\bf
FAMIMO Common Software Documentation ($\alpha$2.0)\vspace{1cm}}
\author{\bf Antoine Duch\^ateau, Edy Bertolissi\vspace{.6cm}\\
\small IRIDIA, Universit\'e Libre de Bruxelles\\
\small 50, av. Franklin Roosevelt\\
\small 1050 Brussels, Belgium\\
\small Tel: +32 2 650 27 29, Fax: 32 2 650 27 15\\
\small email: bersini@ulb.ac.be}  

\date{}      % Deleting this command produces today's date.


\begin{document}             % End of preamble and beginning of text.
\parindent=0cm

\maketitle                   % Produces the title.

\include{abstract}

\include{abstract}
\include{introduction}
\include{tutorial}

\chapter{API Reference}

\section{Data Manipulation Routines}
Data collected from real processes often need to be pre-processed
before being used for modelisation or control. Filtering, elimination
of outliers, selection and combination of data coming from different
sources are common crucial tasks that have to be carried on at an
early srage in the modelisation process.\\

The {\tt dataset} class is used to handle the data. Once an object
belonging to this class is initialized with raw data, it is possible
to perform two basic set of operations:\\

Basic data manipulation: concatenation of data sets, simple filtering,
data normalisation, extraction of a data subset, separation of data
into subsets on the basis of criteria, split of the data set for cross
validation purposes.\\

Complex data analysis: statistical analysis, crisp clustering, fuzzy
clustering, supervised classification using neural networks, decision
iridia0trees, nearest neighbours and bayesian methods.\\

The {\tt dataset} class takes care of transparently loading and
storing parts of the data set from and to the hard disk, allowing the
whole toolbox to deal with very large data sets: as long as the user
respects the calling convention of the dataset class for retrieving and
storing data points, and he does not have to worry about the
physical location of the data. If the needed data point is not in RAM,
it is loaded from the hard disk and changes will be saved.\\

Besides the functionalities it provides, the use of a separate class
for storing and manipulating data sets insure that every pieces of
information related to the data - name of the variables, possible
discrete values, range of the variables, origin of a piece of data -
are kept in the same place. Moreover, the dataset methods continuously
check the coherence of the dataset properties and guarantee that the
information always makes sense.\\

\include{dataset}

\section{High Level Routines}

\subsection{{\tt System} Abstract Class}
The {\tt system} class centralizes the information needed to describe
a general static or dynamic system. It defies the number of inputs and
outputs of the system and their limits. All the input-output data
associated with the system is stores in an attribute of type {\tt
dataset}.  Since a system can be represented using an internal (state
space based) or external (input-output based) representation the {\tt
system} class has been defined as an abstract class and captures
the common features of these two alternative representations. 
Two subclasses, the {\tt external} and the {\tt internal} one, implement 
these alternative descriptions of a system.\\

\include{system}

\subsection{{\tt Internal} Class}
In the case of a system which uses an external representation the
input-output relation is of the type:
\[\frac{dy}{dt}=f(y,u)\]
in the case of a continuous system, and 
\[y(t+1)=f(y(t),u(t))\]
in the case of a discrete time system.\\

The different input-output relations, which are represented using an
objects belonging to one of the subclasses of the {\tt mapping} class,
are stored inside an attribute of the {\tt external} class.
If a system is multi-outputs, a different relation (a linear one, a
fuzzy representation, a non-parametric approach) can be defined for
each output. This allows a very general representation and lets the
user experiment different types and sizes of models.\\

\include{internal}

\subsection{{\tt External} Class}
In the case of internal representation the system is described 
by the following input output relationships:
\[\dot{x}=a(x) + b(u)\]
\[y=c(x)+ d(u)\]
or in the case of a continuous time system
\[x(t+1)=a(x(t)) + b(u(t))\]
\[y(t)=c(x(t))+ d(u(t))\]
in the case of a discrete time system.\\ 

In the class {\tt internal}, {\tt mapping} objects are used to describe
the evolution of the state variables in the same way they are used for
the outputs. It is possible to choose a different model for
each of the internal variables of the system as well as for its
outputs.\\

\include{external}

\section{Low Level Routines}

\subsection{{\tt Mapping} Abstract Class}
The {\tt mapping} class is used to define the modeling of a process.
Since, as describe in section \label{sec:learning_and_modeling}, it is
possible to choose among different possible alternatives for defining
the mapping between a specified number of inputs and outputs, the {\tt
mapping} abstract class, defines all the features which are common to
all the possible mapping, leaving the definition of the details, which
are dependent from the particular descriptor, to its subclasses which
define the different approaches which can be used for process
modeling.  The {\tt mapping} class stores in its attributes the number
of input and the number of output parameters of the mapping, as well
as their limits. The main abstract methods defined by the class are
{\tt identify}, for performing the identification of the model, {\tt
eval} which is used to evaluate the mapping given the input values,
and {\tt jacob\_inputs} and {\tt jacob\_params}, for computing the
jacobian of the model respect the inputs and the parameters. The
mapping could be normalizing and denormalizing as needed.  All these
are defined as abstract methods, that is define only the calling
convention and the returned values, and need to be implemented at the
level of the subclasses since the operations which they perform depend
on the particular descriptor.\\

Virtually every type of system representation can be implemented as a
subclass of the {\tt mapping} class. At present the following mappings
are available in the toolbox:
\begin{itemize}
\item linear systems 
\item Mamdani fuzzy systems
\item Takagi-Sugeno fuzzy systems
\item lazy learning local modeling (integrated from \cite{birattari_tc99})
\item mixture of experts modeling
\end{itemize}
New mappings are easily added by defining other subclasses of the {\tt
mapping} class.\\ 
\indent 
With the exception of the linear models, all the mappings may be
used to define nonlinearities. They can be seen as different ways of
parametrising nonlinearities. These methods have been shown to be
universal approximators for certain classes of functions. For example
fuzzy logic systems are universal approximators for continuous
functions defined on compact sets \cite{kosko94}. This means that
these methods are equivalent with the respect to the nonlinearity
which they approximate. From the process point of view
it is the nonlinearity that matters, not the way in which it is
parametrized. However from the point of view of the designer, which
parameterization is used can be very important.\\

\include{mapping}

\subsection{{\tt Lazy} Class}
the lazy learning is a local modeling
technique that defers processing of the training data until a query
explicitly needs to be answered. This means that all the training data
needs to be stored in memory and accessed when a query is made.  Given
two variables $ {\bf x} \in \Re_m $ and $ y \in \Re$, and considering
the mapping $ f: \Re_m \to \Re$ which is known only through a set of
examples $ \{({\bf x}_i,y_i)\}^{n}_{i=1}$, which could be affected by
noise, the aim is to find the best possible estimate $\hat{y} = \beta
{\bf q}$ associated the the query ${\bf q}$ using a model linear in
the parameters $\beta$. This can be done by minimizing the following
cost expression
\[
C({\bf q}) = \sum_{i=1}^n [(y_i-{\bf x}_i^T\beta)^2 
            K(d({\bf x}_i,{\bf q}))]
\]
where $d({\bf x}_i,{\bf q})$ is the distance from the query point of
the $i^{th}$ example, and $K(.)$ is a weighting function, or kernel
function, which is used to calculate a weight for that data point from
the distance. In this way the linear local model can be specialized to
the query by emphasizing nearby points and discarding more distant
ones. In matrix notation the solution of the previous weighted least
squares problem is given by:
\[
\beta = ({\bf X}^T{\bf W}^T {\bf W}{\bf X})^{-1}{\bf X}^T{\bf W}^T
{\bf W} {\bf y}
\]
where ${\bf X}$ is a matrix whose $i^{th}$ row is $x_i^T$, {\bf y} is a
vector whose $i^{th}$ element is $y_i$, ${\bf W}$ is a diagonal matrix
whose $i^{th}$ diagonal elements is $w_{ii} =\sqrt{k(d({\bf x}_i,
{\bf x}_q))}$. Once the local approximation of the polynomial has been
computed the prediction is given by 
\[ \hat{y} = \beta {\bf q} \]
In these systems, no parameter identification process is needed since 
the answer is produced upon request of information, and it is
calculated starting from the database of examples which must be
retained into memory.\\ 
The {\tt lazy} class is based on the {\em lazy learning toolbox}
\cite{birattari_tc99, birattari_NIPS99}, whose functions have been
integrated into an implementation of the {\tt mapping} class.\\
The complete documentation of the {\em lazy learning toolbox} is joined
in the annexes.\\
\include{lazy}

\subsection{{\tt Taksug} Class}
the Takagi-Sugeno fuzzy systems are
a different type of fuzzy systems which use rules of the form:
\begin{center}
${\cal R}^{(l)}:$ IF $x_1$ IS $A_1^{(l)}$ AND \dots AND  $x_n$ IS $A_n^{(l)}$
THEN $y=h^{(l)}(x)$
\end{center}
where ${\bf x_i} \in \Re$ are the inputs (antecedent) variables, and
$ y \in \Re$ is the output (consequent). $A_i^{(l)}$ are antecedent 
fuzzy sets of the $l$th rule, defined by a membership function
\[
\mu_{A_i^{(l)}}(x_i): \Re \to [0,1]
\]
In many cases the output function $h^{(l)}(x)$ is a linear combination
of the input variables plus a constant term.
\[
h^{(l)}(x) = a^{(l)}_0+a^{(l)}_1 x_1 + \dots +a^{(l)}_n x_n
\] 
In the special case when all the $l_1 \dots l_n =0$, and therefore the
consequents are constant functions, this becomes a special case of the
Mandami system with the consequent fuzzy sets being fuzzy singletons. 
The system output is a weighted average of the individual rule
outputs, similar to the fuzzy-mean defuzzification formula:
\[
y=\sum^M_{l=1}\frac{\mu_{A^{(l)}}(x)}{\sum^M_{k=1}\mu_{A^{(k)}}(x)}
h^{(l)}(x)
\]
where the weights $\mu_{A^{(l)}}(x)$ are computed according to 
\[
\mu_{A^{(l)}}(x)= \prod^n_{i=1} \mu_{A_i^{(l)}}(x_i)
\]
This approach allows to model a system by means of the decomposition
of a nonlinear system into a collection of local linear models. It
allows a more accurate representation of systems, since the rules are
usually simple linear subsystems, and not constant values like in the
case of the Mamdani fuzzy systems. Since this approach imposes to
structure the problem in a series of local models, Takagi-Sugeno
models can be more easily constructed from numerical data than Mamdani
models, which are essentially structure free.\\

The Features of this class are similar to the ones defined in the {\tt
mamdani} class. It is possible to add and delete rules, and to define
their shape. Since these type of models are usually constructed from
numerical data several identification procedures are available for
performing the parameter identification procedure.\\ 

\include{taksug}

\subsection{{\tt Mamdani} Class}
Mamdani fuzzy systems, also known as
linguistics fuzzy systems, are composed by four different elements: a
fuzzifier, a rule base, an inference engine, and a defuzzifier. The
inputs are transformed in fuzzy numbers by the fuzzifier, which
converts numeric values $x^\star$ into fuzzy sets
$\mu_{A}(x)$. Usually singleton fuzzifiers are the predominant ones,
since their use simplifies the computations in the fuzzy system
considerably. This means that the inputs of the fuzzy system are real
numbers and not fuzzy sets. The fuzzy rule base consists of fuzzy
IF-THEN rules and membership functions, characterizing the fuzzy
set. More precisely a fuzzy rule base, $\cal{R}$, is a set of rules
${\cal R}^{(l)} $, $l=1,2,\dots, K$ of the form:
\begin{center}
${\cal R}^{(l)}:$ IF $x_1$ IS $A_1^{(l)}$ AND \dots AND  $x_n$ IS $A_n^{(l)}$
THEN $y$ IS $B^{(l)}$
\end{center}
where ${\bf x_i} \in \Re$ are the input (antecedent) variables, and
$ y \in {\cal Y} \in \Re$ is the output (consequent). $A_i^{(l)}$ and $B^{(l)}$ are
{\em linguistic terms} (labels) defined by fuzzy sets
$\mu_{A_i^{(l)}}(x_i): \Re \to [0,1]$ and 
$\mu_{B^{(l)}}(y): {\cal Y} \to [0,1]$. The inference
engine takes the input of the fuzzy system, and  uses the rule based
systems to calculate the output of the system. Each rule is
evaluated and the output suggested by the rule is inferred using fuzzy
operators. Then the predictions of each rules are aggregated together
in a fuzzy set. The defuzzifier has the task of taking transforming
this fuzzy set into a numeric value which is the output of the system.
These type of fuzzy systems are well suited for encoding imprecise
knowledge expressed in the form of  IF-THEN rules. These rules are
easily understood by the user since they do not imply any mathematical
representation. The type of interpolation between the rules depends on
their shape and and the choice of the fuzzy logic operators and
defuzzification procedures \cite{babuska_thesis}.\\

The class defines methods for adding, and removing rules. It is
possible to choose the shape of the membership functions by
composition of simpler shapes. In this way it is possible to select
gaussian, sigmoidal, triangular, and several variations of rectangular
membership functions. The inference engine uses .... for calculating
the contribution of each rule.  Freedom about the defuzzification
method is also granted. Center of gravity, center average, and maximum
defuzzifiers are implemented in this class.\\

\include{mamdani}

\include{gui}

%\include{lazydoc}
\tableofcontents
\end{document}

