[TMVA][Preprocessing] - Additional normalisation method

Add scaling VarTransform functionality (like normalisation it linearly scales the data but the sign of the input and output data is retained).

Add scaling VarTransform functionality to TMVA preproccessing (like normalisation it linearly scales the data but the sign of the input and output data is retained).

I have added to the functionality of the VariableNormalizeTransform class in the style of the VariableGaussTransform class to transform data such that it remains in the range of [-1,1], there is no offset, so the sign of the input data is unchanged by the transformation.

This is proving essential for my neural network analyses that treat a detector hit data like an image classification problem and use ReLU activation functions at the beginning of my network.

I have also added a description to the TMVA documentation

Author: josephmckenna

documentation/tmva/UsersGuide/DataPreprocessing.tex

@@ -13,16 +13,17 @@ \section{Data Preprocessing}
 decomposition are available for input and target variables, gaussianization, uniformization and decorrelation 
 discussed below can only be used for input variables. 
 
-Apart from five variable transformation methods mentioned above, an unsupervised variable selection method Variance Threshold is also implemented in TMVA. It follows a completely different processing pipeline. It is discussed in detail in section \ref{sec:varianceThreshold}. 
+Apart from six variable transformation methods mentioned above, an unsupervised variable selection method Variance Threshold is also implemented in TMVA. It follows a completely different processing pipeline. It is discussed in detail in section \ref{sec:varianceThreshold}. 
 
 \subsection{Transforming input variables}
 \label{sec:variableTransform}
 
-Currently five preprocessing\index{Discriminating variables!preprocessing of}  
+Currently six preprocessing\index{Discriminating variables!preprocessing of}  
 transformations\index{Discriminating variables!transformation of}
 are implemented in TMVA:
 \begin{itemize}
 \item variable normalisation;
+\item variable scaling;
 \item decorrelation via the square-root of the covariance matrix ;
 \item decorrelation via a principal component decomposition;
 \item transformation of the variables into Uniform distributions (``Uniformization'').
@@ -100,6 +101,17 @@ \subsubsection{Variable normalisation\index{Discriminating variables!normalisati
 Normalisation may also render minimisation processes, such as the adjustment of 
 neural network weights, more effective. 
 
+\subsubsection{Variable scaling\index{Discriminating variables!scaling of}}
+\label{sec:scaling}
+
+The larger absolute value of the minimum and maximum values is determined from the training events 
+and used to scale the dataset to lie within $[-1,1]$. There is no offset added and thus the original 
+sign of the input is maintained. E.g Input data with a range $[x,y]$ where $|y|>|x|$ will transform 
+to the range $[x/|y|,1]$. 
+As with Normalisation, this may also render minimisation processes, such as the adjustment of 
+neural network weights, more effective especially if you are using sign sensitive activation 
+functions such as the a rectified linear unit ( ReLU ).
+
 \subsubsection{Variable decorrelation\index{Discriminating variables!decorrelation of}}
 \label{sec:decorrelation}
 
@@ -225,10 +237,10 @@ \subsubsection{Booking and chaining transformations for some or all input variab
 Variable transformations to be applied prior to the MVA training (and application) 
 can be defined independently for each MVA method with the booking option 
 {\tt VarTransform=<type>}, where {\tt <type>} denotes the desired transformation 
-(or chain of transformations). The available transformation types are normalisation, 
+(or chain of transformations). The available transformation types are normalisation, scaling, 
 decorrelation, principal component analysis and Gaussianisation, which are labelled by 
-\code{Norm}, \code{Deco}, \code{PCA}, \code{Uniform}, \code{Gauss}, respectively, or, equivalently, 
-by the short-hand notations \code{N}, \code{D}, \code{P}, \code{U} , \code{G}.
+\code{Norm}, \code{Scale}, \code{Deco}, \code{PCA}, \code{Uniform}, \code{Gauss}, respectively, or, equivalently, 
+by the short-hand notations \code{N}, \code{S}, \code{D}, \code{P}, \code{U} , \code{G}.
 
 Transformations can be {\em chained} allowing the consecutive application of all defined 
 transformations to the variables for each event.
@@ -326,7 +338,7 @@ \subsection{Variable selection based on variance}
 \label{eq:meanecalculation}
 \mu_j = \frac{\sum_{i=1}^N w_i x_{j}(i)}{\sum_{i=1}^N w_i}
 \eeq
-Unlike above five variable transformation method, this Variance Threshold method is implemented in DataLoader class. After loading dataset in the DataLoader object, we can apply this method. It returns a new DataLoader with the selected variables which have variance strictly greater than the threshold value passed by user. Default value of threshold is zero i.e. remove the variables which have same value in all the events. 
+Unlike above six variable transformation method, this Variance Threshold method is implemented in DataLoader class. After loading dataset in the DataLoader object, we can apply this method. It returns a new DataLoader with the selected variables which have variance strictly greater than the threshold value passed by user. Default value of threshold is zero i.e. remove the variables which have same value in all the events. 
 
 \begin{codeexample}
 \begin{tmvacode}

tmva/tmva/inc/TMVA/VariableNormalizeTransform.h

@@ -51,7 +51,7 @@ namespace TMVA {
 
       typedef std::vector<Float_t>       FloatVector;
       typedef std::vector< FloatVector > VectorOfFloatVectors;
-      VariableNormalizeTransform( DataSetInfo& dsi );
+      VariableNormalizeTransform( DataSetInfo& dsi, TString strcor=""  );
       virtual ~VariableNormalizeTransform( void );
 
       void   Initialize() override;
@@ -77,6 +77,8 @@ namespace TMVA {
 
    private:
 
+      Bool_t           fNoOffset;
+
       void CalcNormalizationParams( const std::vector< Event*>& events);
 
       //      mutable Event*           fTransformedEvent;

tmva/tmva/src/VariableNormalizeTransform.cxx

@@ -54,9 +54,13 @@ Linear interpolation class
 ////////////////////////////////////////////////////////////////////////////////
 /// constructor
 
-TMVA::VariableNormalizeTransform::VariableNormalizeTransform( DataSetInfo& dsi )
-: VariableTransformBase( dsi, Types::kNormalized, "Norm" )
+TMVA::VariableNormalizeTransform::VariableNormalizeTransform( DataSetInfo& dsi, TString strcor )
+: VariableTransformBase( dsi, Types::kNormalized, "Norm" ),
+   fNoOffset(kFALSE)
 {
+   if (strcor=="Scale") {fNoOffset = kTRUE;
+      SetName("Scale");
+   }
 }
 
 ////////////////////////////////////////////////////////////////////////////////
@@ -143,10 +147,16 @@ const TMVA::Event* TMVA::VariableNormalizeTransform::Transform( const TMVA::Even
 
       min = minVector.at(iidx);
       max = maxVector.at(iidx);
-      Float_t offset = min;
-      Float_t scale  = 1.0/(max-min);
 
-      Float_t valnorm = (val-offset)*scale * 2 - 1;
+      Float_t valnorm;
+      if (!fNoOffset) {
+         Float_t offset = min;
+         Float_t scale  = 1.0/(max-min);
+         valnorm        = (val-offset)*scale * 2 - 1;
+      } else {
+         fabs(max)>fabs(min) ? valnorm=val/fabs(max) : valnorm=val/fabs(min);
+      }
+
       output.push_back( valnorm );
 
       ++iidx;
@@ -188,10 +198,16 @@ const TMVA::Event* TMVA::VariableNormalizeTransform::InverseTransform(const TMVA
 
       min = minVector.at(iidx);
       max = maxVector.at(iidx);
-      Float_t offset = min;
-      Float_t scale  = 1.0/(max-min);
 
-      Float_t valnorm = offset+((val+1)/(scale * 2));
+      Float_t valnorm;
+      if (!fNoOffset) {
+         Float_t offset = min;
+         Float_t scale  = 1.0/(max-min);
+         valnorm = offset+((val+1)/(scale * 2));
+      } else {
+         fabs(max)>fabs(min) ? valnorm=val*fabs(max) : valnorm=val*fabs(min);
+      }
+
       output.push_back( valnorm );
 
       ++iidx;
@@ -282,8 +298,15 @@ std::vector<TString>* TMVA::VariableNormalizeTransform::GetTransformationStrings
 
       Char_t type = (*itGet).first;
       UInt_t idx  = (*itGet).second;
-      Float_t offset = min;
-      Float_t scale  = 1.0/(max-min);
+      Float_t offset;
+      Float_t scale;
+      if (!fNoOffset) {
+         offset = min;
+         scale  = 1.0/(max-min);
+      } else {
+         offset = 0.;
+         fabs(max)>fabs(min) ? scale=.5/fabs(max) : scale=.5/fabs(min);
+      }
       TString str("");
       VariableInfo& varInfo = (type=='v'?fDsi.GetVariableInfo(idx):(type=='t'?fDsi.GetTargetInfo(idx):fDsi.GetSpectatorInfo(idx)));
 
@@ -329,6 +352,7 @@ void TMVA::VariableNormalizeTransform::AttachXMLTo(void* parent)
 {
    void* trfxml = gTools().AddChild(parent, "Transform");
    gTools().AddAttr(trfxml, "Name", "Normalize");
+   gTools().AddAttr(trfxml, "UseOffsetOrNot", (fNoOffset?"NoOffset":"UseOffset") );
    VariableTransformBase::AttachXMLTo( trfxml );
 
    Int_t numC = (GetNClasses()<= 1)?1:GetNClasses()+1;
@@ -353,6 +377,13 @@ void TMVA::VariableNormalizeTransform::AttachXMLTo(void* parent)
 
 void TMVA::VariableNormalizeTransform::ReadFromXML( void* trfnode )
 {
+   TString UseOffsetOrNot;
+
+   gTools().ReadAttr(trfnode, "UseOffsetOrNot", UseOffsetOrNot );
+
+   if (UseOffsetOrNot == "NoOffset") fNoOffset = kTRUE;
+   else                              fNoOffset = kFALSE;
+
    Bool_t newFormat = kFALSE;
 
    void* inpnode = NULL;

tmva/tmva/src/VariableTransform.cxx

@@ -158,6 +158,10 @@ void CreateVariableTransforms(const TString& trafoDefinitionIn,
          if (variables.Length() == 0) variables = "_V_,_T_";
          transformation = new VariableNormalizeTransform(dataInfo);
       }
+      else if (trName == "S" || trName == "Scale" || trName == "ScaleNorm" ) {
+         if (variables.Length() == 0) variables = "_V_,_T_";
+         transformation = new VariableNormalizeTransform(dataInfo,"Scale");
+      }
       else
          log << kFATAL << Form("Dataset[%s] : ",dataInfo.GetName())
              << "<ProcessOptions> Variable transform '"