Inference mode

Inference mode used for Dropout and Batch Normalization.

This is an introduction to the “inference mode”. Dropout and Batch Normalization , and some probablistic method has the inference mode and learning mode.
For example, you have not to use the dropout when you predict the data after training.


In [1]:
import numpy as np
import renom as rm # >= v2.0.0

What is inference mode?

There are some functions which may exhibit different behavior between training and testing. For example, dropout and batch normalization .

The dropout function randomly sets some of the data to zero, during training. During testing however, nothing is dropped-out.

To control this, we have to set the “inference” mode flag in our model.

How to set the model to inference mode

In ReNom, there is a flag that switches-on the inference mode.

The following code shows how to switch the mode:

In [2]:
x = np.random.rand(2, 3)
model = rm.Sequential([

# If Model is set to the "training mode",
# then the Dropout function drops some of the data.
print("Training mode. Some data are dropped.")

#  If Model is set to the "inference mode",
#  then the Dropout function doesn't drop part of data.
print("Inference mode. Any data are not dropped.")
Training mode. Some data are dropped.
[[ 0.          0.15597625  1.60988951]
 [ 0.          0.52721781  0.62134868]]

Inference mode. Any data are not dropped.
[[ 0.43925468  0.07798813  0.80494478]
 [ 0.34552573  0.26360891  0.31067435]]