Inference mode

Inference mode used for Dropout and Batch Normalization.

This is an introduction to the “inference mode”. Dropout and Batch Normalization , and some probablistic method has the inference mode and learning mode.
For example, you have not to use the dropout when you predict the data after training.

Requirements

In [1]:
import numpy as np
import renom as rm # >= v2.0.0

What is inference mode?

There are some functions which may exhibit different behavior between training and testing. For example, dropout and batch normalization .

The dropout function randomly sets some of the data to zero, during training. During testing however, nothing is dropped-out.

To control this, we have to set the “inference” mode flag in our model.

How to set the model to inference mode

In ReNom, there is a flag that switches-on the inference mode.

The following code shows how to switch the mode:

In [2]:
x = np.random.rand(2, 3)
model = rm.Sequential([
    rm.Dropout(dropout_ratio=0.5),
])

model.set_models(inference=False)
# If Model is set to the "training mode",
# then the Dropout function drops some of the data.
print("Training mode. Some data are dropped.")
print(model(x))
print()

model.set_models(inference=True)
#  If Model is set to the "inference mode",
#  then the Dropout function doesn't drop part of data.
print("Inference mode. Any data are not dropped.")
print(model(x))
Training mode. Some data are dropped.
[[ 0.          0.15597625  1.60988951]
 [ 0.          0.52721781  0.62134868]]

Inference mode. Any data are not dropped.
[[ 0.43925468  0.07798813  0.80494478]
 [ 0.34552573  0.26360891  0.31067435]]