Inference mode ¶
Inference mode used for Dropout and Batch Normalization.
import numpy as np import renom as rm # >= v2.0.0
What is inference mode? ¶
There are some functions which may exhibit different behavior between training and testing. For example, dropout and batch normalization .
The dropout function randomly sets some of the data to zero, during training. During testing however, nothing is dropped-out.
To control this, we have to set the “inference” mode flag in our model.
How to set the model to inference mode ¶
In ReNom, there is a flag that switches-on the inference mode.
The following code shows how to switch the mode:
x = np.random.rand(2, 3) model = rm.Sequential([ rm.Dropout(dropout_ratio=0.5), ]) model.set_models(inference=False) # If Model is set to the "training mode", # then the Dropout function drops some of the data. print("Training mode. Some data are dropped.") print(model(x)) print() model.set_models(inference=True) # If Model is set to the "inference mode", # then the Dropout function doesn't drop part of data. print("Inference mode. Any data are not dropped.") print(model(x))
Training mode. Some data are dropped. [[ 0. 0.15597625 1.60988951] [ 0. 0.52721781 0.62134868]] Inference mode. Any data are not dropped. [[ 0.43925468 0.07798813 0.80494478] [ 0.34552573 0.26360891 0.31067435]]