renom_rl.utility.logger : Logger Series ¶
This section shows description for logger class series.
About Iteration
- For modules using a single agent and non advanced steps, 1 iteration is equivalent to 1 step.
- For modules using advantage steps, 1 iteration is equivalent to the advantage step.
- For modules using multiple agents, 1 agent is logged.
About Keys
Keys are string values which refer to variables inside algorithms. For example, if users set “reward” as a key value, “reward” variables will be viewed (or recorded.)
View Logger Key Table for variables available to log.
-
AVAILABLE_KEYS
[dict] ¶ -
AVAIABLE_KEYS
Allows users to view keys that are avaiable for logging.
key description dqn dqn epoch / epoch_logger keys ddqn ddqn epoch / epoch_logger keys ddpg ddpg epoch / epoch_logger keys a2c a2c epoch / epoch_logger keys doc AVAIABLE_KEYS document Examples
>>> from renom_rl.utility.logger import Logger >>> # from renom_rl.utility.logger import AVAILABLE_KEYS >>> Logger.available_keys["dqn"] # or AVAILABLE_KEYS["dqn"] {'logger': ['state', 'action', 'reward', 'terminal', 'next_state', 'total_step', 'epoch_step', 'max_step', 'total_episode', 'epoch_episode', 'steps_per_episode', 'epoch', 'max_epoch', 'loss', 'sum_reward', 'epsilon'], 'logger_epoch': ['total_episode', 'epoch_episode', 'epoch', 'max_epoch', 'test_reward', 'epsilon']}
-
class
Logger
( log_key=None , log_key_epoch=None , record=True , record_episode_base=True , show_bar=True , disable=False ) ¶ -
Logger Module
This class logs various data of each module.
By setting
log_key
,log_key_epoch
, this class will record data based on keys for every iteration. Keys specified for these argument are mainly used for plotting graph and csv output.log_key
,log_key_epoch
argument must be a list of strings which exist in the algorithm.logger(**log)
function reads all keys that are available to read at every iteration and returns as a progress bar message. Simultaneously, data specified atlog_key
will be memorized as iteration data. (Overriding Required)logger_epoch(**log)
function reads all keys that are available to read at every end of epoch and returns as a progress bar message. Simultaneously, data specified atlog_key_epoch
will be memorized as epoch data. (Not Mandatory)Users must also call super class
super().__init__
when initializing.Parameters: - log_key ( list ) – List of logging keys for each iteration.
- log_key_epoch ( list ) – List of logging keys for each epoch.
- record ( boolean ) – Keeps data for graph and csv. Default is True.
-
record_episode_base
(
boolean
) – Keeps data when
record
is True and episode changes. Default is True. - show_bar ( boolean ) – Shows bar. Default is True.
- disable ( boolean ) – Disables tqdm. Default is False.
Examples
>>> import numpy as np >>> class Original(Logger): ... def __init__(self,log_key): ... super(Original,self).__init__(log_key,record_episode_base=False) ... self.reward_previous = 0 ... self.reward = 0 ... self.total_list = [] ... self.state = 0 ... self.total = 0 ... ... def logger(self,**log): ... self.state = log["state"] ... self.reward = log["reward"] ... self.total += log["reward"] ... ... return "state----{}/reward---{}/total----{}".format(self.state, self.reward, self.total) ... >>> >>> >>> import renom as rm >>> >>> from renom_rl.environ.openai import CartPole00 >>> from renom_rl.discrete.dqn import DQN >>> >>> network = rm.Sequential([rm.Dense(32),rm.Relu(),rm.Dense(32),rm.Relu(),rm.Dense(2)]) >>> >>> logger = Original(["reward"]) >>> >>> dqn=DQN(env=CartPole00(),q_network=network,logger=logger) state----[-0.00528582 0.76312646 -0.00763515 -1.1157825 ]/reward---0/total-----39: 100%|██████████████████████████████████████| 500/500 [00:01<00:00, 438.39it/s]
Note
Note that keys
log_key
does not suppress logger from reading values. For example, users can specifylog_key
as:log_k = ["state","reward"] class Original(Logger): def __init__(self,log_key): super(Original,self).__init__(log_key) ..... original = Original(log_key = log_k)
and still view other keys such as “next_state”, “terminal” at
logger(**log)
function, as shown below:def logger(**log): log_key = log["next_state"] terminal = log["terminal"] return .....
However, users cannot graph data for “next_state”, “terminal” at
graph
function etc.. Users will only be able to graph using key “state”, “reward” atgraph
function etc.graph(y_key="reward") # Pass graph(y_key="terminal") # Error
-
logger
( **log ) ¶ -
This function will be called for every iteration. Override this function when creating custom logger
Parameters: log ( dictionary ) – Data input from every iteration. Keys are logging keys. Returns: Message required for progress view. Return type: ( str ) Examples
>>> class Some_Logger(Logger): ... def __init__(self,log_key): ... ... ... ... def logger(self,**log): ... self.state = log["state"] ... self.reward = log["reward"] ... self.total += log["reward"] ... ... return "making original reward:{}".format(self.reward) ... >>> >>> dqn=DQN(env=... , q_network= ... ,logger=Some_Logger) making original reward:0: 50%|████████████████████ | 250/500 [00:01<00:00, 438.39it/s]
-
logger_epoch
( **log ) ¶ -
This function will be called when 1 epoch is done. Due to its similiarity, view
logger
function for detail. Override this function when creating custom logger
-
result
( *args ) ¶ -
Returns dictionary of data that were specified as log_key. If argument is blank, then all output will be shown.
Parameters: *args ( string ) – Strings of arguments. Returns: Dictionary of logging data. Return type: ( dict ) Examples
>>> logger.result() {'reward': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1, ....]} >>> >>> logger.result("reward") {'reward': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1, ....]} >>> >>> logger.result("reward","state") {'reward': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1, ....], 'state': [....],}
-
result_epoch
( *args ) ¶ -
Returns dictionary of data that were specified as log_key_epoch. If argument is blank, then all output will be shown.
-
graph
( y_key , x_key=None , x_lim=None , y_lim=None , x_interval=0 , y_interval=0 , figsize=None , dpi=100 , average_range=0 , plot_label=None , legend=None , grid=True ) ¶ -
Shows plot from recorded data. Keys must be from
log_key
. ifx_key
is None,y_key
will be plot based on its length. Note that this function is focused on quick view so if detailed view is required, save data as csv.(refer toto_csv
function)Parameters: - y_key ( string ) – Key for Y (vertical) axis. 2-D data is allowed.
- x_key ( string ) – Key for X (horizontal) axis. This must be 1-D data. Default is None.
- x_lim ( list ) – [min,max] range for X axis. Default is min,max of x_key data.
- y_lim ( list ) – [min,max] range for Y axis. Default is min,max of y_key data.
- x_interval ( float ) – Interval of ticks for Y axis. Default is None (ticks: auto).
- y_interval ( float ) – Interval of ticks for X axis. Default is None (ticks: auto).
- figsize ( tuple ) – When (a,b) is input, plot increase to a*x_axis, b*y_axis.
- dpi ( int ) – Digital Pixel Image. Default is 100.
- average_length ( int or list ) – Creates average plot in [i - min, i + max] range (i being plotted point), when [min,max] is set. If int type is set, it becomes [average_length,average_length]. Default is 0.
- plot_label ( string ) – Names label of plot when legend appears. Default is None.
- legend ( bool or dictionary ) – Shows Legend. If dictionary, legend’s property will be set based on its value. Default is None (False).
- grid ( boolean ) – Shows grid based on ticks. Default is True.
Examples
>>> logger.graph(y_key="reward",figsize=(6,6),dpi=100,average_range=[1,1])
-
graph_epoch
( y_key , x_key=None , x_lim=None , y_lim=None , x_interval=0 , y_interval=0 , figsize=None , dpi=100 , average_range=0 , plot_label=None , legend=None , grid=True ) ¶ -
Shows plot from recorded data at every epoch. View the function above for details.
-
classmethod
graph_custom
( y_data , x_data=None , y_label='' , x_label='' , x_lim=None , y_lim=None , x_interval=0 , y_interval=0 , figsize=None , dpi=100 , average_range=0 , plot_label=None , legend=None , grid=True ) ¶ -
This function allows users to quickly create graph when custom creating own data.
Refer to
graph
for other arguments.Parameters: - y_data ( numpy ) – Y (vertical) axis data. 2-D data is allowed.
- x_data ( numpy ) – X (horizontal) axis data. This must be 1-D data. Default is None.
- y_label ( string ) – Y (vertical) axis label.
- x_label ( string ) – X (vertical) axis label.
Examples
>>> # suppose logger.total has a 2D list >>> array_list=np.array(logger.total_list)[:,1] >>> logger.graph_custom(array_list,y_label="this is y",x_label="this is x",x_interval=5)
-
classmethod
graph_attribute
( plt_sub , y_data , x_data=None , y_label='' , x_label='' , x_lim=None , y_lim=None , x_interval=0 , y_interval=0 , dpi=100 , average_range=0 , plot_label=None , legend=None , grid=True ) ¶ -
This function allows users to generate graph properties more easily.
Refer to
graph
for other arguments.Parameters: plt ( matplotlib.pyplot ) – plt object. Examples
>>> import numpy as np >>> from renom_rl.utility.logger import Logger >>> import matplotlib.pyplot as plt >>> >>> data_list={} >>> data_list["param1"]=np.random.random((100,2)) >>> data_list["param2"]=np.random.random((100,2)) >>> data_list["param3"]=np.random.random((100,2)) >>> >>> plt.figure(figsize=(10,10)) >>> for i , k in enumerate(data_list,1): ... plt.subplot(len(data_list),1,i) ... Logger.graph_attribute(plt,data_list[k],plot_label=["plt_a", "plt_b"],y_label=k,legend={"loc":"upper right"}) >>> >>> plt.show()
-
to_csv
( filename , overwrite=False , epoch_data=True ) ¶ -
Stores csv file based on filename. Epoch data are stored as filename +”_e.csv”
Parameters: - filename ( string ) – Filename of the string.
- overwrite ( boolean ) – Overwrites if exist. Appends number if exist. Default is False.
- epoch_data ( boolean ) – Stores epoch data if True. Default is True.
Examples
>>> logger.to_csv("./test.csv", overwrite=True)
-
from_csv
( filename ) ¶ -
Loads csv file based on filename. If file ends with ‘_e’,csv file will be loaded as epoch data.
Parameters: filename ( string ) – Filename of the string. Examples
>>> logger.from_csv("./test.csv")
-
class
SimpleLogger
( log_key , log_key_epoch=None , msg='' , msg_epoch='' , record=True , record_episode_base=True , show_bar=True , disable=False ) ¶ -
Simple Logger Module
This class logs various data for each module.
log_key
,log_key_epoch
argument must be a list of strings which exist in the algorithm.msg
is required.msg_epoch
is optional.Parameters: - log_key ( list ) – Logging values.
- log_key_epoch ( list ) – Logging values at end of epoch.
- msg ( string ) – Printing message for each iteration. Use curly braces ‘{}’.
- msg_epoch ( string ) – Printing message for each epoch. Use curly braces ‘{}’.
- record ( boolean ) – Keeps data for graph and csv. Default is True.
-
record_episode_base
(
boolean
) – Keeps data when
record
is True and episode changes. Default is True. - show_bar ( boolean ) – Shows bar. Default is True.
- disable ( boolean ) – Disables tqdm. Default is False.
Examples
>>> logger = SimpleLogger(log_key = ["state","reward"] , msg="this is {state} reward:{reward}") >>> logger = SimpleLogger(log_key = ["state","reward"] , msg="this is {} reward:{}")