renom_rl.utility.logger : Logger Series ¶

This section shows description for logger class series.

About Iteration

For modules using a single agent and non advanced steps, 1 iteration is equivalent to 1 step.

For modules using advantage steps, 1 iteration is equivalent to the advantage step.

For modules using multiple agents, 1 agent is logged.

About Keys

Keys are string values which refer to variables inside algorithms. For example, if users set “reward” as a key value, “reward” variables will be viewed (or recorded.)

View Logger Key Table for variables available to log.

AVAILABLE_KEYS [dict] ¶

AVAIABLE_KEYS

Allows users to view keys that are avaiable for logging.

key	description
dqn	dqn epoch / epoch_logger keys
ddqn	ddqn epoch / epoch_logger keys
ddpg	ddpg epoch / epoch_logger keys
a2c	a2c epoch / epoch_logger keys
doc	AVAIABLE_KEYS document

Examples

             >>> from renom_rl.utility.logger import Logger
>>> # from renom_rl.utility.logger import AVAILABLE_KEYS
>>> Logger.available_keys["dqn"] # or AVAILABLE_KEYS["dqn"]
{'logger': ['state', 'action', 'reward', 'terminal', 'next_state',
'total_step', 'epoch_step', 'max_step', 'total_episode',
'epoch_episode', 'steps_per_episode', 'epoch', 'max_epoch',
'loss', 'sum_reward', 'epsilon'],
'logger_epoch': ['total_episode', 'epoch_episode',
'epoch', 'max_epoch', 'test_reward', 'epsilon']}

            

class Logger ( log_key=None , log_key_epoch=None , record=True , record_episode_base=True , show_bar=True , disable=False ) ¶

Logger Module

This class logs various data of each module.

By setting log_key , log_key_epoch , this class will record data based on keys for every iteration. Keys specified for these argument are mainly used for plotting graph and csv output. log_key , log_key_epoch argument must be a list of strings which exist in the algorithm.

logger(**log) function reads all keys that are available to read at every iteration and returns as a progress bar message. Simultaneously, data specified at log_key will be memorized as iteration data. (Overriding Required) logger_epoch(**log) function reads all keys that are available to read at every end of epoch and returns as a progress bar message. Simultaneously, data specified at log_key_epoch will be memorized as epoch data. (Not Mandatory)

Users must also call super class super().__init__ when initializing.

Parameters:

log_key ( list ) – List of logging keys for each iteration.
log_key_epoch ( list ) – List of logging keys for each epoch.
record ( boolean ) – Keeps data for graph and csv. Default is True.
record_episode_base ( boolean ) – Keeps data when record is True and episode changes. Default is True.
show_bar ( boolean ) – Shows bar. Default is True.
disable ( boolean ) – Disables tqdm. Default is False.

Examples

             >>> import numpy as np
>>> class Original(Logger):
...     def __init__(self,log_key):
...         super(Original,self).__init__(log_key,record_episode_base=False)
...         self.reward_previous = 0
...         self.reward = 0
...         self.total_list = []
...         self.state = 0
...         self.total = 0
...
...     def logger(self,**log):
...         self.state = log["state"]
...         self.reward = log["reward"]
...         self.total += log["reward"]
...
...         return "state----{}/reward---{}/total----{}".format(self.state, self.reward, self.total)
...
>>>
>>>
>>> import renom as rm
>>>
>>> from renom_rl.environ.openai import CartPole00
>>> from renom_rl.discrete.dqn import DQN
>>>
>>> network = rm.Sequential([rm.Dense(32),rm.Relu(),rm.Dense(32),rm.Relu(),rm.Dense(2)])
>>>
>>> logger = Original(["reward"])
>>>
>>> dqn=DQN(env=CartPole00(),q_network=network,logger=logger)
state----[-0.00528582  0.76312646 -0.00763515 -1.1157825 ]/reward---0/total-----39: 100%|██████████████████████████████████████| 500/500 [00:01<00:00, 438.39it/s]

            

Note

Note that keys log_key does not suppress logger from reading values. For example, users can specify log_key as:

              log_k = ["state","reward"]

class Original(Logger):
     def __init__(self,log_key):
         super(Original,self).__init__(log_key)
         .....

original = Original(log_key = log_k)

             

and still view other keys such as “next_state”, “terminal” at logger(**log) function, as shown below:

              def logger(**log):
    log_key = log["next_state"]
    terminal = log["terminal"]

    return .....

However, users cannot graph data for “next_state”, “terminal” at graph function etc.. Users will only be able to graph using key “state”, “reward” at graph function etc.

              graph(y_key="reward")  # Pass
graph(y_key="terminal")  # Error

logger ( **log ) ¶

This function will be called for every iteration. Override this function when creating custom logger

Parameters:	log ( dictionary ) – Data input from every iteration. Keys are logging keys.
Returns:	Message required for progress view.
Return type:	( str )

Examples

               >>> class Some_Logger(Logger):
...     def __init__(self,log_key):
...            ...
...
...     def logger(self,**log):
...         self.state = log["state"]
...         self.reward = log["reward"]
...         self.total += log["reward"]
...
...         return "making original reward:{}".format(self.reward)
...
>>>
>>> dqn=DQN(env=... , q_network= ... ,logger=Some_Logger)
making original reward:0: 50%|████████████████████                  | 250/500 [00:01<00:00, 438.39it/s]

              

logger_epoch ( **log ) ¶: This function will be called when 1 epoch is done. Due to its similiarity, view logger function for detail. Override this function when creating custom logger

result ( *args ) ¶

Returns dictionary of data that were specified as log_key. If argument is blank, then all output will be shown.

Parameters:	args ( string* ) – Strings of arguments.
Returns:	Dictionary of logging data.
Return type:	( dict )

Examples

               >>> logger.result()
{'reward': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1, ....]}
>>>
>>> logger.result("reward")
{'reward': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1, ....]}
>>>
>>> logger.result("reward","state")
{'reward': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1, ....],
'state': [....],}

              

result_epoch ( *args ) ¶: Returns dictionary of data that were specified as log_key_epoch. If argument is blank, then all output will be shown.

graph ( y_key , x_key=None , x_lim=None , y_lim=None , x_interval=0 , y_interval=0 , figsize=None , dpi=100 , average_range=0 , plot_label=None , legend=None , grid=True ) ¶

Shows plot from recorded data. Keys must be from log_key . if x_key is None, y_key will be plot based on its length. Note that this function is focused on quick view so if detailed view is required, save data as csv.(refer to to_csv function)

Parameters:

y_key ( string ) – Key for Y (vertical) axis. 2-D data is allowed.
x_key ( string ) – Key for X (horizontal) axis. This must be 1-D data. Default is None.
x_lim ( list ) – [min,max] range for X axis. Default is min,max of x_key data.
y_lim ( list ) – [min,max] range for Y axis. Default is min,max of y_key data.
x_interval ( float ) – Interval of ticks for Y axis. Default is None (ticks: auto).
y_interval ( float ) – Interval of ticks for X axis. Default is None (ticks: auto).
figsize ( tuple ) – When (a,b) is input, plot increase to a*x_axis, b*y_axis.
dpi ( int ) – Digital Pixel Image. Default is 100.
average_length ( int or list ) – Creates average plot in [i - min, i + max] range (i being plotted point), when [min,max] is set. If int type is set, it becomes [average_length,average_length]. Default is 0.
plot_label ( string ) – Names label of plot when legend appears. Default is None.
legend ( bool or dictionary ) – Shows Legend. If dictionary, legend’s property will be set based on its value. Default is None (False).
grid ( boolean ) – Shows grid based on ticks. Default is True.

Examples

               >>> logger.graph(y_key="reward",figsize=(6,6),dpi=100,average_range=[1,1])

              

graph_epoch ( y_key , x_key=None , x_lim=None , y_lim=None , x_interval=0 , y_interval=0 , figsize=None , dpi=100 , average_range=0 , plot_label=None , legend=None , grid=True ) ¶: Shows plot from recorded data at every epoch. View the function above for details.

classmethod graph_custom ( y_data , x_data=None , y_label='' , x_label='' , x_lim=None , y_lim=None , x_interval=0 , y_interval=0 , figsize=None , dpi=100 , average_range=0 , plot_label=None , legend=None , grid=True ) ¶

This function allows users to quickly create graph when custom creating own data.

Refer to graph for other arguments.

Parameters:	y_data ( numpy ) – Y (vertical) axis data. 2-D data is allowed. x_data ( numpy ) – X (horizontal) axis data. This must be 1-D data. Default is None. y_label ( string ) – Y (vertical) axis label. x_label ( string ) – X (vertical) axis label.

Examples

               >>> # suppose logger.total has a 2D list
>>> array_list=np.array(logger.total_list)[:,1]
>>> logger.graph_custom(array_list,y_label="this is y",x_label="this is x",x_interval=5)

              

classmethod graph_attribute ( plt_sub , y_data , x_data=None , y_label='' , x_label='' , x_lim=None , y_lim=None , x_interval=0 , y_interval=0 , dpi=100 , average_range=0 , plot_label=None , legend=None , grid=True ) ¶

This function allows users to generate graph properties more easily.

Refer to graph for other arguments.

Parameters:	plt ( matplotlib.pyplot ) – plt object.

Examples

               >>> import numpy as np
>>> from renom_rl.utility.logger import Logger
>>> import matplotlib.pyplot as plt
>>>
>>> data_list={}
>>> data_list["param1"]=np.random.random((100,2))
>>> data_list["param2"]=np.random.random((100,2))
>>> data_list["param3"]=np.random.random((100,2))
>>>
>>> plt.figure(figsize=(10,10))
>>> for i , k in enumerate(data_list,1):
...     plt.subplot(len(data_list),1,i)
...     Logger.graph_attribute(plt,data_list[k],plot_label=["plt_a", "plt_b"],y_label=k,legend={"loc":"upper right"})
>>>
>>> plt.show()

              

to_csv ( filename , overwrite=False , epoch_data=True ) ¶

Stores csv file based on filename. Epoch data are stored as filename +”_e.csv”

Parameters:	filename ( string ) – Filename of the string. overwrite ( boolean ) – Overwrites if exist. Appends number if exist. Default is False. epoch_data ( boolean ) – Stores epoch data if True. Default is True.

Examples

               >>> logger.to_csv("./test.csv", overwrite=True)

              

from_csv ( filename ) ¶

Loads csv file based on filename. If file ends with ‘_e’,csv file will be loaded as epoch data.

Parameters:	filename ( string ) – Filename of the string.

Examples

               >>> logger.from_csv("./test.csv")

              

class SimpleLogger ( log_key , log_key_epoch=None , msg='' , msg_epoch='' , record=True , record_episode_base=True , show_bar=True , disable=False ) ¶

Simple Logger Module

This class logs various data for each module.

log_key , log_key_epoch argument must be a list of strings which exist in the algorithm.

msg is required. msg_epoch is optional.

Parameters:

log_key ( list ) – Logging values.
log_key_epoch ( list ) – Logging values at end of epoch.
msg ( string ) – Printing message for each iteration. Use curly braces ‘{}’.
msg_epoch ( string ) – Printing message for each epoch. Use curly braces ‘{}’.
record ( boolean ) – Keeps data for graph and csv. Default is True.
record_episode_base ( boolean ) – Keeps data when record is True and episode changes. Default is True.
show_bar ( boolean ) – Shows bar. Default is True.
disable ( boolean ) – Disables tqdm. Default is False.

Examples

             >>> logger = SimpleLogger(log_key = ["state","reward"] , msg="this is {state} reward:{reward}")
>>> logger = SimpleLogger(log_key = ["state","reward"] , msg="this is {} reward:{}")