4.2. Action Adapters

4.2.1. Action Adapter Base Class

class rlgraph.components.action_adapters.action_adapter.ActionAdapter(action_space, add_units=0, units=None, weights_spec=None, biases_spec=None, activation=None, scope='action-adapter', **kwargs)

Bases: rlgraph.components.component.Component

A Component that cleans up a neural network’s flat output and gets it ready for parameterizing a Distribution Component. Processing steps include: - Sending the raw, flattened NN output through a Dense layer whose number of units matches the flattened action space. - Reshaping (according to the action Space). - Translating the reshaped outputs (logits) into probabilities (by softmaxing) and log-probabilities (log).

check_input_spaces(input_spaces, action_space=None)

Should check on the nature of all in-Sockets Spaces of this Component. This method is called automatically by the Model when all these Spaces are know during the Model’s build time.

Args:

input_spaces (Dict[str,Space]): A dict with Space/shape information.

keys=in-Socket name (str); values=the associated Space

action_space (Optional[Space]): The action Space of the Agent/GraphBuilder. Can be used to construct and

connect more Components (which rely on this information). This eliminates the need to pass the action Space information into many Components’ constructors.

get_action_layer_output(*args, **kwargs)

get_logits(*args, **kwargs)

get_logits_probabilities_log_probs(*args, **kwargs)

4.2.2. Dueling Action Adapter

class rlgraph.components.action_adapters.dueling_action_adapter.DuelingActionAdapter(units_state_value_stream, units_advantage_stream, weights_spec_state_value_stream=None, biases_spec_state_value_stream=None, activation_state_value_stream='relu', weights_spec_advantage_stream=None, biases_spec_advantage_stream=None, activation_advantage_stream='relu', scope='dueling-action-adapter', **kwargs)

Bases: rlgraph.components.action_adapters.action_adapter.ActionAdapter

An ActionAdapter that adds a dueling Q calculation to the flattened output of a neural network.

API:

get_dueling_output(nn_output) (Tuple[SingleDataOp x 3]): The state-value, advantage-values

(reshaped) and q-values (reshaped) after passing action_layer_output through the dueling layer.

get_action_layer_output(*args, **kwargs)

get_logits_probabilities_log_probs(*args, **kwargs)

4.2.3. Baseline Action Adapter

class rlgraph.components.action_adapters.baseline_action_adapter.BaselineActionAdapter(scope='baseline-action-adapter', **kwargs)

Bases: rlgraph.components.action_adapters.action_adapter.ActionAdapter

An ActionAdapter that adds 1 node to its action layer for an additional state-value output per batch item.

API:

get_state_values_and_logits(nn_output) (Tuple[SingleDataOp x 2]): The state-value and action logits (reshaped).

check_input_spaces(input_spaces, action_space=None)

Should check on the nature of all in-Sockets Spaces of this Component. This method is called automatically by the Model when all these Spaces are know during the Model’s build time.

Args:

input_spaces (Dict[str,Space]): A dict with Space/shape information.

keys=in-Socket name (str); values=the associated Space

action_space (Optional[Space]): The action Space of the Agent/GraphBuilder. Can be used to construct and

connect more Components (which rely on this information). This eliminates the need to pass the action Space information into many Components’ constructors.

get_logits(*args, **kwargs)

get_logits_probabilities_log_probs(*args, **kwargs)

get_state_values_and_logits(*args, **kwargs)