Common Components

Batch Splitter

class rlgraph.components.common.batch_splitter.BatchSplitter(num_shards, shard_size, **kwargs)

Bases: rlgraph.components.component.Component

Splits a number of incoming DataOps along their batch dimension.

Container Splitter

class rlgraph.components.common.container_splitter.ContainerSplitter(*output_order, **kwargs)

Bases: rlgraph.components.component.Component

Splits an incoming container Space into all its single primitive Spaces.

check_input_spaces(input_spaces, action_space=None)

Should check on the nature of all in-Sockets Spaces of this Component. This method is called automatically by the Model when all these Spaces are know during the Model’s build time.

Args:

input_spaces (Dict[str,Space]): A dict with Space/shape information.

keys=in-Socket name (str); values=the associated Space

action_space (Optional[Space]): The action Space of the Agent/GraphBuilder. Can be used to construct and

connect more Components (which rely on this information). This eliminates the need to pass the action Space information into many Components’ constructors.

create_variables(input_spaces, action_space=None)

Should create all variables that are needed within this component, unless a variable is only needed inside a single _graph_fn-method, in which case, it should be created there. Variables must be created via the backend-agnostic self.get_variable-method.

Note that for different scopes in which this component is being used, variables will not(!) be shared.

Args:

input_spaces (Dict[str,Space]): A dict with Space/shape information.

keys=in-Socket name (str); values=the associated Space

action_space (Optional[Space]): The action Space of the Agent/GraphBuilder. Can be used to construct and

connect more Components (which rely on this information). This eliminates the need to pass the action Space information into many Components’ constructors.

Dict Merger

class rlgraph.components.common.dict_merger.DictMerger(*input_names, **kwargs)

Bases: rlgraph.components.component.Component

Merges incoming items into one FlattenedDataOp.

API:

merge(*inputs) -> DataOpDict with keys=`self.input_names` and values=inputs

check_input_spaces(input_spaces, action_space=None)

Should check on the nature of all in-Sockets Spaces of this Component. This method is called automatically by the Model when all these Spaces are know during the Model’s build time.

Args:

input_spaces (Dict[str,Space]): A dict with Space/shape information.

keys=in-Socket name (str); values=the associated Space

action_space (Optional[Space]): The action Space of the Agent/GraphBuilder. Can be used to construct and

connect more Components (which rely on this information). This eliminates the need to pass the action Space information into many Components’ constructors.

Environment Stepper

class rlgraph.components.common.environment_stepper.EnvironmentStepper(environment_spec, actor_component_spec, num_steps=20, state_space=None, reward_space=None, internal_states_space=None, add_action_probs=False, action_probs_space=None, add_previous_action=False, add_previous_reward=False, **kwargs)

Bases: rlgraph.components.component.Component

A Component that takes an Environment object, a PreprocessorStack and a Policy to step n times through the environment, each time picking actions depending on the states that the environment produces.

API:

reset(): Resets the Environment stepper including its environment and gets everything ready for stepping.

Resets the stored state, return and terminal of the env.

step(): Performs n steps through the environment and returns some

collected stats: preprocessed_states, actions taken, (optional: action log-probabilities)?, rewards, accumulated episode returns, terminals, next states (un-preprocessed), (optional: internal states, only for RNN based ActorComponents).

create_variables(input_spaces, action_space=None)

Should create all variables that are needed within this component, unless a variable is only needed inside a single _graph_fn-method, in which case, it should be created there. Variables must be created via the backend-agnostic self.get_variable-method.

Note that for different scopes in which this component is being used, variables will not(!) be shared.

Args:

input_spaces (Dict[str,Space]): A dict with Space/shape information.

keys=in-Socket name (str); values=the associated Space

action_space (Optional[Space]): The action Space of the Agent/GraphBuilder. Can be used to construct and

connect more Components (which rely on this information). This eliminates the need to pass the action Space information into many Components’ constructors.

rlgraph.components.common.environment_stepper.force_tuple(elements=None, *, to_tuple=True)

Makes sure elements is returned as a list, whether elements is a single item, already a list, or a tuple.

Args:

elements (Optional[any]): The inputs as single item, list, or tuple to be converted into a list/tuple.

If None, returns empty list/tuple.

to_tuple (bool): Whether to use tuple (instead of list).

Returns:

Union[list,tuple]: All given elements in a list/tuple depending on to_tuple’s value. If elements is None,

returns an empty list/tuple.

Multi-GPU-Sync Synchronizer

Repeater Stack

class rlgraph.components.common.repeater_stack.RepeaterStack(sub_component, repeats=2, scope='repeater', **kwargs)

Bases: rlgraph.components.neural_networks.stack.Stack

A repeater is a special Stack that copies one(!) sub-Component n times and calls certain API-method(s) n times. n is the number of repeats.

API:

apply(input_) -> call’s some API-method on the “repeat-unit” (another Component) n times, each time passing the

result of the previous repeat and then returning the result of the last repeat.

Sampler

class rlgraph.components.common.sampler.Sampler(sampling_strategy='uniform', scope='sampler', **kwargs)

Bases: rlgraph.components.component.Component

A Sampling component can be used to sample entries from an input op, e.g. to repeatedly perform sub-sampling.

Slice

class rlgraph.components.common.slice.Slice(squeeze=False, scope='slice', **kwargs)

Bases: rlgraph.components.component.Component

A simple slicer layer. Slices off a piece from the input along the 0th rank returns it.

Staging Area

class rlgraph.components.common.staging_area.StagingArea(num_data=1, device='/device:GPU:0', scope='staging-area', **kwargs)

Bases: rlgraph.components.component.Component

Stages an arbitrary number of incoming ops for next-step processing. This allows for more efficient handling of dequeued (or otherwise pipelined) data: The data can be prepared and then staged while a training step is still taking place, the next training step can then immediately take the staged data, aso.asf..

create_variables(input_spaces, action_space=None)

Should create all variables that are needed within this component, unless a variable is only needed inside a single _graph_fn-method, in which case, it should be created there. Variables must be created via the backend-agnostic self.get_variable-method.

Note that for different scopes in which this component is being used, variables will not(!) be shared.

Args:

input_spaces (Dict[str,Space]): A dict with Space/shape information.

keys=in-Socket name (str); values=the associated Space

action_space (Optional[Space]): The action Space of the Agent/GraphBuilder. Can be used to construct and

connect more Components (which rely on this information). This eliminates the need to pass the action Space information into many Components’ constructors.

Synchronizable

class rlgraph.components.common.synchronizable.Synchronizable(*args, **kwargs)

Bases: rlgraph.components.component.Component

The Synchronizable Component adds a simple synchronization API to arbitrary Components to which this Synchronizable is added (and connected via connections=CONNECT_ALL). This is useful for constructions like a target network in DQN or for distributed setups where e.g. local policies need to be sync’d from a global model from time to time.

check_input_completeness()

Checks whether this Component is “input-complete” and stores the result in self.input_complete. Input-completeness is reached (only once and then it stays that way) if all API-methods of this component (whose must_be_complete field is not set to False) have all their input Spaces defined.

Returns:

bool: Whether this Component is input_complete or not.

Synchronizable

class rlgraph.components.common.synchronizable.Synchronizable(*args, **kwargs)

Bases: rlgraph.components.component.Component

The Synchronizable Component adds a simple synchronization API to arbitrary Components to which this Synchronizable is added (and connected via connections=CONNECT_ALL). This is useful for constructions like a target network in DQN or for distributed setups where e.g. local policies need to be sync’d from a global model from time to time.

check_input_completeness()

Checks whether this Component is “input-complete” and stores the result in self.input_complete. Input-completeness is reached (only once and then it stays that way) if all API-methods of this component (whose must_be_complete field is not set to False) have all their input Spaces defined.

Returns:

bool: Whether this Component is input_complete or not.