4.8. Memories

4.8.1. Memory Base Class

class rlgraph.components.memories.memory.Memory(capacity=1000, scope='memory', **kwargs)

Bases: rlgraph.components.component.Component

Abstract memory component.

API:

insert_records(records) -> Triggers an insertion of records into the memory. get_records(num_records) -> Returns num_records records from the memory.

create_variables(input_spaces, action_space=None)

Should create all variables that are needed within this component, unless a variable is only needed inside a single _graph_fn-method, in which case, it should be created there. Variables must be created via the backend-agnostic self.get_variable-method.

Note that for different scopes in which this component is being used, variables will not(!) be shared.

Args:

input_spaces (Dict[str,Space]): A dict with Space/shape information.

keys=in-Socket name (str); values=the associated Space

action_space (Optional[Space]): The action Space of the Agent/GraphBuilder. Can be used to construct and

connect more Components (which rely on this information). This eliminates the need to pass the action Space information into many Components’ constructors.

4.8.2. ReplayMemory

class rlgraph.components.memories.replay_memory.ReplayMemory(capacity=1000, scope='replay-memory', **kwargs)

Bases: rlgraph.components.memories.memory.Memory

Implements a standard replay memory to sample randomized batches.

create_variables(input_spaces, action_space=None)

Should create all variables that are needed within this component, unless a variable is only needed inside a single _graph_fn-method, in which case, it should be created there. Variables must be created via the backend-agnostic self.get_variable-method.

Note that for different scopes in which this component is being used, variables will not(!) be shared.

Args:

input_spaces (Dict[str,Space]): A dict with Space/shape information.

keys=in-Socket name (str); values=the associated Space

action_space (Optional[Space]): The action Space of the Agent/GraphBuilder. Can be used to construct and

connect more Components (which rely on this information). This eliminates the need to pass the action Space information into many Components’ constructors.

4.8.3. PrioritizedReplay

class rlgraph.components.memories.prioritized_replay.PrioritizedReplay(capacity=1000, alpha=1.0, beta=0.0, scope='prioritized-replay', **kwargs)

Bases: rlgraph.components.memories.memory.Memory

Implements pure TensorFlow prioritized replay.

API:

update_records(indices, update) -> Updates the given indices with the given priority scores.

create_variables(input_spaces, action_space=None)

Should create all variables that are needed within this component, unless a variable is only needed inside a single _graph_fn-method, in which case, it should be created there. Variables must be created via the backend-agnostic self.get_variable-method.

Note that for different scopes in which this component is being used, variables will not(!) be shared.

Args:

input_spaces (Dict[str,Space]): A dict with Space/shape information.

keys=in-Socket name (str); values=the associated Space

action_space (Optional[Space]): The action Space of the Agent/GraphBuilder. Can be used to construct and

connect more Components (which rely on this information). This eliminates the need to pass the action Space information into many Components’ constructors.

4.8.4. FIFOQueue

class rlgraph.components.memories.fifo_queue.FIFOQueue(record_space=None, only_insert_single_records=False, **kwargs)

Bases: rlgraph.components.memories.memory.Memory

A wrapper for a simple in-graph FIFOQueue.

create_variables(input_spaces, action_space=None)

Should create all variables that are needed within this component, unless a variable is only needed inside a single _graph_fn-method, in which case, it should be created there. Variables must be created via the backend-agnostic self.get_variable-method.

Note that for different scopes in which this component is being used, variables will not(!) be shared.

Args:

input_spaces (Dict[str,Space]): A dict with Space/shape information.

keys=in-Socket name (str); values=the associated Space

action_space (Optional[Space]): The action Space of the Agent/GraphBuilder. Can be used to construct and

connect more Components (which rely on this information). This eliminates the need to pass the action Space information into many Components’ constructors.

4.8.5. QueueRunner

class rlgraph.components.memories.queue_runner.QueueRunner(queue, api_method_name, return_slot, env_output_splitter, fifo_input_merger, next_states_slicer, internal_states_slicer, *data_producing_components, **kwargs)

Bases: rlgraph.components.component.Component

A queue runner that contains n sub-components, of which an API-method is called. The return values are bundled into a FIFOQueue as inputs. Queue runner uses multi-threading and is started after session creation.

API: enqueue() -> Returns a noop, but creates the enqueue ops for enqueuing data into the queue and hands these

to the underlying queue-runner object.