Transition Functions#

In this section we describe the transition function protocol, the transition function registry, and how to write your own custom transition functions.

The TransitionFunction Protocol#

A transition function is a generative function which represents a (stochastic) mapping from a state-action pair to a next states. Using the typing standard library, the transition function type is defined as a callable typing.Protocol which receives a State, an gym_gridverse.action.Action, and an optional numpy.random.Generator, and edits the input State.

class TransitionFunction(*args, **kwargs)[source]

Signature that all reset functions must follow

__call__(state, action, *, rng=None)[source]

Call self as a function.

Return type: None

Note

A transition function may (and often does) accept additional arguments; this is possible so long as the extra arguments either have default values, or are binded to specific values later on, e.g., using functools.partial().

The Transition Function Registry#

The transition_functions module contains some pre-defined transition functions and the transition_function_registry, a dictionary-like object through which to register and retrieve transition functions. Transition functions are registered using the register() method, which can be used as a decorator (also see Custom Transition Functions). As a dictionary, transition_function_registry has a keys() method which returns the names of registered functions.

Custom Transition Functions#

Custom transition functions can be defined so long as they satisfy some basic rules; A custom transition function:

MUST satisfy the TransitionFunction protocol.
MUST edit the input state, rather than return a new state altogether.
SHOULD use the rng argument as the source for any stochasticity.
MUST use get_gv_rng_if_none() (only if the rng is used at all).

Warning

The rng argument is used to control the source of randomness and allow for the environment to be seeded via set_seed(), which in turn guarantees the reproducibility of traces, runs, and experiments; if you wish to use external sources of randomness, you will have to manage them and their seeding yourself.

Practical Example 1#

Note

The examples shown here can be found in the examples/ folder.

In this example, we are going to write a transition function in which at every time-step one of the Floor tiles turns into a Wall tile.

from typing import Optional

import numpy.random as rnd

from gym_gridverse.action import Action
from gym_gridverse.envs.transition_functions import transition_function_registry
from gym_gridverse.grid_object import Floor, Wall
from gym_gridverse.rng import get_gv_rng_if_none
from gym_gridverse.state import State


@transition_function_registry.register
def creeping_walls(
    state: State,
    action: Action,
    *,
    rng: Optional[rnd.Generator] = None,
) -> None:
    """randomly converts a Floor into a Wall"""

    rng = get_gv_rng_if_none(rng)  # necessary to use rng object!

    # all Floor positions
    floor_positions = [
        position
        for position in state.grid.area.positions()
        if isinstance(state.grid[position], Floor)
    ]

    try:
        # choose a random Floor position ...
        i = rng.choice(len(floor_positions))
    except ValueError:
        # (floor_positions was an empty list)
        pass
    else:
        # ... and turn it into a Wall
        position = floor_positions[i]
        state.grid[position] = Wall()

Practical Example 2#

In this example, we are going to write a transition function in which the agent moves until it hits an obstacle.

from typing import Optional

import numpy.random as rnd

from gym_gridverse.action import Action
from gym_gridverse.envs.transition_functions import transition_function_registry
from gym_gridverse.geometry import Orientation, Position
from gym_gridverse.state import State

_action_orientations = {
    Action.MOVE_FORWARD: Orientation.F,
    Action.MOVE_LEFT: Orientation.L,
    Action.MOVE_RIGHT: Orientation.R,
    Action.MOVE_BACKWARD: Orientation.B,
}


@transition_function_registry.register
def chessrook_movement(
    state: State,
    action: Action,
    *,
    rng: Optional[rnd.Generator] = None,
) -> None:
    """moves the agent like a chess rook, until it hits an obstacle"""

    # get agent's movement direction
    try:
        action_orientation = _action_orientations[action]
    except KeyError:
        # not a movement action
        return

    movement_orientation = state.agent.orientation * action_orientation
    position_delta = Position.from_orientation(movement_orientation)

    # check positions until a blocking cell is found
    position = state.agent.position
    while not state.grid[position + position_delta].blocks_movement:
        position = position + position_delta

    state.agent.position = position

Practical Example 3#

In this example, we are going to write a transition function which randomizes the execution of another transition function.

from typing import Optional

import numpy.random as rnd

from gym_gridverse.action import Action
from gym_gridverse.envs.transition_functions import (
    TransitionFunction,
    transition_function_registry,
)
from gym_gridverse.rng import get_gv_rng_if_none
from gym_gridverse.state import State


@transition_function_registry.register
def random_transition(
    state: State,
    action: Action,
    *,
    transition_function: TransitionFunction,
    p_success: float,
    rng: Optional[rnd.Generator] = None,
) -> None:
    """randomly determines whether to perform a transition"""

    rng = get_gv_rng_if_none(rng)  # necessary to use rng object!

    # flip coin to run the transition_function
    if rng.random() <= p_success:
        transition_function(state, action, rng=rng)