Reset Functions#

In this section we describe the reset function protocol, the reset function registry, and how to write your own custom reset functions.

The ResetFunction Protocol#

A reset function is a generative function which represents a (stochastic) distribution over initial states. Using the typing standard library, the reset function type is defined as a callable typing.Protocol which receives an optional numpy.random.Generator and returns a State.

class ResetFunction(*args, **kwargs)[source]

Signature that all reset functions must follow

__call__(*, rng=None)[source]

Call self as a function.

Return type: State

Note

A reset function may (and often does) accept additional arguments; this is possible so long as the extra arguments either have default values, or are binded to specific values later on, e.g., using functools.partial().

The Reset Function Registry#

The reset_functions module contains some pre-defined reset functions and the reset_function_registry, a dictionary-like object through which to register and retrieve reset functions. Reset functions are registered using the register() method, which can be used as a decorator (also see Custom Reset Functions). As a dictionary, reset_function_registry has a keys() method which returns the names of registered functions.

Custom Reset Functions#

Custom reset functions can be defined so long as they satisfy some basic rules; A custom reset function:

MUST satisfy the ResetFunction protocol.
SHOULD use the rng argument as the source for any stochasticity.
MUST use get_gv_rng_if_none() (only if the rng is used at all).

Warning

The rng argument is used to control the source of randomness and allow for the environment to be seeded via set_seed(), which in turn guarantees the reproducibility of traces, runs, and experiments; if you wish to use external sources of randomness, you will have to manage them and their seeding yourself.

Practical Example 1#

Note

The examples shown here can be found in the examples/ folder.

In this example, we are going to write an extremely simple reset function in which the agent is positioned in front of a Exit.

from typing import Optional

import numpy.random as rnd

from gym_gridverse.agent import Agent
from gym_gridverse.envs.reset_functions import reset_function_registry
from gym_gridverse.geometry import Orientation, Position
from gym_gridverse.grid import Grid
from gym_gridverse.grid_object import Exit, Floor, Wall
from gym_gridverse.state import State


@reset_function_registry.register
def simplest(*, rng: Optional[rnd.Generator] = None) -> State:
    """smallest possible room with exit right in front of agent"""

    # constructed the grid directly from objects
    grid = Grid(
        [
            [Wall(), Wall(), Wall()],
            [Wall(), Exit(), Wall()],
            [Wall(), Floor(), Wall()],
            [Wall(), Wall(), Wall()],
        ]
    )

    # positioning the agent in the above grid
    agent = Agent(Position(2, 1), Orientation.F)

    return State(grid, agent)

Practical Example 2#

In this example, we are going to write a reset function for an environment in which the agent needs to pick up the correctly colored Key to open a randomly colored Door and reach the Exit.

from typing import Optional

import numpy.random as rnd

from gym_gridverse.agent import Agent
from gym_gridverse.envs.reset_functions import reset_function_registry
from gym_gridverse.geometry import Orientation, Position
from gym_gridverse.grid import Grid
from gym_gridverse.grid_object import Color, Door, Exit, Floor, Key, Wall
from gym_gridverse.rng import choice, get_gv_rng_if_none, shuffle
from gym_gridverse.state import State


@reset_function_registry.register
def choose_key(*, rng: Optional[rnd.Generator] = None) -> State:
    """the agent has to pick the correct key to open a randomly colored door"""

    rng = get_gv_rng_if_none(rng)  # necessary to use rng object!

    # only consider these colors
    colors = [Color.RED, Color.GREEN, Color.BLUE, Color.YELLOW]

    # randomly select key locations
    keys = shuffle(rng, [Key(color) for color in colors])
    # randomly select door color
    color = choice(rng, colors)

    # grids can be constructed directly from objects
    door = Door(Door.Status.LOCKED, color)
    grid = Grid(
        [
            [Wall(), Wall(), Wall(), Wall(), Wall()],
            [Wall(), Wall(), Exit(), Wall(), Wall()],
            [Wall(), Wall(), door, Wall(), Wall()],
            [Wall(), keys[0], Floor(), keys[1], Wall()],
            [Wall(), keys[2], Floor(), keys[3], Wall()],
            [Wall(), Wall(), Wall(), Wall(), Wall()],
        ]
    )

    # positioning the agent in the above grid
    agent = Agent(Position(4, 2), Orientation.F)

    return State(grid, agent)