Actions, States, and Observations

Before seeing how the environments glue everything together, let us have a brief look into how actions, states, and observations are represented by their respective classes.

class Action(value)

Actions available to the agent.

There are (up to) 8 actions:

• 4 movement actions (forward, backwards, left & right)

• 2 turn actions (left & right)

• 1 actuate action, which can actuate objects (e.g., the one in front)

• 1 pick and drop action, to pick up objects (e.g., the one in front)

class State(grid, agent)

A state is a pure data-class containing grid and agent components.

This class offers little functionality, and is just a holder for grid and agent. The grid is an instance of class Grid, which represents the global state of the gridworld. The agent is an instance of class Agent, which represents the agent’s location, orientation, and held item.

class Observation(grid, agent)

An observation is a pure data-class containing grid and agent components.

This class offers little functionality, and is just a holder for grid and agent. The grid is an instance of class Grid, which represents the gridworld from the agent’s POV. The agent is an instance of class Agent, which represents the agent’s location, orientation, and held item, all from the agent’s POV.

States and observations are very simple container classes, and the details of what they contain are all in the respective Grid and Agent fields. However, for the time being, we will postpone looking into those classes, and move onto a very important design choice: the inner-outer environment separation.