tfurmston / tfrlrl Goto Github PK

Currently the project has Isort installed, but it is not configured in the pyproject.toml file.

Currently various assumptions are made about the data types of certain outputs of an environment. For example, it is currently assumed that actions are integers. Likewise, it is assumed that observations are in vector form. Instead, it would be preferable the data types of these data types were either configurable, and thus can be specified when initialising an environment, or use some form of generics.

The exact form of the approach taken to resolve this issues is currently open and needs to be scoped.

The existing implementation of the ReplayBuffer is inefficient in terms of memory consumption. In particular, it stores the same observation twice, once as an observation and once as a corresponding next_observation.

The purpose of this issue is to construct a more efficient implementation of the buffer class.

The current implementation of the validator descriptor classes only checks that the value is a NumPy ndarray. It would be better if it were possible to validate the types of the elements themselvees, e.g., through a generics approach.