Comments (7)
Glad you agree with me on this ^^. I'm not sure whether anywhere in the code the retrieval of the slice with empty values is used. For me it's fine to completely remove it, however, many tests will need to be adjusted, as now many of them rely on this somehow weird retrieval mechanism.
We could live without the full retrieval method until someone actually needs it. It's a good practice to keep the public interface as small as possible
from tianshou.
Sorry, I'm afraid I don't understand what you are asking. We can have a call on Friday if you want to :)
from tianshou.
I can look into this.
Currently this is the signature of ReplayBuffer
's __getitem__(index: slice | int | list[int] | np.ndarray)
.
So the idea here is to not support full-buffer (filled and non-filled entries) retrieval, when slice is provided, for buffer[:].obs
because it can be computationally expensive?
What about the cases when user gives a list[int]
or np.ndarray
with all the indices? Practically it is the same as slice. Should we allow retrieving the full buffer in this case?
from tianshou.
I'm not sure when we should ever be retrieving the full buffer. Tbh I haven't given too much thought about the best way of resolving this, it just seems very confusing and arbitrary that buffer.obs[:]
and buffer[:].obs
would be semantically wildly different entities, right?
from tianshou.
I have reviewed this issue again. The different way of retrieving with slicing seems arbitrary to me. The user can check the maximum size via the buffer.max_size
attribute, no need to return the empty values as well. A special method to retrieve the full buffer (if anyone ever needs it!?) as you mentioned is more appropriate.
We should probably update buffer.obs[:]
(and other slicing methods like start:stop
) to retrieve only non-empty values like buffer[:].obs
does.
Here are some examples of how indexing is currently different between the two:
In [52]: dummy_buf = ReplayBuffer(size=10)
...: for i in range(6):
...: dummy_buf.add(
...: Batch(obs=i, act=i, rew=i, terminated=0, truncated=0, done=0, obs_next=i + 1, info={}),
...: )
...:
In [53]: dummy_buf.obs[2:8]
Out[53]: array([2, 3, 4, 5, 0, 0])
In [54]: dummy_buf[2:8].obs
Out[54]: array([2, 3, 4, 5])
In [55]: dummy_buf.obs[:]
Out[55]: array([0, 1, 2, 3, 4, 5, 0, 0, 0, 0])
In [56]: dummy_buf[:].obs
Out[56]: array([0, 1, 2, 3, 4, 5])
In [57]: dummy_buf.obs[np.arange(10)]
Out[57]: array([0, 1, 2, 3, 4, 5, 0, 0, 0, 0])
In [58]: dummy_buf[np.arange(10)].obs
Out[58]: array([0, 1, 2, 3, 4, 5, 0, 0, 0, 0])
from tianshou.
Closed the issue accidentally, sry for that
from tianshou.
![image](https://private-user-images.githubusercontent.com/1534513/324856219-2694b3f3-6445-475c-a2f9-8296469a4b85.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM4MzU0NzMsIm5iZiI6MTcyMzgzNTE3MywicGF0aCI6Ii8xNTM0NTEzLzMyNDg1NjIxOS0yNjk0YjNmMy02NDQ1LTQ3NWMtYTJmOS04Mjk2NDY5YTRiODUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDgxNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA4MTZUMTkwNjEzWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MjFkY2ViZmI3ZDgyZDg3NWFjYTYxMmM2MzRkMGE1Njg5NjA1MDJmYzM4MzNjZDU3ZGYzMzg0NmQ0YjQ2YjhjYSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.Pjbd8jJLqZ6ISMfr560ZPoM1HprPgnMt5ZaOm3e2wxA)
Something is off here. As seen above, index
is of type str
so Batch.__getitem__
immediately returns. How can we get access on the last .[]
?
EDIT: dummy_buffer.obs
is of type np.ndarray
so we can't have direct access on that slicing.
from tianshou.
Related Issues (20)
- will add dreamerv3 ? HOT 1
- Documentation for multi-agent needs fixing HOT 1
- No minibatch for computation of logp_old in PPOPolicy HOT 1
- MPO Implementation HOT 1
- Improve interface of BasePolicy.compute_action HOT 1
- Suggestion - Redesign RayEnvWorker for Improved Performance HOT 12
- tianshou v1.0.0 failed to install on python 3.12.4 HOT 2
- How to add a obs with multiple dimensions to the buffer?
- TypeError: isinstance() argument 2 cannot be a parameterized generic HOT 1
- Make Batch.__eq__ work also for scalar values (0-dimensional arrays) HOT 2
- EOFError on training HOT 8
- ModuleNotFoundError: No module named 'tianshou.highlevel' HOT 3
- Cannot unify float64 and array(float64, 1d, C) for 'gae.2'
- Discrete distributions are broken HOT 2
- Actor and distribution function should not be specified independently HOT 1
- Learning rate scheduler confusion HOT 1
- Handling LR scheduling when episodes instead of steps are collected
- Release 1.1.0 to pypi HOT 1
- Refactoring: make Trainer more flexible
- Support custom scores for Trainer in high-level interfaces
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tianshou.