Batch: don't just strip off empty entries when creating batches about tianshou HOT 5 OPEN

I'm quite sure these assignments shouldn't be valid for a batch, should they? If so, what should be the expected behavior in your given examples, especially if another batch item with the same key/structure should be concatenated or stacked? Maybe some kind of sanity check should run before constructing a batch, although this kind of data probably doesn't occur in the RL context.

from tianshou.

Yes, one way to solve this issue is to raise errors on invalid assignments. It's generally better to be strict. So many things can go wrong inside an RL training, we should help the users by eliminating as many possible sources of errors as we can

from tianshou.

We shouldn't completely eliminate object-type arrays though. For example our current vector envs return an array of dicts for the info entry. Custom envs are also free to return whatever they want as observations

from tianshou.

Arrays of dicts should be fine (such as info entries), as long as the dict keys are consistent.

from tianshou.

I agree. It might not make sense to solve the batch related issues independently, maybe a single PR will have to address multiple of them

from tianshou.