expected to be in range of [-1, 0], but got 1 about tianshou HOT 4 OPEN

I noticed that if I change batch size to 32, then the weird "obs: torch.Size([64])" becomes "obs: torch.Size([32])"

so the error is somehow connected to the batch size

from tianshou.

buffer = PrioritizedVectorReplayBuffer(
    args.buffer_size,
    buffer_num=len(train_env),
    # ignore_obs_next=True,
    # save_only_last_obs=True,
    alpha=args.alpha,
    beta=args.beta,
)

when I comment out those two lines, problem solved.

Can someone help me explain what happens here? Really appreciate.

from tianshou.

@WillInvest I can look into it, but can you format your posts above to make the code more readable?

from tianshou.

buffer = PrioritizedVectorReplayBuffer(
    args.buffer_size,
    buffer_num=len(train_env),
    # ignore_obs_next=True,
    # save_only_last_obs=True,
    alpha=args.alpha,
    beta=args.beta,
)

when I comment out those two lines, problem solved.

Can someone help me explain what happens here? Really appreciate.

The buffer is currently very complicated, I was working on simplifying it and extending documentation in the last months. Will be merged to master soon.

ignore_obs_next should essentially never be set to True, it means that obs rolled forwards will be used instead of obs_next, which means making errors at the boundaries of episodes and at interrupted collections. It's only an option if one really really needs to save RAM, like maybe with atari envs, where observations are large.

save_only_last_obs should also not be touched.

from tianshou.