Git Product home page Git Product logo

Comments (3)

Toni-SM avatar Toni-SM commented on August 24, 2024 1

Hi @sjYoondeltar

I assume the regularization noise is always defined by the user for training.
I will fix the implementation to ignore its use when it's None (and show a warning message in this case)

A typical configuration can be found in the following example (td3_gym_pendulum.py):

skrl/docs/source/examples/gym/td3_gym_pendulum.py

Lines 87 to 88 in b760dc7

cfg_td3["smooth_regularization_noise"] = GaussianNoise(0, 0.2, device=device)
cfg_td3["smooth_regularization_clip"] = 0.5

In this example, the parameters are the same as those used by the TD3 authors to evaluate the algorithm in the paper

Screenshot from 2023-01-28 17-21-59

from skrl.

Toni-SM avatar Toni-SM commented on August 24, 2024 1

Hi @sjYoondeltar

Many thanks for reporting this issue.

A warning message has been added to the development version (develop branch) when the regularization noise is not specified and its availability is checked before applying it.

skrl/skrl/agents/torch/td3/td3.py

Lines 160 to 161 in bca2173

if self._smooth_regularization_noise is None:
logger.warning("agents:TD3: No smooth regularization noise specified to reduce variance during training")

skrl/skrl/agents/torch/td3/td3.py

Lines 357 to 361 in bca2173

if self._smooth_regularization_noise is not None:
noises = torch.clamp(self._smooth_regularization_noise.sample(next_actions.shape),
min=-self._smooth_regularization_clip,
max=self._smooth_regularization_clip)
next_actions.add_(noises)

from skrl.

sjYoondeltar avatar sjYoondeltar commented on August 24, 2024

Thanks for your answering @Toni-SM

from skrl.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.