Is there a way to log the loss during training? about mushroom-rl HOT 9 CLOSED

we are working into it right now. In the next few days, the support will be added in version 1.7.0.

from mushroom-rl.

I've implemented the feature in the above-mentioned commit.
It still experimental, and it's available in the dev branch.
Please feel free to report bugs or propose enhancement.

from mushroom-rl.

is there an example of this? I would like to get the loss value to plot it in tensorboard.
thanks!

from mushroom-rl.

Yes! TorchApproximator has a property called loss_fit, that returns the average value of the loss of the last fit call.

Unfortunately, we don't log the value of the loss of every epoch in the fit, if multiple epochs are required in the fit call.

However, if you are working with algorithms such as DQN, you just need to write a callback that does the following:

1. take the q function approximator
2. use the "model" property to get access to TorchApproximator
3. use the "loss_fit" property to get the loss of the last fit call (in DQN is just one step on a single minibatch)
4. log the data as you prefer

Maybe we would add more functionalities to log the loss in the future, but right now we have other priorities: we are porting to the new version of Mujoco and soon the new version of gym, so our time for other improvements is limited.

from mushroom-rl.

@davidenitti I've also updated the documentation, which was not up to date. Probably we need a tutorial, but I guess you cannot expect it soon due to time reasons explained above.

from mushroom-rl.

@boris-il-forte thanks, I added a writer (from tensorboard in core and added this after the if fit_condition():

if self.writer is not None and self.agent.approximator.model.loss_fit is not None:
    self.writer.add_scalar("train/loss", self.agent.approximator.model.loss_fit, self._total_steps_counter)

if there are multiple losses would loss_fit contain them all? is there another way to do it? maybe in the regressor?
thanks!

from mushroom-rl.

You don't need to modify the core, you could just use a fit callback (passing it to core). It can also be an object.

Unfortunately, we didn't test multiple losses, but I guess it will not work. However, if you are passing a class loss to the DQN fit, that contains any loss you want, you could simply log your loss there manually.

from mushroom-rl.

I don't know if it's possible to access self.agent.approximator.model.loss_fit from the callback_fit which gets the dataset only as input. (unless I modify the callbacks_fit to get other variables as input so I can get loss_fit

for c in self.callbacks_fit:
    c(dataset)

from mushroom-rl.

the callback can be an object. Just pass .model attribute to the callback constructor, and then you have access to it.

Anyways, this is for sure another topic that needs improvements for the logging as is more relevant nowadays. I'll try to look at it in the near future

from mushroom-rl.