First Epoch Loss is Bigger than Expected about basset HOT 2 CLOSED

Hey Dadi, I'm a bit too tied up right now to dig in here myself. In Basset, the loss is defined here: https://github.com/davek44/Basset/blob/master/src/convnet.lua#L251. The binary cross entropy loss is described here: https://github.com/torch/nn/blob/master/doc/criterion.md#nn.BCECriterion. I'd open a torch session, and try some things to test your assumptions and get a feel for how these work.

from basset.

Hi David,

Sorry my assumption was totally wrong. By generating a dummy set with random targets labels either 0 or 1 and normally distributed probability between 0.0001 and 0.99, the defined BCE loss function loss(o, t) = - 1/n sum_i (t[i] * log(o[i]) + (1 - t[i]) * log(1 - o[i])) can exceed my expectation. It makes sense now.

BTW, seems the nucleotide order in motif heat map is reversed. I've compared the weight matrix and the heat map matrix. both agree to the reverse of current setting. I've reported it in a separate thread.

Apologize for disturbing you a lot.

Best,
Dadi

from basset.