Class balancing in FineTune BERT about transformer-models HOT 4 CLOSED

I think you need to rearrange the arguments since the class weights argument is positional:

loss = crossentropy(Y,T,classWeights,"DataFormat","CB","WeightsFormat","C");

Otherwise that looks like what I'd expect.

from transformer-models.

One way would be to add class weights to the crossentropy call in the modelGradients function, documentation here.

from transformer-models.

Thanks @bwdGitHub for your quick response. Is the below call correct? Suppose my classWeights array is [2.5 0.5], where 2.5 corresponds to the minority class weight.

function [loss,gradients] = modelGradients(X,T,parameters, classWeights)

dropout = 0.1;
Y = model(X,parameters,dropout);
Y = softmax(Y,"DataFormat","CB");
    
loss = crossentropy(Y,T,"DataFormat","CB", classWeights, 'WeightsFormat','C');   % This line
gradients = dlgradient(loss,parameters.Weights);

end

from transformer-models.

Great thanks!

from transformer-models.