Type-based NCE about rsn HOT 3 CLOSED

In fact, we create two different weight matrices for entity prediction and relation prediction, respectively. When the target is a relation, we only need to call the standard NCE function tf.nn.nce_loss with the corresponding relation weight matrix as input.

from rsn.

Thanks for the explaination but I am still confused about the NCE-based negative sampling. My understanding is: in the entity prediction, the input is the entity embedding matrix and the tf.nn.nce_loss output is the negative samples, is it right?

from rsn.

Almost right. tf.nn.nce_loss is a high-level API that computes (include negative sampling) and returns the loss. If you want to customize the sampling algorithm, I suggest you have a look at log_uniform_candidate_sampler. tf.nn.nce_loss has such an augment sampled_values to receive sampled entities:

a tuple of (sampled_candidates, true_expected_count, sampled_expected_count) returned by a *_candidate_sampler function.

Thanks for your attention to our work.

from rsn.