About CodeBERT code2nl task about codebert HOT 3 CLOSED

In the pre-training stage, we use NL-Code Pair to learn both NL and Code representation. Although we use bimodal data in the pre-training stage, you can obtain NL or Code representation by taking only unimodal data as the input on downstream tasks, which is similar as XLM.

from codebert.

thank you for your reply.
The following picture describes the difference between the input and output of the pre-training phase and the fine-tuning phase. Is this correct?

from codebert.

The input of pre-training phase is [<s>,nl,</s>,code,</s>]. But in fine-tune phase, the input can be [<s>,nl,</s>,code,</s>], [<s>, nl, </s>], or [<s>, code,</s>], which depends on your downstream tasks. The output is representation of each token in both pre-training and fine-tune phases.

from codebert.