Hi, when I am testing a pretrained model, I get very different accuracy numbers in different runs. Let's say I am using UFLD-ADDA. Below is the config file I used. Even before I train, the validate and test accuracy numbers vary quite a lot between different runs (sometimes by 4-6%).
Also, what is the difference between using 'pretrained' vs. 'finetune' in a config file? If I just want to finetune a pretrained model, should I just specify that model in pretrained or also in finetune?
molane config file:
DATA
data_root = '../UFLD-DANN/MoLane/data/'
source_train = '../UFLD-DANN/MoLane/splits/source_train.txt'
target_train = '../UFLD-DANN/MoLane/splits/target_train.txt'
source_val = '../UFLD-DANN/MoLane/splits/source_val.txt'
target_val = '../UFLD-DANN/MoLane/splits/target_val.txt'
target_test = '../UFLD-DANN/MoLane/splits/target_test.txt'
TRAIN
epoch = 1
batch_size = 16
optimizer = 'Adam' # ['SGD','Adam']
learning_rate = 1e-6
learning_rate_disc = 1e-3
weight_decay = 2.5e-5
momentum = 0.9
slope = 0.2
scheduler = 'cos' # ['multi', 'cos']
steps = [50,75]
gamma = 0.1
warmup = 'linear'
warmup_iters = 100
NETWORK
backbone = '18'
griding_num = 100
num_lanes = 2
cls_num_per_lane = 56
use_aux = False
num_workers = 4
LOSS
sim_loss_w = 1.0
shp_loss_w = 0.0
EXP
note = ''
log_path = './ufld_adda_log/'
FINETUNE or RESUME MODEL PATH
pretrained = '../pretrained/ufld_so_resnet18_ep149.pth'
finetune = None
resume = None
TEST
test_model = None
test_work_dir = None
Run 1 output:
[Validate on source]: 100%|ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ| 250/250 [01:37<00:00, 2.56it/s, top1=0.879, top2=0.968, top3=0.978]
[Validate on target]: 100%|ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ| 125/125 [00:49<00:00, 2.52it/s, top1=0.752, top2=0.868, top3=0.891]
[Validate Disc]: 100%|ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ| 125/125 [00:49<00:00, 2.53it/s, acc=0.422, correct=1687, total=4000]
[Test]: 100%|ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ| 63/63 [00:25<00:00, 2.48it/s, top1=0.759, top2=0.873, top3=0.882]
Run 2 output (without any change from run 1):
[Validate on source]: 100%|ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ| 250/250 [01:38<00:00, 2.55it/s, top1=0.866, top2=0.959, top3=0.972]
[Validate on target]: 100%|ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ| 125/125 [00:49<00:00, 2.55it/s, top1=0.753, top2=0.854, top3=0.873]
[Validate Disc]: 100%|ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ| 125/125 [00:49<00:00, 2.53it/s, acc=0.527, correct=2107, total=4000]
[Test]: 100%|ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ| 63/63 [00:24<00:00, 2.53it/s, top1=0.790, top2=0.901, top3=0.911]