์๋
ํ์ธ์!
NLP๋ฅผ ๊ณต๋ถํ๊ณ ์๋ ํ์์
๋๋ค.
๊ธฐ์ฌ ๋๊ธ ๋ฐ์ดํฐ๋ฅผ ์ฒ๋ฆฌํ๋๋ฐ ์์ฃผ ์ ํฉํ KcBERT๋ฅผ ์ฐพ์์ ๋งค์ฐ ์ ์ฌ์ฉํ๊ณ ์์ต๋๋ค. ๊ฐ์ฌํฉ๋๋ค :)
๊ทธ๋ฐ๋ฐ, ๊ณต์ ํด์ฃผ์ NSMC ๋ฐ์ดํฐ์
fine-tuningํ๋ ์ฝ๋์์ ๋ค๋ฅธ ๋ฐ์ดํฐ์
์ ์ฌ์ฉํ ๋ ๋ฐ์ํ ์๋ฌ๋ฅผ ์ก์ง ๋ชปํ๊ณ ์์ต๋๋ค.
์ฒ์์๋ RuntimeError: CUDA error: device-side assert triggered
์๋ฌ๊ฐ ๋์ ํด๋น ์ฌ์ดํธ์์ Runtime type์ None์ผ๋ก ๋ณ๊ฒฝํ๋ฉด ์ค์ ๋ฌธ์ ๋ฅผ ๋ณด์ฌ์ค๋ค๊ณ ํฉ๋๋ค. ๊ทธ ์ดํ ๋ง์ฃผ์น ์๋ฌ๋ ๋ค์๊ณผ ๊ฐ์ต๋๋ค: IndexError: Target 2 is out of bounds.
-
์ฝ๋ฉํ๊ฒฝ(๊ณต์ ํด์ฃผ์ ๋ค์ด๋ฒ ์ํํ ๋ฐ์ดํฐ์
fine-tuning Large Model): https://colab.research.google.com/drive/1dFC0FL-521m7CL_PSd8RLKq67jgTJVhL?usp=sharing
-
์๋ฌ: IndexError: Target 2 is out of bounds.
-
์ํฉ: ์ ๊ณตํด์ฃผ์ ์ฝ๋ฉ ํ๊ฒฝ์์ ๋ค์ด๋ฒ ์ํํ ๋ฐ์ดํฐ์
์ด ์๋ 5๊ฐ์ ๊ฐ์ฑ ํด๋์ค(๋งค์ฐ ๋ถ์ 0, ๋ถ์ 1, ์ค๋ฆฝ 2, ๊ธ์ 3, ๋งค์ฐ ๊ธ์ 4)๋ฅผ ๊ฐ์ง๊ณ ์๋ ๋ฐ์ดํฐ์
์ผ๋ก fine-tuning ํ๋ ๊ณผ์ ์์ ํด๋น ์๋ฌ๊ฐ ๊ณ์ ์๊น๋๋ค.
-
๋ฐ์ดํฐ์
๊ฐ์: ๋๊ธ ๋ฐ์ดํฐ train
: 11,281๊ฐ / test
: 1,253๊ฐ
-
๋ฐ์ดํฐ์
๊ตฌ์ฑ: NSMC ๋ฐ์ดํฐ์
๊ณผ ๋์ผํ๊ฒ id
, document
, label
๋ก ์์ ํ์๊ณ , tab์ผ๋ก ๋์ด์ด txt ํ์ผ์
๋๋ค.
๋ฐ์ดํฐ ๊ฒฝ๋ก๋ง ์์ ํ์๊ณ , ๋ค๋ฅธ ๋ถ๋ถ์ ๊ฑด๋ค์ง ์์์ต๋๋ค.
5๊ฐ์ ๋ ์ด๋ธ์ ๊ฐ์ง ๋ฐ์ดํฐ์
์ ํด๋น ์ฝ๋์์ ์ฌ์ฉํ๋ ค๋ฉด ์ด๋ป๊ฒ ํ๋ฉด ์ข์๊น์?
์ ๊ฐ ๋ฌด์์ ๋์น๊ณ ์๋ ๊ฒ์ธ์ง ๋์ ์ฃผ์ค์ ์์ผ์ค๊น์?
์๋ฌ ์ฝ๋ ์ ๋ฌธ ๊ณต์ ๋๋ฆฝ๋๋ค.
IndexError Traceback (most recent call last)
in ()
----> 1 main()
18 frames
in main()
18 # tpu_cores=args.tpu_cores if args.tpu_cores else None,
19 )
---> 20 trainer.fit(model)
/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/states.py in wrapped_fn(self, *args, **kwargs)
46 if entering is not None:
47 self.state = entering
---> 48 result = fn(self, *args, **kwargs)
49
50 # The INTERRUPTED state can be set inside the run function. To indicate that run was interrupted
/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/trainer.py in fit(self, model, train_dataloader, val_dataloaders, datamodule)
1082 self.accelerator_backend = CPUBackend(self)
1083 self.accelerator_backend.setup(model)
-> 1084 results = self.accelerator_backend.train(model)
1085
1086 # on fit end callback
/usr/local/lib/python3.6/dist-packages/pytorch_lightning/accelerators/cpu_backend.py in train(self, model)
37
38 def train(self, model):
---> 39 results = self.trainer.run_pretrain_routine(model)
40 return results
/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/trainer.py in run_pretrain_routine(self, model)
1237
1238 # CORE TRAINING LOOP
-> 1239 self.train()
1240
1241 def _run_sanity_check(self, ref_model, model):
/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py in train(self)
392 # RUN TNG EPOCH
393 # -----------------
--> 394 self.run_training_epoch()
395
396 if self.max_steps and self.max_steps <= self.global_step:
/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py in run_training_epoch(self)
489 # TRAINING_STEP + TRAINING_STEP_END
490 # ------------------------------------
--> 491 batch_output = self.run_training_batch(batch, batch_idx)
492
493 # only track outputs when user implements training_epoch_end
/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py in run_training_batch(self, batch, batch_idx)
842 opt_idx,
843 optimizer,
--> 844 self.hiddens
845 )
846 using_results_obj = isinstance(opt_closure_result.training_step_output, Result)
/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py in optimizer_closure(self, split_batch, batch_idx, opt_idx, optimizer, hiddens)
1013 else:
1014 training_step_output = self.training_forward(split_batch, batch_idx, opt_idx,
-> 1015 hiddens)
1016
1017 # ----------------------------
/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py in training_forward(self, batch, batch_idx, opt_idx, hiddens)
1224 # CPU forward
1225 else:
-> 1226 output = self.model.training_step(*args)
1227
1228 is_result_obj = isinstance(output, Result)
in training_step(self, batch, batch_idx)
15 def training_step(self, batch, batch_idx):
16 data, labels = batch
---> 17 loss, logits = self(input_ids=data, labels=labels)
18 preds = logits.argmax(dim=-1)
19
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
720 result = self._slow_forward(*input, **kwargs)
721 else:
--> 722 result = self.forward(*input, **kwargs)
723 for hook in itertools.chain(
724 _global_forward_hooks.values(),
in forward(self, **kwargs)
11
12 def forward(self, **kwargs):
---> 13 return self.bert(**kwargs)
14
15 def training_step(self, batch, batch_idx):
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
720 result = self._slow_forward(*input, **kwargs)
721 else:
--> 722 result = self.forward(*input, **kwargs)
723 for hook in itertools.chain(
724 _global_forward_hooks.values(),
/usr/local/lib/python3.6/dist-packages/transformers/modeling_bert.py in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, labels, output_attentions, output_hidden_states, return_dict)
1340 else:
1341 loss_fct = CrossEntropyLoss()
-> 1342 loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
1343
1344 if not return_dict:
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
720 result = self._slow_forward(*input, **kwargs)
721 else:
--> 722 result = self.forward(*input, **kwargs)
723 for hook in itertools.chain(
724 _global_forward_hooks.values(),
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/loss.py in forward(self, input, target)
946 def forward(self, input: Tensor, target: Tensor) -> Tensor:
947 return F.cross_entropy(input, target, weight=self.weight,
--> 948 ignore_index=self.ignore_index, reduction=self.reduction)
949
950
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
2420 if size_average is not None or reduce is not None:
2421 reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 2422 return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
2423
2424
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
2216 .format(input.size(0), target.size(0)))
2217 if dim == 2:
-> 2218 ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
2219 elif dim == 4:
2220 ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
IndexError: Target 2 is out of bounds.