And can you provide the train.csv file in the form of dataset_root/video_1.mp4 label_1

Hi <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="

Why should the video corresponding label be added to the annotations? about videomae HOT 1 CLOSED

HaleYan commented on June 8, 2024

Why should the video corresponding label be added to the annotations?

from videomae.

Comments (1)

yztongzhan commented on June 8, 2024

Hi @HaleYan!
We didn't use the corresponding label during the pre-training phase.
You can check the code below:

VideoMAE/kinetics.py

Lines 471 to 493 in e93f386

 def __getitem__(self, index): 

 directory, target = self.clips[index] 

 if self.video_loader: 

 if '.' in directory.split('/')[-1]: 

 # data in the "setting" file already have extension, e.g., demo.mp4 

 video_name = directory 

 else: 

 # data in the "setting" file do not have extension, e.g., demo 

 # So we need to provide extension (i.e., .mp4) to complete the file name. 

 video_name = '{}.{}'.format(directory, self.video_ext) 

 decord_vr = decord.VideoReader(video_name, num_threads=1) 

 duration = len(decord_vr) 

 segment_indices, skip_offsets = self._sample_train_indices(duration) 

 images = self._video_TSN_decord_batch_loader(directory, decord_vr, duration, segment_indices, skip_offsets) 

 process_data, mask = self.transform((images, None)) # T*C,H,W 

 process_data = process_data.view((self.new_length, 3) + process_data.size()[-2:]).transpose(0,1) # T*C,H,W -> T,C,H,W -> C,T,H,W 

 return (process_data, mask)

for details.

from videomae.

Why should the video corresponding label be added to the annotations? about videomae HOT 1 CLOSED

Comments (1)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	def __getitem__(self, index):

	directory, target = self.clips[index]
	if self.video_loader:
	if '.' in directory.split('/')[-1]:
	# data in the "setting" file already have extension, e.g., demo.mp4
	video_name = directory
	else:
	# data in the "setting" file do not have extension, e.g., demo
	# So we need to provide extension (i.e., .mp4) to complete the file name.
	video_name = '{}.{}'.format(directory, self.video_ext)

	decord_vr = decord.VideoReader(video_name, num_threads=1)
	duration = len(decord_vr)

	segment_indices, skip_offsets = self._sample_train_indices(duration)

	images = self._video_TSN_decord_batch_loader(directory, decord_vr, duration, segment_indices, skip_offsets)

	process_data, mask = self.transform((images, None)) # T*C,H,W
	process_data = process_data.view((self.new_length, 3) + process_data.size()[-2:]).transpose(0,1) # T*C,H,W -> T,C,H,W -> C,T,H,W

	return (process_data, mask)