ms-dot-k / visual-context-attentional-gan Goto Github PK
View Code? Open in Web Editor NEWPyTorch implementation of "Lip to Speech Synthesis with Visual Context Attentional GAN" (NeurIPS2021)
License: Other
PyTorch implementation of "Lip to Speech Synthesis with Visual Context Attentional GAN" (NeurIPS2021)
License: Other
After run the preprocess.py, i got the video the the top left part of the video rather than the lip centered. Is there bug in preprocess.py?
Hi All,
How are you?
Thank you for your wonderful and interesting work!
I'm trying to train VCA-GAN on LRS3 and wonder what steps I should follow for doing that. (what is the correct dir structure, etc.)
Cheers,
Traceback (most recent call last):
File "train.py", line 479, in
train_net(args)
File "train.py", line 122, in train_net
_ = validate(v_front, gen, post, fast_validate=True)
File "train.py", line 396, in validate
wav_spec = val_data.inverse_spec(gs[:, :, :, :mel_len[0]].detach(), stft)
File "XXXX/vid_aud_grid.py", line 216, in inverse_spec
wav = griffin_lim(spec.squeeze(1), stft.stft_fn, 60).squeeze(1) # B,L
File "XXXX/audio_processing.py", line 63, in griffin_lim
signal = stft_fn.inverse(magnitudes, angles).squeeze(1)
File "XXXX/Visual-Context-Attentional-GAN/src/data/stft.py", line 108, in inverse
padding=0)
RuntimeError: Given transposed=1, weight of size [1026, 1, 1024], expected input[44, 642, 298] to have 1026 channels, but got 642 channels instead
Hello, after I run preprocess.py, he will report this error after processing 989 pieces of data. Why is it
########## 982 / 32822 ##########
########## 983 / 32822 ##########
########## 984 / 32822 ##########
########## 985 / 32822 ##########
########## 986 / 32822 ##########
########## 987 / 32822 ##########
########## 988 / 32822 ##########
########## 989 / 32822 ##########
Traceback (most recent call last):
File "p1.py", line 121, in
for kk, data in enumerate(Data_loader):
File "/home/imu_liuxin/.conda/envs/hhs1/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 517, in next
data = self._next_data()
File "/home/imu_liuxin/.conda/envs/hhs1/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1199, in _next_data
return self._process_data(data)
File "/home/imu_liuxin/.conda/envs/hhs1/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1225, in _process_data
data.reraise()
File "/home/imu_liuxin/.conda/envs/hhs1/lib/python3.8/site-packages/torch/_utils.py", line 429, in reraise
raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 2.
Original Traceback (most recent call last):
File "/home/imu_liuxin/.conda/envs/hhs1/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 202, in _worker_loop
data = fetcher.fetch(index)
File "/home/imu_liuxin/.conda/envs/hhs1/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/imu_liuxin/.conda/envs/hhs1/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "p1.py", line 75, in getitem
v = np.stack(frames, 0)
File "<array_function internals>", line 200, in stack
File "/home/imu_liuxin/.conda/envs/hhs1/lib/python3.8/site-packages/numpy/core/shape_base.py", line 460, in stack
raise ValueError('need at least one array to stack')
ValueError: need at least one array to stack
Hello, i have downloaded your Pre-trained model checkpoints and then run the test_LRS.py. However, the stoi score we tested is 0.208, which is different from the provided score.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.