wangsuzhen / audio2head Goto Github PK

Dear suzhen,

Thank you for publishing the inference code. It is an amazing work. But how can I train a new model with higher resolution. I would like to ask whether you will publish the train code.

Looking forward to your kind reply.

Regards,
Bolin

issue：Traceback (most recent call last):
File "inference.py", line 251, in
audio2head(parse.audio_path,parse.img_path,parse.model_path,parse.save_path)
File "inference.py", line 233, in audio2head
imageio.mimsave(video_path, predictions_gen,fps=25.0)
File "D:\miniconda3\envs\a2\lib\site-packages\imageio\v2.py", line 495, in mimwrite
return file.write(ims, is_batch=True, **kwargs)
File "D:\miniconda3\envs\a2\lib\site-packages\imageio\plugins\tifffile_v3.py", line 244, in write
self._fh.write(image, **kwargs)
TypeError: write() got an unexpected keyword argument 'fps'

I've read your paper and am excited to test it myself.

However, I have a question below just for double-checking purpose :
Is the released pretrained model trained on voxceleb1?

thank you

Hi

How to train a new model?

How to prepare the dataset?
and how to invoke the training code? what are the optimal hyperparameters? etc.

C:\Users\flyingree\Downloads\Audio2Head-main>python inference.py --audio_path temp.wav --img_path 1.jpg ffmpeg version 6.0-essentials_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers built with gcc 12.2.0 (Rev10, Built by MSYS2 project) configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libvpl --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband libavutil 58. 2.100 / 58. 2.100 libavcodec 60. 3.100 / 60. 3.100 libavformat 60. 3.100 / 60. 3.100 libavdevice 60. 1.100 / 60. 1.100 libavfilter 9. 3.100 / 9. 3.100 libswscale 7. 1.100 / 7. 1.100 libswresample 4. 10.100 / 4. 10.100 libpostproc 57. 1.100 / 57. 1.100 Guessed Channel Layout for Input Stream #0.0 : mono Input #0, wav, from 'temp.wav': Duration: 00:00:10.00, bitrate: 352 kb/s Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 22050 Hz, 1 channels, s16, 352 kb/s Stream mapping: Stream #0:0 -> #0:0 (pcm_s16le (native) -> pcm_s16le (native)) Press [q] to stop, [?] for help Output #0, wav, to './results/temp.wav': Metadata: ISFT : Lavf60.3.100 Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 16000 Hz, mono, s16, 256 kb/s Metadata: encoder : Lavc60.3.100 pcm_s16le size= 313kB time=00:00:10.00 bitrate= 256.1kbits/s speed=1.5e+03x video:0kB audio:313kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.024367% torch.Size([1, 250, 6]) Traceback (most recent call last): File "C:\Users\flyingree\Downloads\Audio2Head-main\inference.py", line 251, in <module> audio2head(parse.audio_path,parse.img_path,parse.model_path,parse.save_path) File "C:\Users\flyingree\Downloads\Audio2Head-main\inference.py", line 141, in audio2head config = yaml.load(f) TypeError: load() missing 1 required positional argument: 'Loader'

So what is the matter?

RuntimeError: shape '[-1, 1, 4, 41]' is invalid for input of size 186780

It is real cool work! However I see it generates 256X256 video.

Is that possible to generate 512X512 video?

Thanks.

Thank you for your research.
Can i use this code and pretrained model for commercial use?
I wanna know about license.

mac 环境，直接运行报错

python3 inference.py --audio_path input.wav --img_path input.png
/bin/sh: ffmpeg: command not found
Traceback (most recent call last):
File "/Users/xxx/Documents/GitHub_Source/Audio2Head/inference.py", line 251, in
audio2head(parse.audio_path,parse.img_path,parse.model_path,parse.save_path)
File "/Users/xxx/Documents/GitHub_Source/Audio2Head/inference.py", line 125, in audio2head
audio_feature = get_audio_feature_from_audio(temp_audio)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/xxx/Documents/GitHub_Source/Audio2Head/inference.py", line 99, in get_audio_feature_from_audio
sample_rate, audio = wavfile.read(audio_path)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/scipy/io/wavfile.py", line 647, in read
fid = open(filename, 'rb')
^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: './results/temp.wav'