extract_kp_videos.py中读取图片关键点时索引超出界限 about video-retalking HOT 5 OPEN

Good lord. I had to.

Uninstall Torch.
Re-install Torch with Cuda (latest) (Through Conda not Pip)
Install Numba
Finally it worked.

from video-retalking.

The inference script will save preprocessing results in ./temp folder. When the input video has been preprocessed, the key points are loaded directly from the folder to save inference time.
Did you test two videos with the same name? This can cause the loaded data to be inconsistent with the input video.

You can try deleting ./temp folder or adding the parameter --re_preprocess.

from video-retalking.

landmark Det:: 1%|▏ | 1/135 [00:05<12:15, 5.49s/it]
Traceback (most recent call last):
File "inference.py", line 342, in
main()
File "inference.py", line 79, in main
lm = kp_extractor.extract_keypoint(frames_pil, './temp/'+base_name+'_landmarks.txt')
File "/home/xiaoduo/cll/videoretalk/1/video-retalking-main/third_part/face3d/extract_kp_videos.py", line 27, in extract_keypoint
current_kp = self.extract_keypoint(image)
File "/home/xiaoduo/cll/videoretalk/1/video-retalking-main/third_part/face3d/extract_kp_videos.py", line 55, in extract_keypoint
return keypoints
UnboundLocalError: local variable 'keypoints' referenced before assignment
直接训练会出现这个问题当我修改了之后会出现索引超出界限，请问一下是那里的问题

from video-retalking.

Same error here...

(video_retalking) C:\Users\chlyw\Desktop\video-retalking>python inference.py --face examples/Carlin.mp4 --audio examples/Carlin.wav --outfile results/Carlin.mp4
[Info] Using cuda for inference.
[Step 0] Number of frames available for inference: 168
[Step 1] Landmarks Extraction in Video.
Downloading: "https://www.adrianbulat.com/downloads/python-fan/2DFAN4-cd938726ad.zip" to C:\Users\chlyw/.cache\torch\hub\checkpoints\2DFAN4-cd938726ad.zip
100%|██████████████████████████████████████████████████████████████████████████████| 91.9M/91.9M [00:11<00:00, 8.70MB/s]
landmark Det:: 1%|▍ | 1/168 [00:06<16:55, 6.08s/it]nvrtc: error: invalid value for --gpu-architecture (-arch)

nvrtc compilation failed:

#define NAN __int_as_float(0x7fffffff)
#define POS_INFINITY __int_as_float(0x7f800000)
#define NEG_INFINITY __int_as_float(0xff800000)

template
device T maximum(T a, T b) {
return isnan(a) ? a : (a > b ? a : b);
}

template
device T minimum(T a, T b) {
return isnan(a) ? a : (a < b ? a : b);
}

extern "C" global
void fused_cat_cat(float* tinput0_42, float* tinput0_46, float* tout3_67, float* tinput0_60, float* tinput0_52, float* tout3_71, float* aten_cat, float* aten_cat_1) {
{
if (blockIdx.x<512 ? 1 : 0) {
aten_cat_1[512 * blockIdx.x + threadIdx.x] = ((((512 * blockIdx.x + threadIdx.x) / 1024) % 256<192 ? 1 : 0) ? ((((512 * blockIdx.x + threadIdx.x) / 1024) % 256<128 ? 1 : 0) ? __ldg(tinput0_60 + (512 * blockIdx.x + threadIdx.x) % 262144) : __ldg(tinput0_52 + (512 * blockIdx.x + threadIdx.x) % 262144 - 131072)) : __ldg(tout3_71 + (512 * blockIdx.x + threadIdx.x) % 262144 - 196608));
}
aten_cat[512 * blockIdx.x + threadIdx.x] = ((((512 * blockIdx.x + threadIdx.x) / 4096) % 256<192 ? 1 : 0) ? ((((512 * blockIdx.x + threadIdx.x) / 4096) % 256<128 ? 1 : 0) ? __ldg(tinput0_42 + (512 * blockIdx.x + threadIdx.x) % 1048576) : __ldg(tinput0_46 + (512 * blockIdx.x + threadIdx.x) % 1048576 - 524288)) : __ldg(tout3_67 + (512 * blockIdx.x + threadIdx.x) % 1048576 - 786432));
}
}

landmark Det:: 1%|▍ | 1/168 [00:06<17:30, 6.29s/it]
Traceback (most recent call last):
File "inference.py", line 342, in
main()
File "inference.py", line 79, in main
lm = kp_extractor.extract_keypoint(frames_pil, './temp/'+base_name+'_landmarks.txt')
File "C:\Users\chlyw\Desktop\video-retalking\third_part\face3d\extract_kp_videos.py", line 27, in extract_keypoint
current_kp = self.extract_keypoint(image)
File "C:\Users\chlyw\Desktop\video-retalking\third_part\face3d\extract_kp_videos.py", line 55, in extract_keypoint
return keypoints
UnboundLocalError: local variable 'keypoints' referenced before assignment

(video_retalking) C:\Users\chlyw\Desktop\video-retalking>python inference.py --face examples/Carlin.mp4 --audio examples/Carlin.wav --outfile results/Carlin.mp4 --re_preprocess
[Info] Using cuda for inference.
[Step 0] Number of frames available for inference: 168
[Step 1] Landmarks Extraction in Video.
landmark Det:: 1%|▍ | 1/168 [00:06<16:52, 6.07s/it]nvrtc: error: invalid value for --gpu-architecture (-arch)

nvrtc compilation failed:

#define NAN __int_as_float(0x7fffffff)
#define POS_INFINITY __int_as_float(0x7f800000)
#define NEG_INFINITY __int_as_float(0xff800000)

template
device T maximum(T a, T b) {
return isnan(a) ? a : (a > b ? a : b);
}

template
device T minimum(T a, T b) {
return isnan(a) ? a : (a < b ? a : b);
}

extern "C" global
void fused_cat_cat(float* tinput0_42, float* tinput0_46, float* tout3_67, float* tinput0_60, float* tinput0_52, float* tout3_71, float* aten_cat, float* aten_cat_1) {
{
if (blockIdx.x<512 ? 1 : 0) {
aten_cat_1[512 * blockIdx.x + threadIdx.x] = ((((512 * blockIdx.x + threadIdx.x) / 1024) % 256<192 ? 1 : 0) ? ((((512 * blockIdx.x + threadIdx.x) / 1024) % 256<128 ? 1 : 0) ? __ldg(tinput0_60 + (512 * blockIdx.x + threadIdx.x) % 262144) : __ldg(tinput0_52 + (512 * blockIdx.x + threadIdx.x) % 262144 - 131072)) : __ldg(tout3_71 + (512 * blockIdx.x + threadIdx.x) % 262144 - 196608));
}
aten_cat[512 * blockIdx.x + threadIdx.x] = ((((512 * blockIdx.x + threadIdx.x) / 4096) % 256<192 ? 1 : 0) ? ((((512 * blockIdx.x + threadIdx.x) / 4096) % 256<128 ? 1 : 0) ? __ldg(tinput0_42 + (512 * blockIdx.x + threadIdx.x) % 1048576) : __ldg(tinput0_46 + (512 * blockIdx.x + threadIdx.x) % 1048576 - 524288)) : __ldg(tout3_67 + (512 * blockIdx.x + threadIdx.x) % 1048576 - 786432));
}
}

landmark Det:: 1%|▍ | 1/168 [00:06<17:30, 6.29s/it]
Traceback (most recent call last):
File "inference.py", line 342, in
main()
File "inference.py", line 79, in main
lm = kp_extractor.extract_keypoint(frames_pil, './temp/'+base_name+'_landmarks.txt')
File "C:\Users\chlyw\Desktop\video-retalking\third_part\face3d\extract_kp_videos.py", line 27, in extract_keypoint
current_kp = self.extract_keypoint(image)
File "C:\Users\chlyw\Desktop\video-retalking\third_part\face3d\extract_kp_videos.py", line 55, in extract_keypoint
return keypoints
UnboundLocalError: local variable 'keypoints' referenced before assignment

from video-retalking.

Package Version

absl-py 1.4.0
addict 2.4.0
aiofiles 23.1.0
aiohttp 3.8.4
aiosignal 1.3.1
altair 5.0.1
anyio 3.7.0
async-timeout 4.0.2
attrs 23.1.0
audioread 3.0.0
basicsr 1.4.2
cachetools 5.3.1
certifi 2023.5.7
cffi 1.15.1
charset-normalizer 3.1.0
click 8.1.3
colorama 0.4.6
contourpy 1.1.0
cycler 0.11.0
decorator 5.1.1
dlib 19.24.0
einops 0.4.1
exceptiongroup 1.1.1
face-alignment 1.3.5
facexlib 0.2.5
fastapi 0.97.0
ffmpy 0.3.0
filelock 3.12.2
filterpy 1.4.5
fonttools 4.40.0
frozenlist 1.3.3
fsspec 2023.6.0
future 0.18.3
google-auth 2.20.0
google-auth-oauthlib 1.0.0
gradio 3.35.2
gradio_client 0.2.7
grpcio 1.54.2
h11 0.14.0
httpcore 0.17.2
httpx 0.24.1
huggingface-hub 0.15.1
idna 3.4
imageio 2.31.1
importlib-metadata 6.6.0
importlib-resources 5.12.0
Jinja2 3.1.2
joblib 1.2.0
jsonschema 4.17.3
kiwisolver 1.4.4
kornia 0.5.1
lazy_loader 0.2
librosa 0.9.2
linkify-it-py 2.0.2
llvmlite 0.40.1rc1
lmdb 1.4.1
Markdown 3.4.3
markdown-it-py 2.2.0
MarkupSafe 2.1.3
matplotlib 3.7.1
mdit-py-plugins 0.3.3
mdurl 0.1.2
multidict 6.0.4
networkx 3.1
ninja 1.10.2.3
numba 0.57.0
numpy 1.23.1
oauthlib 3.2.2
opencv-contrib-python 4.7.0.72
opencv-python 4.7.0.72
orjson 3.9.1
packaging 23.1
pandas 2.0.2
Pillow 9.5.0
pip 23.1.2
pkgutil_resolve_name 1.3.10
platformdirs 3.5.3
pooch 1.7.0
protobuf 4.23.3
pyasn1 0.5.0
pyasn1-modules 0.3.0
pycparser 2.21
pydantic 1.10.9
pydub 0.25.1
Pygments 2.15.1
pyparsing 3.0.9
pyrsistent 0.19.3
python-dateutil 2.8.2
python-multipart 0.0.6
pytz 2023.3
PyWavelets 1.4.1
PyYAML 6.0
requests 2.31.0
requests-oauthlib 1.3.1
resampy 0.4.2
rsa 4.9
scikit-image 0.21.0
scikit-learn 1.2.2
scipy 1.10.1
semantic-version 2.10.0
setuptools 67.8.0
six 1.16.0
sniffio 1.3.0
soundfile 0.12.1
starlette 0.27.0
tb-nightly 2.14.0a20230616
tensorboard-data-server 0.7.1
threadpoolctl 3.1.0
tifffile 2023.4.12
tomli 2.0.1
toolz 0.12.0
torch 1.13.0+cu117
torchvision 0.14.0+cu117
tqdm 4.65.0
typing_extensions 4.6.3
tzdata 2023.3
uc-micro-py 1.0.2
urllib3 1.26.16
uvicorn 0.22.0
websockets 11.0.3
Werkzeug 2.3.6
wheel 0.38.4
yapf 0.40.0
yarl 1.9.2
zipp 3.15.0

升级一下cuda117 的，我用的是4070TI的，升级后，就可以了。

from video-retalking.