s9roll7 / ebsynth_utility Goto Github PK

I'm on the latest version of the plugin, and the latest version of the webui.

When I'm on step 3, and performing img2img using the script, the images outputted are really dark, and applying color matcher after causes it to become oversaturated.

This bug does not occur when i have Generation TEST!!(Ignore Project directory and use the image and mask specified in the main UI) enabled. But obviously that won't process all my images.

I'm mostly using default settings, no auto tagging, and using inpaint area: whole picture, but that doesn't seem to affect it, neither does denoising strength.

I have also tried changing model, height, width, cfg scale and denoising strength, but this doesn't fix the issue.

Ubuntu 22.04
Commit hash: 22bcc7be428c94e9408f589966c2040187245d81
current transparent-background 1.2.3

Error completing request
Arguments: (5, '/mnt/data/stable_diffusion/Ebsynth/', '/tmp/a148ef1d58164eddf8eb471d0e20778531b4bc6d/my-video.mp4', -1, -1, 0, 0, False, False, '', '', 0.4, 11, 11, 10, 300, 8.5, True, 'hm-mkl-hm', True, False, False, 0, None, 1, 'mp4', '', 'Fit video length', 5, 0, 0, 'Normal') {}
Traceback (most recent call last):
  File "/home/lawrencexu/stable-diffusion-webui/modules/call_queue.py", line 56, in f
    res = list(func(*args, **kwargs))
  File "/home/lawrencexu/stable-diffusion-webui/modules/call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "/home/lawrencexu/stable-diffusion-webui/extensions/ebsynth_utility/ebsynth_utility.py", line 157, in ebsynth_utility_process
    ebsynth_utility_stage5(dbg, project_args, is_invert_mask)
  File "/home/lawrencexu/stable-diffusion-webui/extensions/ebsynth_utility/stage5.py", line 206, in ebsynth_utility_stage5
    rename_keys(img2img_upscale_key_path)
  File "/home/lawrencexu/stable-diffusion-webui/extensions/ebsynth_utility/stage5.py", line 28, in search_key_dir
IndexError: list index out of range

when you want to cancel the img2img and have some control nets enabled it loops and I have to kill the auto111 window and restart.

output does this

Loading model from cache: control_canny-fp16 [e3fe7712]
Loading preprocessor: canny
3050549512
Loading model from cache: control_depth-fp16 [400750f6]
Loading preprocessor: depth_leres
Loading model from cache: control_canny-fp16 [e3fe7712]
Loading preprocessor: canny
3050549512
Loading model from cache: control_depth-fp16 [400750f6]
Loading preprocessor: depth_leres
Loading model from cache: control_canny-fp16 [e3fe7712]
Loading preprocessor: canny
3050549512
Loading model from cache: control_depth-fp16 [400750f6]
Loading preprocessor: depth_leres
Loading model from cache: control_canny-fp16 [e3fe7712]
Loading preprocessor: canny
3050549512
Loading model from cache: control_depth-fp16 [400750f6]
Loading preprocessor: depth_leres
Loading model from cache: control_canny-fp16 [e3fe7712]
Loading preprocessor: canny
3050549512```

Hi !
This is really amazing, a tutorial video is much needed though, there are many bells and whistles on this one.

Hi:
Why stage4 upscaled keyframes only? The video original frames have not been upscaled?
If I use Ebsynth, I will got an error said frame resolution not match.

Worked till Step 5 Video_key folder generated and img2img_key folders generated.
Also generated 3 separate .ebs fies.

How to run those .ebs files in Ubuntu? Which repo or software/ui are we using for this.

Is there any plan to support other masking options besides transparent-background?

Love the plugin, btw, thanks for the great work!

[image2 @ 00000200b869ebc0] Could not open file : Diffusion\ebs-medo\video_frame\00001.pngs speed=N/A
av_interleaved_write_frame(): I/O error
[out#1/image2 @ 00000200b83eca80] Error muxing a packet
[image2 @ 00000200b849e980] The specified filename 'D:\Stable' does not contain an image sequence pattern or a pattern is invalid.
[image2 @ 00000200b849e980] Use a pattern such as %03d for an image sequence or use the -update option (with -frames:v 1 if needed) to write a single image.
[image2 @ 00000200b849e980] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
av_interleaved_write_frame(): Invalid argument
[out#0/image2 @ 00000200b83fe380] Error muxing a packet
frame= 2 fps=0.0 q=-0.0 Lq=-0.0 size=N/A time=00:00:00.73 bitrate=N/A speed=7.97x
video:1425kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!
frame extracted
usage: transparent-background [-h] [--source SOURCE] [--dest DEST] [--type TYPE] [--fast] [--jit] [--device DEVICE]
[--ckpt CKPT]
transparent-background: error: unrecognized arguments: Diffusion\ebs-medo\video_frame Diffusion\ebs-medo\video_mask
mask created

completed.

Keep getting this error during stage 3 img2img. Double checked I had everything right. Any ideas?

I am wondering if it is possible to have a different seed for each key frame when the img2img script is run. When I set the seed to "-1" it picks a random seed but it uses it for for each key frame.

Thanks!

Love this extension. Thanks for making it!

I generated the frames. They are very consistent

How can I use existing frames to reduce flicker?

Im getting this error i did a fresh clone of everything including automatic1111 as well as ffmpeg.

stage1

Skip frame extraction
Settings -> Mode=base, Device=cuda:0, Torchscript=disabled
Traceback (most recent call last):
File "C:\Users\Pete\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Pete\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "E:\01_AI\stable-diffusion-webui\venv\Scripts\transparent-background.exe_main.py", line 7, in
File "E:\01_AI\stable-diffusion-webui\venv\lib\site-packages\transparent_background\Remover.py", line 181, in console
loader = eval(_format + 'Loader')(args.source)
File "", line 1, in
NameError: name 'Loader' is not defined
mask created

completed.

Is it possible add an option to replace underscores with a space when auto tagging with deepbooru? Because the prompts with underscores are ineffective when used with the anime models.

For example, "closed_eyes" doesn't seem to work whereas "closed eyes" works.

Or use the waifu diffusion 1.4 tagger https://github.com/toriato/stable-diffusion-webui-wd14-tagger

I know it's a simple fix but just stating the issue here.

On setup
python: 3.9.13  •  torch: 1.13.1+cu117  •  xformers: N/A  •  gradio: 3.23.0  •  commit: 22bcc7be  •  checkpoint: c35782bad8

I'm getting the error below. I don't have any spaces in my paths.

stage1

Skip frame extraction
Downloading...
From: https://drive.google.com/uc?id=13oBl5MTVcWER3YU4fSxW3ATlVfueFQPY
To: /home/admin/.transparent-background/ckpt_base.pth
100%|███████████████████████████████████████████████████████████████████████████████████████| 368M/368M [01:25<00:00, 4.30MB/s]
Settings -> Mode=base, Device=cuda:0, Torchscript=disabled
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/admin/.conda/envs/automatic/bin/transparent-background:8 in │
│ │
│ 5 from transparent_background import console │
│ 6 if name == 'main': │
│ 7 │ sys.argv[0] = re.sub(r'(-script.pyw|.exe)?$', '', sys.argv[0]) │
│ ❱ 8 │ sys.exit(console()) │
│ 9 │
│ │
│ /home/admin/.conda/envs/automatic/lib/python3.10/site-packages/transparent_background/Remover.py │
│ :181 in console │
│ │
│ 178 │ if save_dir is not None: │
│ 179 │ │ os.makedirs(save_dir, exist_ok=True) │
│ 180 │ │
│ ❱ 181 │ loader = eval(_format + 'Loader')(args.source) │
│ 182 │ frame_progress = tqdm.tqdm(total=len(loader), position=1 if (_format == 'Video' and │
│ 183 │ sample_progress = tqdm.tqdm(total=len(loader), desc='Total:', position=0, bar_format │
│ 184 │ writer = None │
│ in :1 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
NameError: name 'Loader' is not defined
mask created

completed.

When activating stage7, It ends up giving me an input command and hanging there waiting.

'Input #0, image2, from 'C:\Stable_Diffusion\Training\Video\project\crossfade_tmp%05d.png':
Duration: 00:00:03.13, start: 0.000000, bitrate: N/A
Stream #0:0: Video: png, rgb24(pc, gbr/unknown/unknown), 1152x864, 30 fps, 30 tbr, 30 tbn
Stream mapping:
Stream #0:0 -> #0:0 (png (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 000001b5fb47e780] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 000001b5fb47e780] profile High, level 3.2, 4:2:0, 8-bit
[libx264 @ 000001b5fb47e780] 264 - core 164 r3106 eaa68fa - H.264/MPEG-4 AVC codec - Copyleft 2003-2023 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=24 lookahead_threads=4 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'C:\Stable_Diffusion\Training\Video\project\20230322-202343.mp4':
Metadata:
encoder : Lavf60.4.100
Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv420p(tv, progressive), 1152x864, q=2-31, 30 fps, 15360 tbn
Metadata:
encoder : Lavc60.6.101 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame= 0 fps=0.0 q=0.0 size= 0kB time=-577014:32:22.77 bitrate= -0.0kbits/s speed=N/A
Enter command: |all |-1 [ ]'

Seems like it wants an additional command line argument here for some reason?

I have install WebUI and the extension when my files where on the desktop , but then i moved everything to the Drive root , looks like its using an old path for some reason ( at 3rd last line in the logs below ) , tried restarted the machine but didn't seem to help.
Video frame folder is being populated correctly , but the video_mask folder is empty , logs :

...
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> png (native))
Press [q] to stop, [?] for help
Output #0, image2, to 'C:\StableDiffusion\Projects\one\video_frame\%05d.png':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.4.100
  Stream #0:0(und): Video: png, rgb24(pc, gbr/unknown/unknown, progressive), 852x480 [SAR 640:639 DAR 16:9], q=2-31, 200 kb/s, 23.98 fps, 23.98 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.6.100 png
frame=   38 fps=0.0 q=-0.0 Lsize=N/A time=00:00:01.54 bitrate=N/A speed=9.09x    ts/s speed=N/A
video:9268kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
frame extracted
Fatal error in launcher: Unable to create process using '"C:\Users\Stealth\Desktop\SDWebUI\venv\Scripts\python.exe"  "C:\StableDiffusion\SDWebUI\venv\Scripts\transparent-background.exe" --source C:\StableDiffusion\Projects\one\video_frame --dest C:\StableDiffusion\Projects\one\video_mask --type map': The system cannot find the file specified.

mask created

completed.

Hi there,

first of all thank you for your work. I really enjoy figuring out your extension and it seems to work for most of the features.
The only feature that is not working for me is the implementation of your face crop. I get the following message when trying to generate with face crop activated:

Error completing request Arguments: ([...], [], <PIL.Image.Image image mode=RGBA size=348x538 at 0x7FD2FD1132E0>, None, None, None, None, None, None, 35, 0, 4, 0, 1, False, False, 1, 1, 15.5, 1.5, 0.35, 1241424.0, -1.0, 0, 0, 0, False, 512, 512, 0, 0, 32, 0, '', '', '', [], 9, True, True, 'openpose', 'control_openpose-fp16 [9ca67cc5]', 1, None, False, 'Scale to Fit (Inner Fit)', False, False, 512, 64, 64, 0, 1, False, '<ul>\n<li><code>CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True, '', '', True, 50, True, 1, 0, False, 4, 1, 'None', '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 1, '', 0, '', 0, '', True, False, False, False, 0, 'testZwei', 'Normal', 1, True, 5, 1, 'None', False, True, 'YuNet', 512, 0.24, 1.5, False, 'Nico Robin', 0.5, 0.1, False, True, 'YuNet', 1024, 0.5, 1.5, False, 'face close up,', True, True, True, 0, 0.1, 1, 'None', False, 0, 2, 512, 512, False, None, 50) {}
Traceback (most recent call last):
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 56, in f
res = list(func(*args, **kwargs))
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/img2img.py", line 169, in img2img
processed = modules.scripts.scripts_img2img.run(p, *args)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/scripts.py", line 376, in run
processed = script.run(p, *script_args)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/ebsynth_utility/scripts/custom_script.py", line 851, in run
face_coords = self.detect_face(image, mask_image, face_detection_method, max_crop_size)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/ebsynth_utility/scripts/custom_script.py", line 207, in detect_face
img_array = mask_array * img_array
ValueError: operands could not be broadcast together with shapes (538,348,3) (538,348,4)

Is this a bug or am i doing something wrong?

Hello,

I have been able to execute steps 1-5 on Linux. I deferred the full project folder to a windows machine and executed the indications from step 6 (running the .ebs file). The result was multiple out-00xxx folders, one for each keyframe. I then proceeded to move the project folder back to linux to run step 7 in stable diffusion. The folder path is the same as the original folder, yet step 7 results in 'no out dir' which makes me think the recursive directory search does not work on a linux filesystem or something.

Error:
blend_rate: 1
export_type: mp4
fps: 30.0
-1
no out dir

Would be great if the concatenation worked under Linux, even if the EbSynth process has to be deferred to a windows machine which returns the project folder.

Project folder structure after step 6:

Of course, there is always the option I did something wrong, in which case, I'm just dumb and you can close the issue. :)
Great work, btw! The stability is pretty good. The only thing missing is being able to use the img2img alternative test script to feed the keyframe image as input noise. That would help a lot in stability and consistency of keyframes output from img2img.

[Edit] Installing SD on the Windows machine and your extension, then just running step 7 on the same folder worked perfectly

Tried to install via extensions page and applied and resart UI but there isn't anything there. Restarted A1111 and still not there. Have uninstalled / reinstalled and installed using git. Still nothing.

Any ideas what might cause this?

Error: source guide C:\Users\cpgsa\AppData\Local\Temp\87730C91-DDF1-4189-8C12-EAD60 06389B8.A2V4uc doesn't match the resolution of 'C: \Users\cpgsa\AppData\local\Temp\74517F86-E5B7-4406-BBFF-F513FA 3555D2.A2V4uc

How can i fix this?

Does it work properly under Linux?thx！

I just got ebsynth and have been having a couple of issues. It doesn't seem to show all the videos I choose when I drag it into the video box. So I dragged it in manually. (I have the latest ffmpeg I also have deforum extension installed and it can break apart the same videos into photos there.). Stage 1 makes the mask but stage 2 generates an error. I'm still pretty new so it's probably me, but what should I be looking for with this error?

fps: 30.0
key_min_gap: 10
key_max_gap: 300
key_th: 8.5
Error completing request
Arguments: (1, 'ebsynth', 'C:\Users\ASUS CUSTOM\AppData\Local\Temp\7c0f3a98cf7b70e12d97287d38931950ede61124\kenalpha.mp4', -1, -1, 0, 0, False, False, '', '', 0.4, 11, 11, 10, 300, 8.5, True, 'hm-mkl-hm', True, False, False, 0, None, 1, 'mp4', '', 'Fit video length', 5, 0, 0, 'Normal') {}
Traceback (most recent call last):
File "D:\stable-diffusion-webui\modules\call_queue.py", line 56, in f
res = list(func(*args, **kwargs))
File "D:\stable-diffusion-webui\modules\call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "D:\stable-diffusion-webui\extensions\ebsynth_utility\ebsynth_utility.py", line 85, in ebsynth_utility_process
ebsynth_utility_stage2(dbg, project_args, key_min_gap, key_max_gap, key_th, key_add_last_frame, is_invert_mask)
File "D:\stable-diffusion-webui\extensions\ebsynth_utility\stage2.py", line 153, in ebsynth_utility_stage2
keys = analyze_key_frames(frame_path, frame_mask_path, key_th, key_min_gap, key_max_gap, key_add_last_frame, is_invert_mask)
File "D:\stable-diffusion-webui\extensions\ebsynth_utility\stage2.py", line 80, in analyze_key_frames
key_frame = frames[0]
IndexError: list index out of range

Ubuntu 22.04
when I click generate, I got:

__init__() missing 2 required positional arguments: 'p' and 'images_list'

Whenever I try to set the project path, whether it be in C:\ or on my D:\ drive, it does not work. I have this (see image) as the project path. Those double // were to test whether that would work or not.
In the second image, the console window, you can see how I tried it multiple times.

Is this an known issue, or is this me doing something wrong?

I was following this tutorial: https://www.youtube.com/watch?v=Qph7A3UaVds&t=596s

The video frames are made but the masks folder remains empty. I am using 1e91627

I just have a couple questions about how these features are actually invoked.

controlnet -- Is each controlnet module invoked by ebsynth_utility in order? Does order even matter? In the readme, it shows the use of canny and normal modules. Can I use others modules like depth instead of using the depth_maps script? How about just canny? I tried using a depth module in controlnet, but it failed. This could be for any number of reasons, including my shoddy GPU. That's why I'm asking.

depth images (in video_key_depth) -- I am generating these manually before a run. Is that the correct workflow? How are these images actually used by your script?

clipseg -- are these actually used in image generation, or are they just used in .ebs file generation? If they are used in the actual img2img operation, how is it different than face crop? Also please explain face crop :)

Great tool. This is not an issue. More of a thought. Would it be possible to incorporate depth mapping as the mask somewhere in the pipeline? It tends to hold coherency for characters and the overall scene a bit stronger that just masking alone. Just a suggestion. I really appreciate the kickass tool!

I ran into this issue a few times while trying to debug why the script wasn't working.. any chance to have it check if empty extract, else not?

Hi, when trying stage 2 of the process, I get:
stage2

fps: 25.0
key_min_gap: 8
key_max_gap: 250
key_th: 8.5
Error completing request
Arguments: (1, 'H:\Ebsynth_Animations\01_MaxWomanTest\ebsynth', 'C:\Users\USER\AppData\Local\Temp\1132xk9u1.mp4', -1, -1, 0, 0, False, False, '', '', 0.4, 11, 11, 10, 300, 8.5, True, 'hm-mkl-hm', True, False, False, 0, None, 1, 'mp4', '', 'Fit video length', 5, 0, 0, 'Normal') {}
Traceback (most recent call last):
File "C:\stable-diffusion-webui\modules\call_queue.py", line 56, in f
res = list(func(*args, **kwargs))
File "C:\stable-diffusion-webui\modules\call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "C:\stable-diffusion-webui\extensions\ebsynth_utility\ebsynth_utility.py", line 85, in ebsynth_utility_process
ebsynth_utility_stage2(dbg, project_args, key_min_gap, key_max_gap, key_th, key_add_last_frame, is_invert_mask)
File "C:\stable-diffusion-webui\extensions\ebsynth_utility\stage2.py", line 153, in ebsynth_utility_stage2
keys = analyze_key_frames(frame_path, frame_mask_path, key_th, key_min_gap, key_max_gap, key_add_last_frame, is_invert_mask)
File "C:\stable-diffusion-webui\extensions\ebsynth_utility\stage2.py", line 80, in analyze_key_frames
key_frame = frames[0]
IndexError: list index out of range

Please help

How do ?

Skip frame extraction
'transparent-background' �����ڲ����ⲿ���Ҳ���ǿ����еij���
���������ļ���
mask created

completed.

I made a slight change if that's okay? If you'd like I can send it to you. Though, you probably already can do way more than me based on your code. Pretty awesome job! Thank you for making this.

After step 1, the mask folder is empty. Should there be images there?

frame extracted
'venv\Scripts\transparent-background' is not recognized as an internal or external command,
operable program or batch file.
mask created

completed.

frame extracted
Access denied with the following error:

        Cannot retrieve the public link of the file. You may need to change
        the permission to 'Anyone with the link', or have had many accesses.

You may still be able to access the file from the browser:

         https://drive.google.com/uc?id=13oBl5MTVcWER3YU4fSxW3ATlVfueFQPY

Traceback (most recent call last):
  File "C:\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\stable-diffusion-webui\venv\Scripts\transparent-background.exe\__main__.py", line 7, in <module>
  File "C:\stable-diffusion-webui\venv\lib\site-packages\transparent_background\Remover.py", line 151, in console
    remover = Remover(fast=args.fast, jit=args.jit, device=args.device, ckpt=args.ckpt)
  File "C:\stable-diffusion-webui\venv\lib\site-packages\transparent_background\Remover.py", line 79, in __init__
    self.model.load_state_dict(torch.load(os.path.join(ckpt_dir, ckpt_name), map_location='cpu'), strict=True)
  File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\serialization.py", line 771, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\serialization.py", line 270, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\serialization.py", line 251, in __init__
    super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\MainUser\\.transparent-background\\ckpt_base.pth'
mask created

completed.

Went to the link, tried to download "latest.pth" but got: HTTP ERROR 429.

First, amazing work on this plugin - seriously, one of the most comprehensive plugins given the complexity of making the ebsynth/sd connection frictionless.

Is there a reason why the current ebs output is limited to only 10 keyframes per file? It's a bit annoying to have to manually open and run each ebs in ebsynth, so I was wondering if there's an easier way to run multiple ebs files and/or a way to export larger ebs files.

Thanks!

Traceback (most recent call last):
File "K:\Misc\Automatic1111\stable-diffusion-webui\modules\scripts.py", line 195, in load_scripts
module = script_loading.load_module(scriptfile.path)
File "K:\Misc\Automatic1111\stable-diffusion-webui\modules\script_loading.py", line 13, in load_module
exec(compiled, module.dict)
File "K:\Misc\Automatic1111\stable-diffusion-webui\extensions\ebsynth_utility\scripts\ui.py", line 4, in
from ebsynth_utility import ebsynth_utility_process
File "K:\Misc\Automatic1111\stable-diffusion-webui\extensions\ebsynth_utility\ebsynth_utility.py", line 8, in
from extensions.ebsynth_utility.stage1 import ebsynth_utility_stage1,ebsynth_utility_stage1_invert
File "K:\Misc\Automatic1111\stable-diffusion-webui\extensions\ebsynth_utility\stage1.py", line 7, in
from transformers import AutoProcessor, CLIPSegForImageSegmentation
ImportError: cannot import name 'CLIPSegForImageSegmentation' from 'transformers' (K:\Misc\Automatic1111\stable-diffusion-webui\venv\lib\site-packages\transformers_init_.py)

the tab also doesn't show up in the UI

I have check the https://github.com/s9roll7/ebsynth_utility#installation, i see should install ebsynth, i download it. but there is no install programe, only .exe, how can i install? or i should place to somewhere?

First let me just say this extension is so phenomenal. I was using EBsynth separately and this makes life much easier. One issue I just ran into was using the extension with SDV2.1_768 model. The face crop always makes 512ish image. It's great for 1.5 and 512 models but it can't make a good face for the 768 model. I tried adjusting various setting but couldnt get it right. A separate slider to adjust the resolution of the face image would be great. One slider if it needs to keep the same ratio or 2 sliders if it doesnt. Thanks so much

So, I was wondering if it was possible to use swap a face from frames that are already made in a folder somewhere instead of the face created with img2img. I mean run the face detection and all that but swap another cropped face on each frame. Thanks

What would your feature do ?
A Stage (optional) that convert single images into high dynamic range Image for 10bit display monitor.
e.x.* https://github.com/VinAIResearch/single_image_hdr, https://github.com/vinthony/awesome-deep-hdr.
I probably miss more recent exciting papers with code.

I noticed you have st1_masking_method_index which is always set to 0 , can you add a dropdown / slider for step1 so we can choose alternative masking methods ?

Hey @s9roll7,

I've been working on this extension: https://github.com/djbielejeski/video-veil-automatic1111-extension and would love to bring some of your features over to it (especially the ebsynth stuff).

Let me know if you want to collaborate.

Script available on Img2Img tab after install.

Thanks!

Hi, this is really silly but for whatever reason, ebsynth is just refusing to pick up the video folder, for example, project dir is users\project, which it picks up just fine but user\project\movie with the video file in it and it just won't pick up for the movie file path. I have tried different folder locations but it just won't work for whatever reason.

I used it for a day yesterday, and there was no such problem!
But when I used it this morning, there was a problem of garbled characters. I didn’t do anything. How can I solve it?

Stage 2 generates directories called video_frame and video_mask. However, stage 3 expects a directory named video_key. I believe this might be the same intended location as renaming video_frame -> video_key enables the recipe to move forward.

Related error, which occurs after stage 2 succeeds, and results from following stage 3 recipe:

C:\Users\GameBox\Pictures\Ebsynth\video_key not found
Generate key frames first.

I saw \, ffmpeg.exe, etc in stage1.py, which means it doesn't seem to work on linux or Macos? Is there any plan to support?

Hello,
I encountered an issue when using this plugin. The "video_frame" and "video_mask" directories are created under the project_dir as expected. The "video_frame" directory contains disassembled video frames, but the "video_mask" directory is empty. I have already installed FFmpeg and transparent_background.

When running the Stable Diffusion web UI, I get the following output:

frame extracted
'venv\Scripts\transparent-background' is not recognized as an internal or external command, operable program or batch file.
Creating mask file
mask created
completed.
Please let me know if there's any additional information I can provide to help troubleshoot this issue. Thank you!

how can I put the negative prompts into the key frame prompt text file?