[Bug]: Stable Lora args missing in API calls about sd-webui-text2video HOT 6 OPEN

Hi. I'm unable to resolve this at the moment, but I think I could lead you in the right direction (I haven't looked at it in full). I don't know if it hooks into the current UI process through Gradio, or strictly does a POST directly to FastAPI.

I had a look in the API code, and it seems that the instance and/or args isn't passed to the API endpoint.

You would have to do an import something along the lines of:

from stable_lora.scripts.lora_webui import StableLoraScriptInstance
StableLoraScript = StableLoraScriptInstance

Here is how the script process works:

# You would use the default args (as a list) + stable_lora_ui.
stable_lora_ui = StableLoraScript.ui()

stable_lora_processor = StableLoraScriptInstance

# extra_args = args + stable_lora_ui  
stable_lora_args = stable_lora_processor.process_extension_args(all_args=extra_args) 

# pipe = ModelScope model (pipe.unet, pipe.text_encoder, etc.)
stable_lora_processor.process(pipe, *stable_lora_args)

The script relies on Gradio to process the args, which is why I can't give an assured fix at the moment for the API without looking into it.

from sd-webui-text2video.

I will try looking into that this week. Thank you. I want to start testing the Stable Lora stuff, but will need to find a fix for the API first.

from sd-webui-text2video.

I have the same issue while sending a request via API. I'm also looking into it.

I'm running the webui on an A40.

Here are my logs:

text2video — The model selected is: base (ModelScope-like)
text2video extension for auto1111 webui
Git commit: 8f0af8c
Starting text2video
Pipeline setup
config namespace(framework='pytorch', task='text-to-video-synthesis', model={'type': 'latent-text-to-video-synthesis', 'model_args': {'ckpt_clip': 'open_clip_pytorch_model.bin', 'ckpt_unet': 'text2video_pytorch_model.pth', 'ckpt_autoencoder': 'VQGAN_autoencoder.pth', 'max_frames': 16, 'tiny_gpu': 1}, 'model_cfg': {'unet_in_dim': 4, 'unet_dim': 320, 'unet_y_dim': 768, 'unet_context_dim': 1024, 'unet_out_dim': 4, 'unet_dim_mult': [1, 2, 4, 4], 'unet_num_heads': 8, 'unet_head_dim': 64, 'unet_res_blocks': 2, 'unet_attn_scales': [1, 0.5, 0.25], 'unet_dropout': 0.1, 'temporal_attention': 'True', 'num_timesteps': 1000, 'mean_type': 'eps', 'var_type': 'fixed_small', 'loss_type': 'mse'}}, pipeline={'type': 'latent-text-to-video-synthesis'})
Traceback (most recent call last):
File "/home/far/sd/stable-diffusion-webui/extensions/sd-webui-text2video/scripts/t2v_helpers/render.py", line 30, in run
vids_pack = process_modelscope(args_dict, args)
File "/home/far/sd/stable-diffusion-webui/extensions/sd-webui-text2video/scripts/modelscope/process_modelscope.py", line 70, in process_modelscope
stable_lora_processor.process(pipe, *stable_lora_args)
File "/home/far/sd/stable-diffusion-webui/venv/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
TypeError: process() missing 8 required positional arguments: 'lora_files_selection', 'lora_alpha', 'use_bias', 'use_linear', 'use_conv', 'use_emb', 'use_multiplier', and 'use_time'
Exception occurred: process() missing 8 required positional arguments: 'lora_files_selection', 'lora_alpha', 'use_bias', 'use_linear', 'use_conv', 'use_emb', 'use_multiplier', and 'use_time'

from sd-webui-text2video.

Any update on this. I'm unable to resolve the issue.

from sd-webui-text2video.

We are also looking for a fix for this issue. Earlier it was working for us. So, I tried using a older version of the extension. That didnt work even on the ui side.
Any date on when we can expect a fix?

from sd-webui-text2video.

Hey guys, I encountered this bug which prevented me from being able to run the extension and I determined a temporary hack to get around it. I commented out lines 68-70 in process_modelscope.py so it looks like this:

#TODO Wrap this in a list so that we can process this for future extensions.
#stable_lora_processor = StableLoraScriptInstance
#stable_lora_args = stable_lora_processor.process_extension_args(all_args=extra_args)
#stable_lora_processor.process(pipe, *stable_lora_args)

As far as I can tell this disables the processing of the LORA model which isn't ideal but seems to allow the extension to continue working. Obviously would be great to be able to keep using the LORAs module but this seems to work for the time being. Hopefully this helps someone.

from sd-webui-text2video.