Same model I was trying to train on issue <a class="issue-link js-issue-link" data-err

Training of a 512x512 model did not go well.,about aliaksandrsiarohin/motion-cosegmentation

Comments (11)

AliaksandrSiarohin commented on June 17, 2024

Have you tried to do a face swap in supervised mode?

from motion-cosegmentation.

adeptflax commented on June 17, 2024

How would I do that?

from motion-cosegmentation.

AliaksandrSiarohin commented on June 17, 2024

For the reference we also provide fully-supervised segmentation. For fully-supervised add --supervised option. And run git clone https://github.com/AliaksandrSiarohin/face-makeup.PyTorch face_parsing which is a fork of @zllrunning.

from motion-cosegmentation.

adeptflax commented on June 17, 2024

That's what I am doing in the code I posted. I'm using that face parser in the code.

from motion-cosegmentation.

AliaksandrSiarohin commented on June 17, 2024

Motion segmentation network is not needed than. You can use fomm.

from motion-cosegmentation.

adeptflax commented on June 17, 2024

what's fomm?

from motion-cosegmentation.

AliaksandrSiarohin commented on June 17, 2024

First order motion model.

from motion-cosegmentation.

adeptflax commented on June 17, 2024

This is the code for running and loading the model. I have first_order_motion_model=True on load_checkpoints().

from part_swap import load_checkpoints
cpu = True
reconstruction_module, segmentation_module = load_checkpoints(config='config/vox-512-sem-10segments.yaml', 
                                               checkpoint='log/vox-512-sem-10segments 26-04-21 19:25:33/00000005-checkpoint.pth.tar',
                                               blend_scale=0.125, first_order_motion_model=True,cpu=cpu)

from part_swap import make_video, load_face_parser
face_parser = load_face_parser(cpu=cpu)

def swap(source_image, target_image):
    shape = source_image.shape

    source_image = resize(source_image, (512, 512))[..., :3]
    target_video = [resize(target_image, (512, 512))[..., :3]]


    out = make_video(swap_index=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], source_image = source_image,
         target_video = target_video, use_source_segmentation=True, segmentation_module=segmentation_module,
         reconstruction_module=reconstruction_module, face_parser=face_parser, cpu=cpu)[0]
    
    return resize(out, (shape[0], shape[1]))