Bulk generation of variations of your favorite image about invokeai HOT 14 CLOSED

from invokeai.

Thinking about it more - even "hack" feature automated would be very interesting to play with :)
I imagine it work this way:

dream "an elephant <<3>> walking on <<5>> another planet <<7>>" -n 100 -S 385838583

<<3>> means that it will be replaced by random 3 digit number.

So the "real" prompt passed to AI will be:

dream "an elephant 387 walking on 06310 another planet 4756102" -n 100 -S 385838583

from invokeai.

This may be of interest here -- but I'm currently working on a plugin/scripting capability on a branch in my fork:

https://github.com/verderog/stable-diffusion/tree/plugin-scripting

See the plugins/ path for info.

Nickel tour:

• Added --plugin plugins.<plugin-name-here> option
• Plugins are simple classes derived from plugins._dreamplugin.DreamPlugin with three methods to automate operations
• I've currently got two examples defined for randomizing input prompts as mentioned in this thread and sweeping the cfg_scale

Operation:

• launch dream.py
• at the dream> prompt, enter your prompt, plugin reference, and plugin input.
• Example#1: dream> an elephant walking on {} another planet {} -S 385838583 -count 10 --plugin plugins.promptrandomizer
• Example#2: dream> dragon --plugin plugins.iteratecfgscale --cs_tart 7.5 --cs_stop 10.0 --cs_step 0.5

Looking for people to test before I submit a PR.

from invokeai.

@lstein Is there any interest in me submitting my plugin-scripting branch as PR? If so, I can integrate the latest main branch changes into my fork and submit that. (Reference: https://github.com/verderog/stable-diffusion/tree/plugin-scripting)

On a related note, I made a new plugin today named "artistrendering". Inspired by https://old.reddit.com/r/StableDiffusion/comments/x32dif/study_of_500_artists_using_the_same_prompt/, I wrote this plugin that iterates through 442 different artists with a common prompt and seed, changing only the artist name. Includes an option to use a randomly selected subset of artists from the full list.

Example invocation: a beautiful elf princess, ethereal face. Painted by {} --steps 15 --plugin plugins.artistrendering

from invokeai.

This is a cool idea. So the random numbers in the prompt are just there to stir the pot a bit? This should be way easy to implement under the current framework. Did you want the variations all to be written into the output directory, or should there be a subdirectory named ".variations" ?

from invokeai.

This hack was discovered by users in SD discord - you add a combination of random numbers and letters in random places of your prompt to get variations of the original image but the hack has few problems.

Consider you have this original prompt:

• dream "an elephant walking on 532 another planet 761" -n 1

Then you want to make variations of it, so you use its seed and append one digit to last number:

• dream "an elephant walking on 532 another planet 7615" -n 1 -S 385838583

You could get a slight variation of the original image or you might get pretty much identical one.

Then you continue to change numbers further:

• dream "an elephant walking on 6872 another planet 4215" -n 1 -S 385838583

But it did not work - you get a totally different image, so you try to understand why it did not work (it is safer to add digits at the end of existing number than prepend a digit) and rinse and repeat this hacky process.

But today I accidentally discovered that tweet and was surprised how good author's code is to generate "true" variations of the initial image where any two images are very similar to each other but differ a little. For this Github Issue I was hoping you could integrate author's code to add this powerful "variations engine" to your branch.

I think that "variations engine" will provide more value to users than "hacky" approach with random numbers.

I don't need .variations folder - let a script write variations images into the output directory but to quickly find variation files among others it could be useful to append ".v" string to filename like 000008.2976531757.v.png

from invokeai.

Additionally, I think that "hack" variation feature should work in 2 modes:

1. Default mode. In this mode all new images will use seed of the original image. It will result of many versions of original image altered a bit.

Prompt example:

dream "an elephant <<3>> walking on <<5>> another planet <<7>>" -n 100 -S 385838583

2. Sequence (movie) mode. This mode is activated by passing --sequence argument.
   It will use seed of the original image only once. Then it will pass Seed of the last generated image. I expect this mode to be able to generate "movies" - collection of frames that will vary greatly from original image the longer it works but two subsequent images will vary a little.

Prompt example:

dream "an elephant <<3>> walking on <<5>> another planet <<7>>" -n 100 -S 385838583 --sequence

from invokeai.

This is interesting - there could be a syntax for token replacement as well. something like ‘’’a ${[old, young, silly, worried]} elephant wearing a ${[gold, red, pink, blue]} ${[dress, tutu, kimono]} <<7>>’’’ to generate all variants and the sequence

from invokeai.

@Oceanswave I believe you are talking about a different feature - prompt generator.
For example, these two prompts differ "only" in color but they produce totally different images (second prompt used Seed from 1st image).

In this thread I am talking about a different feature - creation of "variants" (I don't know how to call it correctly in English).
By variant I mean almost identical image, like 95-99% identical to the source. Look at blue elephant image - a variant would be another image where front leg would be rotated a little different.

Prompts:

elephant wearing green hat -n 1
elephant wearing blue hat -n 1 -S934675406

from invokeai.

The technique in the linked gist is neat, if I've understood it correctly: it takes two random seeds, uses those to generate the initial noise matrices, and interpolates between those two matrices.

That suggests a way of doing variations, as long as you have the original seed and prompt: take that seed and generate the same initial matrix, then add a small amount of noise to that (i.e. generate another random matrix, multiply it by a weight < 1, and add it to the original), then re-run the rest of the process with the same prompt using your slightly-different matrix. How much noise you add determines how different the result will be.

It looks like all that's necessary to enable this (for programmatic usage) would be to allow the start_code to be specified.

(Forgive me if I'm using the wrong terms here; I have no idea what I'm doing.)

It would be fun if it could read the initial parameters out of the png metadata, so that all you had to do was scripts/re-run.py previous-output.png --strength 0.1 or whatever and it would figure out the rest.

Also, if you have two images generated with the same prompt but different seeds, you can interpolate between them using the same technique.

from invokeai.

I have a script which can variations in #81. It's not ready for general use but if you know a little python and want this feature, give it a try.

from invokeai.

Fork now includes an example of "img2imgchain" plugin where a base image is morphed for a number of generations using a fixed strength setting.

Example: black dragon --count 10 --chain_base outputs/img-samples/000001.3302085031.png --chain_strength 0.5 --plugin plugins.img2imgchain -> starts with the 000001..image, and runs 10 iterations, using the previous iteration's output for a subsequent iteration's input.

from invokeai.

@bakkot, @morganavr; Can we close this as complete based on #277? Or is this a request for something completely different?

@verderog; Your work is really cool, and I think people here would be very interested in taking a look at it, but it's pretty OT for this issue. Please either open a PR with your contributions, or have a look at #155 and see if what you have is what's being asked for there (or can at least get the ball rolling).

Really, you might as well open a PR and see what people think of it 😁

from invokeai.

Closed as completed in #277

from invokeai.