When I read your paper, I found ConvBlock(K1,S1) in DSConv.

But I couldn't find ConvBlock(K1,S1) in your structure of networks.

def Separable_conv2d(inputs, filters, kernel_size=3, strides=1, padding='VALID', Use_bias = None):
    if kernel_size==3 and strides==1:
        inputs = tf.pad(inputs, [[0, 0], [1, 1], [1, 1], [0, 0]], mode="REFLECT")
    if strides == 2:
        inputs = tf.pad(inputs, [[0, 0], [0, 1], [0, 1], [0, 0]], mode="REFLECT")
    return tf.contrib.layers.separable_conv2d(
        inputs,
        num_outputs=filters,
        kernel_size=kernel_size,
        depth_multiplier=1,
        stride=strides,
        biases_initializer=Use_bias,
        normalizer_fn=tf.contrib.layers.instance_norm,
        activation_fn=lrelu,
        padding=padding)

I thought Separable_conv2d means DSConv block, am I right?
What can I think about this?

Thanks again!

三次元风景照秒变宫崎骏动画，还能把石原里美带入吉卜力世界，AnimeGAN已开源 - 量子位的文章 - 知乎
https://zhuanlan.zhihu.com/p/118538002

A research from WHU reported by this public account involves the project and is suspected of infringement. May I ask if it is the research involving the author of the project.

It is possible to add citation support to Github. When referencing AnimeGAN, it would be nice to be able to not only reference the Paper, but also the code. I would suggest adding that to AnimeGAN, AnimeGANv2 and AnimeGANv3

More information here:
https://citation-file-format.github.io/

I installed tf-gpu 1.15.0, my GPU utilization is about 20%, an iter will run for 10s, and a data set will run for about 13 days

Hi @TachibanaYoshino ,

Wonderful work! AnimeGAN is really amazing to me. Those images on the README looks really good. However, for those people who do not have a lot of knowledge in the field of deep learning or even Python but just want to use the work to create an animated version of their photos, installing Tensorflow on a platform such as Windows/Linux could be challenging, which can limit people's enthusiasm. Therefore, from the pre-trained model that you released, I migrated AnimeGAN onto TensorFlow.js, allowing anyone who is able to access the Internet through a browser to do photo animation.

The project is here: https://github.com/TonyLianLong/AnimeGAN.js.
You can have a try here: https://animegan.js.org/.

The implementation uses WebGL or, if not supported, other backends to accelerate computing, but does not require the user to install anything.

Writing this message, I wonder if it's possible that you add a link of AnimeGAN.js to AnimeGAN's README as a demo, which will benefit this project since people are able to upload their own photo and try AnimeGAN out in their browser in addition to viewing the examples that you give, and they will have a deeper feeling than viewing a collection of example images. By listing a demo link, I mean in a way that this project lists other implementations, but suggestions are always welcomed. If you'd like, we could discuss further steps and work together.

3. Do edge_smooth
   eg. python edge_smooth.py --dataset Hayao --img_size 256

4. Calculate the three-channel(BGR) color difference
   eg. python data_mean.py --dataset Hayao
   1，这两项是训练必须的步骤吗？ 搜遍全网也没搞清楚这两个步骤的意义，可能是我太菜了。是否预测的时候也需要执行这两个程序？
   2，是我尝试迁移预测一些黑暗场景和爆炸场景，得到十分糟糕的效果，是训练集缺失的原因吧？假设我去热血漫搞点爆炸镜头能否改善这个问题呢？
   3，训练集图片都挺小的，能否自定义输入图片的形状，比如直接输入1080p？
   感谢作者，期望能得到你的教导。

hi,
I am not sure where i'm wrong but when i try inference with test.py file as read.me file, each image need more than 2 second for build fake images. When reading your paper, image inference just 50ms, what im wrong. Does image size has big impact for slow inference,
thanks you.

tf.config 报错是什么情况

is there any tutorial on how to install in wondows10

my tensorflow version is 1.13.1 because tf==1.8 is too old ,I can not install it
I can not load AnimeGAN.model-60 weight by tf.train.get_check_state

In my way:
download: Haoyao-style.zip
and unzip into "checkpoint/"
so, the checkpoint folder including:

checkpoint
AnimeGAN.model-60.index
AnimeGAN.model-60.meta
AnimeGAN.model-60.data-00000-of-00001

and using test scripts:
python test.py --checkpoint_dir checkpoint/AnimeGAN.model-60 --test_dir dataset/test/real --style_name H

but it failed
ckpt is None by tf.train.get_check_state(checkpoint/AnimeGAN.model-60)

Hi, I tried training but an error occurred. How do i fix it?

[] Reading checkpoints...
[] Failed to find a checkpoint
[!] Load failed...
2020-12-08 21:29:36.529360: W T:\src\github\tensorflow\tensorflow\core\framework\op_kernel.cc:1306] Invalid argument: TypeError: Cannot cast ufunc add output from dtype('<U32') to dtype('float32') with casting rule 'same_kind'
Traceback (most recent call last):

File "C:\Users\VGLAB\Anaconda3\lib\site-packages\tensorflow\python\ops\script_ops.py", line 157, in call
ret = func(*args)

File "C:\Users\VGLAB\Desktop\GAN\AnimeGAN-master\AnimeGAN-master\tools\data_loader.py", line 64, in load_image
image1, image2 = self.read_image(img1)

File "C:\Users\VGLAB\Desktop\GAN\AnimeGAN-master\AnimeGAN-master\tools\data_loader.py", line 45, in read_image
image1[:,:,0] += self.data_mean[2]

TypeError: Cannot cast ufunc add output from dtype('<U32') to dtype('float32') with casting rule 'same_kind'

您好，请问论文哪里可以看呢？在arxiv上好像没找到

Is there any way we can get tensorflow.contrib or any idea how to download older tensorflow, 1.8.0 or 1.15?

Hi @TachibanaYoshino,

Great work! Your project is really interesting, so I decided to implement the algorithm in Pytorch based on your Tensorflow code.

I tried to make the architecture of G and D to be the same as yours, and train on the same config (loss weights) but the result is a little bit different. So I tuned the loss weights to get the best result.

Currently, my results look like anime, but some generation images still have some artifacts. Maybe I will need to train longer.

Could you add this implementation into your README.md, I think It helps other people when looking for a Pytorch version 😄

thank you very much.

my repo: https://github.com/ptran1203/pytorch-animeGAN

Some of my results:

shows 404 error when clicking the link to your pre-trained model

Hi

I ran python test.py --checkpoint_dir checkpoint/generator_Hayao_weight --test_dir dataset/test/real --style_name H and got a nice results as you shown!

There are three styles, Hayao, Paprika, and Shinkai, I notice that you used style_name=H. Does this mean that you use capital letters for other ones too? Shinkai for S and paprika for P?

I tried setting --style_name S and --style_name Shinkai, but it still gives the same result as --style_name H

I use 1070 8gb.
I try batch is 2. ok
I try batch is 3. ok
I try batch is 4. overflow.

Is the author's video memory 8gb? Is it higher (11gb)? The author uses batch = 4, will it report an error? Is my graphics card set incorrectly?

I try run example point file AnimeGAN.model-60.meta. from epoch=60. start run train.
d_loss show start value 2. gogogo, My train result d_loss>2, =20. The number d_loss has high and the situation has deteriorated.

I just failed to use the default parameters. Adjustments must be made at any time.
If this program wants to be great, the d-loss must be maintained at least to <n? 5? 7?

Hi,

Love the work! I have managed to train the model with the existing dateset. I want to train it to generate better images with people. As you suggested, I've added about 2000 photos of various people in the train_photo. However it looks like the discriminator loss is converging to 0 after a few epoch and the generator loss gets stuck at a high number when I retrain the model.

This seems to be the discriminator is learning too fast? Any parameter I should tune or is my choice of data not good enough?
Am I suppose to replace the photos in the train_photo folder or just add new ones?

Excuse me if this sounds really amateurish, I'm quite new at this.

Hi,
What a great work!
Those results look really amazing.

Seems Baidu and vgg19.npy links are dead. I can't download them.
Can you reupload them?

Thanks.

Hello

I would like to know if it possible to adjust the config so as to retain full the original colors, so that just the texture changes.

Thanks

the result are just like a filtered pic.

i tried both checkpoints:

Haoyao-style.zip

Haoyao-style-initial-checkpoint.zip

When i run

python3 main.py --phase test --dataset Hayao  --checkpoint_dir checkpoint/AnimeGAN.model-60

I get strange results like

or

What might be causing it?

With all due respect, I believe the Brightness_tool directory needs to be outside the AnimeGAN+ directory. I tried to resolve the issue by introducing the init.py file, but that did not work. I am not really good at python. But after the commit 4 hours ago, the code does not run until and unless I put the Brightness_tool directory outside of AnimeGAN+.

谢谢，我自己解决啦。感谢这么好的素材

if I want convert a picture to anime picture. What is command should use?

I install environment using a docker image, and put my pictures in folder "samples", and run
docker run -it xxxx python main.py

I except AnimeGAN can handle my pictures, but he start train the pictures in folder dataset. one step and one step, slowly, I close it using ctrl+c until it runing step 100+.

I just want convert my pictures to anime picture. What is wrong with me? And what should I do correct？？

Hello, Can you help me?

I rewrite AnimeGAN by tensorflow2.0. All hyperparameters are the same as your code.

But I found that in the training process, at first the generator generated the correct image, but later the loss of the discriminator gradually decreased, resulting in the loss of anime style of the generated image.

loss value

generate image

my opinions

At first, I thought the loss method was wrong, and then I tried Dragan, but the effect was not as good as lsgan.

I'm currently trying to increase training rate : Generator : Discriminator = 3 : 1.

I'd appreciate it if you could give me some tips

can the weights be exported to a frozen_graph.pb or model.onnx file?

Dear author:
thanks for sharing this greate repo. it's helpful!
During reproduce your exp, i noticed the d&g loss is like this:

Epoch: 108 Step:   456  time: 0.595382 s d_loss: 11.78779125, g_loss: 1069.37170410 -- mean_d_loss: 15.22266197, mean_g_loss: 1128.97387695
Epoch: 108 Step:   457  time: 0.598991 s d_loss: 21.59355354, g_loss: 1145.24707031 -- mean_d_loss: 15.33250523, mean_g_loss: 1129.25439453
Epoch: 108 Step:   458  time: 0.592521 s d_loss: 6.78862953, g_loss: 1130.46630859 -- mean_d_loss: 15.18769360, mean_g_loss: 1129.27490234
Epoch: 108 Step:   459  time: 0.594440 s d_loss: 1.89800203, g_loss: 1081.30371094 -- mean_d_loss: 14.96619892, mean_g_loss: 1128.47534180
Epoch: 108 Step:   460  time: 0.595921 s d_loss: 65.07347870, g_loss: 1352.42736816 -- mean_d_loss: 15.78763008, mean_g_loss: 1132.14672852
Epoch: 108 Step:   461  time: 0.595723 s d_loss: 60.10780716, g_loss: 1378.10986328 -- mean_d_loss: 16.50247002, mean_g_loss: 1136.11389160
Epoch: 108 Step:   462  time: 0.596035 s d_loss: 2.82108736, g_loss: 1654.43518066 -- mean_d_loss: 16.28530502, mean_g_loss: 1144.34130859
Epoch: 108 Step:   463  time: 0.595019 s d_loss: 2.04391837, g_loss: 857.95996094 -- mean_d_loss: 16.06278419, mean_g_loss: 1139.86657715
Epoch: 108 Step:   464  time: 0.596555 s d_loss: 10.40408993, g_loss: 1090.06201172 -- mean_d_loss: 15.97572708, mean_g_loss: 1139.10034180
Epoch: 108 Step:   465  time: 0.595400 s d_loss: 7.73724365, g_loss: 965.87994385 -- mean_d_loss: 15.85090065, mean_g_loss: 1136.47583008

The g_loss is relatively large and does not decrease steadily. Is this normal? what's your training process looks like? thank you.

Super cool! Since you have released the project as open source, can you clarify on the license you have used for the project? Thank you.

Would like to have a chat or conversation with you @TachibanaYoshino . Can you please tell me in which social media should I follow you or where can we talk about it?

HI I am very interest to know how far AnimGan can beat CartoonGan ,
but the baidu link (dateset) is dead .
Can you put it at google drive or mega drive for everyone?
i think that is a problem for everyone interest in your project~

How to fix this ?
and you did a great work ! it's very fun ! Thanks!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main.py", line 112, in
main()
File "main.py", line 104, in main
gan.train()
File "/home/tom/workspace/AnimeGAN/AnimeGAN.py", line 232, in train
anime, anime_smooth, real = self.sess.run([anime_img_op, anime_smooth_op, real_img_op])
File "/root/anaconda3/envs/testa/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 900, in run
run_metadata_ptr)
File "/root/anaconda3/envs/testa/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File "/root/anaconda3/envs/testa/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1316, in _do_run
run_metadata)
File "/root/anaconda3/envs/testa/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1335, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: buffer_size must be greater than zero.
[[Node: ShuffleDataset = ShuffleDataset[output_shapes=[[]], output_types=[DT_FLOAT], reshuffle_each_iteration=true](RepeatDataset, ShuffleDataset/buffer_size, ShuffleDataset/seed, ShuffleDataset/seed2)]]
[[Node: OneShotIterator = OneShotIteratorcontainer="", dataset_factory=_make_dataset_25WVjNoEiQY[], output_shapes=[, ], output_types=[DT_FLOAT, DT_FLOAT], shared_name="", _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

Hi
I download and open project with pycharm. I configured the environment as README required(1080Ti), running edge-smooth.py and data_mean.py has no problem, however , when I ran main.py, something wrong:
[*] Reading checkpoints...
[*] Failed to find a checkpoint
[!] Load failed...
2021-04-12 13:15:23.718658: W tensorflow/core/framework/op_kernel.cc:1306] Invalid argument: UFuncTypeError: Cannot cast ufunc 'add' output from dtype('<U32') to dtype('float32') with casting rule 'same_kind'
Traceback (most recent call last):
File "****/anaconda3/envs/anime/lib/python3.6/site-packages/tensorflow/python/ops/script_ops.py", line 157, in __call__ ret = func(*args)
File "*****/AnimeGAN-master/tools/data_loader.py", line 70, in load_image image1, image2 = self.read_image(img1)
File "*****/AnimeGAN-master/tools/data_loader.py", line 45, in read_image image1[:,:,0] += self.data_mean[2]
numpy.core._exceptions._UFuncOutputCastingError: Cannot cast ufunc 'add' output from dtype('<U32') to dtype('float32') with casting rule 'same_kind'
Could you please help solve this problem or give some advice?
Best wishes!

Ur instruction doesn't work. luk for my screenshot. what to do? how to transform my photo into anime? nya?
i did everythonng by instruction.

Hi @TachibanaYoshino , your animegan is amazing, thanks for your great works!
Could you consider upgrade this repo with newer tensorflow version, eg. 1.14 or 1.15. since tensorflow 1.8 only support cuda to 9.x version, but cuda 9.x does not supppot newer nvidia rtx graphic card like rtx 2080ti, I run into this problem and it stuck me for a few days. here are to reference:

https://stackoverflow.com/questions/50622525/which-tensorflow-and-cuda-version-combinations-are-compatible

tensorflow/tensorflow#23341

btw here are error output when train animegan in my 2080ti cuda9.0 machine:

# ignore...

2020-04-20 11:45:56.252939: E tensorflow/stream_executor/cuda/cuda_blas.cc:654] failed to run cuBLAS routine cublasSgemm_v2: CUBLAS_STATUS_EXECUTION_FAILED
Traceback (most recent call last):
  File "/home/bigdata/anaconda3/envs/animegan/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1322, in _do_call
    return fn(*args)
  File "/home/bigdata/anaconda3/envs/animegan/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1307, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/home/bigdata/anaconda3/envs/animegan/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1409, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InternalError: Blas SGEMM launch failed : m=65536, n=128, k=64

# ignore

please add function: output pb file

I do it,but I don't how do. (convert checkpoint to pb file)

do you open a message about convert OR add output pb file function.

example other software message(convert checkpoint to pb file message):

input = tf.placeholder(tf.float32, shape=[None, 36], name="Input")
output = tf.layers.dense(inputs=self.layer2, units=1, activation=tf.nn.relu, name="Prediction")

0: op name = import/Input, op type = ( Placeholder ), inputs = , outputs = import/Input:0
@input shapes:
@output shapes:
name = import/Input:0 : (?, 36)

19: op name = import/Prediction, op type = ( Relu ), inputs = import/add_5:0, outputs = import/Prediction:0
@input shapes:
name = import/add_5:0 : (?, 1)
@output shapes:
name = import/Prediction:0 : (?, 1)

Hello，TachibanaYoshino

I want to know the host configuration of your research algorithm and the time when animegan runs on this host?Thanks.

Great job!
I ran the code with python 3.7.4 and tf 1.15.0 to reproducing the results. And I found it hard to train the Paprika style well with default hyperparameter settings. It seems that the GAN just transforms the image close to grayscale, and there are also some vertical lines.


Is it caused by Paprika data?
Note that I trained 130 epoches. The results after 100th epoch looks better, but the green region still goes grayscale.

It means that I just need to select a proper checkpoint during training? If yes, how?

I try move to iphone. I test it. ram very high.The system crashed.
I just used the verification part, but looking at the structure it seems to call the original training program. This may be the reason for his large memory requirements?

I tried trimming the model to make it smaller, but I failed. Do you have a plan to reduce memory（low)?
example:
self.test_generated = self.generator(self.test_real, reuse=False)
Of course I failed, the system reported an error

In addition, I have run other gan projects, the memory requirements are not so large, why is this project so high memory requirements? Even outside of training, test part of application part

File "D:\develop\workspace\AnimeGAN\AnimeGAN.py", line 232, in train
anime, anime_smooth, real = self.sess.run([anime_img_op, anime_smooth_op, real_img_op])

tensorflow.python.framework.errors_impl.InvalidArgumentError: [Derived]buffer_size must be greater than zero.
[[{{node OptimizeDataset/ShuffleDataset}}]]
[[OneShotIterator]]

I use the whole new conda environment, but when run the example video transfer command, error occurs:

    from tensorflow_estimator.python.estimator import estimator
  File "C:\Users\XU\.conda\envs\anime\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 36, in <module>
    from tensorflow.python.profiler import trace
ImportError: cannot import name 'trace'

(anime) E:\AI_art\AnimeGANv2-master>

How to solve it please? Thanks

When i run the edge_smooth.py #(usage 3)# , the terminal return 'No module named 'tools'' . how can i fix this ?
Traceback (most recent call last): File "edge_smooth.py", line 2, in <module> from tools.utils import check_folder ModuleNotFoundError: No module named 'tools'

tensorflow计算计算量，需要固定输入尺寸，固定情况下输出才是对的。
固定256size，FLOPs: 62650916141
不固定，FLOPs: 7937325
总之，和论文贴的都不太一样，论文和不固定的情况比较接近吧

I used more shinkai data and real images to train AnimeGAN, and got artifacts on my generated image as follow

I trained the model with default parameters in your main.py

I simply removed instance norm according to stylegan2. Not worked.

I was wondering if you had encountered this and what was the solution? thanks

亲，看啦你们模型效果很赞，十分好奇数据的样子。
但是给的百度链接已经失效了，请问能再发一个吗？感谢

[] Reading checkpoints...
[] Failed to find a checkpoint
[!] Load failed...