parc / intricate-art-neural-transfer Goto Github PK

Hi,
So I was trying to run it on AWS instance with 1gb gpu.
But got below error:
2018-10-17 04:42:19.158952: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA 2018-10-17 04:42:19.247149: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2018-10-17 04:42:19.247507: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties: name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235 pciBusID: 0000:00:1e.0 totalMemory: 11.17GiB freeMemory: 11.10GiB 2018-10-17 04:42:19.247549: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: Tesla K80, pci bus id: 0000:00:1e.0, compute capability: 3.7) Model loaded. Starting iteration 1 of 10 2018-10-17 04:42:23.707640: E tensorflow/stream_executor/cuda/cuda_dnn.cc:385] could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2018-10-17 04:42:23.707693: E tensorflow/stream_executor/cuda/cuda_dnn.cc:352] could not destroy cudnn handle: CUDNN_STATUS_BAD_PARAM 2018-10-17 04:42:23.707724: F tensorflow/core/kernels/conv_ops.cc:667] Check failed: stream->parent()->GetConvolveAlgorithms( conv_parameters.ShouldIncludeWinogradNonfusedAlgo<T>(), &algorithms) Aborted (core dumped)

How much GPU resource would be required for running it.

Hi, I tried putting a content image of dim 1400 * 600 , then it's breaking.
But I rotated the image and its now having dim 600 * 1400 , then it's working.
I couldn't find any reshaping or rescaling operation in the code which could have done this.

stackTrace:
`Starting iteration 1 of 25
2018-11-26 09:28:09.905759: W tensorflow/core/common_runtime/bfc_allocator.cc:273] Allocator (GPU_0_bfc) ran out of memory trying to allocate 304.10MiB. Current allocation summary follows.
2018-11-26 09:28:09.905827: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (256): Total Chunks: 104, Chunks in use: 104. 26.0KiB allocated for chunks. 26.0KiB in use in bin. 1.4KiB client-re
quested in use in bin.
2018-11-26 09:28:09.905845: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (512): Total Chunks: 4, Chunks in use: 4. 2.0KiB allocated for chunks. 2.0KiB in use in bin. 2.0KiB client-requeste
d in use in bin.
2018-11-26 09:28:09.905864: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (1024): Total Chunks: 7, Chunks in use: 7. 7.2KiB allocated for chunks. 7.2KiB in use in bin. 7.0KiB client-requeste
d in use in bin.
2018-11-26 09:28:09.905888: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (2048): Total Chunks: 12, Chunks in use: 12. 24.0KiB allocated for chunks. 24.0KiB in use in bin. 24.0KiB client-req
uested in use in bin.
2018-11-26 09:28:09.905910: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (4096): Total Chunks: 2, Chunks in use: 1. 11.5KiB allocated for chunks. 6.8KiB in use in bin. 6.8KiB client-request
ed in use in bin.
2018-11-26 09:28:09.905930: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (8192): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in
bin.
2018-11-26 09:28:09.905950: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (16384): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in
use in bin.
2018-11-26 09:28:09.905964: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (32768): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in
use in bin.
2018-11-26 09:28:09.905992: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (65536): Total Chunks: 1, Chunks in use: 0. 126.2KiB allocated for chunks. 0B in use in bin. 0B client-reques
ted in use in bin.
2018-11-26 09:28:09.906017: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (131072): Total Chunks: 1, Chunks in use: 1. 144.0KiB allocated for chunks. 144.0KiB in use in bin. 144.0KiB c
lient-requested in use in bin.
2018-11-26 09:28:09.906036: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (262144): Total Chunks: 8, Chunks in use: 5. 2.12MiB allocated for chunks. 1.34MiB in use in bin. 1.28MiB clie
nt-requested in use in bin.
2018-11-26 09:28:09.906049: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (524288): Total Chunks: 2, Chunks in use: 1. 1.44MiB allocated for chunks. 576.0KiB in use in bin. 576.0KiB cl
ient-requested in use in bin.
2018-11-26 09:28:09.906072: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (1048576): Total Chunks: 4, Chunks in use: 4. 4.62MiB allocated for chunks. 4.62MiB in use in bin. 4.12MiB clie
nt-requested in use in bin.
2018-11-26 09:28:09.906087: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (2097152): Total Chunks: 3, Chunks in use: 2. 7.00MiB allocated for chunks. 4.50MiB in use in bin. 4.50MiB client-requested in use in bin.
2018-11-26 09:28:09.906099: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (4194304): Total Chunks: 2, Chunks in use: 2. 10.71MiB allocated for chunks. 10.71MiB in use in bin. 10.71MiB client-requested in use in bin.
2018-11-26 09:28:09.906108: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (8388608): Total Chunks: 15, Chunks in use: 15. 139.51MiB allocated for chunks. 139.51MiB in use in bin. 133.43MiB client-requested in use in bin.
2018-11-26 09:28:09.906120: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (16777216): Total Chunks: 13, Chunks in use: 12. 310.15MiB allocated for chunks. 291.02MiB in use in bin. 274.87MiB client-requested in use in bin.
2018-11-26 09:28:09.906130: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (33554432): Total Chunks: 21, Chunks in use: 20. 1.01GiB allocated for chunks. 982.33MiB in use in bin. 943.62MiB client-requested in use in bin.
2018-11-26 09:28:09.906146: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (67108864): Total Chunks: 7, Chunks in use: 7. 608.20MiB allocated for chunks. 608.20MiB in use in bin. 557.52MiB client-requested in use in bin.
2018-11-26 09:28:09.906156: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (134217728): Total Chunks: 7, Chunks in use: 5. 1.19GiB allocated for chunks. 810.94MiB in use in bin. 709.57MiB client-requested in use in bin.
2018-11-26 09:28:09.906170: I tensorflow/core/common_runtime/bfc_allocator.cc:627] Bin (268435456): Total Chunks: 15, Chunks in use: 15. 7.29GiB allocated for chunks. 7.29GiB in use in bin. 6.83GiB client-requested in use in bin.
2018-11-26 09:28:09.906179: I tensorflow/core/common_runtime/bfc_allocator.cc:643] Bin for 304.10MiB was 256.00MiB, Chunk State:
2018-11-26 09:28:09.906198: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1204fc0000 of size 1280
2018-11-26 09:28:09.906206: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1204fc0500 of size 9964800
2018-11-26 09:28:09.906217: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1205941200 of size 9964800
2018-11-26 09:28:09.906236: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c1f00 of size 256
2018-11-26 09:28:09.906247: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c2000 of size 256
2018-11-26 09:28:09.906263: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c2100 of size 256
2018-11-26 09:28:09.906274: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c2200 of size 256
2018-11-26 09:28:09.906283: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c2300 of size 256
2018-11-26 09:28:09.906294: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c2400 of size 256
2018-11-26 09:28:09.906300: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c2500 of size 256
2018-11-26 09:28:09.906305: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c2600 of size 256
2018-11-26 09:28:09.906312: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c2700 of size 512
2018-11-26 09:28:09.906319: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c2900 of size 256
2018-11-26 09:28:09.906325: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c2a00 of size 256
2018-11-26 09:28:09.906331: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c2b00 of size 512
2018-11-26 09:28:09.906338: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c2d00 of size 256
2018-11-26 09:28:09.906343: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c2e00 of size 256
2018-11-26 09:28:09.906354: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c2f00 of size 1024
2018-11-26 09:28:09.906360: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c3300 of size 256
2018-11-26 09:28:09.906366: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c3400 of size 256
2018-11-26 09:28:09.906381: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c3500 of size 1024
2018-11-26 09:28:09.906394: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c3900 of size 256
2018-11-26 09:28:09.906405: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c3a00 of size 1024
2018-11-26 09:28:09.906421: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c3e00 of size 256
2018-11-26 09:28:09.906427: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c3f00 of size 256
2018-11-26 09:28:09.906435: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c4000 of size 2048
2018-11-26 09:28:09.906445: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c4800 of size 256
2018-11-26 09:28:09.906451: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c4900 of size 256
2018-11-26 09:28:09.906456: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c4a00 of size 2048
2018-11-26 09:28:09.906463: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c5200 of size 256
2018-11-26 09:28:09.906470: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c5300 of size 2048
2018-11-26 09:28:09.906475: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c5b00 of size 2048
2018-11-26 09:28:09.906487: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c6300 of size 256
2018-11-26 09:28:09.906493: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c6400 of size 2048
2018-11-26 09:28:09.906498: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c6c00 of size 256
2018-11-26 09:28:09.906504: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c6d00 of size 2048
2018-11-26 09:28:09.906512: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c7500 of size 256
2018-11-26 09:28:09.906518: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12062c7600 of size 9964800
2018-11-26 09:28:09.906531: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1206c48300 of size 9964800
2018-11-26 09:28:09.906536: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9000 of size 256
2018-11-26 09:28:09.906542: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9100 of size 256
2018-11-26 09:28:09.906548: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9200 of size 256
2018-11-26 09:28:09.906555: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9300 of size 256
2018-11-26 09:28:09.906560: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9400 of size 256
2018-11-26 09:28:09.906570: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9500 of size 256
2018-11-26 09:28:09.906576: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9600 of size 256
2018-11-26 09:28:09.906581: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9700 of size 256
2018-11-26 09:28:09.906588: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9800 of size 256
2018-11-26 09:28:09.906594: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9900 of size 256
2018-11-26 09:28:09.906600: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9a00 of size 256
2018-11-26 09:28:09.906610: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9b00 of size 256
2018-11-26 09:28:09.906616: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9c00 of size 256
2018-11-26 09:28:09.906621: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9d00 of size 256
2018-11-26 09:28:09.906628: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9e00 of size 256
2018-11-26 09:28:09.906634: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075c9f00 of size 256
2018-11-26 09:28:09.906640: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075ca000 of size 256
2018-11-26 09:28:09.906650: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075ca100 of size 256
2018-11-26 09:28:09.906656: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075ca200 of size 256
2018-11-26 09:28:09.906661: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075ca300 of size 256
2018-11-26 09:28:09.906667: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075ca400 of size 256
2018-11-26 09:28:09.906674: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075ca500 of size 256
2018-11-26 09:28:09.906680: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075ca600 of size 256
2018-11-26 09:28:09.906692: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075ca700 of size 256
2018-11-26 09:28:09.906698: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075ca800 of size 256
2018-11-26 09:28:09.906703: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075ca900 of size 256
2018-11-26 09:28:09.906710: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075caa00 of size 256
2018-11-26 09:28:09.906717: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cab00 of size 256
2018-11-26 09:28:09.906722: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cac00 of size 256
2018-11-26 09:28:09.906729: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cad00 of size 256
2018-11-26 09:28:09.906744: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cae00 of size 256
2018-11-26 09:28:09.906754: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075caf00 of size 256
2018-11-26 09:28:09.906764: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cb000 of size 256
2018-11-26 09:28:09.906774: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cb100 of size 256
2018-11-26 09:28:09.906785: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cb200 of size 256
2018-11-26 09:28:09.906794: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cb300 of size 256
2018-11-26 09:28:09.906804: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cb400 of size 256
2018-11-26 09:28:09.906815: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cb500 of size 256
2018-11-26 09:28:09.906824: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cb600 of size 256
2018-11-26 09:28:09.906835: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cb700 of size 256
2018-11-26 09:28:09.906845: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cb800 of size 256
2018-11-26 09:28:09.906853: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cb900 of size 256
2018-11-26 09:28:09.906859: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cba00 of size 256
2018-11-26 09:28:09.906864: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cbb00 of size 256
2018-11-26 09:28:09.906870: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cbc00 of size 256
2018-11-26 09:28:09.906877: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cbd00 of size 256
2018-11-26 09:28:09.906882: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cbe00 of size 256
2018-11-26 09:28:09.906896: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cbf00 of size 256
2018-11-26 09:28:09.906901: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cc000 of size 256
2018-11-26 09:28:09.906906: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cc100 of size 256
2018-11-26 09:28:09.906913: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cc200 of size 256
2018-11-26 09:28:09.906918: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cc300 of size 256
2018-11-26 09:28:09.906930: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cc400 of size 256
2018-11-26 09:28:09.906935: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cc500 of size 256
2018-11-26 09:28:09.906941: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cc600 of size 256
2018-11-26 09:28:09.906947: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cc700 of size 256
2018-11-26 09:28:09.906954: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cc800 of size 256
2018-11-26 09:28:09.906960: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cc900 of size 256
2018-11-26 09:28:09.906971: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cca00 of size 256
2018-11-26 09:28:09.906977: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075ccb00 of size 256
2018-11-26 09:28:09.906982: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075ccc00 of size 256
2018-11-26 09:28:09.906988: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075ccd00 of size 256
2018-11-26 09:28:09.906995: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cce00 of size 256
2018-11-26 09:28:09.907000: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075ccf00 of size 256
2018-11-26 09:28:09.907010: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cd000 of size 256
2018-11-26 09:28:09.907016: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cd100 of size 256
2018-11-26 09:28:09.907021: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cd200 of size 256
2018-11-26 09:28:09.907028: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cd300 of size 256
2018-11-26 09:28:09.907034: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cd400 of size 256
2018-11-26 09:28:09.907040: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cd500 of size 256
2018-11-26 09:28:09.907052: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cd600 of size 256
2018-11-26 09:28:09.907057: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cd700 of size 256
2018-11-26 09:28:09.907063: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cd800 of size 256
2018-11-26 09:28:09.907069: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cd900 of size 256
2018-11-26 09:28:09.907076: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cda00 of size 256
2018-11-26 09:28:09.907082: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cdb00 of size 256
2018-11-26 09:28:09.907093: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cdc00 of size 256
2018-11-26 09:28:09.907099: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cdd00 of size 256
2018-11-26 09:28:09.907104: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cde00 of size 256
2018-11-26 09:28:09.907115: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075cf200 of size 256
2018-11-26 09:28:09.907121: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12075eec00 of size 256
2018-11-26 09:28:09.907126: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1207636d00 of size 512
2018-11-26 09:28:09.907133: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1207676f00 of size 327680
2018-11-26 09:28:09.907140: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12076c6f00 of size 512
2018-11-26 09:28:09.907146: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12076c7100 of size 262144
2018-11-26 09:28:09.907157: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12077e7100 of size 1024
2018-11-26 09:28:09.907163: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12077e7500 of size 2359296
2018-11-26 09:28:09.907169: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1207a27500 of size 1024
2018-11-26 09:28:09.907180: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1207a67900 of size 262144
2018-11-26 09:28:09.907186: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1207aa7900 of size 262144
2018-11-26 09:28:09.907192: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1207ae7900 of size 1572864
2018-11-26 09:28:09.907202: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1207c67900 of size 1024
2018-11-26 09:28:09.907208: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1207c67d00 of size 1048576
2018-11-26 09:28:09.907214: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1207d67d00 of size 1048576
2018-11-26 09:28:09.907223: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12080e7d00 of size 2048
2018-11-26 09:28:09.907230: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12080e8500 of size 9437184
2018-11-26 09:28:09.907235: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12089e8500 of size 2048
2018-11-26 09:28:09.907242: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12089e8d00 of size 9437184
2018-11-26 09:28:09.907249: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12092e8d00 of size 2048
2018-11-26 09:28:09.907255: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12092e9500 of size 9437184
2018-11-26 09:28:09.907265: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1209be9500 of size 2048
2018-11-26 09:28:09.907299: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1209be9d00 of size 2048
2018-11-26 09:28:09.907311: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1209bea500 of size 9437184
2018-11-26 09:28:09.907318: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x120a4ea500 of size 2048
2018-11-26 09:28:09.907326: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x120a4ead00 of size 9437184
2018-11-26 09:28:09.907333: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x120adead00 of size 6912
2018-11-26 09:28:09.907344: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x120adec800 of size 147456
2018-11-26 09:28:09.907350: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x120ae10800 of size 294912
2018-11-26 09:28:09.907358: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x120ae58800 of size 589824
2018-11-26 09:28:09.907366: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x120aee8800 of size 1179648
2018-11-26 09:28:09.907377: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x120b008800 of size 2359296
2018-11-26 09:28:09.907384: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x120b248800 of size 4718592
2018-11-26 09:28:09.907392: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x120b6c8800 of size 9437184
2018-11-26 09:28:09.907403: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x120bfc8800 of size 9941248
2018-11-26 09:28:09.907409: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x120c943900 of size 9964800
2018-11-26 09:28:09.907415: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x120d2c4600 of size 19882496
2018-11-26 09:28:09.907426: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x120e5ba800 of size 39835648
2018-11-26 09:28:09.907433: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1210bb8000 of size 637747200
2018-11-26 09:28:09.907441: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1236bec000 of size 667641600
2018-11-26 09:28:09.907452: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x125e8a2700 of size 9964800
2018-11-26 09:28:09.907458: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x125f223400 of size 9964800
2018-11-26 09:28:09.907464: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x125fba4100 of size 9964800
2018-11-26 09:28:09.907471: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1260524e00 of size 212582400
2018-11-26 09:28:09.907483: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x126cfe0e00 of size 53145600
2018-11-26 09:28:09.907497: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x127028fe00 of size 53145600
2018-11-26 09:28:09.907507: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x127353ee00 of size 531456000
2018-11-26 09:28:09.907518: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1293014e00 of size 318873600
2018-11-26 09:28:09.907533: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12a602ee00 of size 53145600
2018-11-26 09:28:09.907543: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12a92dde00 of size 53145600
2018-11-26 09:28:09.907553: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12ac58ce00 of size 53145600
2018-11-26 09:28:09.907562: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12af83be00 of size 53145600
2018-11-26 09:28:09.907581: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12b2aeae00 of size 53145600
2018-11-26 09:28:09.907592: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12b5d99e00 of size 159436800
2018-11-26 09:28:09.907602: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12bf5a6e00 of size 26572800
2018-11-26 09:28:09.907618: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12c0efe600 of size 19550208
2018-11-26 09:28:09.907624: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12c21a3600 of size 26572800
2018-11-26 09:28:09.907632: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12c3afae00 of size 33595392
2018-11-26 09:28:09.907638: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12c5b04e00 of size 53145600
2018-11-26 09:28:09.907650: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12c8db3e00 of size 53145600
2018-11-26 09:28:09.907656: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12cc062e00 of size 318873600
2018-11-26 09:28:09.907662: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12df07ce00 of size 106291200
2018-11-26 09:28:09.907669: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12e55dae00 of size 53145600
2018-11-26 09:28:09.907675: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12e8889e00 of size 26572800
2018-11-26 09:28:09.907681: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12ea1e1600 of size 26572800
2018-11-26 09:28:09.907691: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12ebb38e00 of size 53145600
2018-11-26 09:28:09.907696: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12eede7e00 of size 53145600
2018-11-26 09:28:09.907702: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12f5345e00 of size 26572800
2018-11-26 09:28:09.907709: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12f6c9d600 of size 26572800
2018-11-26 09:28:09.907715: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12f85f4e00 of size 6516736
2018-11-26 09:28:09.907721: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12f9f4c600 of size 79718400
2018-11-26 09:28:09.907733: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x12feb52e00 of size 53145600
2018-11-26 09:28:09.907738: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1301e01e00 of size 26572800
2018-11-26 09:28:09.907744: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1303759600 of size 26572800
2018-11-26 09:28:09.907750: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x13050b0e00 of size 106291200
2018-11-26 09:28:09.907757: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x130b60ee00 of size 106291200
2018-11-26 09:28:09.907763: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1311b6ce00 of size 159436800
2018-11-26 09:28:09.907772: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x131b379e00 of size 53145600
2018-11-26 09:28:09.907778: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x132e393e00 of size 159436800
2018-11-26 09:28:09.907784: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1337ba0e00 of size 53145600
2018-11-26 09:28:09.907790: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x133ae4fe00 of size 53145600
2018-11-26 09:28:09.907797: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x134790be00 of size 159436800
2018-11-26 09:28:09.907802: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1351118e00 of size 53145600
2018-11-26 09:28:09.907812: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x13543c7e00 of size 53145600
2018-11-26 09:28:09.907818: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1357676e00 of size 79718400
2018-11-26 09:28:09.907823: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x135c27d600 of size 26572800
2018-11-26 09:28:09.907830: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x135dbd4e00 of size 26572800
2018-11-26 09:28:09.907837: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x135f52c600 of size 79718400
2018-11-26 09:28:09.907842: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1364132e00 of size 79718400
2018-11-26 09:28:09.907852: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1368d39600 of size 318873600
2018-11-26 09:28:09.907858: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x137bd53600 of size 451737600
2018-11-26 09:28:09.907863: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1396c22e00 of size 637747200
2018-11-26 09:28:09.907874: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x13bcc56e00 of size 637747200
2018-11-26 09:28:09.907879: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x13e2c8ae00 of size 637747200
2018-11-26 09:28:09.907885: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1408cbee00 of size 637747200
2018-11-26 09:28:09.907895: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x142ecf2e00 of size 637747200
2018-11-26 09:28:09.907901: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x1454d26e00 of size 637747200
2018-11-26 09:28:09.907906: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x147ad5ae00 of size 318873600
2018-11-26 09:28:09.907913: I tensorflow/core/common_runtime/bfc_allocator.cc:661] Chunk at 0x148dd74e00 of size 438815488
2018-11-26 09:28:09.907921: I tensorflow/core/common_runtime/bfc_allocator.cc:670] Free at 0x12075cdf00 of size 4864
2018-11-26 09:28:09.907928: I tensorflow/core/common_runtime/bfc_allocator.cc:670] Free at 0x12075cf300 of size 129280
2018-11-26 09:28:09.907934: I tensorflow/core/common_runtime/bfc_allocator.cc:670] Free at 0x12075eed00 of size 294912
2018-11-26 09:28:09.907944: I tensorflow/core/common_runtime/bfc_allocator.cc:670] Free at 0x1207636f00 of size 262144
2018-11-26 09:28:09.907950: I tensorflow/core/common_runtime/bfc_allocator.cc:670] Free at 0x1207707100 of size 917504
2018-11-26 09:28:09.907955: I tensorflow/core/common_runtime/bfc_allocator.cc:670] Free at 0x1207a27900 of size 262144
2018-11-26 09:28:09.907962: I tensorflow/core/common_runtime/bfc_allocator.cc:670] Free at 0x1207e67d00 of size 2621440
2018-11-26 09:28:09.907969: I tensorflow/core/common_runtime/bfc_allocator.cc:670] Free at 0x12f2096e00 of size 53145600
2018-11-26 09:28:09.907975: I tensorflow/core/common_runtime/bfc_allocator.cc:670] Free at 0x12f8c2be00 of size 20056064
2018-11-26 09:28:09.907984: I tensorflow/core/common_runtime/bfc_allocator.cc:670] Free at 0x131e628e00 of size 265728000
2018-11-26 09:28:09.907990: I tensorflow/core/common_runtime/bfc_allocator.cc:670] Free at 0x133e0fee00 of size 159436800
2018-11-26 09:28:09.907996: I tensorflow/core/common_runtime/bfc_allocator.cc:676] Summary of in-use Chunks by size:
2018-11-26 09:28:09.908263: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 104 Chunks of size 256 totalling 26.0KiB
2018-11-26 09:28:09.908282: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 4 Chunks of size 512 totalling 2.0KiB
2018-11-26 09:28:09.908301: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 6 Chunks of size 1024 totalling 6.0KiB
2018-11-26 09:28:09.908320: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 1280 totalling 1.2KiB
2018-11-26 09:28:09.908339: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 12 Chunks of size 2048 totalling 24.0KiB
2018-11-26 09:28:09.908357: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 6912 totalling 6.8KiB
2018-11-26 09:28:09.908376: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 147456 totalling 144.0KiB
2018-11-26 09:28:09.908395: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 3 Chunks of size 262144 totalling 768.0KiB
2018-11-26 09:28:09.908413: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 294912 totalling 288.0KiB
2018-11-26 09:28:09.908432: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 327680 totalling 320.0KiB
2018-11-26 09:28:09.908450: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 589824 totalling 576.0KiB
2018-11-26 09:28:09.908469: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 2 Chunks of size 1048576 totalling 2.00MiB
2018-11-26 09:28:09.908487: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 1179648 totalling 1.12MiB
2018-11-26 09:28:09.908505: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 1572864 totalling 1.50MiB
2018-11-26 09:28:09.908523: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 2 Chunks of size 2359296 totalling 4.50MiB
2018-11-26 09:28:09.908541: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 4718592 totalling 4.50MiB
2018-11-26 09:28:09.908560: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 6516736 totalling 6.21MiB
2018-11-26 09:28:09.908574: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 6 Chunks of size 9437184 totalling 54.00MiB
2018-11-26 09:28:09.908588: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 9941248 totalling 9.48MiB
2018-11-26 09:28:09.908600: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 8 Chunks of size 9964800 totalling 76.03MiB
2018-11-26 09:28:09.908612: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 19550208 totalling 18.64MiB
2018-11-26 09:28:09.908626: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 19882496 totalling 18.96MiB
2018-11-26 09:28:09.908642: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 10 Chunks of size 26572800 totalling 253.42MiB
2018-11-26 09:28:09.908659: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 33595392 totalling 32.04MiB
2018-11-26 09:28:09.908671: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 39835648 totalling 37.99MiB
2018-11-26 09:28:09.908685: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 18 Chunks of size 53145600 totalling 912.30MiB
2018-11-26 09:28:09.908696: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 4 Chunks of size 79718400 totalling 304.10MiB
2018-11-26 09:28:09.908708: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 3 Chunks of size 106291200 totalling 304.10MiB
2018-11-26 09:28:09.908727: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 4 Chunks of size 159436800 totalling 608.20MiB
2018-11-26 09:28:09.908745: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 212582400 totalling 202.73MiB
2018-11-26 09:28:09.908763: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 4 Chunks of size 318873600 totalling 1.19GiB
2018-11-26 09:28:09.908781: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 438815488 totalling 418.49MiB
2018-11-26 09:28:09.908799: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 451737600 totalling 430.81MiB
2018-11-26 09:28:09.908819: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 531456000 totalling 506.84MiB
2018-11-26 09:28:09.908837: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 7 Chunks of size 637747200 totalling 4.16GiB
2018-11-26 09:28:09.908855: I tensorflow/core/common_runtime/bfc_allocator.cc:679] 1 Chunks of size 667641600 totalling 636.71MiB
2018-11-26 09:28:09.908874: I tensorflow/core/common_runtime/bfc_allocator.cc:683] Sum Total of in-use chunks: 10.08GiB
2018-11-26 09:28:09.908895: I tensorflow/core/common_runtime/bfc_allocator.cc:685] Stats:
Limit: 11324823962
InUse: 10821965056
MaxInUse: 11161878784
NumAllocs: 497
MaxAllocSize: 5859666688

2018-11-26 09:28:09.908929: W tensorflow/core/common_runtime/bfc_allocator.cc:277] x*__*******************************************************x
2018-11-26 09:28:09.908956: W tensorflow/core/framework/op_kernel.cc:1192] Resource exhausted: OOM when allocating tensor with shape[3,300,692,128]`

Hi
How long does the processing usually take?
On my GTX 1050 4gb it takes about 5 min to do 10 iterations. There's loads of out of memory errors.
I've been running other style transfer code on same computer and it zips through hundreds of iterations in a number of minutes, with just a few memory errors.
I have keras and tensorflow-gpu installed.

Is there anyway of improving performance?