artoriuz / artcnn Goto Github PK

I'm not sure if this is related to the tint mentioned in 37b7c9c, but I've noticed a slight difference in brightness between the main variant of ArtCNN and the denoise variants, both DN and DS.

For example, this is with ArtCNN_C4F16:

and this is with ArtCNN_C4F16_DS:

The difference is fairly subtle, but if you look at the darker parts of the image, around the bottom left and right corners, the DS variant looks slightly darker, at least on my monitor.

Whether it’s super-resolution after frame interpolation or frame interpolation after super-resolution, the performance requirements are too high. I hope to use ArtCNN_Restore before frame interpolation and then apply other upscaling methods after frame Interpolation, with minimal impact on performance.

test file:
sd.zip
results:
https://slow.pics/c/X8waASMt
all NN scaler fail here. so this filter is not an outliner.
edit: fixed one image.

I am currently using ArtCNN_C4F32_CMP and CFL. My goal is to get the best possible image regardless of how much GPU compute it uses.

I noticed that you recently updated your benchmark results and it shows that ArtCNN_C4F32_Chroma outperforms CFL.

I am planning to switch from CFL to C4F32_Chroma but on your GIT page it says "These are intended to be used on 4:2:0 content and will not work as intended in any other scenario (which means you can't use them after luma doublers)."

Since I am already using ArtCNN_C4F32_CMP, does it mean I can't use C4F32_Chroma? You mentioned that you won't bother wasting resource on this (on the benchmark commentary), so we can just skip the Chroma upscaler altogether?

Thank you

Would you consider making one?

I have an mpv config with lightweight processing needs here and there, especially on material that is already in high resolution, over 30 fps in framerate or if the 2x result is being slightly downscaled, C4F8 used to be below FSRCNNX in processing amount and it wouldn't drop frames, while C4F16 is uncomfortably above on the midrange gpu.
Thanks for the consideration.

Interesting work! I was just going to test it out with MPV on some 360p videos. Added it into input.conf along with other shaders. But it shows a blue screen:
Is there an issue with this? All other work except this. I tried all variations and sizes.

...
...
CTRL+5 no-osd change-list glsl-shaders set "~~/shaders/Anime4K_3DGraphics_Upscale_x2_US.glsl"; show-text "Anime4K 3D: Mode E (HQ)"
CTRL+6 no-osd change-list glsl-shaders set "~~/shaders/ArtCNN_C4F8_LL.glsl"; show-text "ArtCNN: ArtCNN"

Do you plan to generate non-compute shaders? Performance has tanked on macOS for me.

Does one have to use both non chroma and chroma variants to scale both luma and chroma aka:

glsl-shader="~~/shaders/ArtCNN_C4F16.glsl"
glsl-shader="~~/shaders/ArtCNN_C4F16_Chroma.glsl"

or does the Chroma version scale both LUMA and CHROMA?

Hi,
You mentioned the following on your page.

If you plan on using ArtCNN for fractional scaling factors below 2x, it's recommended that you change the default dscale from hermite to a sharper option to avoid blurring the image.

How do you change the dscale from hermite to something else? I tried to look for hermite in the glsl but didn't find it.

Also, what would you recommend changing it too for a sharp picture with the best possible subjective quality if performance isn't a consideration?

Thanks

With the exception of the DIV2K variant, all the other shaders were trained on the Manga109 dataset.

Would it be possible for you to also have the DIV2K variant for ArtCNN_C4F8? (Laptop not powerful enough to run C4F16).

.Alternatively, if most of the media I consume is live action, would you recommend that I use ravu-zoom-ar-r3 or FSRCNNX instead of ArtCNN?

Thank you

Without this pragma, last component (.w) in the output texture might be truncated

Hi, this is not a issue. I have seen that you are pretty familiar with glsl shaders and I'm looking for help with a shader.
So I have this S-Curve (sigmoid) contrast shader and I would like to apply the effect only in the sRGB 0-40/80(smoothstep) range/ treshold.
I would appreciate any help.

#define Thresh 15/255.
#define w 10/255.
#define CoefLuma vec4(0.2126, 0.7152, 0.0722, 0) //sRGB, HDTV
smoothstep(Thresh + w, Thresh - w)

S-Curve (sigmoid)

//!HOOK OUTPUT
//!BIND HOOKED
//!DESC sCurve (sigmoid)

#define S 0.25 

vec4 hook(){
    vec4 c0 = HOOKED_texOff(0);
    
    c0.rgb = mix(c0.rgb, 1/( 1.0 + exp(-14*c0.rgb +7) ), S);
    return c0; 
}

Thank you :)

You have defined ar_strength, why not use ar_strength instead here...

Would be nice to have more in-depth optimization or even compute shader version for speeed. Anyway this micro-optimization of first layer is low-hanging fruit.

• remove 0 from mat4 and use vec4
• multiply only one component that we care about
• use gather to reduce texture fetches 3 vs 9 (this needs to be behind ifdef, because gather not always is available)

example, not tested, but something like that should produce the same result.

//!DESC ArtCNN C4F8 (Conv-0)
//!HOOK LUMA
//!BIND LUMA
//!SAVE conv2d_tf
//!WIDTH LUMA.w
//!HEIGHT LUMA.h
//!COMPONENTS 4
//!WHEN OUTPUT.w LUMA.w / 1.3 > OUTPUT.h LUMA.h / 1.3 > *
#define go_0(x_off, y_off) (LUMA_gather(LUMA_pos + LUMA_pt * vec2(x_off, y_off), 0))
vec4 hook() {
    vec4 a = go_0(-1.0, -1.0);
    vec4 b = go_0(-1.0, 0.0);
    vec4 c = go_0(-1.0, 1.0);
    vec4 result = vec4(0.05641021, -0.1615453, -0.06533657, 0.096413605) * a.x;
    result += vec4(0.1842147, 0.2938016, 0.147652, -0.22436947) * b.x;
    result += vec4(0.07949546, 0.039763954, -0.1269216, 0.12983432) * c.x;
    result += vec4(0.04739664, -0.10706943, 0.23155588, -0.3707046) * a.y;
    result += vec4(0.42271706, 0.009869351, 0.44073802, 0.7824819) * b.y;
    result += vec4(0.12183003, 0.22918288, -0.18820816, -0.24825077) * c.y;
    result += vec4(-0.012375349, -0.22000322, -0.1642485, 0.06789503) * a.z;
    result += vec4(0.045476228, -0.033663496, 0.14933448, -0.08178548) * b.z;
    result += vec4(-0.1617377, 0.029577361, 0.13827337, -0.06188948) * c.z;
    result += vec4(-0.03365639, -0.0049422677, -0.10187809, -0.025332384);
    return result;
}

Hi. Could you help me out with this shader? I'm struggling with it since months... ^^
TLDR: Do you know why this works on SDR but isn't on HDR? The original shader worked on both and only 3 lines of code have changed... I marked them at the end

more info: This is a vibrance shader that has 2 conditions. The first one is the srgb threshold float Thresh = 200.0/255.0; the 2nd one is the saturation of the pixel delta = smoothstep(0.8, 0.9, colorSat);
What I found out is that if the first number is higher than the 2nd one delta = smoothstep(0.3, 0.2, colorSat);, then you can see some changes but this isn't working as expected. This saturation condition is somehow broken.

By "isn't working" I mean that it visually doesn't change anything. The shader is working in the pipeline but there is no visible effect.

Thank you :)

//!PARAM vibr
//!DESC Saturates/deSaturates
//!TYPE float
//!MINIMUM -1
-0.5

//!HOOK MAIN
//!BIND HOOKED
//!DESC Vibrance

#define CoefLuma vec4(0.2126, 0.7152, 0.0722, 0) //sRGB, HDTV
float Thresh = 200.0/255.0;
float w = 10.0/255.0;

vec4 hook() {
	vec4 c0 = HOOKED_texOff(0);
	float lum = (c0.r + c0.g + c0.b)/3.0;

	float colorSat = max(max(c0.r, c0.g), c0.b) - min(min(c0.r, c0.g), c0.b); 
	vec3 sat = mix(vec3(dot(c0, CoefLuma)), c0.rgb, 1+vibr - colorSat*abs(vibr));
	float delta = smoothstep(Thresh-w, Thresh+w, lum);
1	sat = mix(sat, c0.rgb, delta);
2	delta = smoothstep(0.8, 0.9, colorSat);
3	c0.rgb = mix(c0.rgb, sat, delta);
	return c0;
}

The original shader, works on SDR and HDR

//!PARAM vibr
//!DESC Saturates/deSaturates
//!TYPE float
//!MINIMUM -1
-0.5

//!HOOK MAIN
//!BIND HOOKED
//!DESC Vibrance

#define CoefLuma vec4(0.2126, 0.7152, 0.0722, 0) //sRGB, HDTV
float Thresh = 200.0/255.0;
float w = 10.0/255.0;

vec4 hook() {
vec4 c0 = HOOKED_texOff(0);
float lum = (c0.r + c0.g + c0.b)/3.0; 

float colorSat = max(max(c0.r, c0.g), c0.b) -min(min(c0.r, c0.g), c0.b); // >=0, 5 arithmetic
vec3 sat = mix(vec3(dot(c0, CoefLuma)), c0.rgb, 1+vibr -colorSat*abs(vibr));
float delta = smoothstep( Thresh-w, Thresh+w, lum);
c0.rgb = mix( sat, c0.rgb, delta);
return c0;
}

why isnt there a lighter chroma upscale than f32? what happened to 16?

Just the title

Here's what mpv says:

[vo/gpu-next/libplacebo] shader compile log (status=0): 0:67(12): warning: extension `GL_EXT_shader_explicit_arithmetic_types_float16' unsupported in compute shader
[vo/gpu-next/libplacebo] 0:67(1): error: #extension directive is not allowed in the middle of a shader

Thanks in advance

Given that these are "upscaling" shaders, can we combine them with other shaders and use in mpv? I would like to add it to some Restore and Denoising shaders (e.g. from Anime4K) and pipeline them together.

I'm guessing by the name that this scaler is recommend only for anime, am I wrong or can it be used for live action too?

The README says it can be used as a fractional upscaler, I tried 720p source on a 1366x768 screen and it doesn't work, how should I do that? I'm sorry if this is an obvious question.