Git Product home page Git Product logo

Comments (11)

dbsxdbsx avatar dbsxdbsx commented on September 26, 2024 2

That might just be from FilterType::Nearest? Have you tried the other filter types?

I think it is not the reason. When I say "like a mosaic", I mean it is quite unreadable, like this:
revert_image
I guess no one would recognize it is a girl's picture originally.
But if a Dynamic Picture is resized, then the result is readable, code like this:

let img = image::open(r"D:\MEDIA\PICTURE\girl.jpg").unwrap();
let resized_image = img.resize(width, height, filter_type);

Also, as you suggested, I tried other filter types, and the same issue appeared.
So I think there is still something wrong when dealing with raw image buff.

I've also thought this may be due to the shape of the tensor. It is saved in shape (C,H,W). I know the instance of DynamicImage would prefer shape of (C,W,H) and (W,H,C). But as I said, since I could correctly save the tensor buffer of shape (C,H,W) by image::save_buffer, I think the shape is also not the issue.

from image.

fintelia avatar fintelia commented on September 26, 2024

The load_from_memory function takes the bytes of a PNG/JPEG/etc. file and decodes them into an image. You already have the decoded bytes, and so probably want to use RgbImage::from_raw or RgbaImage::from_raw.

from image.

dbsxdbsx avatar dbsxdbsx commented on September 26, 2024

The load_from_memory function takes the bytes of a PNG/JPEG/etc. file and decodes them into an image. You already have the decoded bytes, and so probably want to use RgbImage::from_raw or RgbaImage::from_raw.

I tried this:

let image_buff = Vec::<u8>::from(&rgb_image_t);
let revert_image = image::RgbImage::from_raw(width, height, image_buff).unwrap();
    // way 1
    // let revert_image = image::imageops::resize(
    //     &revert_image,
    //     width + 10,
    //     height + 10,
    //     image::imageops::FilterType::Nearest,
    // );

    // way 2
    let revert_image = DynamicImage::ImageRgb8(revert_image);
    let revert_image = revert_image.resize(
        width + 10,
        height + 10,
        image::imageops::FilterType::Nearest,
    );

 // save
  revert_image.save("revert_image.png").unwrap();
  // revert_image
  //     .save_with_format("revert_image.png", image::ImageFormat::Png)
  //     .unwrap();

No matter which approach I take, the final saved image is like a mosaic. Did I miss something?

from image.

fintelia avatar fintelia commented on September 26, 2024

That might just be from FilterType::Nearest? Have you tried the other filter types?

from image.

fintelia avatar fintelia commented on September 26, 2024

It is even weirder because DynamicImage::resize internally calls imageops::resize...

from image.

dbsxdbsx avatar dbsxdbsx commented on September 26, 2024

It is even weirder because DynamicImage::resize internally calls imageops::resize...

So maybe the core problem is from image::RgbImage::from_raw?

from image.

fintelia avatar fintelia commented on September 26, 2024

The thing is that the from_raw function essentially doesn't do anything.

Your comment about the shape of the tensor sounds off to me though. Both RgbImage and DynamicImage assume that the image is stored in row-major order, and you'll get garbled images if they actually aren't. Perhaps try reordering the elements of the tensor and see if resize starts working?

One difference between DynamicImage::resize and imageops::resize is that the former preserves aspect ratio. Perhaps slight differences in the output image size are causing the result to be garbled or not? You can try DynamicImage::resize_exact to test that. It would also be helpful to know the precise image sizes you are trying so I can test on my end.

from image.

dbsxdbsx avatar dbsxdbsx commented on September 26, 2024

The thing is that the from_raw function essentially doesn't do anything.

Your comment about the shape of the tensor sounds off to me though. Both RgbImage and DynamicImage assume that the image is stored in row-major order, and you'll get garbled images if they actually aren't. Perhaps try reordering the elements of the tensor and see if resize starts working?

One difference between DynamicImage::resize and imageops::resize is that the former preserves aspect ratio. Perhaps slight differences in the output image size are causing the result to be garbled or not? You can try DynamicImage::resize_exact to test that. It would also be helpful to know the precise image sizes you are trying so I can test on my end.

I think any image could be test to show the issue. Here let's just use my icon image as example:
写轮眼
It is an png with transparent channel. And below is my test code with tch-rs:

use image::{DynamicImage, GenericImageView};
use tch::Tensor;

fn main() {
    let width = 40;
    let height = 50;
    let filter_type = image::imageops::FilterType::Triangle;
    let img = image::open(r"D:\MEDIA\PICTURE\iamge.png").unwrap();
    println!("dimensions {:?}", img.dimensions());
    println!("{:?}", img.color());

    // open image from disk
    let grey_img = img.grayscale();
    println!("dimensions {:?}", img.dimensions());
    println!("{:?}", grey_img.color());

    // TODO: refractor try resizing by image crate from tensor
    let resized_image = img.resize(width, height, filter_type);

    // get image tensor
    let rgb_image_t = Tensor::from_image(&img);
    let resize_image_t1 = Tensor::from_image(&resized_image);
    let gray_image_t = Tensor::from_image(&grey_img);

    // test revert to image from tensor
    println!("orig image size:{:?}", rgb_image_t.size());
    let trans_image = rgb_image_t.permute(&[1, 2, 0]);
    let shape = trans_image.size();
    println!("trans image shape:{shape:?}");
    let image_buff = Vec::<u8>::from(&trans_image);
    let revert_image = image::RgbImage::from_raw(width, height, image_buff).unwrap();
    // way 1
    // let revert_image = image::imageops::resize(
    //     &revert_image,
    //     width + 10,
    //     height + 10,
    //     filter_type,
    // );

    // way 2
    let revert_image = DynamicImage::ImageRgb8(revert_image);
    // let revert_image = revert_image.resize_exact(width + 10, height + 10, filter_type);
    // let revert_image = revert_image.resize(width + 10, height + 10, filter_type);
    // let revert_image_t = Tensor::from_image(&revert_image);
    // println!("revert_image dimensions {:?}", revert_image.dimensions());

    // save tensor image
    img.save("orig_image.png").unwrap();
    rgb_image_t.save_image("recover_image.png");
    resize_image_t1.save_image("resize_image1.png");
    // revert_image_t.save_image("revert_image.png");

    revert_image.save("revert_image.png").unwrap();
    // revert_image
    //     .save_with_format("revert_image.", image::ImageFormat::Png)
    //     .unwrap();png
    gray_image_t.save_image("gray_image.png");
}

pub trait TensorExt {
    fn from_image(image: &image::DynamicImage) -> Self;
    fn save_image(&self, path: &str);
}

impl TensorExt for tch::Tensor {
    fn from_image(image: &image::DynamicImage) -> Self {
        let dimensions = image.dimensions();
        let color_type = image.color();
        match color_type {
            image::ColorType::L8 => {
                return tch::Tensor::of_data_size(
                    image.as_bytes(),
                    &[1, dimensions.1 as i64, dimensions.0 as i64],
                    tch::Kind::Uint8,
                )
                .to_kind(tch::Kind::Float);
            }
            image::ColorType::La8 => {
                let new_image = image.to_luma8();
                let dimensions = new_image.dimensions();
                return tch::Tensor::of_data_size(
                    image.as_bytes(),
                    &[2, dimensions.1 as i64, dimensions.0 as i64],
                    tch::Kind::Uint8,
                )
                .to_kind(tch::Kind::Float);
            }
            image::ColorType::Rgb8 => {
                return tch::Tensor::of_data_size(
                    image.as_bytes(),
                    &[3, dimensions.1 as i64, dimensions.0 as i64],
                    tch::Kind::Uint8,
                )
                .to_kind(tch::Kind::Float);
            }
            image::ColorType::Rgba8 => {
                let new_image = image.to_rgb8(); //TODO:need or not?
                let dimensions = new_image.dimensions();
                return tch::Tensor::of_data_size(
                    image.as_bytes(),
                    &[4, dimensions.1 as i64, dimensions.0 as i64],
                    tch::Kind::Uint8,
                )
                .to_kind(tch::Kind::Float);
            }
            _ => panic!(""),
        }
    }

    fn save_image(&self, path: &str) {
        let shape = self.size();
        let (height, width, color_type) = match shape {
            shape if shape.len() == 2 => (shape[0], shape[1], image::ColorType::L8),
            shape if shape.len() == 3 => {
                let channel_size = shape[0];
                match channel_size {
                    size if size == 1 => (shape[1], shape[2], image::ColorType::L8),
                    size if size == 2 => (shape[1], shape[2], image::ColorType::La8),
                    size if size == 3 => (shape[1], shape[2], image::ColorType::Rgb8),
                    size if size == 4 => (shape[1], shape[2], image::ColorType::Rgba8),
                    _ => panic!(),
                }
            }
            _ => panic!(),
        };

        image::save_buffer(
            path,
            &Vec::<u8>::from(self),
            width as u32,
            height as u32,
            color_type,
        )
        .unwrap();
    }
}

Meanwhile, I still think the issue is not related to resize. Just running the test code above, you would see that only revert_image is saved in wrong way.

from image.

fintelia avatar fintelia commented on September 26, 2024

I was able to get your code working by changing these lines:

     // test revert to image from tensor
     println!("orig image size:{:?}", rgb_image_t.size());
-    let trans_image = rgb_image_t.permute(&[1, 2, 0]);
+    let trans_image = rgb_image_t.permute(&[0, 1, 2]);
     let shape = trans_image.size();
     println!("trans image shape:{shape:?}");
     let image_buff = Vec::<u8>::from(&trans_image);
-    let revert_image = image::RgbImage::from_raw(width, height, image_buff).unwrap();
+    let revert_image = image::RgbImage::from_raw(img.width(), img.height(), image_buff).unwrap();
     // way 1
     // let revert_image = image::imageops::resize(
     //     &revert_image,

from image.

dbsxdbsx avatar dbsxdbsx commented on September 26, 2024

I was able to get your code working by changing these lines:

     // test revert to image from tensor
     println!("orig image size:{:?}", rgb_image_t.size());
-    let trans_image = rgb_image_t.permute(&[1, 2, 0]);
+    let trans_image = rgb_image_t.permute(&[0, 1, 2]);
     let shape = trans_image.size();
     println!("trans image shape:{shape:?}");
     let image_buff = Vec::<u8>::from(&trans_image);
-    let revert_image = image::RgbImage::from_raw(width, height, image_buff).unwrap();
+    let revert_image = image::RgbImage::from_raw(img.width(), img.height(), image_buff).unwrap();
     // way 1
     // let revert_image = image::imageops::resize(
     //     &revert_image,

Thanks for your test. Now I tested again, and I notice that this issue is not related to shape of the image. So no need to do permute with tch-rs tensor, (in my case is: rgb_image_t).
And let revert_image = image::RgbImage::from_raw(img.width(), img.height(), image_buff).unwrap(); is essential with the original image width and height.

The modification above is almost workable for most cases, but not every case. For example, for my icon image I just pasted above, the saved revert_image is quite strange. I don't know way.
(If you want to test my icon image, plz download it for exact the same image data)

from image.

dbsxdbsx avatar dbsxdbsx commented on September 26, 2024

I know what is wrong with my code. That is, for png, which has an alpha channel. The correct way to transfer from tch-rs to dynamicImage should using DynamicImage::ImageRgba8 and image::RgbaImage::from_raw.

from image.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.