Git Product home page Git Product logo

Comments (11)

hakonanes avatar hakonanes commented on August 26, 2024

I believe they are related to these changes in band_detect_cl.py: USNavalResearchLaboratory/PyEBSDIndex@88136e6#diff-b1a7457abbba36e579c00423396bc962373e6fcb044b4c43a3c2ad0384cca8be

from kikuchipy.

drowenhorst-nrl avatar drowenhorst-nrl commented on August 26, 2024

What is the data type of your dask array?

from kikuchipy.

hakonanes avatar hakonanes commented on August 26, 2024

In this example it is uint8.

from kikuchipy.

drowenhorst-nrl avatar drowenhorst-nrl commented on August 26, 2024

Trying to figure out exactly what is up ... I see Numba error, but seems like it is failing on making an opencl buffer?

from kikuchipy.

drowenhorst-nrl avatar drowenhorst-nrl commented on August 26, 2024

Ah - I see what is up ... I think. When it fails to run the OpenCL version, it attempts the numba version. Both are failing for similar reasons. Let me look at what I can do here.

from kikuchipy.

hakonanes avatar hakonanes commented on August 26, 2024

Thank you for having a look, really appreciate it! Any potentially useful information you have about the problem is useful.

from kikuchipy.

drowenhorst-nrl avatar drowenhorst-nrl commented on August 26, 2024

If only all the fixes where this easy (at least I hope it is this easy). Maybe not the most elegant solution, but it looks like throwing an image = np.asarray(image) at the top of the radon functions should work. This will make a copy of the chunk of patterns read in, but I think I was doing that before anyway. While I have made many improvements on making PyEBSDIndex faster, I have not worried much about the memory efficiency.

Some insight:

  • Through some staring some Windows system monitoring tools, it looked like I was spending just about as much time transferring patterns to the GPU as calculating (as is the way with GPU compute). I figured it would not be that bad, that if the patterns are uint8, or uint16, I could just send them as-is to the GPU, and convert to float on the GPU. In fact - I made the decision that if the patterns were float -- I actually convert them to uint16, and then scale the results (anyone who wants 32-bit precision on pattern quality really needs to justify it to me I guess).
  • Then I figured, well, I might as well do my data transpose on the GPU as well.

Thus, I stoped converting to float on the CPU (which would have made a copy in this case), and tried sending a version as is to the GPU. OpenCL and Numba will work with numpy, but not Dask arrays ... thus the issue on both pipelines.

I will send in a PR for you to test.

from kikuchipy.

drowenhorst-nrl avatar drowenhorst-nrl commented on August 26, 2024

BTW - old version on my machine: 780 pats/s, new 1180 pats/s

from kikuchipy.

hakonanes avatar hakonanes commented on August 26, 2024

Your changes in USNavalResearchLaboratory/PyEBSDIndex#60 fixes my issue here, thanks!

from kikuchipy.

drowenhorst-nrl avatar drowenhorst-nrl commented on August 26, 2024

PyEBSDIndex 0.3.2 is now out on PyPi and conda-forge, and should be the fix for this issue.

from kikuchipy.

hakonanes avatar hakonanes commented on August 26, 2024

Thank you so much for the quick handling of my issue here. Have a good weekend!

from kikuchipy.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.