Git Product home page Git Product logo

Comments (2)

jwbrooks0 avatar jwbrooks0 commented on June 18, 2024 1

Particularly with Datasets, I definitely would prefer to just provide a single command/attribute to set the entire thing. I haven't thought through datatrees very carefully yet, but I think the same thing applies. My main goal of compression is to save space on my harddrive and having a single setting seems easier to me.

I also don't really understand why I would ever want to have different compression levels for individual DataArrays in a Dataset or datatree.

For context, I mostly save data as float32/64 and occasionally int64/32.

from datatree.

jhamman avatar jhamman commented on June 18, 2024

Or is recommended practice to update the compression info for each DataArray? (The second answer to the above link)

I'll just say that this is the approach I've been taking.

However, I do think there is some room for improvement to the current api. The challenge of course is that the per-variable or per-dataset encoding dictionary needs to map to tree structure. In practice these nested-dict data structures are unruly to work with and I've found myself finding plenty of functionality in the DataArray.encoding approach.

@jwbrooks0 - I'm curious if you have thoughts on a possible api that meets your use case? Do you agree that a nested-dict of encoding parameters is less than ideal from an end-user perspective?

from datatree.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.