Git Product home page Git Product logo

nd2tools's People

Contributors

fricktobias avatar

Stargazers

avatar

Watchers

avatar avatar

nd2tools's Issues

Make Linux condalock file

Problem

Currently there is a conda problem giving an error when running condalock. When it is resolved the environment.linux.lock.yml file should be remade using bash misc/condalock.sh linux

bash misc/condalock.sh osx works as normal.

The current place holder linux file is just a copy of environment.yml

Error

Verifying transaction: failed

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/Users/tobiasfrick/miniconda/lib/python3.8/site-packages/conda/exceptions.py", line 1079, in __call__
        return func(*args, **kwargs)
      File "/Users/tobiasfrick/miniconda/lib/python3.8/site-packages/conda_env/cli/main.py", line 80, in do_call
        exit_code = getattr(module, func_name)(args, parser)
      File "/Users/tobiasfrick/miniconda/lib/python3.8/site-packages/conda_env/cli/main_create.py", line 118, in execute
        result[installer_type] = installer.install(prefix, pkg_specs, args, env)
      File "/Users/tobiasfrick/miniconda/lib/python3.8/site-packages/conda_env/installers/conda.py", line 40, in install
        unlink_link_transaction.execute()
      File "/Users/tobiasfrick/miniconda/lib/python3.8/site-packages/conda/core/link.py", line 244, in execute
        self.verify()
      File "/Users/tobiasfrick/miniconda/lib/python3.8/site-packages/conda/common/io.py", line 88, in decorated
        return f(*args, **kwds)
      File "/Users/tobiasfrick/miniconda/lib/python3.8/site-packages/conda/core/link.py", line 231, in verify
        exceptions = self._verify(self.prefix_setups, self.prefix_action_groups)
      File "/Users/tobiasfrick/miniconda/lib/python3.8/site-packages/conda/core/link.py", line 590, in _verify
        for exc in self.verify_executor.map(UnlinkLinkTransaction._verify_individual_level,
      File "/Users/tobiasfrick/miniconda/lib/python3.8/site-packages/conda/common/io.py", line 525, in map
        yield func(thing)
      File "/Users/tobiasfrick/miniconda/lib/python3.8/site-packages/conda/core/link.py", line 411, in _verify_individual_level
        error_result = axn.verify()
      File "/Users/tobiasfrick/miniconda/lib/python3.8/site-packages/conda/core/path_actions.py", line 323, in verify
        source_size_in_bytes = getsize(self.source_full_path)
      File "/Users/tobiasfrick/miniconda/lib/python3.8/genericpath.py", line 50, in getsize
        return os.stat(filename).st_size
    OSError: [Errno 62] Too many levels of symbolic links: '/Users/tobiasfrick/miniconda/pkgs/ncurses-6.2-h58526e2_4/share/terminfo/N/NCR260VT300WPP'

`$ /Users/tobiasfrick/miniconda/bin/conda-env create -n blrtmp-7778-linux -f environment.yml`

  environment variables:
                 CIO_TEST=<not set>
  CONDA_AUTO_UPDATE_CONDA=false
        CONDA_DEFAULT_ENV=nd2tools
                CONDA_EXE=/Users/tobiasfrick/miniconda/bin/conda
             CONDA_PREFIX=/Users/tobiasfrick/miniconda/envs/nd2tools
    CONDA_PROMPT_MODIFIER=(nd2tools)
         CONDA_PYTHON_EXE=/Users/tobiasfrick/miniconda/bin/python
               CONDA_ROOT=/Users/tobiasfrick/miniconda
              CONDA_SHLVL=1
           CURL_CA_BUNDLE=<not set>
                     PATH=/Users/tobiasfrick/miniconda/envs/nd2tools/bin:/usr/local/git/bin:/sw/
                          bin:/usr/local/bin:/usr/local:/usr/local/sbin:/usr/local/mysql/bin:/Us
                          ers/tobiasfrick/Library/Python/3.9/bin:/Users/tobiasfrick/external/cel
                          lranger-5.0.1:/Users/tobiasfrick/miniconda/condabin:/usr/local/bin:/us
                          r/bin:/bin:/usr/sbin:/sbin:/Users/tobiasfrick/external/sratoolkit.2.10
                          .9-mac64/bin:/usr/local/bin
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=<not set>

     active environment : nd2tools
    active env location : /Users/tobiasfrick/miniconda/envs/nd2tools
            shell level : 1
       user config file : /Users/tobiasfrick/.condarc
 populated config files : /Users/tobiasfrick/.condarc
          conda version : 4.9.2
    conda-build version : not installed
         python version : 3.8.5.final.0
       virtual packages : __unix=0=0
                          __archspec=1=x86_64
       base environment : /Users/tobiasfrick/miniconda  (writable)
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://conda.anaconda.org/bioconda/linux-64
                          https://conda.anaconda.org/bioconda/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/tobiasfrick/miniconda/pkgs
                          /Users/tobiasfrick/.conda/pkgs
       envs directories : /Users/tobiasfrick/miniconda/envs
                          /Users/tobiasfrick/.conda/envs
               platform : linux-64
             user-agent : conda/4.9.2 requests/2.24.0 CPython/3.8.5 Darwin/20.3.0 OSX/10.16
                UID:GID : 501:20
             netrc file : None
           offline mode : False


An unexpected error has occurred. Conda has prepared the above report.

If submitted, this report will be used by core maintainers to improve
future releases of conda.

timestamp for non-t iter axes

If iter_axes is not t you can run into issues with timestamps since it's take from timesteps[image_number].

Should be solved by taking image t info and use timesteps[t] or time information directly from metadata.

Traceback (most recent call last):
  File "/Users/tobiasfrick/miniconda3/envs/ndt/envs/ndt/bin/nd2tools", line 33, in <module>
    sys.exit(load_entry_point('nd2tools', 'console_scripts', 'nd2tools')())
  File "/Users/tobiasfrick/PycharmProjects/nd2tools/src/nd2tools/__main__.py", line 77, in main
    module.main(args)
  File "/Users/tobiasfrick/PycharmProjects/nd2tools/src/nd2tools/cli/image.py", line 39, in main
    image(input=args.input, output=args.output, clip_start=args.clip_start,
  File "/Users/tobiasfrick/PycharmProjects/nd2tools/src/nd2tools/cli/image.py", line 71, in image
    acquisition_time = timesteps[image_number]
IndexError: list index out of range

Add MyPy checks

MyPy can do more advanced code style checking, easy to add and can be good for reducing risks for bugs or unreadable code.

Handle more than one channel

Look into images.iter_axes = "t" and how to properly choose one channel and loop over time.

Maybe something like

images.iter_axes = "c"
images = images[1]
images.iter_axes = "t"

Utilize pyproject.toml more

the pyproject.toml file can be used to predetermine things for MyPy and pytest, like pre-defining path to python test scripts etc.

Add coverage report to testing

A coverage report shows how much of the total code is being touched by the tests supplied in pytest. This can indicate how much of the code is actually being tested when changes are made.

image.size does not correspond to image.shape sometimes

Some nd2 images has images seem to miss one data point, making it impossible to reshape their array which also makes the file non-iterable.

Notably 18105583 == 18105584 - 1, where 18105584 == 3892 * 4651 so it would seem one data point is missing.

NIS-viewer can open the file without issues.

Error

ValueError: cannot reshape array of size 18105583 into shape (3892,4651)

Code to reproduce

#! /bin/python3

from nd2reader import ND2Reader

INPUT = "/Users/tobiasfrick/PycharmProjects/demos/test-data/multiple-channels.nd2"


def iterate():
    with ND2Reader(INPUT) as images:
        for _ in images:
            pass


if __name__ == '__main__':
    iterate()

Versions

% python --version
Python 3.8.10
% conda list | grep nd2reader
nd2reader                 3.3.0              pyhd8ed1ab_0    conda-forge

Traceback

python main.py
Traceback (most recent call last):
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/nd2reader/parser.py", line 277, in _get_raw_image_data
    image_data = np.reshape(image_group_data[image_data_start::number_of_true_channels], (height, width))
  File "<__array_function__ internals>", line 5, in reshape
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 298, in reshape
    return _wrapfunc(a, 'reshape', newshape, order=order)
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 54, in _wrapfunc
    return _wrapit(obj, method, *args, **kwds)
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 43, in _wrapit
    result = getattr(asarray(obj), method)(*args, **kwds)
ValueError: cannot reshape array of size 18105583 into shape (3892,4652)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 18, in <module>
    main()
  File "main.py", line 13, in main
    for i in images:
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/slicerator/__init__.py", line 227, in <genexpr>
    return (self._get(i) for i in self.indices)
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/slicerator/__init__.py", line 207, in _get
    return self._ancestor[key]
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/slicerator/__init__.py", line 188, in __getitem__
    return self._get(indices)
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/pims/base_frames.py", line 98, in __getitem__
    return self.get_frame(key)
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/pims/base_frames.py", line 592, in get_frame
    result = self._get_frame_wrapped(**coords)
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/nd2reader/reader.py", line 88, in get_frame_2D
    return self._parser.get_image_by_attributes(t, v, c, z, y, x)
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/nd2reader/parser.py", line 103, in get_image_by_attributes
    timestamp, raw_image_data = self._get_raw_image_data(image_group_number, channel,
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/nd2reader/parser.py", line 280, in _get_raw_image_data
    image_data = np.reshape(image_group_data[image_data_start::number_of_true_channels], (height, new_width))
  File "<__array_function__ internals>", line 5, in reshape
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 298, in reshape
    return _wrapfunc(a, 'reshape', newshape, order=order)
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 54, in _wrapfunc
    return _wrapit(obj, method, *args, **kwds)
  File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 43, in _wrapit
    result = getattr(asarray(obj), method)(*args, **kwds)
ValueError: cannot reshape array of size 18105583 into shape (3892,4651)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.