Git Product home page Git Product logo

astroaugmentations's Introduction

  • ๐Ÿ‘‹ Hi, Iโ€™m Micah a Schmidt AI in Science Postdoctoral Fellow at the University of Oxford.
  • ๐ŸŒฑ I am researching various aspects of large neural networks in astronomy.
  • ๐Ÿค” I think about representation learning, pre-training, inductive biases, explainability, large language models and more.
  • ๐ŸŽ“ See my Google Scholar!
  • ๐Ÿ“ซ Email me at micah.bowles (at) physics.ox.ac.uk
  • ๐Ÿ‘‰ More details on my personal website!

astroaugmentations's People

Contributors

inigoval avatar mb010 avatar patrikasvanagas avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

astroaugmentations's Issues

Broken imports

The current version (0.1.0) installs Albumentations most recent release, which does not contain certain functions which were used. The current work around is to install from source. I.e. install the development version of Albumentations and use the cloned version of this repo.
This will need to be fixed for next update.

AA.composed.ImgOptical fails with Tensorflow dataset

AA.composed.ImgOptical requires a dataset to sample from for AA.image_domain.optical.CroppedTemplateOverlap. This indexes the dataset to pull a random galaxy. Prefetched Tensorflow datasets cannot be indexed so requesting a random image fails.

I would suggest having the dataset an optional requirement in AA.composed.ImgOptical and either skipping the call to AA.image_domain.optical.CroppedTemplateOverlap or calling it with a template if dataset == None

Full traceback:

<ipython-input-7-cf2b44823a21> in <module>
     30 train_iter = iter(train)
     31 
---> 32 img = next(train_iter)['image']
     33 print(img.shape)

~/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py in __next__(self)
    759   def __next__(self):
    760     try:
--> 761       return self._next_internal()
    762     except errors.OutOfRangeError:
    763       raise StopIteration

~/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py in _next_internal(self)
    745           self._iterator_resource,
    746           output_types=self._flat_output_types,
--> 747           output_shapes=self._flat_output_shapes)
    748 
    749       try:

~/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_dataset_ops.py in iterator_get_next(iterator, output_types, output_shapes, name)
   2726       return _result
   2727     except _core._NotOkStatusException as e:
-> 2728       _ops.raise_from_not_ok_status(e, name)
   2729     except _core._FallbackException:
   2730       pass

~/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in raise_from_not_ok_status(e, name)
   6939   message = e.message + (" name: " + name if name is not None else "")
   6940   # pylint: disable=protected-access
-> 6941   six.raise_from(core._status_to_exception(e.code, message), None)
   6942   # pylint: enable=protected-access
   6943 

~/.local/lib/python3.6/site-packages/six.py in raise_from(value, from_value)

InvalidArgumentError: TypeError: 'PrefetchDataset' object does not support indexing
Traceback (most recent call last):

  File "/home/pearsonw/.local/lib/python3.6/site-packages/tensorflow/python/ops/script_ops.py", line 249, in __call__
    ret = func(*args)

  File "/home/pearsonw/.local/lib/python3.6/site-packages/tensorflow/python/autograph/impl/api.py", line 645, in wrapper
    return func(*args, **kwargs)

  File "<ipython-input-7-cf2b44823a21>", line 13, in aug_fn
    aug_data = transform(**data)

  File "/home/pearsonw/.local/lib/python3.6/site-packages/albumentations/core/composition.py", line 205, in __call__
    data = t(force_apply=force_apply, **data)

  File "/home/pearsonw/.local/lib/python3.6/site-packages/albumentations/core/transforms_interface.py", line 95, in __call__
    return self.apply_with_params(params, **kwargs)

  File "/home/pearsonw/.local/lib/python3.6/site-packages/albumentations/core/transforms_interface.py", line 110, in apply_with_params
    res[key] = target_function(arg, **dict(params, **target_dependencies))

  File "/home/pearsonw/.local/lib/python3.6/site-packages/albumentations/augmentations/transforms.py", line 2219, in apply
    return fn(img, **params)

  File "/home/pearsonw/.local/lib/python3.6/site-packages/astroaugmentations/composed.py", line 183, in __call__
    return self.augmentation(image=image)['image']

  File "/home/pearsonw/.local/lib/python3.6/site-packages/albumentations/core/composition.py", line 205, in __call__
    data = t(force_apply=force_apply, **data)

  File "/home/pearsonw/.local/lib/python3.6/site-packages/albumentations/core/transforms_interface.py", line 95, in __call__
    return self.apply_with_params(params, **kwargs)

  File "/home/pearsonw/.local/lib/python3.6/site-packages/albumentations/core/transforms_interface.py", line 110, in apply_with_params
    res[key] = target_function(arg, **dict(params, **target_dependencies))

  File "/home/pearsonw/.local/lib/python3.6/site-packages/albumentations/augmentations/transforms.py", line 2219, in apply
    return fn(img, **params)

  File "/home/pearsonw/.local/lib/python3.6/site-packages/astroaugmentations/image_domain/optical.py", line 310, in __call__
    datasample, _ = self.dataset[self.rng.integers(0, len(self.dataset))]

TypeError: 'PrefetchDataset' object does not support indexing


	 [[{{node augment}}]] [Op:IteratorGetNext]

(I did promise you I would break it)

Transformation Application

Potentially update transformation handling to be in line with pytorch transforms. The suggestion being that A.Compose([]) objects are called using a lambda function before being passed into the data loader. Something like: transform = lambda x : A.Compose([...])[image=x]['image'].

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.