Comments (8)
I made it a to-do item yesterday: scikit-hep/awkward#127
But you can use the IndexedMaskedArray
constructor to make option-type data (data with Nones
) without changing the length of the content.
What you have to do is make a "mask" that is increasing, non-negative integers for non-masked values and -1
for masked values:
mask = np.full(length_of_table, -1)
mask[selection] = np.arange(np.count_nonzero(selection))
to_put_in_table = awkward.IndexedMaskedArray(mask, events[selection])
The word "mask" is inappropriate here because this is a reshaping index with negative values interpreted as None
. It will be changing to "IndexedOptionArray".
from awkward-0.x.
Indeed it works, I don't know why it did not when I tried, but thanks!
from awkward-0.x.
Sorry to bother again, the masking in this way works, but when trying to compute a deltaR between a Jagged LorentzVectorArray and a flat IndexedMaskedArray I get an exception, the shapes match, I think there is just some assumption in the broadcasting. Now is quite late, I will try to write a simple test case in the next days.
return self.awkward.numpy.sqrt(self.delta_r2(other))
File "/home/mverzett/miniconda3/lib/python3.7/site-packages/uproot_methods/classes/TLorentzVector.py", line 86, in delta_r2
return (self.eta - other.eta)**2 + self.delta_phi(other)**2
File "/home/mverzett/miniconda3/lib/python3.7/site-packages/numpy/lib/mixins.py", line 25, in func
return ufunc(self, other)
File "/home/mverzett/miniconda3/lib/python3.7/site-packages/awkward/array/jagged.py", line 1027, in __array_ufunc__
content = recurse(data)
File "/home/mverzett/miniconda3/lib/python3.7/site-packages/awkward/array/jagged.py", line 1024, in recurse
content[good] = x.reshape(-1)[parents[good]]
File "/home/mverzett/miniconda3/lib/python3.7/site-packages/awkward/array/base.py", line 256, in __getattr__
raise AttributeError("no column named {0}".format(repr(where)))
AttributeError: no column named 'reshape'```
from awkward-0.x.
This might be related to scikit-hep/uproot3#458, though you're not using ChunkedArrays.
Somewhere in there, there's an assumption that an array node's content
is a numpy.ndarray
, and that assumption is not true. (This is exactly the sort of inconsistency that led me to a rewrite—in Awkward0, some of the arrays are Awkward and some of them are NumPy. Overly strong assumptions didn't get caught in normal usage, but they are caught as we get into more complete usage of the Awkward model: IndexedMaskedArrays and ChunkedArrays).
from awkward-0.x.
Here's where it fails:
It's applying a ufunc and needs to broadcast one side of the binary operation to fit the other (or it's just going through the motions, if it already fits). The array it's looking at has a shape
(I think I needed to add that to everything so that Pandas or Dask would be happy) and it has multiple, regularly sized dimension (i.e. not 1D and not using arbitrary-length sublists through JaggedArray). The array isn't a NumPy array, so there's no reshape
.
Maybe the quick fix is to give all Awkward0 arrays a reshape
operation. On any type that allows a non-1D shape, reshape(-1)
would just apply reshape(-1)
to its starts
, stops
, index
, mask
, or other structure array. As a very quick fix, we could just put in an if-statement to catch your array type.
A maybe even quicker fix is to not use regular-sized dimensions in your analysis: use JaggedArrays instead. I think structure1d()
does that.
I'm looking for quick-fixes here because Awkward1 is almost ready. It will be glued into Uproot in April, and then it would be the primary version for many users. (Awkward1 doesn't have this issue because it uses a new RegularArray node type to represent rectilinear dimensions for non-NumPy content.)
from awkward-0.x.
@jpivarski quick fixes are good for me! :) Can you make an example of structure1d()
usage?
from awkward-0.x.
(If I remember right!) It's a zero-argument method you can call on any array, or maybe just JaggedArrays. It returns a certain if the same data with regular dimensions replaced by JaggedArrays.
from awkward-0.x.
Indeed it works, thanks!
from awkward-0.x.
Related Issues (20)
- dynamically created methods are confusing for users HOT 1
- AssertionError when Table is part of a list HOT 5
- Potential bug with subsequent masking HOT 2
- Reduction of empty elements HOT 2
- TLorentzVectorArray yields different values depending on masking order HOT 4
- IndexError when masking empty jaggedArray made from offsets HOT 1
- awkward method names HOT 9
- TypeError when using array.mean(weights) HOT 2
- Cyclic array? HOT 1
- broken link in readme HOT 1
- Installing awkward-numba in usermode breaks awkward HOT 3
- Syntax warning due to comparison of literals using is in Python 3.8 HOT 1
- Inconsistent Filesizes with .awkd Files HOT 6
- Bug in string comparison in StringArray HOT 1
- mean, std fail on ChunkedArrays HOT 1
- AttributeError when trying to read a particular format of awkward array HOT 5
- JaggedArray.fromiter() functions fails for python lists HOT 2
- Small detail; broadcasting seems to work a little different to what is implied in the documentation. HOT 6
- Accumulate numpy arrays inside the loop HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from awkward-0.x.