Comments (12)
So here is where we start to see that issue. The alternative is to give the mini-language indexing
mag[:, 0]
But the missing part is the mapping between 0 1 2 3 .. and urgiz (which is an knowlege in the painter/ reader)
so mag_u mag_r may actually make more sense . What about using mag.u mag.r?
from nbodykit.
I like the mag.u mag.r syntax. I think that makes the most sense for now
from nbodykit.
I am looking in this again. What if we want to do a selection based pos[:, 0]?
from nbodykit.
Hmm, I don't think I have a good solution for this. For the plain text painters, the positions are named columns "x", "y", "z", etc, so you could do it that way, but that only works for those specific cases.
Maybe we could setup something like an "aliases" keyword that would allow you to define "x" as "eval(pos[:,0])" or something along those lines
from nbodykit.
I don't like aliases. It adds another layer of mental complexity. What about just supporting pos[:, 0] type of selections in selection language?
from nbodykit.
@rainwoodman This has come back up in the context of slicing DataSources by Position to create sub-boxes -- did we ever make any progress on this?
from nbodykit.
Nope.
In npyquery I ended up using 'pos[0] pos[1] pos[2]' to represent pos[:, 0], pos[:, 1] pos[:, 2].
The parser in selectionlanguage needs to be extended to support indexing? Do you want me to look at this?
from nbodykit.
I think it should be relatively easy to put in indexing -- I'll take a stab at it.
I think the trickier part is if we can integrate query selection with the new stream / caching interface better. For example, one problem currently is that there's no way to even slice 'Position' for the Pandas datasource, since it is selecting based off the columns read from the file and not any columns created afterwards, i.e., 'Position'.
One potential way forward:
-- selection done based only off of the column names returned by a DataSource
-- have all DataSource classes take the select
keyword, similar to cosmo
and comm
-- selecting would be done before any caching, so only the relevant data is cached
-- for datasources that explicitly read named columns from files, add those columns to the data returned by readall
, so that they are available for slicing
from nbodykit.
If we take "have all DataSource classes take the select keyword, similar to
cosmo and comm" approach, then it smells like a 'mixin' pattern may be of
relevance. But I am not sufficiently familiar with it.
The datasource shall know the selection in order to perform the selection
the most efficient way.
On Wed, Jun 29, 2016 at 8:59 PM, Nick Hand [email protected] wrote:
I think it should be relatively easy to put in indexing -- I'll take a
stab at it.I think the trickier part is if we can integrate query selection with the
new stream / caching interface better. For example, one problem currently
is that there's no way to even slice 'Position' for the Pandas datasource,
since it is selecting based off the columns read from the file and not any
columns created afterwards, i.e., 'Position'.One potential way forward:
-- selection done based only off of the column names returned by a
DataSource
-- have all DataSource classes take the select keyword, similar to cosmo
and comm
-- selecting would be done before any caching, so only the relevant data
is cached
-- for datasources that explicitly read named columns from files, add
those columns to the data returned by readall, so that they are available
for slicing—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#35 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAIbTHpwxOrN7TFSLt1_jKl68YLdwNopks5qQz8ZgaJpZM4FcLOd
.
from nbodykit.
What do you mean by your second comment? You're thinking the user would implement the selection explicitly in the readall
or parallel_read
function?
from nbodykit.
Also, which of these syntaxes do you like more:
- "Position[:,0] < XX"
- "Position[0] < XX"
where in the second syntax we assume the indexing is referring to the second dimension
from nbodykit.
closed via #203
from nbodykit.
Related Issues (20)
- support for indexing in newer `dask` release HOT 2
- Bug in nbodykit.transform.CartesianToSky
- nersc installation page doesn't exist HOT 1
- `CatalogMesh.to_real_field()` output attributes HOT 1
- Help with parallelising nbodykit code HOT 5
- TPCF calculation error: "Corrfunc.mocks.DDsmu_mocks.DDsmu_mock" fails.
- Problem with painting to mesh if value has zero mean HOT 5
- install nbodykit on NERSC Perlmutter HOT 5
- ```HDFCatalog.to_mesh``` generate negative values for the density field 'delta+1'. HOT 6
- FKPCatalog failing in absence of N(z)
- Trouble understanding the matter power spectrum of lognormal mocks
- Unequal shape partition during MPI parallel mesh.
- not read in correct file in FITs format HOT 4
- FFTPower in redshift-space produces nan in the output (related to #663)
- MPI Issue on Perlmutter HOT 1
- Is there any way to parallelize the computation of multiple convolutions? HOT 8
- Does this work on Python 3.7 and later? HOT 2
- Tutorial on readthedocs: FFTPower requires `mode` argument HOT 1
- Kdcount building fail during cython build on NERSC HOT 2
- Creating a linear mesh from a CLASS power spectrum.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nbodykit.