Git Product home page Git Product logo

pysensors's Introduction

PySensors

Build Documentation Status PyPI Codecov Binder JOSS Zenodo

PySensors is a Scikit-learn style Python package for the sparse placement of sensors, either for reconstruction or classification tasks.

Sparse sensor placement concerns the problem of selecting a small subset of sensor or measurement locations in a way that allows one to perform some task nearly as well as if one had access to measurements at every location.

PySensors provides objects designed for the tasks of reconstruction and classification. See Manohar et al. (2018) for more information about the PySensors approach to reconstruction problems and Brunton et al. (2016) for classification. de Silva et al. (2021) contains a full literature review along with examples and additional tips for using PySensors effectively.

Reconstruction deals with predicting the values of a quantity of interest at different locations other than those where sensors are located. For example, one might predict the temperature at a point in the middle of a lake based on temperature readings taken at various other positions in the lake.

PySensors provides the SSPOR (Sparse Sensor Placement Optimization for Reconstruction) class to aid in the solution of reconstruction problems.

Take representative examples of the types of data to be reconstructed (in this case polynomials)

x = numpy.linspace(0, 1, 1001)
data = numpy.vander(x, 11).T  # Create an array whose rows are powers of x

feed them to a SSPOR instance with 10 sensors, and

model = pysensors.reconstruction.SSPOR(n_sensors=10)
model.fit(data)

Use the predict method to reconstruct a new function sampled at the chosen sensor locations:

f = numpy.abs(x[model.selected_sensors]**2 - 0.5)
f_pred = model.predict(f)
A plot showing the function to be reconstructed, the learned sensor locations, and the reconstruction.

Classification is the problem of predicting which category an example belongs to, given a set of training data (e.g. determining whether digital photos are of dogs or cats). The SSPOC (Sparse Sensor Placement Optimization for Classification) class is used to solve classification problems. Users familiar with Scikit-learn will find it intuitive:

model = pysensors.classification.SSPOC()
model.fit(x, y)  # Learn sensor locations and fit a linear classifier
y_pred = model.predict(x_test[:, model.selected_sensors])  #  Get predictions

See our set of classification examples for more information.

The basis in which measurement data are represented can have a dramatic effect on performance. PySensors implements the three bases most commonly used for sparse sensor placement: raw measurements, SVD/POD/PCA modes, and random projections. Bases can be easily incorporated into SSPOR and SSPOC classes:

basis = pysensors.basis.SVD(n_basis_modes=20)
recon_model = pysensors.reconstruction.SSPOR(basis=basis)
class_model = pysensors.classification.SSPOC(basis=basis)

See this example for further discussion of these options.

The high-level dependencies for PySensors are Linux or macOS and Python 3.6-3.8. pip is also recommended as is makes managing PySensors' other dependencies much easier. You can install it by following the instructions here.

PySensors has not been tested on Windows.

If you are using Linux or macOS you can install PySensors with pip from the command line/terminal:

pip install python-sensors

Note: the name you type in here is python-sensors and is not pysensors.

Once you have run the line above, you are ready to get started with PySensors. Have a look at the examples in our documentation to see what PySensors can do.

First clone this repository:

git clone https://github.com/dynamicslab/pysensors.git

Then, to install the package, run

cd pysensors
pip install .

If you do not have pip you can instead use

python setup.py install

If you do not have root access, you should add the --user option to the install commands above.

The primary PySensors objects are the SSPOR and SSPOC classes, which are used to choose sensor locations optimized for reconstruction and classification tasks, respectively. Other implemented objects include

  • basis - submodule implementing different bases in which to represent data
    • Identity - use raw measurement data
    • SVD - efficiently compute first k left singular vectors
    • RandomProjection - Gaussian random projections of measurements
  • Convenience functions to aid in the analysis of error as number of sensors or basis modes are varied

PySensors has a documentation site hosted by readthedocs. Examples are available online, as static Jupyter notebooks and as interactive notebooks. To run the example notebooks locally you should install the dependencies in requirements-examples.txt:

pip install -r requirements-examples.txt

You may create an issue for any questions that aren't answered by the documentation or examples.

If you have used PySensors to solve an interesting problem, please consider submitting an example Jupyter notebook showcasing your work!

We welcome contributions to PySensors. To contribute a new feature please submit a pull request. To get started we recommend installing the packages in requirements-dev.txt via

pip install -r requirements-dev.txt

This will allow you to run unit tests and automatically format your code. To be accepted your code should conform to PEP8 and pass all unit tests. Code can be tested by invoking

pytest

We recommend using pre-commit to format your code. Once you have staged changes to commit

git add path/to/changed/file.py

you can run the following to automatically reformat your staged code

pre-commit

Note that you will then need to re-stage any changes pre-commit made to your code.

If you find a bug in the code or want to request a new feature, please open an issue.

We have published a short paper in the Journal of Open Source Software (JOSS). You can find the paper here.

If you use PySensors in your work, please consider citing it using:

de Silva et al., (2021). PySensors: A Python package for sparse sensor placement. Journal of Open Source Software, 6(58), 2828, https://doi.org/10.21105/joss.02828``

Bibtex:

@article{de Silva2021,
  doi = {10.21105/joss.02828},
  url = {https://doi.org/10.21105/joss.02828},
  year = {2021},
  publisher = {The Open Journal},
  volume = {6},
  number = {58},
  pages = {2828},
  author = {Brian M. de Silva and Krithika Manohar and Emily Clark and Bingni W. Brunton and J. Nathan Kutz and Steven L. Brunton},
  title = {PySensors: A Python package for sparse sensor placement},
  journal = {Journal of Open Source Software}
}
  • de Silva, Brian M., Krithika Manohar, Emily Clark, Bingni W. Brunton, Steven L. Brunton, J. Nathan Kutz. "PySensors: A Python package for sparse sensor placement." arXiv preprint arXiv:2102.13476 (2021). [arXiv]
  • Manohar, Krithika, Bingni W. Brunton, J. Nathan Kutz, and Steven L. Brunton. "Data-driven sparse sensor placement for reconstruction: Demonstrating the benefits of exploiting known patterns." IEEE Control Systems Magazine 38, no. 3 (2018): 63-86. [DOI]
  • Brunton, Bingni W., Steven L. Brunton, Joshua L. Proctor, and J Nathan Kutz. "Sparse sensor placement optimization for classification." SIAM Journal on Applied Mathematics 76.5 (2016): 2099-2122. [DOI]
  • Clark, Emily, Travis Askham, Steven L. Brunton, and J. Nathan Kutz. "Greedy sensor placement with cost constraints." IEEE Sensors Journal 19, no. 7 (2018): 2642-2656. [DOI]

pysensors's People

Contributors

briandesilva avatar emilyclark012 avatar jimmy-inl avatar kmanohar avatar niharika2999 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pysensors's Issues

Sensor Grouping

Dear team,

Firstly, many thanks for your fantastic work on SSPOC and also for providing this package. This is indeed a beneficial resource.

I am currently using PySensors to find the minimal sensors required for a classification task. However, instead of running this directly on the actual sensor count, I intend to input the extracted features and then select the number of sensors.

Specifically, I have data from 20 sensors, and I calculate 100 features/sensor. This results in a matrix with 2000 columns. In other words, the SSPOC algorithm treats this as a problem with 2000 sensors. While the 'selected_sensors' lists the particular feature that can be traced back to the corresponding sensor. I would be grateful if you could please link me to any inbuilt function for grouping the sensors or any other voting mechanism that I may use or implement?

Thanks a lot for your time.

Multi-dimensional SSPOR

Awesome work you guys do in your research groupπŸ™ŒπŸ™ŒπŸ™Œ

I am looking into a case quite similar to what I have seen from your group - The von Karman example.

Suppose I have the von Karman example of flow past a cylinder and wish to do SSPOR based on a minimal number of sensors. The challenge in my case is that I have an additional dimension to my data. Basically, I have the spatial-temporal snapshots for a bunch of different boundary conditions (or flow points if you like) and I am looking to find the best sensor placement to reconstruct the entire transient pressure field (for a given sensor input) and not just an instance. Can pySensor handle that?

Hope it makes sense

Documentation: Vandermonde Example

Dear team,

Thank you for the extensive work that has gone into this package. I wanted to try and clarify the Vandermonde example (polynomial interpolation) in the documentation so that I may better understand the functionality of this package.

  1. The basis modes refer to the number of columns in the matrix but what criteria determines the number we select? Does it refer to every possible sensor location or the regions where sensors can be placed?
  2. For the list of data defined as x, is that referring to a list of normalized values or can we input the exact values of the QoI we obtain from simulations? Similarly, is x only defined as a distance or is that just an example and we can input any kind of data?
  3. My interest with PySensors is seeing if I can find optimal sensors locations based on data I obtain from a 3D simulated model. Typically, these will not have analytical solutions to compare the reconstruction error too. Is there any other form of validation to provide confidence in where the SSPOR places the sensors?

Thank you for your time and help.

-Kennan Hodovic

Oversampling

Dear developer team
First of all, many thanks for the Py Sensors packages and the detailed and very clear examples! I am using SSPOR to optimize a hydrographs observation network. My data set consists of 480 sensors with 1304 features each. Now I would like to make reconstruction with more than 480 SVD basis modes (p>r). In the paper of Manohar e al (2018) I have found out that the oversampled case can be handled with ψrψrT. This should allow a maximum of 1304 basis modes?! Is there a possibility to integrate this into the SSPOR functionality?
Thanks for your help!
Marc

Grid search with SSPOC

Hello, as I have seen in example pysensor supports sklearn based parameters optimization. In the example, it is shown how the parameter tuning is performed for SSPOR. However, I would like to perform it for SSPOC, but for now I was unable to do so. At the end of optimization it throws me an error: ValueError: w must be a 1D vector; received a vector of dimension 0 which is implemented in pysensors\utils\_optimizers.py. Furthermore, scores which are calculated during grid search are all nan values.

I am attaching simple code snippet in case anyone could help. I hope I will not have to code grid search manually.

model_clf = LinearDiscriminantAnalysis()
model_cs = SSPOC(classifier=model_clf)

param_grid = {
    "basis": [ps.basis.Identity(), ps.basis.SVD(), ps.basis.RandomProjection()],
    "basis__n_basis_modes": [1,2,3,4,5],
    "n_sensors": [5,2,3]
}

cross_val = RepeatedStratifiedKFold(n_splits=5, n_repeats=3, random_state=1)
search = GridSearchCV(estimator = model_cs, 
                        param_grid = param_grid, 
                        scoring = "balanced_accuracy", 
                        n_jobs = -1, 
                        cv = cross_val,
                        refit = True,
                        verbose = 0,
                    )

search.fit(X_train, y_train)

print("---------------------------------------")
print("Best parameters:")
for k, v in search.best_params_.items():
    print(f"{k}:  {v}")
print("")

[Defect] In the case of cost_constrained_qr, the algorithm should error out if the number of sensors is larger than the number of samples

In the cost_constrained_qr algorithms, there must be an assertion or an IOError if the number of sensors is greater than the number of samples, this will cause the current implementation after the sensors exceed the number of samples (i.e., after 300 in the notebook example) will start ignoring the constrained area. Of course, this is a rare occasion, primarily since the whole idea is based on sparsity, but some users might have limited samples and might still try this.

To reproduce this issue, go to the ./examples/cost_constrained_qr.ipynb and change n_sensors to 350, you will see sensors start to be placed in the constrained areas.

Package Update

Is there any package update? I tried to use cross validation as your example but no 'SensorSelector' module is assigned to pysensor.

Code example in README.rst is outdated

While reviewing your toolbox for JOSS I found out that the reconstruction example in the README.rst is outdated:

>>> model = pysensors.Sensorselector(n_sensors=10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'pysensors' has no attribute 'Sensorselector'

I think pysensors.Sensorselector has been renamed to pysensors.reconstruction.SSPOR().

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.