Git Product home page Git Product logo

earthmapper's Introduction

EarthMapper

Project repository for EarthMapper. This is a toolbox for the semantic segmentation of non-RGB (i.e., multispectral/hyperspectral) imagery. We will work on adding more examples and better documentation.

Description

This is a classification pipeline from various projects that we have worked on over the past few years. Currently available options include:

Pre-Processing

  • MinMaxScaler - Scale data (per-channel) between a given feature range (e.g., 0-1)
  • StandardScaler - Scale data (per-channel) to zero-mean/unit-variance
  • PCA - Reduce dimensionality via principal component analysis
  • Normalize - Scale data using the per-channel L2 norm

Spatial-Spectral Feature Extraction

  • Stacked Convolutional Autoencoder (SCAE)
  • Stacked Multi-Loss Convolutional Autoencoder (SMCAE)

Classifiers

  • SVMWorkflow - Support vector machine with a given training/validation split
  • SVMCVWorkflow - Support vector machine that uses n-fold cross-validation to find optimal hyperparameters
  • RandomForestWorkflow - Random Forest classifier
  • MLP - Multi-layer Perceptron Neural Network classifier
  • SSMLP - Semi-supervised MLP Neural Network classifier

Post-Processors

  • Markov Random Field (MRF)
  • Fully-Connected Conditional Random Field (CRF)

Dependencies

Instructions

Installation

$ python setup.py

Run example

$ python examples/example_pipeline.py

Citations

If you use our product, please cite:

Points of Contact

Also Check Out

earthmapper's People

Contributors

ubgewali avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

earthmapper's Issues

No module named 'fileIO'

After running setup.py and installing the dependencies. I start with example_pipeline.py and then i got a problem : No module named 'fileIO'
tested on : ubuntu / windows10
python 3.6

ValueError: Node 'gradients/refinement1/bn2/cond/FusedBatchNorm_1_grad/FusedBatchNormGrad' has an _output_shapes attribute inconsistent with the GraphDef for output #3: Dimension 0 in both shapes must be equal, but are 0 and 256. Shapes are [0] and [256].

2023-11-02 15:23:33.318653: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2023-11-02 15:23:33.319697: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
C:\Users\rupes\anaconda3\envs\Env2\lib\site-packages\h5py_init_.py:39: UserWarning: h5py is running against HDF5 1.12.2 when it was built against 1.12.0, this may cause problems
'{0}.{1}.{2}'.format(*version.hdf5_built_version_tuple)
WARNING:tensorflow:From C:\Users\rupes\anaconda3\envs\Env2\lib\site-packages\tensorflow\python\compat\v2_compat.py:101: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
spectral:INFO: No overlap for target band 0 (365.929800 / 9.852108)
spectral:INFO: No overlap for target band 1 (375.594000 / 9.796976)
spectral:INFO: No overlap for target band 2 (385.262500 / 9.744104)
spectral:INFO: No overlap for target band 3 (394.935500 / 9.693492)
spectral:INFO: No overlap for target band 4 (404.612900 / 9.645140)
spectral:INFO: No overlap for target band 5 (414.294600 / 9.599048)
spectral:INFO: No overlap for target band 52 (850.944200 / 10.568910)
spectral:INFO: No overlap for target band 53 (860.647100 / 10.704480)
spectral:INFO: No overlap for target band 54 (870.344800 / 10.849820)
spectral:INFO: No overlap for target band 55 (880.037200 / 11.004930)
spectral:INFO: No overlap for target band 56 (889.724500 / 11.169800)
spectral:INFO: No overlap for target band 57 (899.406600 / 11.344440)
spectral:INFO: No overlap for target band 58 (909.083400 / 11.528850)
spectral:INFO: No overlap for target band 59 (918.755100 / 11.723020)
spectral:INFO: No overlap for target band 60 (928.421400 / 11.926960)
spectral:INFO: No overlap for target band 61 (938.082700 / 10.781530)
spectral:INFO: No overlap for target band 62 (947.738700 / 9.761805)
spectral:INFO: No overlap for target band 63 (957.389500 / 9.760064)
spectral:INFO: No overlap for target band 64 (967.035100 / 9.759324)
spectral:INFO: No overlap for target band 65 (976.675500 / 9.759585)
spectral:INFO: No overlap for target band 66 (986.310600 / 9.760846)
spectral:INFO: No overlap for target band 67 (995.940600 / 9.763108)
spectral:INFO: No overlap for target band 68 (1005.565000 / 9.766370)
spectral:INFO: No overlap for target band 69 (1015.185000 / 9.770633)
spectral:INFO: No overlap for target band 70 (1024.799000 / 9.775896)
spectral:INFO: No overlap for target band 71 (1034.408000 / 9.782160)
spectral:INFO: No overlap for target band 72 (1044.012000 / 9.789424)
spectral:INFO: No overlap for target band 73 (1053.611000 / 9.797689)
spectral:INFO: No overlap for target band 74 (1063.204000 / 9.806954)
spectral:INFO: No overlap for target band 75 (1072.793000 / 9.817220)
spectral:INFO: No overlap for target band 76 (1082.376000 / 9.828486)
spectral:INFO: No overlap for target band 77 (1091.954000 / 9.840753)
spectral:INFO: No overlap for target band 78 (1101.526000 / 9.854020)
spectral:INFO: No overlap for target band 79 (1111.094000 / 9.868288)
spectral:INFO: No overlap for target band 80 (1120.656000 / 9.883556)
spectral:INFO: No overlap for target band 81 (1130.213000 / 9.899825)
spectral:INFO: No overlap for target band 82 (1139.765000 / 9.917094)
spectral:INFO: No overlap for target band 83 (1149.311000 / 9.935364)
spectral:INFO: No overlap for target band 84 (1158.853000 / 9.954635)
spectral:INFO: No overlap for target band 85 (1168.389000 / 9.974905)
spectral:INFO: No overlap for target band 86 (1177.920000 / 9.996177)
spectral:INFO: No overlap for target band 87 (1187.446000 / 10.018450)
spectral:INFO: No overlap for target band 88 (1196.966000 / 10.041720)
spectral:INFO: No overlap for target band 89 (1206.482000 / 10.065990)
spectral:INFO: No overlap for target band 90 (1215.992000 / 10.091270)
spectral:INFO: No overlap for target band 91 (1225.497000 / 10.117540)
spectral:INFO: No overlap for target band 92 (1234.996000 / 10.144810)
spectral:INFO: No overlap for target band 93 (1244.491000 / 10.173090)
spectral:INFO: No overlap for target band 94 (1253.980000 / 10.202360)
spectral:INFO: No overlap for target band 95 (1263.464000 / 10.232640)
spectral:INFO: No overlap for target band 96 (1253.373000 / 10.838260)
spectral:INFO: No overlap for target band 97 (1263.346000 / 10.832670)
spectral:INFO: No overlap for target band 98 (1273.318000 / 10.827210)
spectral:INFO: No overlap for target band 99 (1283.291000 / 10.821880)
spectral:INFO: No overlap for target band 100 (1293.262000 / 10.816700)
spectral:INFO: No overlap for target band 101 (1303.234000 / 10.811650)
spectral:INFO: No overlap for target band 102 (1313.206000 / 10.806740)
spectral:INFO: No overlap for target band 103 (1323.177000 / 10.801960)
spectral:INFO: No overlap for target band 104 (1333.148000 / 10.797330)
spectral:INFO: No overlap for target band 105 (1343.119000 / 10.792830)
spectral:INFO: No overlap for target band 106 (1353.089000 / 10.788470)
spectral:INFO: No overlap for target band 107 (1402.939000 / 10.768730)
spectral:INFO: No overlap for target band 108 (1412.908000 / 10.765190)
spectral:INFO: No overlap for target band 109 (1422.877000 / 10.761800)
spectral:INFO: No overlap for target band 110 (1432.845000 / 10.758530)
spectral:INFO: No overlap for target band 111 (1442.814000 / 10.755410)
spectral:INFO: No overlap for target band 112 (1452.782000 / 10.752430)
spectral:INFO: No overlap for target band 113 (1462.750000 / 10.749580)
spectral:INFO: No overlap for target band 114 (1472.718000 / 10.746870)
spectral:INFO: No overlap for target band 115 (1482.685000 / 10.744290)
spectral:INFO: No overlap for target band 116 (1492.652000 / 10.741860)
spectral:INFO: No overlap for target band 117 (1502.619000 / 10.739560)
spectral:INFO: No overlap for target band 118 (1512.586000 / 10.737400)
spectral:INFO: No overlap for target band 119 (1522.552000 / 10.735380)
spectral:INFO: No overlap for target band 120 (1532.518000 / 10.733490)
spectral:INFO: No overlap for target band 121 (1542.484000 / 10.731740)
spectral:INFO: No overlap for target band 122 (1552.450000 / 10.730130)
spectral:INFO: No overlap for target band 123 (1562.416000 / 10.728660)
spectral:INFO: No overlap for target band 124 (1572.381000 / 10.727320)
spectral:INFO: No overlap for target band 125 (1582.346000 / 10.726120)
spectral:INFO: No overlap for target band 126 (1592.311000 / 10.725060)
spectral:INFO: No overlap for target band 127 (1602.275000 / 10.724140)
spectral:INFO: No overlap for target band 128 (1612.240000 / 10.723350)
spectral:INFO: No overlap for target band 129 (1622.204000 / 10.722700)
spectral:INFO: No overlap for target band 130 (1632.167000 / 10.722190)
spectral:INFO: No overlap for target band 131 (1642.131000 / 10.721820)
spectral:INFO: No overlap for target band 132 (1652.094000 / 10.721590)
spectral:INFO: No overlap for target band 133 (1662.057000 / 10.721490)
spectral:INFO: No overlap for target band 134 (1672.020000 / 10.721530)
spectral:INFO: No overlap for target band 135 (1681.983000 / 10.721700)
spectral:INFO: No overlap for target band 136 (1691.945000 / 10.722020)
spectral:INFO: No overlap for target band 137 (1701.907000 / 10.722470)
spectral:INFO: No overlap for target band 138 (1711.869000 / 10.723060)
spectral:INFO: No overlap for target band 139 (1721.831000 / 10.723780)
spectral:INFO: No overlap for target band 140 (1731.792000 / 10.724650)
spectral:INFO: No overlap for target band 141 (1741.753000 / 10.725650)
spectral:INFO: No overlap for target band 142 (1751.714000 / 10.726790)
spectral:INFO: No overlap for target band 143 (1761.675000 / 10.728070)
spectral:INFO: No overlap for target band 144 (1771.635000 / 10.729480)
spectral:INFO: No overlap for target band 145 (1781.596000 / 10.731030)
spectral:INFO: No overlap for target band 146 (1791.556000 / 10.732720)
spectral:INFO: No overlap for target band 147 (1801.515000 / 10.734550)
spectral:INFO: No overlap for target band 148 (1811.475000 / 10.736510)
spectral:INFO: No overlap for target band 149 (1937.246000 / 11.099030)
spectral:INFO: No overlap for target band 150 (1947.292000 / 11.088110)
spectral:INFO: No overlap for target band 151 (1957.335000 / 11.076870)
spectral:INFO: No overlap for target band 152 (1967.375000 / 11.065310)
spectral:INFO: No overlap for target band 153 (1977.414000 / 11.053440)
spectral:INFO: No overlap for target band 154 (1987.450000 / 11.041250)
spectral:INFO: No overlap for target band 155 (1997.484000 / 11.028750)
spectral:INFO: No overlap for target band 156 (2007.515000 / 11.015920)
spectral:INFO: No overlap for target band 157 (2017.545000 / 11.002780)
spectral:INFO: No overlap for target band 158 (2027.572000 / 10.989330)
spectral:INFO: No overlap for target band 159 (2037.596000 / 10.975550)
spectral:INFO: No overlap for target band 160 (2047.619000 / 10.961460)
spectral:INFO: No overlap for target band 161 (2057.639000 / 10.947050)
spectral:INFO: No overlap for target band 162 (2067.656000 / 10.932330)
spectral:INFO: No overlap for target band 163 (2077.672000 / 10.917290)
spectral:INFO: No overlap for target band 164 (2087.685000 / 10.901930)
spectral:INFO: No overlap for target band 165 (2097.696000 / 10.886260)
spectral:INFO: No overlap for target band 166 (2107.704000 / 10.870260)
spectral:INFO: No overlap for target band 167 (2117.710000 / 10.853960)
spectral:INFO: No overlap for target band 168 (2127.714000 / 10.837330)
spectral:INFO: No overlap for target band 169 (2137.716000 / 10.820390)
spectral:INFO: No overlap for target band 170 (2147.715000 / 10.803130)
spectral:INFO: No overlap for target band 171 (2157.712000 / 10.785550)
spectral:INFO: No overlap for target band 172 (2167.707000 / 10.767660)
spectral:INFO: No overlap for target band 173 (2177.699000 / 10.749450)
spectral:INFO: No overlap for target band 174 (2187.689000 / 10.730920)
spectral:INFO: No overlap for target band 175 (2197.677000 / 10.712080)
spectral:INFO: No overlap for target band 176 (2207.662000 / 10.692920)
spectral:INFO: No overlap for target band 177 (2217.645000 / 10.673440)
spectral:INFO: No overlap for target band 178 (2227.626000 / 10.653650)
spectral:INFO: No overlap for target band 179 (2237.604000 / 10.633540)
spectral:INFO: No overlap for target band 180 (2247.581000 / 10.613110)
spectral:INFO: No overlap for target band 181 (2257.554000 / 10.592360)
spectral:INFO: No overlap for target band 182 (2267.526000 / 10.571300)
spectral:INFO: No overlap for target band 183 (2277.495000 / 10.549920)
spectral:INFO: No overlap for target band 184 (2287.462000 / 10.528230)
spectral:INFO: No overlap for target band 185 (2297.427000 / 10.506220)
spectral:INFO: No overlap for target band 186 (2307.389000 / 10.483890)
spectral:INFO: No overlap for target band 187 (2317.349000 / 10.461240)
spectral:INFO: No overlap for target band 188 (2327.307000 / 10.438280)
spectral:INFO: No overlap for target band 189 (2337.262000 / 10.415000)
spectral:INFO: No overlap for target band 190 (2347.216000 / 10.391400)
spectral:INFO: No overlap for target band 191 (2357.167000 / 10.367490)
spectral:INFO: No overlap for target band 192 (2367.115000 / 10.343260)
spectral:INFO: No overlap for target band 193 (2377.061000 / 10.318710)
spectral:INFO: No overlap for target band 194 (2387.005000 / 10.293850)
spectral:INFO: No overlap for target band 195 (2396.947000 / 10.268670)
spectral:INFO: No overlap for target band 196 (2406.886000 / 10.243170)
spectral:INFO: No overlap for target band 197 (2416.823000 / 10.217360)
spectral:INFO: No overlap for target band 198 (2426.758000 / 10.191230)
spectral:INFO: No overlap for target band 199 (2436.690000 / 10.164780)
spectral:INFO: No overlap for target band 200 (2446.620000 / 10.138010)
spectral:INFO: No overlap for target band 201 (2456.548000 / 10.110930)
spectral:INFO: No overlap for target band 202 (2466.473000 / 10.083530)
spectral:INFO: No overlap for target band 203 (2476.396000 / 10.055820)
spectral:INFO: No overlap for target band 204 (2486.317000 / 10.027780)
spectral:INFO: No overlap for target band 205 (2496.236000 / 9.999434)
2023-11-02 15:23:38.015369: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'nvcuda.dll'; dlerror: nvcuda.dll not found
2023-11-02 15:23:38.017523: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2023-11-02 15:23:38.025449: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: RupeshKumar
2023-11-02 15:23:38.026297: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: RupeshKumar
2023-11-02 15:23:38.039515: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
[Pipeline Fit Timer]
Elapsed: 1.907 seconds
Traceback (most recent call last):
File "C:\Users\rupes\anaconda3\envs\Env2\lib\site-packages\tensorflow\python\framework\importer.py", line 497, in _import_graph_def_internal
graph._c_graph, serialized, options) # pylint: disable=protected-access
tensorflow.python.framework.errors_impl.InvalidArgumentError: Node 'gradients/refinement1/bn2/cond/FusedBatchNorm_1_grad/FusedBatchNormGrad' has an _output_shapes attribute inconsistent with the GraphDef for output #3: Dimension 0 in both shapes must be equal, but are 0 and 256. Shapes are [0] and [256].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File ".\examples\example_pipeline.py", line 111, in
pipe.fit(X , y_train, X, y_val)
File "C:\rupesh\4th_year\Final_project\EarthMapper-master\EarthMapper\pipeline.py", line 205, in fit
X_train = fe.transform(X_train)
File "C:\rupesh\4th_year\Final_project\EarthMapper-master\EarthMapper\feature_extraction\scae.py", line 106, in transform
clear_devices=True)
File "C:\Users\rupes\anaconda3\envs\Env2\lib\site-packages\tensorflow\python\training\saver.py", line 1467, in import_meta_graph
**kwargs)[0]
File "C:\Users\rupes\anaconda3\envs\Env2\lib\site-packages\tensorflow\python\training\saver.py", line 1491, in _import_meta_graph_with_return_elements
**kwargs))
File "C:\Users\rupes\anaconda3\envs\Env2\lib\site-packages\tensorflow\python\framework\meta_graph.py", line 806, in import_scoped_meta_graph_with_return_elements
return_elements=return_elements)
File "C:\Users\rupes\anaconda3\envs\Env2\lib\site-packages\tensorflow\python\util\deprecation.py", line 549, in new_func
return func(*args, **kwargs)
File "C:\Users\rupes\anaconda3\envs\Env2\lib\site-packages\tensorflow\python\framework\importer.py", line 405, in import_graph_def
producer_op_list=producer_op_list)
File "C:\Users\rupes\anaconda3\envs\Env2\lib\site-packages\tensorflow\python\framework\importer.py", line 501, in _import_graph_def_internal
raise ValueError(str(e))
ValueError: Node 'gradients/refinement1/bn2/cond/FusedBatchNorm_1_grad/FusedBatchNormGrad' has an _output_shapes attribute inconsistent with the GraphDef for output #3: Dimension 0 in both shapes must be equal, but are 0 and 256. Shapes are [0] and [256].

How did you extract patches from IndianPines dataset?

Thank you for sharing the code.
I have few queries regarding extracting data patches from Indian Pines data.

  1. Have you added zero values on edges data while extracting patches?
  2. I couldn't understand the meaning of this line in the paper. "The SCAE was trained by extracting 2000 16 ร— 16 patches from 20 unlabeled data sets" What is the 20 unlabeled data sets?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.