asreimer / lmfit2 Goto Github PK

View Code? Open in Web Editor NEW

3.0 2.0 3.0 21.88 MB

Non-Linear Levenburg-Marquardt Least-Squares Fitting of SuperDARN ACFs

License: GNU General Public License v3.0

Makefile 0.84% C 78.67% Objective-C 0.70% Python 19.11% C++ 0.29% Shell 0.39%

acf marquardt python c superdarn levenburg-marquardt levenburg rawacf

lmfit2's Introduction

Levenburg-Marquardt fitting of SuperDARN auto-correlation functions (ACFs). With NO ad hoc assumptions/conditions!

As described by Reimer et. al. (2018).

Fitted SuperDARN data with accurate error bars. Filter data based on fitted error:

Pictured above is 8 hours of fitted velocity data for the Saskatoon SuperDARN radar for none, 800 m/s, 400 m/s, 200 m/s, 100 m/s, and 50 m/s velocity error thresholds.

Summary

This repository contains code to that can fit the SuperDARN rawacf files using an error weighted non-linear least-squares fitting algorithm called the First-Principles Fitting Methodology (FPFM). The details of this algorithm as described in Reimer et. al. (2018). The real and imaginary components are fitted against a decaying complex sinusoidal model of the ACF. The Levenburg-Marquardt technique is used to minimize the chi-squared sum. No ad hoc conditions are utilized. The errors in the real and imaginary components of the ACF lag estimates are based on the work by Reimer et. al. (2016). Self-clutter due to the multiple-pulse technique is accounted for using the Mean Power-based Self-clutter Estimator (MPSE) detailed in Reimer and Hussey (2015).

C Code

The C code requires RSTLite to be installed (need the C dmap library and the hdw.dat files). For both this software and RSTLite you will need to have zlib-dev installed. It also utilizes the cmpfit library written by Craig B. Marquardt. The codebase has been structured in the same way as the fitacf3.0 code written by Keith Kotyk.

Once the RSTLite and zlib dependencies are installed, to compile the lmfit2 code, simply change into the C code directory and use the make command. The compiled binary will be in the bin directory.

Sourcing RSTLite .profile.bash

Remember to source the .profile.bash file in the RSTLite directory. If you do not, the make file for lmfit2 will not work.

Python Code

The python version of lmfit2 requires the numpy and lmfit python packages to be installed. The code also relies on the python version of the dmap library (pydmap written by Keith Kotyk), which is included in this repository.

Limitations and TO DO

The current version of the C code is not supported for usage with anything but RSTLite. A version compatible the new RST>=4.0 is currently under development.

Currently, this code only fits the ACF not the XCF, so no elevation angles are fit for yet. Also, only the exponential model of the ACF is fit. A version of the code that fits both gaussian and exponential and ACF and XCF is under development.

Speed

The C code takes approximately 9.5 minutes for 1 hour worth of raw data. Python hasn't yet been speed tested.

lmfit2's People

Contributors

Stargazers

Watchers

Forkers

aburrell remingtonrohel gerhobbelt

lmfit2's Issues

Adding lmfit2 to the SuperDARN RST

Hei @asreimer!

I'm reaching out because there have been several requests to add this library to the RST. We can take care of the implementation, but I just wanted to touch base first. Are you happy for us to copy the code? And do you recommend that we copy the develop branch (which seems to include some bug fixes)? RST and lmfit2 are both licensed under the GPLv3 so there will be no problems there.

Thanks!
Emma

c flags and dependencies

I'm having trouble getting the C code to compile. Problems I've had:

Bstatic/dynamic flag unknown (-static/dynamic work)
The library for "-lz" isn't found. Which library is this referencing for you? I didn't see anything about this in the README.

[ENH]: Analytic Jacobians

Instead of allowing mpfit to numerically estimate the jacobian, construct the jacobian analytically and provide this to mpfit.

mpfit is called here.

Documentation on how to provide user-specified derivatives to mpfit is here

[ENH]: Add elevation angle fitting

XCF fitting is currently missing and needs to be implemented.

Migrate to dependency on SuperDARN/rst

Currently C code depends on deprecated VTRST3.5 or RSTLite. Should be migrated to new supported SuperDARN/rst.

docstrings and documentation

Gave up on installing the C version and went to use the python. There are no docstrings on the routines, and the README doesn't describe how the routines are supposed to be used. Help?

[ENH]: Add Gaussian envelope model

Standard SuperDARN processing usually fits both an Exponential and Gaussian envelope models. Currently, LMFIT2 only supports Exponential envelope fitting.

Issue creating the lmfit2 file with the python version

Hi Ashton,
I've been trying to run your lmfit2 code and I've been having some problems. I'm using python 2.7.13 and numpy 1.11.2. I'm trying to create the lmfit2 file from the rawacf file (20120614.0001.00.sas.rawacf) for the saskatoon data from Reimer et al. (2018) paper and keep I getting the following error:

Traceback (most recent call last):
  File "lmfit2.py", line 648, in <module>
    fitted_records = [p.get() for p in temp]
  File "/cm/shared/apps/python/gcc/2.7.13/lib/python2.7/multiprocessing/pool.py", line 567, in get
    raise self._value
TypeError: only integer arrays with one element can be converted to an index

Any help would be brilliant!
Thanks,
Samuel

python lacking init.py and setup.py

Python version is missing some of the structure essential to a python module.