sciris / sciris Goto Github PK

Fast, flexible tools to simplify scientific Python

License: MIT License

Python 99.84% Shell 0.16%

computational-science containers data-science matplotlib numpy parallelization plotting python scientific-computing utilities

sciris's Introduction

Welcome to Sciris

What is Sciris?

Sciris is a library of tools that can help make writing scientific Python code easier and more pleasant. Built on top of NumPy and Matplotlib, Sciris provides functions covering a wide range of common math, file I/O, and plotting operations. This means you can get more done with less code, and spend less time looking things up on Stack Overflow. It was originally written to help epidemiologists and neuroscientists focus on doing science, rather than on writing code, but Sciris is applicable across scientific domains (and some nonscientific ones too).

For more information, see the full documentation, the paper, or GitHub.

If you have questions, feature suggestions, or would like some help getting started, please reach out to us at [email protected] or open an issue.

Installation

Using pip: pip install sciris

Using conda: conda install -c conda-forge sciris

Requires Python >= 3.7.

Tests

Sciris comes with an automated test suite covering all functions. You almost certainly don't need to run these, but if you want to, go to the tests folder and run pytest. See the readme in that folder for more information.

sciris's People

Contributors

Stargazers

Watchers

Forkers

thekerrlab cwiswell-idm chjones-idm phbt thecapitalistcycle appsplash99 kjappelbaum theelx tacaswell roshan2004 vladh kelvinburke

sciris's Issues

Write figure annotation GUI

Possibly better as a separate project, but noting here for posterity!

Use PySimpleGUI to interactively insert text, arrows, etc. into a Matplotlib plot

Add checkexists kwarg to makefilepath

Default None, options 1 to ensure it exists, 0 to ensure it doesn't

Add data_thief()

See process_figure_data.py

Change default json indent

Maximize on Macs

Warning: maximizing the figure failed: ‘FigureManagerMac’

Allow readdate() to handle lists

Allow float for uuid()

Fails with float, e.g. n=10e3

Update documentation

https://sciris.readthedocs.io/en/latest/

Add toctic() function

Add general-purpose dict-to-obj converter

def asobj(obj):
    '''
    Convert any object for which you would normally do a['b'] to one where you
    can do a.b.
    '''

    objtype = type(obj)

    class objobj(objtype):
        def __getattribute__(self, attr):
            try: # First, try to get the attribute as an attribute
                output = objtype.__getattribute__(self, attr)
                return output
            except Exception as E: # If that fails, try to get it as a dict item
                try:
                    output = objtype.__getitem__(self, attr)
                    return output
                except: # If that fails, raise the original exception
                    raise E

        def __setattr__(self, name, value):
            ''' Set key in dict, not attribute! '''

            try:
                objtype.__getattribute__(self, name) # Try retrieving this as an attribute, expect AttributeError...
            except AttributeError:
                return objtype.__setitem__(self, name, value) # If so, simply return

            # Otherwise, raise an exception
            errormsg = '"%s" exists as an attribute, so cannot be set as key; use setattribute() instead' % name
            raise Exception(errormsg)

            return None

        def setattribute(self, name, value):
            ''' Set attribute if truly desired '''
            return objtype.__setattr__(self, name, value)

    return objobj(obj)

Remove six dependency

Not required now that Python2 is not supported

Implement Turbo colormap

https://gist.github.com/mikhailov-work/ee72ba4191942acecc03fe6da94fc73f

Add to conda

https://conda-forge.org/docs/maintainer/adding_pkgs.html

Add helpful error message for freeze_support

Save with nothing supplied should be exception

saveobj(), savejson() with nothing to save should raise an exception

Handle PNG metadata

Add randround()

For arrays, handle floats and lists as well:

x = np.random.random(10)*10
r  = np.floor(x+np.random.random(x.size))

Update getfilelist()

sc.getfilelist() with nopath=True has a leading character. Use os.path.basename() instead.

Potentially port Atomica utilities

parent_dir()
nested_loop()
fast_gitinfo()
parallel_progress()
datetime_to_year() (maybe datetime_to() and then have options for year, day, etc. with an optional 'since' arg?)

Improve checkmem()

Currently can only handle iterables with descend>1, not objects. See sc.prepr() for better parsing of objects

Add memory profiler

Move functions to Sciris

randround()
get_rows_cols()

Fix prepr() for large objects

Currently, sc_utils.prepr() parses the entire object and then trims the output if it's too long. This is extremely slow on large objects. It should have a limit on how much of the object it parses.

Limit recursion in prepr

By default don't allow more than 2 levels of recursion

Issue reading date formats

sc.readdate("Tue 2020-09-29 15:48:05", dateformat='%a %Y-%m-%d %T')

prepr() skips properties

prepr() gets methods and attributes, but misses properties (and potentially other things?)

Lock pickle protocol to 4

Fix path in thisdir()

Currently can't prepend path, which might be useful (or maybe not actually since it gives an absolute path...)

Make Failed, Empty public

Missing from __all__

Allow programmatic setting of missing modules in sc.loadobj()

Automate this:

sc.loadobj('my_old_pickle.obj', remapping={'fp_data.DHS.calobj.CalObj':fp_utils.calobj.CalObj})

e.g. via

def load(*args, **kwargs):
    ''' Tiny alias to sc.loadobj() to load saved calendar objects '''
    import sys
    import types
    import fp_data
    # First, add any placeholder modules that have been subsequently removed
    fp_data.DHS = types.ModuleType('DHS') 
    fp_data.DHS.calobj = types.ModuleType('calobj') 
    sys.modules['fp_data.DHS'] = fp_data.DHS
    sys.modules['fp_data.DHS.calobj'] = fp_data.DHS.calobj
    fp_data.DHS.calobj.CalObj = CalObj
    cal = sc.loadobj(*args, **kwargs)
    assert isinstance(cal, CalObj)
    return cal

Add default odict

a = sc.odict().default(list)

equivalent to

a = defaultdict(list)

Add sc.cat()

Like np.concatenate(), but takes anything and returns the same type, e.g.

arr = sc.cat(np.array([1,2,3]), [4,5], 6)

Consider also adding intersect(), setdiff(), and union() as more flexible versions of their numpy counterparts

Add isarray(), isdate() functions

Fix warnings

/home/cliffk/unsw/sciris/sciris/sc_fileio.py:202
  /home/cliffk/unsw/sciris/sciris/sc_fileio.py:202: DeprecationWarning: invalid escape sequence \!
    filtername = re.sub('[\!\?\"\'<>]', '', rawfilename) # Erase certain characters we don't want at all: !, ?, ", ', <, >

/home/cliffk/unsw/sciris/sciris/sc_fileio.py:203
  /home/cliffk/unsw/sciris/sciris/sc_fileio.py:203: DeprecationWarning: invalid escape sequence \*
    filtername = re.sub('[:/\\\*\|,]', '_', filtername) # Change certain characters that might be being used as separators from what they were to underscores: space, :, /, \, *, |, comma

Fix KeyError repr

class KeyNotFoundError(KeyError):
    ''' A tiny class to fix repr for KeyErrors '''

    def __str__(self):
        return Exception.__str__(self)

a = 3.0
a = sc.setdefault(a, type=int) # returns 3
b = None
b = sc.setdefault(b, 3, null=None) # returns 3
c = {'a':4}
c = sc.setdefault(c, {'b':5}, merge=True) # returns {'a':4, 'b':5}

Add alternate checkmem() function

def checkmem(unit='mb', fmt='0.2f', start=0, to_string=True):
    ''' For use with logger, check current memory usage '''
    process = psutil.Process(os.getpid())
    mapping = {'b':1, 'kb':1e3, 'mb':1e6, 'gb':1e9}
    try:
        factor = mapping[unit.lower()]
    except KeyError:
        raise sc.KeyNotFoundError(f'Unit {unit} not found')
    mem_use = process.memory_info().rss/factor - start
    if to_string:
        output = f'{mem_use:{fmt}} {unit.upper()}'
    else:
        output = mem_use
    return output

Add maximize()

mng = pl.get_current_fig_manager()
mng.window.showMaximized()

Add pandas serialization

The jsonify() method should check for a to_json() method, which will work for pandas dataframes and many other things.

Add OrangeBlue to Sciris

Code, thanks to @pselvaraj87

bottom = cm.get_cmap('Oranges', 128)
top = cm.get_cmap('Blues_r', 128)
newcolors = np.vstack((top(np.linspace(0, 1, 128)),
                       bottom(np.linspace(0, 1, 128))))
newcmp = ListedColormap(newcolors, name='OrangeBlue')
colormaps = [newcmp]
colormap_labels = ['OrangeBlue']