timkpaine / temporal-cache Goto Github PK

View Code? Open in Web Editor NEW

3.0 3.0 4.0 135 KB

Time-based cache invalidation

License: Apache License 2.0

Makefile 2.39% Python 97.61%

python python3 cache caching finance

temporal-cache's Introduction

temporal-cache

Time-based cache invalidation

Install

From pip

pip install temporal-cache

Or from source

python setup.py install

Why?

I needed something that would automagically refresh at 4:00pm when markets close.

    @expire(hour=16)
    def fetchFinancialData():

Interval Cache

The interval cache expires every time interval since its first use

    @interval(seconds=5, minutes=2)
    def myfoo():
        '''myfoo's lru_cache will expire 2 minutes, 5 seconds after last use'''

Expire Cache

The expire cache expires on the time given, in scheduler/cron style.

    @expire(second=5, minute=2)
    def myfoo():
        '''myfoo's lru_cache will expire on the second minute, fifth second of every hour, every day, etc'''

Caveats

Python hashing symantics persist. Dicts will be frozen, lists will be converted to tuples. Users are advised to pre-freeze to avoid issues.

Development

See CONTRIBUTING.md for guidelines.

License

This software is licensed under the Apache 2.0 license. See the LICENSE and AUTHORS files for details.

temporal-cache's People

Contributors

Stargazers

Watchers

Forkers

majacq fermi37 blshao84 tehcoderer

temporal-cache's Issues

We may want to expand this so that the interval cache expires x duration after last call with a specific key. This is a pretty straightforward feature to implement, just replace foo.cache_clear with the appropriate deleter. But since functools.lru_cache does not support this, we would need to switch to e.g. ring or roll our own.

Port to azurepipelines

Add periodic CI

migrate to nose2/tox/pytest

Add global disable for testing, etc

dist with twine

persistent_lru_cache generates a new key for the same arguments between runtimes when there's more than 1 argument

Describe the bug
The cache persistence only seems to work correctly when there is a single argument in the function call. The cached result is not retrieved for multiple arguments or any keyword arguments in subsequent runtimes (although it does work in the same runtime).

To Reproduce
Make a scatch.py:

    from temporalcache.persistent_lru_cache import persistent_lru_cache

    @persistent_lru_cache('test.cache')
    def foo(*args, **kwargs):
        print(f'running {args, kwargs}')
        return f'{args, kwargs}

Run scratch.py twice.

Expected behavior
The first time scratch.py is run the expected behavior is seen:

running (('1',), {})
(('1',), {})
(('1',), {})
running (('1', '2'), {})
(('1', '2'), {})
(('1', '2'), {})
running ((), {'kw': 'k'})
((), {'kw': 'k'})
((), {'kw': 'k'})

The second time, I would expect to see:

(('1',), {})
(('1',), {})
(('1', '2'), {})
(('1', '2'), {})
((), {'kw': 'k'})
((), {'kw': 'k'})

Instead, the second (and future runs) result in:

(('1',), {})
(('1',), {})
running (('1', '2'), {})
(('1', '2'), {})
(('1', '2'), {})
running ((), {'kw': 'k'})
((), {'kw': 'k'})
((), {'kw': 'k'})

Desktop (please complete the following information):

OS: Windows 10
Version: temporal-cache 0.1.4

Additional context
I believe the issue is that for multiple arguments or keyword arguments (where a kwd_mark object is added as an additional argument) the key is generated from _HashedSeq(key) instead of just key[0].
The hash of _HashedSeq uses hash(tup), but the python built-in hash is only deterministic in the same runtime, not between runtimes.

I think the fix probably requires the use of something like hashlib.sha256(), but then that only takes bytes, and I'm not sure what the implications of that may be. Maybe some combination with json.dumps()?

https://requests-cache.readthedocs.io/en/latest/