Hello!

I am working with a lot of large numpy.ndarray and posting them over a network. I want to use a LRU cache so that if I make a call with the same numpy.ndarray I will get the same answer. numpy.ndarray take up a lot of memory and are unhashable, so I would need to use id(object) to identfiy the images.

I would be happy to do the work to enable some kind of id(object) mode (or md5 etc) for caching if the feature would be accepted. This is very relevant to my use cases and I imagine would help other people working with large unhashable objects.

To be clear this is talking about using @lrudecorator(100, mode=id) or something like that to support caching of large unhashable objects. I can use the current library by interacting with the cache directly to solve my problem but would like to make it even simpler in the future. If above in unclear I will be providing code snippets later on to give more detail to my feature idea.

Let me know what you think

Hi, with 1.0.6, I could do this:

class HTTPDecoder(object):
    @lrudecorator(10000)
    def device(self, ua):
        ....
httpDecoder = HTTPDecoder()
print httpDecoder.device('a string')

With 1.0.8, I now get this error. I haven't changed anything:

File "/usr/local/lib/python2.7/dist-packages/pylru.py", line 523, in __call__
    value = self.func(*args, **kwargs)
TypeError: device() takes exactly 2 arguments (1 given)

Got error in multi-threading usage.

Found py3 has built-in lru cache. See https://hg.python.org/cpython/file/3.5/Lib/functools.py#l438

Steps to reproduce

A simple demonstration:

import pylru
from copy import deepcopy
herp = pylru.lrucache(size=1024)
derp = deepcopy(herp)

Expected result

A copy of the lrucache object, or the lvalue bound to the original

Actual behavior

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)

<...snip ...>

  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/copy.py", line 137, in deepcopy
    d = id(x)
RecursionError: maximum recursion depth exceeded while calling a Python object 

Workaround

A simple subclass of pylru.lrucache that returns a pointer to the original on deepcopy operations gets me where I'd like to be:

class LRUCache(pylru.lrucache):
    def __deepcopy__(self, memodict=None):
        return self

Errata

• Actually performing a deepcopy of a cache is madness, I don't think it is something that should be encouraged.
• However, stack traces and SIGABRT crashes are similarly bad form. The software should provide a lrucache.__deepcopy__ method that either:

1. raises a NotImplementedError, RuntimeError, or similar, OR
2. returns a pointer back to itself, with the docs explaining that's what it does, OR
3. Both, depending on how the object was initialized (i.e., add a new kwarg to lrucache.__init__)

Can you upload new release with improvements to pypi?

I really like your project. I was reading docs, trying stuff and stuck on obtaining control over cache of manager objects, particularly FunctionCacheManager.

It took me a while to understand that manager objects have cache property. My IDE didn't show it. I haven't found a mention about it in docs. Reading docs my first though was that FunctionCacheManager has nothing more than size() and clear().

I think that this a core feature of the project, and clearly writing it in docs could ease the understanding of the project capabilities.

For better security, could you please sign all upcoming git commits?

If required, I can make the case for the usefulness of signing git commits.

See https://docs.python.org/2/library/functools.html#functools.wraps for details.

The following ipython session demonstrates the issue:

In [8]: def s_plain(y):
   ...:     "square a number"
   ...:     return y * y
   ...: 

In [9]: s_plain.__doc__
Out[9]: 'square a number'

In [10]: s_plain.__name__
Out[10]: 's_plain'

In [11]: @lrudecorator(100)
def s(x):
    "square a number"
    return x * x
   ....: 

In [12]: s.__doc__

In [13]: s.__name__
Out[13]: 'wrapped'

Hi,

I am trying to use pylru to cache methods in a class. Say:

class myclass(object):
    def __init__(self,data):
        self.data = data

    @lrudecorator(100)
    def longcomputation(self):
        #do some "hard work" depending on self.data and store results in x
        print "I am working hard"
        x = self.data
        return x

    def update(self,newdata):
        self.data = newdata

Now if I do:

obj = myclass("mydata")
print obj.longcomputation()
obj.update("mychangeddata")
print obj.longcomputation()

I get

I am working hard
mydata
mydata

printed on the console, as it should be.

The question I have is if there is some way to clear the cached method so that if I do:

    def update(self,newdata):
        self.data = newdata
        # "something to clear the cache for longcomputation"

I get

I am working hard
mydata
I am working hard
mynewdata

The function cached.keys returns only partial keys, and many repetitions.

why add the _dlnode when __init__?
if the size is 100000, then I need Consume the memory of 100000 node before i use it.

Suggestion:

1. We can init it with a head, then when We add value to the table and the size is in limit, we can add node dynamic.
2. I think we need allow size = -1, when -1, we do not Eliminate or remove cache key.

thanks for your work, i will use it in https://github.com/hustcc/wrapcache/blob/master/wrapcache/database/init.py, it is a wrap cache, use it for lru memory cache.

It appears that calling cache.clear() doesn't call the eviction callback... Is this by design or an accident?

I'm working on a project where I have to expose a dictionary interface to a data store so that pylru could wrap it. It is not that much of a trouble but it would be nicer and easier to maintain if pylru allowed for generic interface to a data store (e.g., function calls)

I'm willing to submit a pull request for a change like that if you give the green light.

Hi. I want to use pylru in an application and use the callback option. However, my callback would need additional information in order to execute. I would like to hand that information into the init() function as an optional dictionary. The callback would know what to do with it.

Is this feasible or is there already a way to do this that I have missed?

If this is the wrong place to request a feature (or propose a code mod) please let me know. Thanks,

< david

Hi!
https://github.com/jlhutch/pylru/blob/master/pylru.py#L549
value = func(*args, **kwargs)
For python3.4 need check if is coroutine and add yield from
value = yield from func(*args, **kwargs)

if asyncio.iscoroutinefunction(expect_handler)

But maybe it will anyway raise error on python 2.

Could you please upload your OpenPGP / gpg public key somewhere?

related:
#30