Hello. I'm new with Walrus and Redis and I don't know how can I set timeout for Publisher/Subscriber.
My current rough solution is monkey patching for redis.Connection.init and set self.socket_timeout there. But, obviously, I don't like this solution.

Thanks in advance and let me know if I have to move this question to SO or so.

Its better to have an option to specify useStopWords True/False parameter before indexing phrases.
If useStopWords is True, option to provide custom stopwords file

Hi,
Is there any chance of adding TimeDeltaField ?

Cheers
S

Looks like the string being joined into the key is using a bytestring, which causes issues on decode.

Quick script to reproduce:

>>> from walrus import Database
>>> from walrus.graph import Graph
>>> db = Database()
>>> graph = db.graph('foo')
>>> graph.store(s=u'subject', p=u'predicate', o=u'object')
>>> query = graph.query(p=u'predicate', o=u'object')
>>> next(query)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "\site-packages\walrus\graph.py", line 146, in query
    start, end = self.keys_for_query(s, p, o)
  File "\lib\site-packages\walrus\graph.py", line 133, in keys_for_query
    return key(parts + ['']), key(parts + ['\xff'])
  File "\lib\site-packages\walrus\graph.py", line 116, in <lambda>
    key = lambda parts: '::'.join(parts)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)

Hi,
Is there any chance of something like a nestedfield ? I guess, this is sort of equivlent to a OneToOne field in Django or SQLAlchemy:

class ScheduleModel(BaseModel):
    interval = walrus.FloatField()
    start_at = walrus.DateTimeField()
    end_at = walrus.DateTimeField()

class ScheduledJobModel(BaseModel):
    name = walrus.TextField(primary_key=True)
    schedule = NestedField(ScheduleModel)
    .... other fields...

So in this case once I had an instance of ScheduledJobModel I could do something like job.schedule.interval or maybe even job.schedule__interval.

?

I'm trying to update a boolean, but when I pull the data again it returns the old information and default.

# Presence REDIS DB (no-SQL)
class Presence(walrus.Model):
    __database__ = redis_db
    channel_name = walrus.TextField(primary_key=True)
    user = walrus.UUIDField(index=True, default=uuid.uuid4())
    is_accessible = walrus.BooleanField(index=True, default=False)

>>> p = Presence.get(Presence.user == '4fd22924-3a45-479b-90f0-ee49bb89caad')
>>> p.is_accessible
False
>>> p.is_accessible = True
>>> p.is_accessible
True
>>> p.save()
>>> p = Presence.get(Presence.user == '4fd22924-3a45-479b-90f0-ee49bb89caad')
>>> **p.is_accessible**
False

PS: The problem is actually in the Boolean object type. I solved by changing the type to Integer.
is_accessible = walrus.IntegerField(index=True, default=0)

model.py uses basestring and unicode, which are gone in Python 3.

Also, there are a few issues stemming form the unicode-ness of strings in Python 3:

• model.TextField.db_value() converts to encoded strings, which would be bytes in Python 3. model.Query.get_primary_hash_key uses db_value to convert the primary key value into the Redis key suffix. So we end up with keys like user.id.b'joedoe'.
• model.Model.load() iterates over the field names, which are (unicode) strings and looks them up in raw_data, which is a dict with bytes keys, since redis.hgetall() returns all hash field names as bytes objects. Consequently all field values of the Model instance will be None.

What principial difference between walrus and https://github.com/lethain/Redis-Python-Datastructures ?

There is some moment that it is not possible to return an object and everything spoils. When I try to pull a list with all objects nothing else works. I could not detect exactly when this occurs, sometimes it looks like it was after using delete(). But I can say that it is unstable.

# Presence REDIS DB (no-SQL)
class Presence(walrus.Model):
    __database__ = redis_db
    channel_name = walrus.TextField(primary_key=True)
    user = walrus.UUIDField(index=True, default='00000000-0000-0000-0000-000000000000')
    last_seen = walrus.DateTimeField(default=datetime.datetime.now, index=True)
    is_accessible = walrus.IntegerField(index=True, default=0)
    connection_state = walrus.IntegerField(default=1, index=True)

>>> for t in Presence.all():
        print(t.user, t.is_accessible, t.connection_state)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Volumes/MacHD/Users/Git/projectX-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 789, in all
    yield cls.load(result, convert_key=False)
  File "/Volumes/MacHD/Users/Git/projectX-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 889, in load
    raise KeyError('Object not found.')
KeyError: 'Object not found.'

See that in the object list I have an object that is marked as indexable, when I do a query looking specifically for that object it also bursts the same error.

>>> for presence in Presence.all():
        print("pint user ID: {}".format(presence.user))
pint user ID: 46981222-d147-48de-aba2-e63133d2491d

>>> for t in Presence.query(Presence.user == '46981222-d147-48de-aba2-e63133d2491d'):
       print("pint user ID: {}".format(presence.user))
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Volumes/MacHD/Users/Git/projectX-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 839, in query
    yield cls.load(hash_id, convert_key=False)
  File "/Volumes/MacHD/Users/Git/projectX-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 889, in load
    raise KeyError('Object not found.')
KeyError: 'Object not found.'

or: uuid.UUID('46981222-d147-48de-aba2-e63133d2491d')):

After the error starts happening, not even the method to delete all records of that type of object does:

>>> Presence.query_delete()
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Volumes/MacHD/Users/Git/webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 857, in query_delete
    cls.load(hash_id, convert_key=False).delete()
  File "/Volumes/MacHD/Users/Git/-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 889, in load
    raise KeyError('Object not found.')
KeyError: 'Object not found.'

Currently from what I see, I can only delete a concrete instance for fetched model:

http://walrus.readthedocs.org/en/latest/models.html#creating-updating-and-deleting

It would be nice to make a delete by query, without fetching documents from the DB.

Given the following snippet:

from walrus import Database

db = Database(host='localhost', port=6379, db=0)
cache = db.cache(default_timeout=0)

@cache.cached()
def foo():
    pass # return something expensive

the result of bar is cached for 3600s. I'd expected that it is cached without TTL.

To cache without TTL, I need to call

@cache.cached(timeout=0)
def bar():
    pass # return something expensive

This was a bit unexpected since I thought default_timeout=0 will be used always if there is no other timeout specified.

Hello coleifer thank you for good library.
I am trying to use it but I failed to import walrus.graph.
When digging I also found that Lua scripts are not installed during pip or setup.py install.

'pip2 install redis-py
pip2 install redis
pip2 install --upgrade walrus
pip2 install -U --force walrus
python2.7 setup.py build
python2.7 setup.py install
python2.7 setup.py test
'
`test_sort (walrus.tests.TestZSet) ... ok

ERROR: test_basic_apis (walrus.tests.TestArray)

Traceback (most recent call last):
File "/Users/alexandermikhalev/py_code_local/walrus/walrus/tests.py", line 1057, in test_basic_apis
self.assertEquivalent(self.arr.pop(1), 'i2')
File "/Users/alexandermikhalev/py_code_local/walrus/walrus/containers.py", line 803, in pop
args=[idx])
File "/Users/alexandermikhalev/py_code_local/walrus/walrus/database.py", line 131, in run_script
return self._scripts[script_name](keys, args)
File "/usr/local/lib/python2.7/site-packages/redis/client.py", line 2699, in call
return client.evalsha(self.sha, len(keys), _args)
File "/usr/local/lib/python2.7/site-packages/redis/client.py", line 1944, in evalsha
return self.execute_command('EVALSHA', sha, numkeys, *keys_and_args)
File "/usr/local/lib/python2.7/site-packages/redis/client.py", line 573, in execute_command
return self.parse_response(connection, command_name, *_options)
File "/usr/local/lib/python2.7/site-packages/redis/client.py", line 585, in parse_response
response = connection.read_response()
File "/usr/local/lib/python2.7/site-packages/redis/connection.py", line 582, in read_response
raise response
ResponseError: Error running script (call to f_59fe925e1dd307a32fa4168556f6891e67bfad0f): @user_script:12: @user_script: 12: Lua redis() command arguments must be strings or integers

`

re-running with more debugging enabled to see why it only happens for 1 of the entries

Traceback (most recent call last):                                                                                                             File "redis_indexer.py", line 21, in <module>
    populate_index()
  File "redis_indexer.py", line 18, in populate_index
    ac.store(obj_id=product['sku'], title=product['name'])
  File "/root/open-data-set/.venv/lib/python3.6/site-packages/walrus/autocomplete.py", line 136, in store
    clean_title = ' '.join(self.tokenize_title(title))
  File "/root/open-data-set/.venv/lib/python3.6/site-packages/walrus/autocomplete.py", line 60, in tokenize_title
    phrase = re.sub('[^a-z0-9_\-\s]', '', phrase.lower())
AttributeError: 'int' object has no attribute 'lower'

danmu_limit = redis_db.rate_limit('danmu_10s_limit',limit=2,per=10)

if try danmu_limit.limit('key') four times in 10seconds, i will get the result as follows:

1,False
2,False
3,True
4,False

At the time of writing runtests.py produces

FAILED (failures=28, errors=13)

when run under Python 3.6.

Most of these errors look like

======================================================================
FAIL: test_basic_apis (walrus.tests.TestList)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vaultah/walrus/walrus/tests.py", line 890, in test_basic_apis
    self.assertList(['ix', 'i1', 'i2', 'i3'])
  File "/home/vaultah/walrus/walrus/tests.py", line 884, in assertList
    self.assertEqual(list(self.lst), expected)
AssertionError: Lists differ: [b'ix', b'i1', b'i2', b'i3'] != ['ix', 'i1', 'i2', 'i3']

First differing element 0:
b'ix'
ix

- [b'ix', b'i1', b'i2', b'i3']
?  -      -      -      -

+ ['ix', 'i1', 'i2', 'i3']

We should definitely make Walrus more Py3k friendly. Why don't we use six for that?

I want to at least try to initiate a general discussion of the collaborative efforts needed.

If the cache is empty , the flush method

def flush(self):
    """Remove all cached objects from the database."""
    return self.database.delete(*self.keys())

fails

  File "/root/python/project/test/great_test.py", line 22, in setUp
    version_properties.cache.flush()
  File "/root/.virtualenvs/project/lib/python2.7/site-packages/walrus/cache.py", line 80, in flush
    return self.database.delete(*self.keys())
  File "/root/.virtualenvs/project/lib/python2.7/site-packages/redis/client.py", line 841, in delete
    return self.execute_command('DEL', *names)
  File "/root/.virtualenvs/project/lib/python2.7/site-packages/redis/client.py", line 573, in execute_command
    return self.parse_response(connection, command_name, **options)
  File "/root/.virtualenvs/project/lib/python2.7/site-packages/redis/client.py", line 585, in parse_response
    response = connection.read_response()
  File "/root/.virtualenvs/project/lib/python2.7/site-packages/redis/connection.py", line 582, in read_response
    raise response
ResponseError: wrong number of arguments for 'del' command

This is pretty minor, however: thank for providing this awesome library

It would be cool to add support for IN queries on subclasses of ContainerField that support it. For instance,

walrus = Walrus()

class Entry(Model):
    database = walrus
    namespace = 'myapp'
    title = TextField(index=True)
    content = TextField(
        fts=True,
        stemmer=True,
        metaphone=True)
    tags = SetField(index=True)  # "index"

Currently you cannot add an index to a container field, nor do these fields support any operations. It makes sense, though, to support something like:

Entry.query(Entry.tags.contains('python'))

Using the following code I've tried to do a basic benchmark on updating instances of Models. Using the following code I get between 1400 and 1900 updates per second. I've also run benchmarks without as many function calls but see similar results.

Given the advertised Redis numbers around SETs and GETs this seems a couple of orders of magnitude too low. Perhaps I'm doing something obviously wrong?

# Standard Library
import time
from random import random, randint

# Third Party
from walrus import (Walrus,
                    Model,
                    TextField,
                    IntegerField)

class Agent(Model):

    database = None
    agent_id = IntegerField(primary_key=True)
    location = TextField()

def get_random_coordinate():

    return randint(0, 325) + random()

def get_random_coordinates():

    x = get_random_coordinate()
    y = get_random_coordinate()
    theta = get_random_coordinate()

    return str(x) + str(y) + str(theta)

def main():

    redis_host = "localhost"
    redis_port = 6379
    agent_db = Walrus(host=redis_host, port=redis_port, db=0)

    # Provide databases to our models
    Agent.database = agent_db

    # Create 100 agents
    for i in range(1, 101):
        agent = Agent.create(agent_id=i,
                             location=get_random_coordinates())

    # Permute their locations as quickly as possible
    while True:
        start_time = time.time()
        counter = 0
        while time.time() < (start_time + 1):
            aid = randint(1, 100)
            agent = Agent.load(aid)
            agent.location = get_random_coordinates()
            agent.save()
            counter += 1

        print counter

if __name__ == '__main__':
    main()

I'm looking to have a way to have a lock scoped to a single instance of a Model. It seems like a LockField construct might be the best way to handle something like that. But I just thought I'd get your opinion.

If you think it would be best to keep separate as it's own library, that's no problem. I just wanted to make sure I reached out before starting to explore the implementation.

Hello Charles,

I did evaluation of several object mappers for Redis - walrus, rom, redisco, python-stdnet.

Walrus fit best for my needs except one issue below.

Let's save two records:

class User(Model):
    #database = db
    login = TextField(primary_key=True)
    email = TextField(index=True)
    password = TextField()
    roles = SetField()
    created = DateTimeField(default=datetime.utcnow)

@fixture
def db(redis_real):
    db = Database(host='localhost', port=16380, db=0)
    User.database = db

def test_walrus(db):
    User.create(login='alex.smith', email='[email protected]')
    User.create(login='anna.smith', email='[email protected]')
    ...

After that done we will have following data in redis for walrus==0.2.0

> KEYS *
1) "user:container.roles.user:id.alex.smith"
2) "user:[email protected]"
3) "user:all"
4) "user:[email protected]"
5) "user:login.absolute.alex.smith"
6) "user:id.alex.smith"
7) "user:id.anna.smith"
8) "user:login.absolute.anna.smith"

On my project we have 30K+ online users and it would be much easier to explore data in Redis using Web UI if we would have following key convention:

> KEYS *
1) "user:container.roles.user:id:alex.smith"
2) "user:email.absolute:[email protected]"
3) "user:all"
4) "user:email.absolute:[email protected]"
5) "user:login.absolute:alex.smith"
6) "user:id:alex.smith"
7) "user:id:anna.smith"
8) "user:login.absolute:anna.smith"

We are using Redis Commander (and Redis Admin UI in past) which displays flat key-value storage as a tree.

With current convention if I try to expand values collapsed next to "user:" I will see about 90K entries:

• 1 for user:container.roles.user:
• 1 for user:all
• 30K for user:id.{PRIMARY_KEY_VALUE}
• 30K for user:login.absolute.{INDEX_VALUE}
• 30K for user:email.absolute.{INDEX_VALUE}

Is it possible somehow customize current convention, so they have more hierarchical view?

In such case I would see following prefixes (with counters next to them):

• 1 for user:container.roles.user:
• 1 for user:all
• 1 for user: id:
• 1 for user:login.absolute:
• 1 for user:email.absolute:

http://joeferner.github.io/redis-commander/
https://github.com/ServiceStackV3/mythz_blog/blob/master/pages/381.md

And big "thank you" for your effort on this project. It's awesome.

If I search for a particular integer field, I get results. If I search for a Boolean field, I also get it. But if I try to join the two queries the return is a mistake.

#Presence REDIS DB (no-SQL)
db = walrus.Database(host='localhost', port=6379, db=0)

class Presence(walrus.Model):
   __database__ = db
   channel_name = walrus.TextField(primary_key=True)
   is_accessible = walrus.BooleanField(index=True, default=False)
   state = walrus.IntegerField(default=MatchState.CLOSED.value, index=True)

>>> Presence.query(Presence.is_accessible == True).__next__()
<Presence: defaultuaGTz>

>>> Presence.query(Presence.state == 4).__next__()
<Presence: defaultuaGTz>

>>> Presence.query(Presence.state == 4 & Presence.is_accessible == True).__next__()
Traceback (most recent call last):
 File "<input>", line 1, in <module>
 File "/Volumes/MacHD/Users/Git/projectXYZ/.env/lib/python3.6/site-packages/walrus/models.py", line 819, in query
   result = executor.execute(expression)
 File "/Volumes/MacHD/Users/Git/projectXYZ/.env/lib/python3.6/site-packages/walrus/query.py", line 240, in execute
   return self._mapping[op](expression.lhs, expression.rhs)
 File "/Volumes/MacHD/Users/Git/projectXYZ/.env/lib/python3.6/site-packages/walrus/query.py", line 243, in execute_eq
   index = lhs.get_index(OP_EQ)
AttributeError: 'Expression' object has no attribute 'get_index'

Hello,
I'm using your module for my apps and it's awesome. Unfortunately I can't install latest version :(

Error when installing from source:

Traceback (most recent call last):
  File "setup.py", line 17, in <module>
    version=__import__('walrus').__version__,
  File "/root/walrus/walrus/__init__.py", line 34, in <module>
    from walrus.database import Database
  File "/root/walrus/walrus/database.py", line 42, in <module>
    class Database(Redis):
TypeError: Error when calling the metaclass bases
    cannot create 'NoneType' instances

Error when installing from pip:

Collecting walrus==0.2.0 (from -r common/requirements.txt (line 42))
  Downloading walrus-0.2.0.tar.gz (42kB)
    100% |################################| 45kB 502kB/s
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "/tmp/pip-build-Q3v_pi/walrus/setup.py", line 17, in <module>
        version=__import__('walrus').__version__,
      File "walrus/__init__.py", line 34, in <module>
        from walrus.database import Database
      File "walrus/database.py", line 42, in <module>
        class Database(Redis):
    TypeError: Error when calling the metaclass bases
        cannot create 'NoneType' instances
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

      File "<string>", line 20, in <module>

      File "/tmp/pip-build-Q3v_pi/walrus/setup.py", line 17, in <module>

        version=__import__('walrus').__version__,

      File "walrus/__init__.py", line 34, in <module>

        from walrus.database import Database

      File "walrus/database.py", line 42, in <module>

        class Database(Redis):

    TypeError: Error when calling the metaclass bases

        cannot create 'NoneType' instances

Tested in:

Mac OS X 10.10.2
Debian Testing
Ubuntu 14.04.1

EDIT:

After installing hiredis everything works like a charm. 😉

I need to return all inactive users, searching for more than 60 seconds without updating the information. But the return is always empty.

I've tried to put a simpler example by looking for the smaller (or larger) field than the current date. Both queries (greater equal or less equal) return empty.

>>> for user in User.query(User.last_seen <= datetime.datetime.now()):
...     print(user.last_seen)
...     (return NULL)

Listing all the records to ensure there is information:

>>> for user in User.all():
...     print(user.last_seen)
...     
2017-05-04 21:58:12.117880
2017-05-04 21:58:12.372056
2017-05-04 18:59:11.817694
2017-0....+50

Hi,

I'm having a hard time figuring out whether redis-py be a hard dependency or not. What threw me off guard is (database.py):

try:
    from redis import Redis
except ImportError:
    Redis = None

My problem occurs when installing walrus in a brand new env. Quoting the doc:

>>> git clone https://github.com/coleifer/walrus.git
>>> cd walrus
>>> python setup.py install
Traceback (most recent call last):
  File "setup.py", line 17, in <module>
    version=__import__('walrus').__version__,
  File ".../walrus/walrus/__init__.py", line 34, in <module>
    from walrus.database import Database
  File ".../walrus/walrus/database.py", line 42, in <module>
    class Database(Redis):
TypeError: Error when calling the metaclass bases
    cannot create 'NoneType' instances

When the setup conf looks for the version in the init.py, Database is imported and since redis-py isn't installed (yet), the Database class definition fails.

So, here come my two cents on suggestions:

• specify the version in the setup.py, thus avoiding any conflict if redis-py and walrus are installed at the same time
• on ImportError, set Redis to object rather than None
• if Redis is mandatory, why not putting install_requires=["redis"] in the setup conf?
• if the version should stay in the init.py, I guess the Database import can't.

I'd be happy to PR if you let me know what you think!

And, just for the record, I've been a happy user of redis-completion for a few years, and I am looking forward to getting my hands dirty with all the new stuff you concocted for us.

Off topic: since the autocomplete appears to rely on HSCAN, you might want to mention that it's only compatible with Redis >= 2.8.0 (I couldn't set it up on 2.4.14, which is Debian 7 stable version).

Cheers

I would like to know if I can select a random object when doing a query? And also how do I lock a query so I do not have records conflict?

In django models I used the "select_for_update" function to ensure a single query, and an "order_by ('?') For randomization.

queryset = User.objects.select_for_update(nowait=True).filter(state=1).order_by('?')[:1]

How can I prevent an error when I try to query an object that is being saved at the same time? I noticed that at the moment the object is being saved it is not visible to the GET method, then there is error, since I try to get the object and it is not found for a few seconds.

Have a default argument for name seems like an accident(al namespace collision) waiting to happen. Make it compulsory would at least force you to think about it. Its a design decision, not a bug, but I though I would flag it, I know this is an issue that's come up at work.

Updating an attribute of an object does not affect the results of a query for that object. Here is a minimal example demonstrating the unexpected behavior.

python shell

from walrus import *
db = Database(host='127.0.0.1', port=6379, db=0)
class Test(Model):
    database     = db
    myattr = BooleanField(index=True)

# Create an object
test = Test(myattr=True)
# Query for it
list(Test.query(Test.myattr==True)) # [<Test: 1>]
# Update the attribute
test.myattr=False
test.save()

# Query again - UNEXPECTED RESULT
list(Test.query(Test.myattr==True)) # [<Test: 1>]
test.myattr # False

Is it feasible to provide a bust() method to a property used by the cached_property decorator?

When the model is loads from redis, the model field will be 'None' str instead of the python None type. That will cause some trouble when using the value by expecting the Nonetype value

I can't find a way to prevent the None value from saving into redis.

See: https://pypi.python.org/pypi/redislite

Models and cache in one package! I have used it in a couple of Python 2.7 projects.

Unfortunately, my next project has to be Python 3, because the other 2 pieces are only Py3 compatible :(

So I was wondering what the current state of the Python 3 compatibility was (what works, what doesn't), or if I should look at a different package for this one.

I saw ticket #8

Thanks again for releasing this, it made my life a lot easier in the past!

Geoff

I have a model:

class WithListModel(Model):
    name = TextField(index=True)
    values = ListField()

When I do

model.delete()

the keys associated with values, i.e. keys of form withlistmodel:container.values.withlistmodel:id.1 are not deleted.

Hello sir, thanks for the walrus. Truely helpful.
I'm a staruper. and I get a question. Please help.
here it is.
in runtime.py i make things like
runtimedb = Database(host='localhost', port=6379, db=0)
runtimedb['walrus'] = 'tusk'
print runtimedb
print runtimedb['walrus']
h = runtimedb.Hash('flowinfo')
h.update(name='testinfo', testtime='0')

and in another app.py I need get that database.(for test, just print h['testtime'])
how do I make it ?
I tried import everthing, not work.
I guss, maybe I should connect to the database.
how do I make it ?
Ps: I run this script in tornado with django.

fixed in #26

If you have the time, would you mind providing a quick example on how to get/set field values from within a subclass of the Model Class? I cannot seem to figure it out. Maybe I am just overthinking it 😕 Thanks in advance for any advice you can offer!

Cheers!

It seems that the fix from #25 broke the save() method.

Here's what I do:

class WithListModel(Model):
    name = TextField(index=True)
    values = ListField()

c = WithListModel.create(name='x')
c.values.append('x')
list(c.values)  # ['x']
c.save()
list(c.values)  # []

This is because of this part: https://github.com/coleifer/walrus/blob/master/walrus/models.py#L931 -- deleting now also deletes associated list...

Hi,

I have a question regarding the Model class that walrus offers. I need to keep the state of a small set of python complex data structures for a small period of time. I have thought to use
walrus as a nice way to organize those data for easier manipulation when needed (each python object spans into 3 walrus models) and persist them in redis. Since I only care for those objects for
a specific period of time, I was wondering if there is any way to specify an expiration time for every Model instance? I have tried using cache as database but it doesn't work.
Another idea I have is to handle by specifying a maxmemory and maxmemory-policy in redis instance. Finally another idea is to have a cron like job to delete least recent model instances, which to be honest I want to avoid.

Do you have any suggestion on how to handle this ?

Thank you in advance.

Hi,

Is it possible to create a method so that it is possible to return a random object after a query?

Today there is already the method to get the first one in the queue:
user = User.query(filter1=="1").__next__()

It would be interesting, if possible, to create something like:
user = User.query(filter1=="1").__random__()

Hi. Excuse my English, I'm with the translator.

I need to store in memory the UUID (pk) of the relational database user. This needs to be used to locate the user in the redis inside querys and also to "open" the relational database user.

I was wondering if I use walrus.TextField() or walrus.UUIDField ).

Using UUIDField I'm getting an error when I try to retrieve records.

>>> import walrus
>>> import uuid
>>> db = walrus.Database(host='localhost', port=6379, db=0)
>>> class X(walrus.Model):
        database = db
        name = walrus.TextField(primary_key=True)
        oi = walrus.UUIDField()


>>> X.create(name='Charlie', oi=uuid.UUID('2ad3afaf-cd21-4842-b51d-c8e0bc80860d'))
<X: Charlie>
>>> test = X.load('Charlie')
**Traceback (most recent call last):**
  File "<input>", line 1, in <module>
  File "/Volumes/MacHD/Users/Git/projectXYZ/.env/lib/python3.6/site-packages/walrus/models.py", line 887, in load
    data[name] = field.python_value(raw_data[encode(name)])
  File "/Volumes/MacHD/Users/Git/projectXYZ/.env/lib/python3.6/site-packages/walrus/models.py", line 259, in python_value
    return uuid.UUID(value)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/uuid.py", line 137, in __init__
    hex = hex.replace('urn:', '').replace('uuid:', '')
TypeError: a bytes-like object is required, not 'str'

Some details:
I'm in Python 3.6
There will actually be 3 UUID fields, none of them will be primary_key and everyone will need to include in query.

Is there an easy way to return all results if no partial phrase is entered? Something like

>>> phrases = ['aa bb', 'aa cc', 'bb cc', 'bb aa cc', 'cc aa bb']
>>> for phrase in phrases:
...     ac.store(phrase)
>>> ac.search('')
['aa bb', 'aa cc', 'bb cc', 'bb aa cc', 'cc aa bb']

If we create an instance of a Model that doesn't specify a field (and that field is not part of a secondary index), should the behavior be for that attribute to return None on access?

Getting KeyError in this case is a bit unexpected.

>>> from walrus import *
>>> db = Database()
>>> class Foo(Model):
...     name = TextField(primary_key=True)
...     keyerror = TextField()
...
test>>> test = Foo(name='test')
>>> test.name
'test'
>>> test.keyerror
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "...\lib\site-packages\walrus\models.py", line 123, in __get__
    return instance._data[self.name]
KeyError: 'keyerror'

Hello,

I think it will be more "Pythonic" if timeout parameter could be a datetime.timedelta
https://walrus.readthedocs.org/en/latest/cache.html#basic-usage

Kind regards

Hello,

what do you think about integrating your library with aioredis, would it be possible ?

When we update a data struct, a coroutine is scheduled in background and will update redis as soon as possible. One caveat would be to cancel previous updates in redis if we updates the variable multiple time...

I have a class

   class Post(BaseModel):
        title = TextField(index=True)
        slug = TextField()
        body = TextField()
        created = DateTimeField()
        edited = DateTimeField()
        tags = SetField()
       draft     = IntegerField()

When I run

     >>> [i.draft for i in Post.all()]
     [0, 0, 1]

The results are as expected

However

     >>> list(Post.query(Post.draft==0))
      []

There seems to be a bug somewhere, or am I missing someting

Using Walrus' Models feature, I'm getting a Segmentation fault when I set order_by on a query and the query returns no results. Example:

>>> q = User.query(User.username=='nobody',  order_by=User.first_name)
>>> for user in q: do_something()
Segmentation fault (core dumped)

...and then the whole Python REPL exits!

This only happens if the query does not return any matches. So if there is a user with username nobody, then it returns the one result, as expected.

>>> u = User(username='nobody')
>>> u.save()
>>> q = User.query(User.username=='nobody',  order_by=User.first_name)
>>> for user in q:  print user.username # prints 'nobody' and returns to prompt

If I run a query without order_by and there are no results, that works fine, too.

>>> q = User.query(User.username=='nobody',  order_by=User.first_name)
>>> for user in q:  print user.username # executes and returns to prompt, as expected

Note that I'm using RLite, not Redis, as the backend. I've pasted a complete example below:

#! /usr/bin/python2
import walrus

from walrus.tusks.rlite import WalrusLite
db = WalrusLite(':memory:')

class User(walrus.Model):
    database = db
    username = walrus.TextField(primary_key=True)
    password = walrus.TextField()
    is_active = walrus.BooleanField(default=True)
    first_name = walrus.TextField(index=True)
    last_name = walrus.TextField(index=True)
    email = walrus.TextField()
    addresses = walrus.HashField()

q = User.query(User.username=='nobody', order_by=User.first_name)

# Crashes somewhere here:
for user in q:
    print user.username

Any ideas? I don't have Redis, so maybe somebody who does could check that it's not an RLite-specific problem.

Thanks for the fantastic work! Quick question about autocomplete: the docs say: "results are scored by the proximity of the match to the front of the string, then alphabetically." Trying the following, however:

database = Database()
ac = database.autocomplete()
ac.store("aa bb cc")
ac.store("tt cc")
ac.search("cc")

gives:

[u'aa bb cc', u'tt cc']

I was wondering if this was intentional--the behavior described in the docs seems preferable.

First of all - brilliant work. Walrus is one of my favourites tools right now:)

It's not an issue but rather a question / suggestion. Maybe it'd be nice to chave a CHANGELOG file, as I can see walrus is under active development and I'm really interested in how it goes. What do you think ?

I had a need to store JSON encoded data in a Walrus Hash, so I hand coded a class to do that, but I was wondering if there would be more interest for something like this.

Example:

>>> hash = redisconnection.Hash(codec=json)
>>> hash["hello"] = dict(place="world", greeting="hey")
# Dict is stored as a JSON
>>> hash["hello"]
{"place": "world", "greeting": "hey"}
>>>

So my question:
Is there any interest for a PR that would implement that for all containers?

The codec would be any module or object that implements both loads and dumps function/method.

Hi,

I love this project, thanks!

Hash is dict-like, but it could be more dict-like.

Is there a design decision preventing the implementation of a dict.get analogue for Hash? The implementation could be as simple as

def get(self, key, fallback=None):
    return self.as_dict().get(key, fallback)