The Attribute field doc seems to contradict itself. For large bodies of text should indexing be turned off by setting indexed=False or should it be turned on by setting indexed=True?

Stores unicode strings. If used for large bodies of text,
turn indexing of this field off by setting indexed=True.

class Article(models.Model):
       title = models.Attribute()
       tags = models.ListField(unicode)

 new = Article(title='what ever')
 new.tags = [ u'Niña', u'Niñb' ]
 new.save()

While when load：

Article.objects.all()

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 0: ordinal not in range(128)

In redis-cli:

redis 127.0.0.1:6379> GET "Article:tags:5omv5reh"
(error) ERR Operation against a key holding the wrong kind of value

A few comments -

1. Not clear what is the full list of supported meta options. I've seen "key", "indices" and "db". This should be documented
2. "db" meta option does not work. I was able to use different databases by directly overriding db property methods of my model classes, but clearly it is just a hack
3. "key" property that allows renaming Redis key bases is great but it would be helpful to also be able to rename specific attribute fields. It is a waste to force long descriptive names to be repeated over and over in Redis hashes. Best to support renaming them into short aliases

Or facilitate json serialization in some other way, right now doing a naive json.dumps(obj.attributes_dict) missed the object id which is kind of important.

if I use redisco to store a model's instance
How can I expire the object in 15 minunts> automatically????

Is is possible (and useful) to implement a count feature in the modelset?

E.g.

from redisco import models
class Foo(models.Model):
... name = models.Attribute()
...
Foo(name="toto").save()
True
Foo(name="toto").save()
True
Foo.objects.count()
2

I forked this repo and I am interested in building this feature.

In Django get_or_create returns p, created this is useful as you can do different operations in the case a record was created.

Any chance a future version of redisco could do this ?

How do I filter by ids?

I tried filtering by Message.filter(id__gt=0) and its giving me <broken repr (AttributeNotIndexed: Attribute id__gt is not indexed in Message class.>.

using the current machine time to specify the mutex lock timeout is probably a bad idea since the internal time might drift slightly from machine to machine (they should all be using NTP, but I'd rather not rely on that).

https://github.com/kiddouk/redisco/blob/master/redisco/models/base.py#L763

Rather than using an absolute time, why not just set the key with a value of 1 and pipeline in the expire timeout of 1 second. Let redis determine when the key will expire by using it relative to the server time.

http://redis.io/commands/setex

pipeline these commands:
SET 1
SETEX 1

Then you only ever have to see if the lock exists, and don't care about the value.

1048 for i in self.db.zrange(self.key, 0, -1):
1049 print i, self.db.zscore(self.key, i)
1050 print self.key, min, max
1051 return self.db.zrangebyscore(self.key, min, max, **kwargs)

The key and the score:
1471795200.000000 1.0
1471536000.000000 2.0
The min and max for float datetime:
BBSRecord:post_date 1471708800.0 1472313599.0

There is a small typo in models.rst.
Should read supports rather than upports.

In the ge method of SortedSet in containers.py, the score for the min should not have an open parenthesis. With the open parenthesis it becomes an open interval, as described in the documentation for redis: http://redis.io/commands/zrangebyscore.

Is there way to have a composite unique key ?

Can we get an updated version of Redisco up on PyPi?

Hi,
How do I run rediscos tests, on python 3.5 ?

I tried python setup.py test but that didn't work ?

I want to add a feature, but need to make sure I don't break anything ?

Cheers
S

Is there a way to add an expire time to a model or entry in v.0.1? Any suggestions on how one might do it?

Given the following model:

class Thing(models.Model):
    created = models.DateTimeField(auto_now_add=True)

and given the following usecase:

Thing.objects.zfilter(created__lt=datetime.now())

...an IndexError is raised.

Save an object:

Thing().save()

Thing.objects.zfilter(created__lt=datetime.now())

No error.

This is what I meant by self-reference.

class Employee(models.model):
    manager = models.ForeignKey('Employee')

This is how Django implements recursive relationships.

To create a recursive relationship – an object that has a many-to-one relationship with itself – use models.ForeignKey('self', on_delete=models.CASCADE).

Is this available to use?

any example is appreciated.

So if I do something like this:

dog = Dog(name="Daryl",
breed="Construction worker",
description="He told his wife that he liked her sister's cooking better.",
current_doghouse=(couch),
previous_doghouses=[silent_treatment, snide_comments])

I get this:

Traceback (most recent call last):
File "redisco_reference_test.py", line 37, in
dog.save()
File "/usr/local/lib/python2.7/dist-packages/redisco/models/base.py", line 324, in save
self._write(_new)
File "/usr/local/lib/python2.7/dist-packages/redisco/models/base.py", line 581, in _write
pipeline.execute()
File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 1987, in execute
return execute(conn, stack, raise_on_error)
File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 1911, in _execute_transaction
self.raise_first_error(response)
File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 1946, in raise_first_error
raise r
redis.exceptions.ResponseError: ERR wrong number of arguments for 'rpush' command

However, appending an element to that list (like exampled within Redisco's tests),

daryl.previous_doghouses.append([snide_comments, cold_shoulder])
print daryl.previous_doghouses

[<Doghouse:2 {'location': u'the cold side of the bed', 'id': '2'}>, [<Doghouse:7 {'location': 'kitchen table', 'id': '7'}>, <Doghouse:8 {'location': 'everywhere', 'id': '8'}>]]

Works perfectly!

I have multiple dbs to store data. If I specify the db 1 in model class as below:
class Param(models.Model):
Name = models.Attribute(required=True, unique=True)
class Meta:
indices = ['Name']
db = redis.Redis(host="127.0.0.1", port=6379, db='1')

The object creation is successful, the object will be created and stored in db 1. But with Param.objects.filter(Name='xxxx'), the resultset is empty.
If I don't have this db specification in model class meta, the creation and query both work.

Anyone aware of this issue?

Hi,

Is it possible to combine filter and zfilter, for example:

Person.objects.filter(zipcode="123").zfilter(created_at__in=(datetime(2010, 4, 20, 5, 2, 0), datetime(2010, 5, 1)))

Or maybe there is another way to do it?

Having the following models:

class Video(Model):
    title = Attribute()

class Playlist(Model):
    now_playing = ReferenceField(Video)

I can save the playlist model without having to assign the now_playing attribute, but if I assign None to the attribute, I get

line 346, in __set__
    setattr(instance, self.attname, value.id)
AttributeError: 'NoneType' object has no attribute 'id'

I'm currently using redisco 0.1.4.

I think I know where the problem is, should i send a pull request?

Commented on the older issue, could not re-open it -
#4 (comment)

I went through your code, and found that when auto_now_add is set to True you store datetime.now() .
This shall fail in timezone oriented app. Though mostly applications are hosted on cloud servers and they are configured for utc timezone. This library won't fail when server is in utc timezone. This library will definitely fail in development environment if development machine is not set to use utc timezone.

Though your code can store timezone aware datetimes, but auto_now_add will be useless in timezone oriented app.

Hope you would fix this.

Thanks for brilliant hard work.

i'm so happy to find out your good project --redisco
and i try this lib in my personal project
however i think it can be better if it has more feature such as:

1. add options in fields, so we can define the field like this:
   status = Attribute(required=True, options=['draft', 'private', 'publish', 'trash']
2. add Reference List Field, such we can has a reference field list like:
   categories = ListField(ReferenceField('Category'))
3. i found that all index field in redis was saves index as a set, whether can we save it as a string
   if we set this field unique=True

that's all my suggestion, i hope it can help improve redisco...thx..

I'm trying to use ListField of ReferenceField
to maintain a list of user channels of my User model
as well as a list of users in my Channel model.

Model:

# Module:   models
# Date:     16th August 2014
# Author:   James Mills, prologic at shortcircuit dot net dot au


"""Data Models"""


from circuits.protocols.irc import joinprefix

from redisco.models import Model
from redisco.models import (
    Attribute, BooleanField, DateTimeField,
    IntegerField, ListField, ReferenceField
)


class User(Model):

    host = Attribute(default="")
    port = IntegerField(default=0)

    nick = Attribute(default=None)
    away = BooleanField(default=False)

    channels = ListField("Channel")
    userinfo = ReferenceField("UserInfo")

    registered = BooleanField(default=False)
    signon = DateTimeField(auto_now_add=True)

    @property
    def prefix(self):
        userinfo = self.userinfo
        return joinprefix(self.nick, userinfo.user, userinfo.host)

    class Meta:
        indices = ("id", "nick",)


class UserInfo(Model):

    user = Attribute(default=None)
    host = Attribute(default=None)
    server = Attribute(default=None)
    name = Attribute(default=None)

    def __nonzero__(self):
        return all(x is not None for x in (self.user, self.host, self.name))


class Channel(Model):

    name = Attribute(required=True, unique=True)
    users = ListField("User")

    class Meta:
        indices = ("id", "name",)

This seems to work fine when there is one User and one Channel
object(s) in the database, but as soon as there are two different users
that are members of the same channel I get this error:

2014-08-16 22:02:51,218 - charla.main - ERROR - ERROR <handler[*.join] (Commands.join)> (<join[commands] (<socket._socketobject object at 0x7f956fb54c20>, ('', None, None), '#circuits' )>) (<type 'exceptions.RuntimeError'>): RuntimeError('maximum recursion depth exceeded while calling a Python object',)
  File "/home/prologic/work/circuits/circuits/core/manager.py", line 603, in _dispatcher
    value = handler(*eargs, **ekwargs)
  File "/home/prologic/charla/charla/plugins/core.py", line 111, in join
    user.save()
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/models/base.py", line 202, in save
    self._write(_new)
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/models/base.py", line 345, in _write
    self._update_indices(pipeline)
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/models/base.py", line 413, in _update_indices
    self._add_to_indices(pipeline)
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/models/base.py", line 418, in _add_to_indices
    self._add_to_index(att, pipeline=pipeline)
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/models/base.py", line 426, in _add_to_index
    index = self._index_key_for(att)
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/models/base.py", line 474, in _index_key_for
    return self._tuple_for_index_key_attr_list(att, value)
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/models/base.py", line 492, in _tuple_for_index_key_attr_list
    return ('list', [self._index_key_for_attr_val(att, e) for e in val])
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/models/base.py", line 499, in _index_key_for_attr_val
    return self._key[att][_encode_key(val)]
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/models/utils.py", line 5, in _encode_key
    return base64.b64encode(str(s)).replace("\n", "")
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/models/base.py", line 516, in __repr__
    return "<%s %s>" % (self.key(), self.attributes_dict)
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/models/base.py", line 516, in __repr__
    return "<%s %s>" % (self.key(), self.attributes_dict)

... repeated lines delete

  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/models/base.py", line 516, in __repr__
    return "<%s %s>" % (self.key(), self.attributes_dict)
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/models/base.py", line 247, in attributes_dict
    h[k] = getattr(self, k)
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/models/attributes.py", line 273, in __get__
    val = List(key).members
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/containers.py", line 34, in __getattribute__
    return object.__getattribute__(self, att)
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/containers.py", line 217, in all
    return self.lrange(0, -1)
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/containers.py", line 32, in __getattribute__
    return partial(getattr(object.__getattribute__(self, 'db'), att), self.key)
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/containers.py", line 39, in db
    if self.pipeline:
  File "/home/prologic/.virtualenvs/charla/lib/python2.7/site-packages/redisco/containers.py", line 31, in __getattribute__
    if att in object.__getattribute__(self, 'DELEGATEABLE_METHODS'):

Considering the following code :

class A(models.Model):
   att = models.Attribute()

class B(models.Model):
   a = models.ReferenceField()

a = A()
a.att = "test"
a.save()

b = B()
b.a = a
b.save()

B.objects.filter(a_id=a.id)
# Here redis doesn't delete the filter keys as "_expire_or_delete" should. Leaking 2 more keys in the DB.

Redisco end up leaking 2 filter keys (best case) and n filtering keys in case of multiple filtering.

Suggestion

give a timeout to each temporary keys to let say 60 seconds. It will give time to the operation to succeed and let redis handle the clean up for us.

Is it currently possible to return a list of dictionaries instead of a ModelSet of Models? If not how would I go about implementing this?

As of now I'm simply doing [x.attributes_dict for x in my_modelset] in my code, but with big sets this becomes terribly slow of course. Alternatively I could write my own redis-py queries, but of course I'd prefer to keep the filter and zfilter methods.

I've looked through redisco's code and found stored_attrs = self.db.hgetall(self.key()) in base.py. So at one time we actually get plain old Python dicts, but they are then converted to proper models.

p = Person.objects.filter(name="Granny")
# Assuming only one record fetched.
p[0].first_name = "Morgan"
p[0].last_name = "Stanley"
p.save()

Above case do not update the values in DB.
even

p[0].update_attributes(first_name="Morgan", last_name="Stanley")
p[0].save()

doesn't work.

Instead following work as expected.

p = Person.objects.get_by_id('1')
p.first_name = "Morgan"
p.last_name = Stanley
p.save()

I believe this is critical issue.

It looks to me like one can define subclasses that inherit fields from the parent classes, but I'm running into a problem when I use them. Here's a trivial example of what I'm seeing:

>>> from redisco import models
>>> class Person(models.Model):
...     name = models.Attribute(required=True)
...
>>> class CoolPerson(Person):
...     coolness = models.IntegerField(required=True)
...
>>> class People(models.Model):
...     members = models.ListField(Person)
...
>>> people = People()
>>> alice = Person(name="Alice")
>>> alice.save()
True
>>> print alice
<Person:1 {'name': 'Alice', 'id': '1'}>
>>> people.members.append(alice)
>>> people.save()
True
>>> print People.objects.all()
[<People:1 {'id': '1', 'members': [<Person:1 {'name': u'Alice', 'id': '1'}>]}>]
>>> bob = CoolPerson(name="Bob", coolness=6)
>>> bob.save()
True
>>> print bob
<CoolPerson:1 {'coolness': 6, 'name': 'Bob', 'id': '1'}>
>>> people.members.append(bob)
>>> people.save()
True
>>> print People.objects.all()
[<People:1 {'id': '1', 'members': [<Person:1 {'name': u'Alice', 'id': '1'}>, <Person:1 {'name': u'Alice', 'id': '1'}>]}>]
>>>

I would expect the last line to show Bob as a member, but instead Alice is shown twice.

Throughout the entire code base redisco requires that keys and indexed attributes be unicode.

This makes absolutely no sense since Redis supports any binary strings and redisco cannot be used to store / index binary data without unnecessary encoding overhead.

The suggestion is to purge unicode requirements from every place in the code base.

There's an article here about supporting startswith queries, it could be a way for redisco to do this in future

http://geekyjournal.blogspot.co.uk/2011/05/redis-how-to-support-startswith-queries.html

https://github.com/dongweiming/redisco/commit/aa93f31f81bf3f163b29f2783a70748ec4479f22

Without it one gets an Import Error can not import * from base.

Will see if there is anything obvious:

======================================================================

ERROR: test_CharField (redisco.tests.BooleanFieldTestCase)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/travis/build/kiddouk/redisco/redisco/models/basetests.py", line 26, in setUp

    self.client.flushdb()

  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/redis/client.py", line 652, in flushdb

    return self.execute_command('FLUSHDB')

  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/redis/client.py", line 578, in execute_command

    connection.send_command(*args)

  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/redis/connection.py", line 563, in send_command

    self.send_packed_command(self.pack_command(*args))

  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/redis/connection.py", line 538, in send_packed_command

    self.connect()

  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/redis/connection.py", line 442, in connect

    raise ConnectionError(self._error_message(e))

ConnectionError: Error 111 connecting to localhost:6379. Connection refused.

Hi,

I have used redisco model for ORM in production. It help me a lot.

But while I review the source code, i found that there is redis lock to protect Model._write. I think the _write is implemented by redis pipeline which is atomic. So is  it redundant to protect by Mutex.

Thanks for you reply.

Hi,
How easy would it be to add DictField and SetField, it seems a little odd to me, there is Dict and Set in this, and ListField, but no DictField and SetField.

?

S

I tried using a mixin with some common fields, but it doesn't work:

class CeleryTaskMixin:
    celery_status = models.CharField()  # last known status, call update_status
    celery_id = models.CharField()      # celery uuid
    f_name = models.CharField(indexed=True)


class RunningTask(CeleryTaskMixin, models.Model):
    def __init__(self, t=None, *args, **kwargs):
        CeleryTaskMixin.__init__(self)
        models.Model.__init__(self)

    task_type = models.CharField()
    start_time = models.DateTimeField(auto_now=True)
    end_time = models.DateTimeField()

Then when I try and do

models.RunningTask.objects.filter(f_name="blah")

It says

  File "/home/stu/.virtualenvs/ng/src/redisco/redisco/models/modelset.py", line 52, in __iter__
    for id in self._set:
  File "/home/stu/.virtualenvs/ng/src/redisco/redisco/models/modelset.py", line 294, in _set
    s = self._add_set_filter(s)
  File "/home/stu/.virtualenvs/ng/src/redisco/redisco/models/modelset.py", line 317, in _add_set_filter
    (k, self.model_class.__name__))
redisco.models.exceptions.AttributeNotIndexed: Attribute f_name is not indexed in RunningTask class.

This seems like something that should work... (?)

I seeing in the doc / description of v0.2 some form of interoperability is implied as a goal.

Was there much testing yet of how well it works? I have a use case where different systems need to read/write to a shared Redis DB from Python and Java. If Redisco and JOhm were using the same Redis data structures/formats that would be immensely helpful.

Is this the case? Is this the goal? Is there anything missing or needs help with?

pip install redisco==0.1.4

Error:

Downloading/unpacking redisco
  Downloading redisco-0.1.4.tar.gz
  Running setup.py (path:/Users/glyn/Documents/workspace/ctzn-api/env/build/redisco/setup.py) egg_info for package redisco

Installing collected packages: redisco
  Running setup.py install for redisco
      File "/Users/glyn/Documents/workspace/ctzn-api/env/lib/python3.4/site-packages/redisco/containers.py", line 63
        raise KeyError, value
                      ^
    SyntaxError: invalid syntax

      File "/Users/glyn/Documents/workspace/ctzn-api/env/lib/python3.4/site-packages/redisco/models/attributes.py", line 193
        except TypeError, ValueError:
                        ^
    SyntaxError: invalid syntax

      File "/Users/glyn/Documents/workspace/ctzn-api/env/lib/python3.4/site-packages/redisco/models/base.py", line 164
        except FieldValidationError, e:
                                   ^
    SyntaxError: invalid syntax

      File "/Users/glyn/Documents/workspace/ctzn-api/env/lib/python3.4/site-packages/redisco/models/utils.py", line 6
        except UnicodeError, e:
                           ^
    SyntaxError: invalid syntax


Successfully installed redisco

Considering the following case :

from redisco.containers import Set

foo = Set('s1')
foo.add('1')
foo.add('2')

bar = Set('s2')
bar.add('1')
bar.add('2')

# Expected False
print bar != foo
True

ListField allready support this feature and it would be nice to have ReferenceField to support this feature too.

Hi,

I found something strange when I try to save an instance which as a unique Attribute field email and a List of Reference fields rooms_visited.

As soon as I add an entry to my list the save() of my instance fails on the uniqueness of the Attribute field 'email'.

Strangely, I am able to update the email field then save() without problems.
Removing unique=True works and I only have one user created, not two.

from redisco import models

import string
import redisco

redisco.connection_setup(host='localhost', db=5)

class UserRoomVisit(models.Model):
    code = models.Attribute(required=True)
    joined_date = models.DateTimeField(auto_now_add=True)

class User(models.Model):
    # for some reasons unique True fails to validate on save !
    email = models.Attribute(required=True, indexed=True, unique=True)
    rooms_visited = models.ListField(UserRoomVisit)

class Room(models.Model):
    code = models.Attribute(required=True, indexed=True, unique=True)       
    owner = models.ReferenceField(User)

u = User(email="[email protected]")
u.save()
r = Room(owner=u, code="xxx")
r.save()
e = UserRoomVisit(code="xxx")
e.save()

print u # <User:1 {'rooms_visited': [], 'email': '[email protected]'}>
u.rooms_visited.append(e)
print u # <User:1 {'rooms_visited': [<UserRoomVisit:1 {'joined_date': datetime.datetime(2012, 10, 5, 19, 26, 55, 382587), 'code': 'ppp'}>], 'email': '[email protected]'}>
errors = u.save()

print errors # [('email', 'not unique')] 

support redis3.0 ?

I've done some tracing of Redis calls and was very unpleasantly surprised with the number of queries generated by Redisco and the resulting latency increase.

One of the root causes is the fact that Redisco does not support natural keys (e.g. IP address, etc). Instead every model requires an integer primary key and a natural key field, both with unique indices. Every index relies on clunky/slow lists/sets and expensive operations to keep them running.

Overall what should be a simple HMGET call turns into ~10 various commands that cannot even be pipelined. Obviously this completely kills any latency-sensitive applications.

The question is whether there are any plans / code snippets to implement natural keys. I may have to hack it myself quick, but wanted to see if anyone tried it before and/or may have any input.

With Redisco 0.1.4

class Foo(models.Model):
Field1 = models.Attribute(indexed = True)
Field2 = models.Attribute(unique = True)

Foo(Field1="1", Field2 = "blah").save(); Foo(Field1="Alpha", Field2 = "Beta").save()

Q = Foo.objects.all()[0]
Q.update_attributes(Field1 = "Whatever")
Q.save()

This returns a failed validation message that Field2 is not unique.

If i am not missing something here, this is a simple update, not even touching the field that is declared as unique and after that operation the "unique" condition would still hold.

BTW, in the above example, if the field is not declared as unique then everything works as expected...but then i really want the field to be declared as unique :)

Right now object loading generates an unnecessary EXISTS command. It is wasteful because the subsequent call to HGETALL can be interpreted to ascertain object's existence.

I've made a change to the model id setter and the method that calls it to fix the problem:

    @id.setter
    def id(self, val):
        """
        Setting the id for the object will fetch it from the datastorage;
        If nothing in the datasore, do nothing
        """
        self._id = str(val)
        stored_attrs = self.db.hgetall(self.key())
        if not stored_attrs:
            self._id = None     # key does not exist, set ID to nothing
        else:
            attrs = self.attributes.values()
            for att in attrs:
                if att.name in stored_attrs and not isinstance(att, Counter):
                    att.__set__(self, att.typecast_for_read(stored_attrs[att.name]))

def get_by_id(self, id):
    if (self._filters or self._exclusions or self._zfilters) and str(id) not in self._set:
        return
    obj = self._get_item_with_id(id)
    return obj if obj.id else None

import redisco.models.modelset
redisco.models.modelset.ModelSet.get_by_id = get_by_id

Additionally @classmethod exists currently does not respect DB setting per class. It should use the classes' DB setting instead of always relying on a global object.

I've implemented simple pipelined fetching of multiple objects from ModelSet. Not sure if it's useful for others in this form, but you can see the code bellow. For efficiency reason, you can also pass raw=True to get plain dicts instead of model instances.

class ModelSet(Set):

    def get_in_pipeline(self, raw=False):        
        key = self.model_class._key
        ids = list(self._set)
        pipe = self.db.pipeline()

        for id in ids:
            pipe.hgetall('%s:%s' % (key, id))

        for i, hash in enumerate(pipe.execute()):
            id = ids[i]
            if raw:
                hash['_id'] = id
                yield hash
            else:
                model = self.model_class(**hash)
                model._id = id
                yield model

is_valid() will return False.