Fields

The core module of limpyd provides 6 fields types, matching the ones in Redis:

You can also manage primary keys with these too fields:

All these fields can be indexed, and they manage the keys for you (they take the same arguments as the real Redis ones, as defined in the StrictRedis class of redis-py, but without the key parameter).

Another thing all fields have in common, is the way to delete them: use the delete method on a field, and both the field and its value will be removed from Redis.

Field attributes

When adding fields to a model, you can configure it with some attributes:

default

It’s possible to set default values for fields of type StringField and InstanceHashField:

class Example(model.RedisModel):
    database = main_database
    foo = fields.StringField(default='FOO')
    bar = fields.StringField()

>>> example = Example(bar='BAR')
>>> example.foo.get()
'FOO'

When setting a default value, the field will be saved when creating the instance. If you defined a PKField (not AutoPKField), don’t forget to pass a value for it when creating the instance, it’s needed to store other fields.

indexable

Sometimes getting objects from Redis by its primary key is not what you want. You may want to search for objects with a specific value for a specific field.

By setting the indexable argument to True when defining the field, this feature is automatically activated, and you’ll be able to retrieve objects by filtering on this field using Collections.

To activate it, just set the indexable argument to True:

class Example(model.RedisModel):
    database = main_database
    foo = fields.StringField(indexable=True)
    bar = fields.StringField()

In this example you will be able to filter on the field foo but not on bar.

When updating an indexable field, a lock is acquired on Redis on this field, for all instances of the model. It isn’t possible for this to use pipeline or redis scripting, because both need to know in advance the keys to update, but we don’t always know since keys for indexes may be based on values. So all writing operations on an indexable field are protected, to ensure consistency if many threads, process, servers are working on the same Redis database.

If you are sure you have only one thread, or you don’t want to ensure consistency, you can disable locking by setting to False the lockable argument when creating a field, or the lockable attribute of a model to inactive the lock for all of its fields.

unique

The unique argument is the same as the indexable one, except it will ensure that you can’t have multiple objects with the same value for some fields. unique fields are also indexed, and can be filtered, as for the indexable argument.

Example:

class Example(model.RedisModel):
    database = main_database
    foo = fields.StringField(indexable=True)
    bar = fields.StringField(unique=True)

>>> example1 = Example(foo='FOO', bar='BAR')
True
>>> example2 = Example(foo='FOO', bar='BAR')
UniquenessError: Key :example:bar:BAR already exists (for instance 1)

See Collections to know how to filter objects, as for indexable.

indexes

This allow to change the default index used, or use many of them. See the “Indexing” section in Collections to know more.

lockable

You can set this argument to False if you don’t want a lock to be acquired on this field for all instances of the model. See indexable for more information about locking.

If not specified, it’s default to True, except if the lockable attribute of the model is False, in which case it’s forced to False for all fields.

Field types

StringField

StringField based fields allow the storage of strings, but some Redis string commands allow to treat them as integer, float [1] or bits.

Example:

from limpyd import model, fields

class Example(model.RedisModel):
    database = main_database

    name = fields.StringField()

You can use this model like this:

>>> example = Example(name='foo')
>>> example.name.get()
'foo'
>>> example.name.set('bar')
>>> example.name.get()
'bar'
>> example.name.delete()
True

The StringField type support these Redis string commands:

Getters

  • bitcount
  • get
  • getbit
  • getrange
  • getset
  • strlen

Modifiers

  • append
  • decr
  • getset
  • incr
  • incrbyfloat [1]
  • set
  • setbit
  • setnx
  • setrange

HashField

HashField allows storage of a dict in Redis.

Example:

class Email(model.RedisModel):
    database = main_database
    headers = fields.HashField()

>>> email = Email()
>>> headers = {'from': 'foo@bar.com', 'to': 'me@world.org'}
>>> email.headers.hmset(**headers)
>>> email.headers.hget('from')
'foo@bar.com'

The HashField type support these Redis hash commands:

Getters

  • hget
  • hgetall
  • hmget
  • hkeys
  • hvals
  • hexists
  • hlen
  • hscan (returns a generator with all/matching key/value pairs, you don’t have to manage the cursor)

Modifiers

  • hdel
  • hmset
  • hsetnx
  • hset
  • hincrby
  • hincrbyfloat [1]

InstanceHashField

As for StringField, InstanceHashField based fields allow the storage of strings. But all the InstanceHashField fields of an instance are stored in the same Redis hash, the name of the field being the key in the hash.

To fully use the power of Redis hashes, we also provide two methods to get and set multiples field in one operation (see hmget and hmset). It’s usually cheaper to store fields in hash that in strings. And it’s faster to set/retrieve them using these two commands.

Example with simple commands:

class Example(model.RedisModel):
    database = main_database

    foo = fields.InstanceHashField()
    bar = fields.InstanceHashField()

>>> example.foo.hset('FOO')
1  # 1 because the hash field was created
>>> example.foo.hget()
'FOO'

The InstanceHashField type support these Redis hash commands:

Getters

  • hget

Modifiers

  • hincrby
  • hincrbyfloat [1]
  • hset
  • hsetnx

Deleter

To delete the value of a InstanceHashField, you can use the hdel command, which do the same as the main `delete` one.

See also hdel on the model to delete many InstanceHashField at once

Multi

The following commands are not called on the fields themselves, but on an instance:

hmget

hmget is called directly on an instance, and expects a list of field names to retrieve.

The result will be, as in Redis, a list of all values, in the same order.

If no names are provided, nothing will be fetched. Use hvals, or better, hgetall to get values for all InstanceHashFields

It’s up to you to associate names and values, but you can find an example below:

class Example(model.RedisModel):
    database = main_database

    foo = fields.InstanceHashField()
    bar = fields.InstanceHashField()
    baz = fields.InstanceHashField()
    qux = fields.InstanceHashField()

    def hmget_dict(self, *args):
        """
        A call to hmget but which return a dict with field names as keys, instead
        of only a list of values
        """
        values = self.hmget(*args)
        keys = args or self._hashable_fields
        return dict(zip(keys, values))


>>> example = Example(foo='FOO', bar='BAR')
>>> example.hmget('foo', 'bar')
['FOO', 'BAR']
>>> example.hmget_dict('foo', 'bar')
{'bar': 'BAR', 'foo': 'FOO'}
hmset

hmset is the reverse of hmget, and also called directly on an instance, and expects named arguments with field names as keys, and new values to set as values.

Example (with same model as for hmget):

>>> example = Example()
>>> example.hmset(foo='FOO', bar='BAR')
True
>>> example.hmget('foo', 'bar')
['FOO', 'BAR']
hdel

hdel is called directly on an instance, and expects a list of field names to delete.

The result will be, as in Redis, the number of field really deleted (ie fields without any stored value won’t be taken into account).

>>> example = Example()
>>> example.hmset(foo='FOO', bar='BAR', baz='BAZ')
True
>>> example.hmget('foo', 'bar', 'baz')
['FOO', 'BAR', 'BAZ']
>>> example.hdel('foo', 'bar', 'qux')
2
>>> example.hmget('foo', 'bar', 'baz')
[None, None, 'BAZ']

Note that you can also call hdel on an InstanceHashField itself, without parameters, to delete this very field.

>>> example.baz.hdel()
1
hgetall

hgetall must be called directly on an instance, and will return a dictionary containing names and values of all InstanceHashField with a stored value.

If a field has no stored value, it will not appear in the result of hgetall.

Example (with same model as for hmget):

>>> example = Example(foo='FOO', bar='BAR')
>>> example.hgetall()
{'foo': 'FOO', 'bar': 'BAR'}
>>> example.foo.hdel()
>>> example.hgetall()
{bar': 'BAR'}
hkeys

hkeys must be called on an instance and will return the name of all the InstanceHashField with a stored value.

If a field has no stored value, it will not appear in the result of hkeys.

Note that the result is not ordered in any way.

Example (with same model as for hmget):

>>> example = Example(foo='FOO', bar='BAR')
>>> example.hkeys()
['foo', 'bar']
>>> example.foo.hdel()
>>> example.hkeys()
['bar']
hvals

hkeys must be called on an instance and will return the value of all the InstanceHashField with a stored value.

If a field has no stored value, it will not appear in the result of hvals.

Note that the result is not ordered in any way.

Example (with same model as for hmget):

>>> example = Example(foo='FOO', bar='BAR')
>>> example.hvals()
['FOO', 'BAR']
>>> example.foo.hdel()
>>> example.hvals()
['BAR']
hlen

hlen must be called on an instance and will return the number of InstanceHashField with a stored value.

If a field has no stored value, it will not be count in the result of hlen.

Example (with same model as for hmget):

>>> example = Example(foo='FOO', bar='BAR')
>>> example.hlen()
2
>>> example.foo.hdel()
>>> example.hlen()
1

SetField

SetField based fields can store many values in one field, using the set data type of Redis, an unordered set (with unique values).

Example:

from limpyd import model, fields

class Example(model.RedisModel):
    database = main_database

    stuff = fields.SetField()

You can use this model like this:

>>> example = Example()
>>> example.stuff.sadd('foo', 'bar')
2  # number of values really added to the set
>>> example.stuff.smembers()
set(['foo', 'bar'])
>>> example.stuff.sismember('bar')
True
>>> example.stuff.srem('bar')
True
>>> example.stuff.smembers()
set(['foo'])
>>> example.stuff.delete()
True

The SetField type support these Redis set commands:

Getters

  • scard
  • sismember
  • smembers
  • srandmember
  • sscan (returns a generator with all/matching values, you don’t have to manage the cursor)
  • sort (with arguments like in redis-py, see redis-py-sort)

Modifiers

  • sadd
  • spop
  • srem

ListField

ListField based fields can store many values in one field, using the list data type of Redis. Values are ordered, and are not unique (you can push many times the same value).

Example:

from limpyd import model, fields

class Example(model.RedisModel):
    database = main_database

    stuff = fields.ListField()

You can use this model like this:

>>> example = Example()
>>> example.stuff.rpush('foo', 'bar')
2  # number of values added to the list
>>> example.stuff.lrange(0, -1)
['foo', 'bar']
>>> example.stuff.lindex(1)
'bar'
>>> example.stuff.lrem(1, 'bar')
1  # number of values really removed
>>> example.stuff.lrange(0, -1)
['foo']
>>> example.stuff.delete()
True

The ListField type support these Redis list commands:

Getters

Modifiers

  • linsert
  • lpop
  • lpush
  • lpushx
  • lrem
  • lset
  • ltrim
  • rpop
  • rpush
  • rpushx

SortedSetField

SortedSetField based fields can store many values, each scored, in one field using the sorted-set data type of Redis. Values are unique (it’s a set), and are ordered by their score.

Example:

from limpyd import model, fields

class Example(model.RedisModel):
    database = main_database

    stuff = fields.SortedSetField()

You can use this model like this:

>>> example = Example()
>>> example.stuff.zadd(foo=2.5, bar=1.1)
2  # number of values added to the sorted set
>>> example.stuff.zrange(0, -1)
['bar', 'foo']
>>> example.stuff.zrangebyscore(1, 2, withscores=True)
[('bar', 1.1)]
>>> example.stuff.zrem('bar')
1  # number of values really removed
>>> example.stuff.zrangebyscore(1, 2, withscores=True)
[]
>>> example.stuff.delete()
True

The SortedSetField type support these Redis sorted set commands:

Getters

  • zcard
  • zcount
  • zrange
  • zrangebyscore
  • zrank
  • zrevrange
  • zrevrangebyscore
  • zrevrank
  • zscore
  • zscan (returns a generator with all/matching key/score pairs, you don’t have to manage the cursor)
  • sort (with arguments like in redis-py, see redis-py-sort)

Modifiers

  • zadd
  • zincrby
  • zrem
  • zremrangebyrank
  • zremrangebyscore

PKField

PKField is a special subclass of StringField that manage primary keys of models. The PK of an object cannot be updated, as it serves to create keys of all its stored fields. It’s this PK that is returned, with others, in Collections.

A PK can contain any sort of string you want: simple integers, float [1], long uuid, names…

If you want a PKField which will be automatically filled, and auto-incremented, see AutoPKField. Otherwise, with standard PKField, you must assign a value to it when creating an instance.

By default, a model has a AutoPKField attached to it, named pk. But you can redefine the name and type of PKField you want.

Examples:

class Foo(model.RedisModel):
    """
    The PK field is ``pk``, and will be auto-incremented.
    """
    database = main_database

class Bar(model.RedisModel):
    """
    The PK field is ``id``, and will be auto-incremented.
    """
    database = main_database
    id = fields.AutoPKField()

class Baz(model.RedisModel):
    """
    The PK field is ``name``, and won't be auto-incremented, so you must assign it a value when creating an instance.
    """
    database = main_database
    name = fields.PKField()

Note that whatever name you use for the PKField (or AutoPKField), you can always access it via the name pk (but also we its real name). It’s easier for abstraction:

class Example(model.RedisModel):
    database = main_database
    id = fields.AutoPKField()
    name = fields.StringField()

>>> example = Example(name='foobar')
>>> example.pk.get()
1
>>> example.id.get()
1

AutoPKField

A AutoPKField field is a PKField filled with auto-incremented integers, starting to 1. Assigning a value to of AutoPKField is forbidden.

It’s a AutoPKField that is attached by default to every model, if no other PKField is defined.

See PKField for more details.

[1](1, 2, 3, 4, 5) When working with floats, pass them as strings to avoid precision problems.