Git Product home page Git Product logo

redis-om-node's Introduction



Redis OM

Object mapping, and more, for Redis and Node.js. Written in TypeScript.


Discord Twitch YouTube Twitter

NPM Build License

Redis OM for Node.js makes it easy to model Redis data in your Node.js applications.

Redis OM .NET | Redis OM Node.js | Redis OM Python | Redis OM Spring

Table of contents

Redis OM for Node.js

Redis OM (pronounced REDiss OHM) makes it easy to add Redis to your Node.js application by mapping the Redis data structures you know and love to simple JavaScript objects. No more pesky, low-level commands, just pure code with a fluent interface.

Define a schema:

const schema = new Schema('album', {
  artist: { type: 'string' },
  title: { type: 'text' },
  year: { type: 'number' }
})

Create a JavaScript object and save it:

const album = {
  artist: "Mushroomhead",
  title: "The Righteous & The Butterfly",
  year: 2014
}

await repository.save(album)

Search for matching entities:

const albums = await repository.search()
  .where('artist').equals('Mushroomhead')
  .and('title').matches('butterfly')
  .and('year').is.greaterThan(2000)
    .return.all()

Pretty cool, right? Read on for details.

⚠️ Warning: This Version Has Breaking Changes from 0.3.6

Redis OM 0.4 is new, improved, and includes breaking changes. If you're trying it for the first time, no worries. Just follow what's in this README and you'll be fine.

However, you might be a user of Redis OM already. If that is the case, you'll want to review this document to understand those changes.

Of course, you don't have to upgrade. If this is you, you'll want to check out the README for that version over on NPM.

However, I hope you choose to try the new version. It has many changes that have been frequently requested that are documented in the CHANGELOG. And more, non-breaking changes will follow these.

Getting Started

First things first, get yourself a Node.js project. There are lots of ways to do this, but I'm gonna go with a classic:

$ npm init

Once you have that sweet, sweet package.json, let's add our newest favorite package to it:

$ npm install redis-om

Redis OM for Node.js uses Node Redis. So you should install that too:

$ npm install redis

And, of course, you'll need some Redis, preferably Redis Stack as it comes with RediSearch and RedisJSON ready to go. The easiest way to do this is to set up a free Redis Cloud instance. But, you can also use Docker:

$ docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:latest

Excellent. Setup done. Let's write some code!

Connect to Redis with Node Redis

Before you can use Redis OM, you need to connect to Redis with Node Redis. Here's how you do that, stolen straight from the top of the Node Redis README:

import { createClient } from 'redis'

const redis = createClient()
redis.on('error', (err) => console.log('Redis Client Error', err));
await redis.connect()

Node Redis is a powerful piece of software with lots and lots of capabilities. Its details are way beyond the scope of this README. But, if you're curious—or if you need that power—you can find all the info in the Node Redis documentation.

Regardless, once you have a connection to Redis you can use it to execute Redis commands:

const aString = await redis.ping() // 'PONG'
const aNumber = await redis.hSet('foo', 'alfa', '42', 'bravo', '23') // 2
const aHash = await redis.hGetAll('foo') // { alfa: '42', bravo: '23' }

You might not need to do this, but it's always handy to have the option. When you're done with a Redis connection, you can let the server know by calling .quit:

await redis.quit()

Redis Connection Strings

By default, Node Redis connects to localhost on port 6379. This is, of course, configurable. Just pass in a url with the hostname and port that you want to use:

const redis = createClient({ url: 'redis://alice:[email protected]:6380' })

The basic format for this URL is:

redis://username:password@host:port

This will probably cover most scenarios, but if you want something more, the full specification for the URL is defined with the IANA. And yes, there is a TLS version as well.

Node Redis has lots of other ways you can create a connection. You can use discrete parameters, UNIX sockets, and all sorts of cool things. Details can be found in the client configuration guide for Node Redis and the clusterting guide.

Entities and Schemas

Redis OM is all about saving, reading, and deleting entities. An Entity is just data in a JavaScript object that you want to save or retrieve from Redis. Almost any JavaScript object is a valid Entity.

Schemas define fields that might be on an Entity. It includes a field's type, how it is stored internally in Redis, and how to search on it if you are using RediSearch. By default, they are mapped to JSON documents using RedisJSON, but you can change it to use Hashes if want (more on that later).

Ok. Let's start doing some object mapping and create a Schema:

import { Schema } from 'redis-om'

const albumSchema = new Schema('album', {
  artist: { type: 'string' },
  title: { type: 'text' },
  year: { type: 'number' },
  genres: { type: 'string[]' },
  songDurations: { type: 'number[]' },
  outOfPublication: { type: 'boolean' }
})

const studioSchema = new Schema('studio', {
  name: { type: 'string' },
  city: { type: 'string' },
  state: { type: 'string' },
  location: { type: 'point' },
  established: { type: 'date' }
})

The first argument is the Schema name. It defines the key name prefix that entities stored in Redis will have. It should be unique for your particular instance of Redis and probably meaningful to what you're doing. Here we have selected album for our album data and studio for data on recording studios. Imaginative, I know.

The second argument defines fields that might be stored in that key. The property name is the name of the field that you'll be referencing in your Redis OM queries. The type property tells Redis OM what sort of data is in that field. Valid types are: string, number, boolean, string[], number[], date, point, and text.

The first three types do exactly what you think—they define a field that is a String, a Number, or a Boolean. string[] and number[] do what you'd think as well, specifically describing an Array of Strings or Numbers respectively.

date is a little different, but still more or less what you'd expect. It describes a property that contains a Date and can be set using not only a Date but also a String containing an ISO 8601 date or a number with the UNIX epoch time in seconds (NOTE: the JavaScript Date object is specified in milliseconds).

A point defines a point somewhere on the globe as a longitude and a latitude. It is expressed as a simple object with longitude and latitude properties. Like this:

const point = { longitude: 12.34, latitude: 56.78 }

A text field is a lot like a string. If you're just reading and writing objects, they are identical. But if you want to search on them, they are very, very different. I'll cover that in detail when I talk about searching but the tl;dr is that string fields can only be matched on their exact value and are best for keys and discrete data—like postal codes or status indicators—while text fields have full-text search enabled on them, are optimized for human-readable text, and can take advantage of stemming and stop words.

JSON and Hashes

As I mentioned earlier, by default Redis OM stores your entities in JSON documents using RedisJSON. You can make this explicit in code if you like:

const albumSchema = new Schema('album', {
  artist: { type: 'string' },
  title: { type: 'string' },
  year: { type: 'number' },
  genres: { type: 'string[]' },
  songDurations: { type: 'number[]' },
  outOfPublication: { type: 'boolean' }
}, {
  dataStructure: 'JSON'
})

But you can also store your entities as Hashes instead. Just change the dataStructure property to reflect it:

const albumSchema = new Schema('album', {
  artist: { type: 'string' },
  title: { type: 'string' },
  year: { type: 'number' },
  genres: { type: 'string[]' },
  outOfPublication: { type: 'boolean' }
}, {
  dataStructure: 'HASH'
})

And that's it.

Of course, Hashes and JSON are somewhat different data structures. Hashes are flat with fields containing values. JSON documents, however, are trees and can have depth and—most excitingly—can be nested. This difference is reflected in how Redis OM maps data to entities and how you configure your Schema.

Note that I have not included the songDurations in the Hash. This is because number[] is only possible when working with JSON, it will generate an error if you try to use it with Hashes.

Configuring JSON

When you store your entities as JSON, the path to the properties in your JSON document and your JavaScript object default to the name of your property in the schema. In the above example, this would result in a document that looks like this:

{
  "artist": "Mushroomhead",
  "title": "The Righteous & The Butterfly",
  "year": 2014,
  "genres": [ "metal" ],
  "songDurations": [ 204, 290, 196, 210, 211, 105, 244, 245, 209, 252, 259, 200, 215, 219 ],
  "outOfPublication": true
}

However, you might not want your JavaScript object and your JSON to map this way. So, you can provide a path option in your schema that contains a JSONPath pointing to where that field actually exists in the JSON and your entity. For example, we might want to store some of the album's data inside of an album property like this:

{
  "album": {
    "artist": "Mushroomhead",
    "title": "The Righteous & The Butterfly",
    "year": 2014,
    "genres": [ "metal" ],
    "songDurations": [ 204, 290, 196, 210, 211, 105, 244, 245, 209, 252, 259, 200, 215, 219 ]
  },
  "outOfPublication": true
}

To do this, we'll need to specify the path property for the nested fields in the schema:

const albumSchema = new Schema('album', {
  artist: { type: 'string', path: '$.album.artist' },
  title: { type: 'string', path: '$.album.title' },
  year: { type: 'number', path: '$.album.year' },
  genres: { type: 'string[]', path: '$.album.genres[*]' },
  songDurations: { type: 'number[]', path: '$.album.songDurations[*]' },
  outOfPublication: { type: 'boolean' }
})

There are two things to note here:

  1. We haven't specified a path for outOfPublication as it's still in the root of the document. It defaults to $.outOfPublication.
  2. Our genres field points to a string[]. When using a string[] the JSONPath must return an array. If it doesn't, an error will be generated.
  3. Same for our songDurations.

Configuring Hashes

When you store your entities as Hashes there is no nesting—all the entities are flat. In Redis, the properties on your entity are stored in fields inside a Hash. The default name for each field is the name of the property in your schema and this is the name that will be used in your entities. So, for the following schema:

const albumSchema = new Schema('album', {
  artist: { type: 'string' },
  title: { type: 'string' },
  year: { type: 'number' },
  genres: { type: 'string[]' },
  outOfPublication: { type: 'boolean' }
}, {
  dataStructure: 'HASH'
})

In your code, your entities would look like this:

{
  artist: 'Mushroomhead',
  title: 'The Righteous & The Butterfly',
  year: 2014,
  genres: [ 'metal' ],
  outOfPublication: true
}

Inside Redis, your Hash would be stored like this:

Field Value
artist Mushroomhead
title The Righteous & The Butterfly
year 2014
genres metal
outOfPublication 1

However, you might not want the names of your fields and the names of the properties on your entity to be exactly the same. Maybe you've got some existing data with existing names or something.

Fear not! You can change the name of the field used by Redis with the field property:

const albumSchema = new Schema('album', {
  artist: { type: 'string', field: 'album_artist' },
  title: { type: 'string', field: 'album_title' },
  year: { type: 'number', field: 'album_year' },
  genres: { type: 'string[]' },
  outOfPublication: { type: 'boolean' }
}, {
  dataStructure: 'HASH'
})

With this configuration, your entities will remain unchanged and will still have properties for artist, title, year, genres, and outOfPublication. But inside Redis, the field will have changed:

Field Value
album_artist Mushroomhead
album_title The Righteous & The Butterfly
album_year 2014
genres metal
outOfPublication 1

Reading, Writing, and Removing with Repository

Now that we have a client and a schema, we have what we need to make a repository. A repository provides the means to write, read, and remove entities. Creating a repository is pretty straightforward—just instantiate one with a schema and a client:

import { Repository } from 'redis-om'

const albumRepository = new Repository(albumSchema, redis)
const studioRepository = new Repository(studioSchema, redis)

Once we have a repository, we can use .save to, well, save entities:

let album = {
  artist: "Mushroomhead",
  title: "The Righteous & The Butterfly",
  year: 2014,
  genres: [ 'metal' ],
  songDurations: [ 204, 290, 196, 210, 211, 105, 244, 245, 209, 252, 259, 200, 215, 219 ],
  outOfPublication: true
}

album = await albumRepository.save(album)

This saves your entity and returns a copy, a copy with some additional properties. The primary property we care about right now is the entity ID, which Redis OM will generate for you. However, this isn't stored and accessed like a typical property. After all, you might have a property in your data with a name that conflicts with the name Redis OM uses and that would create all sorts of problems.

So, Redis OM uses a Symbol to access it instead. You'll need to import this symbol from Redis OM:

import { EntityId } from 'redis-om'

Then you can access the entity ID using that symbol:

album = await albumRepository.save(album)
album[EntityId] // '01FJYWEYRHYFT8YTEGQBABJ43J'

The entity ID that Redis OM generates is a ULID and is a unique id representing that object. If you don't like using generated IDs for some reason and instead want to provide your own, you can totally do that:

album = await albumRepository.save('BWOMP', album)

Regardless, once you have an object's entity ID you can .fetch with it:

const album = await albumRepository.fetch('01FJYWEYRHYFT8YTEGQBABJ43J')
album.artist // "Mushroomhead"
album.title // "The Righteous & The Butterfly"
album.year // 2014
album.genres // [ 'metal' ]
album.songDurations // [ 204, 290, 196, 210, 211, 105, 244, 245, 209, 252, 259, 200, 215, 219 ]
album.outOfPublication // true

If you call .save with an entity that already has an entity ID, probably because you fetched it, .save will update it instead of creating a new Entity:

let album = await albumRepository.fetch('01FJYWEYRHYFT8YTEGQBABJ43J')
album.genres = [ 'metal', 'nu metal', 'avantgarde' ]
album.outOfPublication = false

album = await albumRepository.save(album)

You can even use .save to clone an Entity. Just pass in a new entity ID to .save and it'll save the data to that entity ID:

const album = await albumRepository.fetch('01FJYWEYRHYFT8YTEGQBABJ43J')
album.genres = [ 'metal', 'nu metal', 'avantgarde' ]
album.outOfPublication = false

const clonedEntity = await albumRepository.save('BWOMP', album)

And, of course, you need to be able to delete things. Use .remove to do that:

await albumRepository.remove('01FJYWEYRHYFT8YTEGQBABJ43J')

You can also set an entity to expire after a certain number of seconds. Redis will automatically remove that entity when the time's up. Use the .expire method to do this:

const ttlInSeconds = 12 * 60 * 60  // 12 hours
await albumRepository.expire('01FJYWEYRHYFT8YTEGQBABJ43J', ttlInSeconds)

Missing Entities and Null Values

Redis, and by extension Redis OM, doesn't differentiate between missing and null—particularly for Hashes. Missing fields in Redis Hashes are returned as null. Missing keys also return null. So, if you fetch an entity that doesn't exist, it will happily return you an empty entity, complete with the provided entity ID:

const album = await albumRepository.fetch('TOTALLY_BOGUS')
album[EntityId] // 'TOTALLY_BOGUS'
album.artist // undefined
album.title // undefined
album.year // undefined
album.genres // undefined
album.outOfPublication // undefined

Conversely, if you remove all the properties on an entity and then save it, it will remove the entity from Redis:

const album = await albumRepository.fetch('01FJYWEYRHYFT8YTEGQBABJ43J')
delete album.artist
delete album.title
delete album.year
delete album.genres
delete album.outOfPublication

const entityId = await albumRepository.save(album)

const exists = await redis.exists('album:01FJYWEYRHYFT8YTEGQBABJ43J') // 0

It does this because Redis doesn't distinguish between missing and null. You could have an entity that is empty. Or you could not have an entity at all. Redis doesn't know which is your intention, and so always returns something when you call .fetch.

Searching

Using RediSearch with Redis OM is where the power of this fully armed and operational battle station starts to become apparent. If you have RediSearch installed on your Redis server you can use the search capabilities of Redis OM. This enables commands like:

const albums = await albumRepository.search()
  .where('artist').equals('Mushroomhead')
  .and('title').matches('butterfly')
  .and('year').is.greaterThan(2000)
    .return.all()

Let's explore this in full.

Build the Index

To use search you have to build an index. If you don't, you'll get errors. To build an index, just call .createIndex on your repository:

await albumRepository.createIndex();

If you change your schema, no worries. Redis OM will automatically rebuild the index for you. Just call .createIndex again. And don't worry if you call .createIndex when your schema hasn't changed. Redis OM will only rebuild your index if the schema has changed. So, you can safely use it in your startup code.

However, if you have a lot of data, rebuilding an index can take some time. So, you might want to explicitly manage the building and rebuilding of your indices in some sort of deployment code script thing. To support those devops sorts of things, Redis OM includes a .dropIndex method to explicitly remove an index without rebuilding it:

await albumRepository.dropIndex();

You probably won't use this in your application, but if you come up with a cool use for it, I'd love to hear about it!

Finding All The Things (and Returning Them)

Once you have an index created (or recreated) you can search. The most basic search is to just return all the things. This will return all of the albums that you've put in Redis:

const albums = await albumRepository.search().return.all()

Pagination

It's possible you have a lot of albums; I know I do. In that case, you can page through the results. Just pass in the zero-based offset and the number of results you want:

const offset = 100
const count = 25
const albums = await albumRepository.search().return.page(offset, count)

Don't worry if your offset is greater than the number of entities. If it is, you just get an empty array back. No harm, no foul.

First Things First

Sometimes you only have one album. Or maybe you only care about the first album you find. You can easily grab the first result of your search with .first:

const firstAlbum = await albumRepository.search().return.first();

Note: If you have no albums, this will return null. And I feel sorry for you.

Counting

Sometimes you just want to know how many albums you have. For that, you can call .count:

const count = await albumRepository.search().return.count()

Finding Specific Things

It's fine and dandy to return all the things. But that's not what you usually want to do. You want to find specific things. Redis OM will let you find those specific things by strings, numbers, and booleans. You can also search for strings that are in an array, perform full-text search within strings, search by date, and search for points on the globe within a particular area.

And it does it with a fluent interface that allows—but does not demand—code that reads like a sentence. See below for exhaustive examples of all the syntax available to you.

Searching on Strings

When you set the field type in your schema to string, you can search for a particular value in that string. You can also search for partial strings (no shorter than two characters) that occur at the beginning, middle, or end of a string. If you need to search strings in a more sophisticated manner, you'll want to look at the text type and search it using the Full-Text Search syntax.

let albums

// find all albums where the artist is 'Mushroomhead'
albums = await albumRepository.search().where('artist').eq('Mushroomhead').return.all()

// find all albums where the artist is *not* 'Mushroomhead'
albums = await albumRepository.search().where('artist').not.eq('Mushroomhead').return.all()

// find all albums using wildcards
albums = await albumRepository.search().where('artist').eq('Mush*').return.all()
albums = await albumRepository.search().where('artist').eq('*head').return.all()
albums = await albumRepository.search().where('artist').eq('*room*').return.all()

// fluent alternatives that do the same thing
albums = await albumRepository.search().where('artist').equals('Mushroomhead').return.all()
albums = await albumRepository.search().where('artist').does.equal('Mushroomhead').return.all()
albums = await albumRepository.search().where('artist').is.equalTo('Mushroomhead').return.all()
albums = await albumRepository.search().where('artist').does.not.equal('Mushroomhead').return.all()
albums = await albumRepository.search().where('artist').is.not.equalTo('Mushroomhead').return.all()

Searching on Numbers

When you set the field type in your schema to number, you can store both integers and floating-point numbers. And you can search against it with all the comparisons you'd expect to see:

let albums

// find all albums where the year is ===, >, >=, <, and <= 1984
albums = await albumRepository.search().where('year').eq(1984).return.all()
albums = await albumRepository.search().where('year').gt(1984).return.all()
albums = await albumRepository.search().where('year').gte(1984).return.all()
albums = await albumRepository.search().where('year').lt(1984).return.all()
albums = await albumRepository.search().where('year').lte(1984).return.all()

// find all albums where the year is between 1980 and 1989 inclusive
albums = await albumRepository.search().where('year').between(1980, 1989).return.all()

// find all albums where the year is *not* ===, >, >=, <, and <= 1984
albums = await albumRepository.search().where('year').not.eq(1984).return.all()
albums = await albumRepository.search().where('year').not.gt(1984).return.all()
albums = await albumRepository.search().where('year').not.gte(1984).return.all()
albums = await albumRepository.search().where('year').not.lt(1984).return.all()
albums = await albumRepository.search().where('year').not.lte(1984).return.all()

// find all albums where year is *not* between 1980 and 1989 inclusive
albums = await albumRepository.search().where('year').not.between(1980, 1989);

// fluent alternatives that do the same thing
albums = await albumRepository.search().where('year').equals(1984).return.all()
albums = await albumRepository.search().where('year').does.equal(1984).return.all()
albums = await albumRepository.search().where('year').does.not.equal(1984).return.all()
albums = await albumRepository.search().where('year').is.equalTo(1984).return.all()
albums = await albumRepository.search().where('year').is.not.equalTo(1984).return.all()

albums = await albumRepository.search().where('year').greaterThan(1984).return.all()
albums = await albumRepository.search().where('year').is.greaterThan(1984).return.all()
albums = await albumRepository.search().where('year').is.not.greaterThan(1984).return.all()

albums = await albumRepository.search().where('year').greaterThanOrEqualTo(1984).return.all()
albums = await albumRepository.search().where('year').is.greaterThanOrEqualTo(1984).return.all()
albums = await albumRepository.search().where('year').is.not.greaterThanOrEqualTo(1984).return.all()

albums = await albumRepository.search().where('year').lessThan(1984).return.all()
albums = await albumRepository.search().where('year').is.lessThan(1984).return.all()
albums = await albumRepository.search().where('year').is.not.lessThan(1984).return.all()

albums = await albumRepository.search().where('year').lessThanOrEqualTo(1984).return.all()
albums = await albumRepository.search().where('year').is.lessThanOrEqualTo(1984).return.all()
albums = await albumRepository.search().where('year').is.not.lessThanOrEqualTo(1984).return.all()

albums = await albumRepository.search().where('year').is.between(1980, 1989).return.all()
albums = await albumRepository.search().where('year').is.not.between(1980, 1989).return.all()

Searching on Booleans

You can search against fields that contain booleans if you defined a field type of boolean in your schema:

let albums

// find all albums where outOfPublication is true
albums = await albumRepository.search().where('outOfPublication').true().return.all()

// find all albums where outOfPublication is false
albums = await albumRepository.search().where('outOfPublication').false().return.all()

You can negate boolean searches. This might seem odd, but if your field is null, then it would match on a .not query:

// find all albums where outOfPublication is false or null
albums = await albumRepository.search().where('outOfPublication').not.true().return.all()

// find all albums where outOfPublication is true or null
albums = await albumRepository.search().where('outOfPublication').not.false().return.all()

And, of course, there's lots of syntactic sugar to make this fluent:

albums = await albumRepository.search().where('outOfPublication').eq(true).return.all()
albums = await albumRepository.search().where('outOfPublication').equals(true).return.all()
albums = await albumRepository.search().where('outOfPublication').does.equal(true).return.all()
albums = await albumRepository.search().where('outOfPublication').is.equalTo(true).return.all()

albums = await albumRepository.search().where('outOfPublication').true().return.all()
albums = await albumRepository.search().where('outOfPublication').false().return.all()
albums = await albumRepository.search().where('outOfPublication').is.true().return.all()
albums = await albumRepository.search().where('outOfPublication').is.false().return.all()

albums = await albumRepository.search().where('outOfPublication').not.eq(true).return.all()
albums = await albumRepository.search().where('outOfPublication').does.not.equal(true).return.all()
albums = await albumRepository.search().where('outOfPublication').is.not.equalTo(true).return.all()
albums = await albumRepository.search().where('outOfPublication').is.not.true().return.all()
albums = await albumRepository.search().where('outOfPublication').is.not.false().return.all()

Searching on Dates

If you have a field type of date in your schema, you can search on it using Dates, ISO 8601 formatted strings, or the UNIX epoch time in seconds:

studios = await studioRepository.search().where('established').on(new Date('2010-12-27')).return.all()
studios = await studioRepository.search().where('established').on('2010-12-27').return.all()
studios = await studioRepository.search().where('established').on(1293408000).return.all()

There are several date comparison methods to use. And they can be negated:

const date = new Date('2010-12-27')
const laterDate = new Date('2020-12-27')

studios = await studioRepository.search().where('established').on(date).return.all()
studios = await studioRepository.search().where('established').not.on(date).return.all()
studios = await studioRepository.search().where('established').before(date).return.all()
studios = await studioRepository.search().where('established').not.before(date).return.all()
studios = await studioRepository.search().where('established').after(date).return.all()
studios = await studioRepository.search().where('established').not.after(date).return.all()
studios = await studioRepository.search().where('established').onOrBefore(date).return.all()
studios = await studioRepository.search().where('established').not.onOrBefore(date).return.all()
studios = await studioRepository.search().where('established').onOrAfter(date).return.all()
studios = await studioRepository.search().where('established').not.onOrAfter(date).return.all()
studios = await studioRepository.search().where('established').between(date, laterDate).return.all()
studios = await studioRepository.search().where('established').not.between(date, laterDate).return.all()

More fluent variations work too:

const date = new Date('2010-12-27')
const laterDate = new Date('2020-12-27')

studios = await studioRepository.search().where('established').is.on(date).return.all()
studios = await studioRepository.search().where('established').is.not.on(date).return.all()

studios = await studioRepository.search().where('established').is.before(date).return.all()
studios = await studioRepository.search().where('established').is.not.before(date).return.all()

studios = await studioRepository.search().where('established').is.onOrBefore(date).return.all()
studios = await studioRepository.search().where('established').is.not.onOrBefore(date).return.all()

studios = await studioRepository.search().where('established').is.after(date).return.all()
studios = await studioRepository.search().where('established').is.not.after(date).return.all()

studios = await studioRepository.search().where('established').is.onOrAfter(date).return.all()
studios = await studioRepository.search().where('established').is.not.onOrAfter(date).return.all()

studios = await studioRepository.search().where('established').is.between(date, laterDate).return.all()
studios = await studioRepository.search().where('established').is.not.between(date, laterDate).return.all()

And, since dates are really just numbers, all the numeric comparisons work too:

const date = new Date('2010-12-27')
const laterDate = new Date('2020-12-27')

studios = await studioRepository.search().where('established').eq(date).return.all()
studios = await studioRepository.search().where('established').not.eq(date).return.all()
studios = await studioRepository.search().where('established').equals(date).return.all()
studios = await studioRepository.search().where('established').does.equal(date).return.all()
studios = await studioRepository.search().where('established').does.not.equal(date).return.all()
studios = await studioRepository.search().where('established').is.equalTo(date).return.all()
studios = await studioRepository.search().where('established').is.not.equalTo(date).return.all()

studios = await studioRepository.search().where('established').gt(date).return.all()
studios = await studioRepository.search().where('established').not.gt(date).return.all()
studios = await studioRepository.search().where('established').greaterThan(date).return.all()
studios = await studioRepository.search().where('established').is.greaterThan(date).return.all()
studios = await studioRepository.search().where('established').is.not.greaterThan(date).return.all()

studios = await studioRepository.search().where('established').gte(date).return.all()
studios = await studioRepository.search().where('established').not.gte(date).return.all()
studios = await studioRepository.search().where('established').greaterThanOrEqualTo(date).return.all()
studios = await studioRepository.search().where('established').is.greaterThanOrEqualTo(date).return.all()
studios = await studioRepository.search().where('established').is.not.greaterThanOrEqualTo(date).return.all()

studios = await studioRepository.search().where('established').lt(date).return.all()
studios = await studioRepository.search().where('established').not.lt(date).return.all()
studios = await studioRepository.search().where('established').lessThan(date).return.all()
studios = await studioRepository.search().where('established').is.lessThan(date).return.all()
studios = await studioRepository.search().where('established').is.not.lessThan(date).return.all()

studios = await studioRepository.search().where('established').lte(date).return.all()
studios = await studioRepository.search().where('established').not.lte(date).return.all()
studios = await studioRepository.search().where('established').lessThanOrEqualTo(date).return.all()
studios = await studioRepository.search().where('established').is.lessThanOrEqualTo(date).return.all()
studios = await studioRepository.search().where('established').is.not.lessThanOrEqualTo(date).return.all()

Searching String Arrays

If you have a field type of string[] you can search for whole strings that are in that array:

let albums

// find all albums where genres contains the string 'rock'
albums = await albumRepository.search().where('genres').contain('rock').return.all()

// find all albums where genres contains the string 'rock', 'metal', or 'blues'
albums = await albumRepository.search().where('genres').containOneOf('rock', 'metal', 'blues').return.all()

// find all albums where genres does *not* contain the string 'rock'
albums = await albumRepository.search().where('genres').not.contain('rock').return.all()

// find all albums where genres does *not* contain the string 'rock', 'metal', and 'blues'
albums = await albumRepository.search().where('genres').not.containOneOf('rock', 'metal', 'blues').return.all()

// alternative syntaxes
albums = await albumRepository.search().where('genres').contains('rock').return.all()
albums = await albumRepository.search().where('genres').containsOneOf('rock', 'metal', 'blues').return.all()
albums = await albumRepository.search().where('genres').does.contain('rock').return.all()
albums = await albumRepository.search().where('genres').does.not.contain('rock').return.all()
albums = await albumRepository.search().where('genres').does.containOneOf('rock', 'metal', 'blues').return.all()
albums = await albumRepository.search().where('genres').does.not.containOneOf('rock', 'metal', 'blues').return.all()

Wildcards work here too:

albums = await albumRepository.search().where('genres').contain('*rock*').return.all()

Searching Arrays of Numbers

If you have a field of type number[], you can search on it just like a number. If any number in the array matches your criteria, then it'll match and the document will be returned.

let albums

// find all albums where at least one song is at least 3 minutes long
albums = await albumRepository.search().where('songDuration').gte(180).return.all()

// find all albums where at least one song is at exactly 3 minutes long
albums = await albumRepository.search().where('songDuration').eq(180).return.all()

// find all albums where at least one song is between 3 and 4 minutes long
albums = await albumRepository.search().where('songDuration').between(180, 240).return.all()

I'm not going to include all the examples again. Just go check out the section on searching on numbers.

Full-Text Search

If you've defined a field with a type of text in your schema, you can store text in it and perform full-text searches against it. Full-text search is different from how a string is searched. With full-text search, you can look for words, partial words, fuzzy matches, and exact phrases within a body of text.

Full-text search is optimized for human-readable text and it's pretty clever. It understands that certain words (like a, an, or the) are common and ignores them. It understands how words relate to each other and so if you search for give, it matches gives, given, giving, and gave too. It ignores punctuation and whitespace.

Here are some examples of doing full-text search against some album titles:

let albums

// finds all albums where the title contains the word 'butterfly'
albums = await albumRepository.search().where('title').match('butterfly').return.all()

// finds all albums using fuzzy matching where the title contains a word which is within 3 Levenshtein distance of the word 'buterfly'
albums = await albumRepository.search().where('title').match('buterfly', { fuzzyMatching: true, levenshteinDistance: 3 }).return.all()

// finds all albums where the title contains the words 'beautiful' and 'children'
albums = await albumRepository.search().where('title').match('beautiful children').return.all()

// finds all albums where the title contains the exact phrase 'beautiful stories'
albums = await albumRepository.search().where('title').matchExact('beautiful stories').return.all()

If you want to search for a part of a word. To do it, just tack a * on the beginning or end (or both) of your partial word and it'll match accordingly:

// finds all albums where the title contains a word that contains 'right'
albums = await albumRepository.search().where('title').match('*right*').return.all()

Do not combine partial-word searches or fuzzy matches with exact matches. Partial-word searches and fuzzy matches with exact matches are not compatible in RediSearch. If you try to exactly match a partial-word search or fuzzy match a partial-word search, you'll get an error.

// THESE WILL ERROR
albums = await albumRepository.search().where('title').matchExact('beautiful sto*').return.all()
albums = await albumRepository.search().where('title').matchExact('*buterfly', { fuzzyMatching: true, levenshteinDistance: 3 }).return.all()

As always, there are several alternatives to make this a bit more fluent and, of course, negation is available:

albums = await albumRepository.search().where('title').not.match('butterfly').return.all()
albums = await albumRepository.search().where('title').matches('butterfly').return.all()
albums = await albumRepository.search().where('title').does.match('butterfly').return.all()
albums = await albumRepository.search().where('title').does.not.match('butterfly').return.all()

albums = await albumRepository.search().where('title').exact.match('beautiful stories').return.all()
albums = await albumRepository.search().where('title').not.exact.match('beautiful stories').return.all()
albums = await albumRepository.search().where('title').exactly.matches('beautiful stories').return.all()
albums = await albumRepository.search().where('title').does.exactly.match('beautiful stories').return.all()
albums = await albumRepository.search().where('title').does.not.exactly.match('beautiful stories').return.all()

albums = await albumRepository.search().where('title').not.matchExact('beautiful stories').return.all()
albums = await albumRepository.search().where('title').matchesExactly('beautiful stories').return.all()
albums = await albumRepository.search().where('title').does.matchExactly('beautiful stories').return.all()
albums = await albumRepository.search().where('title').does.not.matchExactly('beautiful stories').return.all()

Searching on Points

RediSearch, and therefore Redis OM, both support searching by geographic location. You specify a point in the globe and a radius and it'll gleefully return all the entities within that radius:

let studios

// finds all the studios with 50 miles of downtown Cleveland
studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).radius(50).miles).return.all()

Note that coordinates are specified with the longitude first, and then the latitude. This might be the opposite of what you expect but is consistent with how Redis implements coordinates in RediSearch and with GeoSets.

If you don't want to rely on argument order, you can also specify longitude and latitude more explicitly:

// finds all the studios within 50 miles of downtown Cleveland using a point
studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin({ longitude: -81.7758995, latitude: 41.4976393 }).radius(50).miles).return.all()

// finds all the studios within 50 miles of downtown Cleveland using longitude and latitude
studios = await studioRepository.search().where('location').inRadius(
  circle => circle.longitude(-81.7758995).latitude(41.4976393).radius(50).miles).return.all()

Radius can be in miles, feet, kilometers, and meters in all the spelling variations you could ever want:

// finds all the studios within 50 miles
studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).radius(50).miles).return.all()

studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).radius(50).mile).return.all()

studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).radius(50).mi).return.all()

// finds all the studios within 50 feet
studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).radius(50).feet).return.all()

studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).radius(50).foot).return.all()

studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).radius(50).ft).return.all()

// finds all the studios within 50 kilometers
studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).radius(50).kilometers).return.all()

studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).radius(50).kilometer).return.all()

studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).radius(50).km).return.all()

// finds all the studios within 50 meters
studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).radius(50).meters).return.all()

studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).radius(50).meter).return.all()

studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).radius(50).m).return.all()

If you don't specify the origin, Redis OM will use a longitude 0.0 and a latitude 0.0, also known as Null Island:

// finds all the studios within 50 miles of Null Island (probably ain't much there)
studios = await studioRepository.search().where('location').inRadius(
  circle => circle.radius(50).miles).return.all()

If you don't specify the radius, it defaults to 1 and if you don't provide units, it defaults to meters:

// finds all the studios within 1 meter of downtown Cleveland
studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393)).return.all()

// finds all the studios within 1 kilometer of downtown Cleveland
studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).kilometers).return.all()

// finds all the studios within 50 meters of downtown Cleveland
studios = await studioRepository.search().where('location').inRadius(
  circle => circle.origin(-81.7758995, 41.4976393).radius(50)).return.all()

And there are plenty of fluent variations to help make your code pretty:

studios = await studioRepository.search().where('location').not.inRadius(
  circle => circle.longitude(-81.7758995).latitude(41.4976393).radius(50).miles).return.all()

studios = await studioRepository.search().where('location').is.inRadius(
  circle => circle.longitude(-81.7758995).latitude(41.4976393).radius(50).miles).return.all()

studios = await studioRepository.search().where('location').is.not.inRadius(
  circle => circle.longitude(-81.7758995).latitude(41.4976393).radius(50).miles).return.all()

studios = await studioRepository.search().where('location').not.inCircle(
  circle => circle.longitude(-81.7758995).latitude(41.4976393).radius(50).miles).return.all()

studios = await studioRepository.search().where('location').is.inCircle(
  circle => circle.longitude(-81.7758995).latitude(41.4976393).radius(50).miles).return.all()

studios = await studioRepository.search().where('location').is.not.inCircle(
  circle => circle.longitude(-81.7758995).latitude(41.4976393).radius(50).miles).return.all()

Chaining Searches

So far we've been doing searches that match on a single field. However, we often want to query on multiple fields. Not a problem:

const albums = await albumRepository.search()
  .where('artist').equals('Mushroomhead')
  .or('title').matches('butterfly')
  .and('year').is.greaterThan(1990).return.all()

These are executed in order from left to right, and ignore any order of operations. So this query will match an artist of "Mushroomhead" OR a title matching "butterfly" before it goes on to match that the year is greater than 1990.

If you'd like to change this you can nest your queries:

const albums = await albumRepository.search()
  .where('title').matches('butterfly').return.all()
  .or(search => search
    .where('artist').equals('Mushroomhead')
    .and('year').is.greaterThan(1990)
  ).return.all()

This query finds all Mushroomhead albums after 1990 or albums that have "butterfly" in the title.

Running Raw Searches

The fluent search interface is nice, but sometimes you need to do something just a bit more. If you want, you can execute a search against your entities using the native RediSearch query syntax. I'm not going to explain the syntax here as it's a bit involved, but you can read it for yourself in the RediSearch documentation.

To execute a raw search, just call .searchRaw on the repository with your query:

// finds all the Mushroomhead albums with the word 'beautiful' in the title from 1990 and beyond
const query = "@artist:{Mushroomhead} @title:beautiful @year:[1990 +inf]"
const albums = albumRepository.searchRaw(query).return.all();

The nice thing here is that it returns the same entities that you've been using for everything else. It's just a lower-level way of executing a query for when you need that extra bit of power.

Sorting Search Results

RediSearch provides a basic mechanism for sorting your search results and Redis OM exposes it. You can sort on a single field and can sort on the following types: string, number, boolean, date, and text. To sort, simply call .sortBy, .sortAscending, or .sortDescending:

const albumsByYear = await albumRepository.search()
  .where('artist').equals('Mushroomhead')
    .sortAscending('year').return.all()

const albumsByTitle = await albumRepository.search()
  .where('artist').equals('Mushroomhead')
    .sortBy('title', 'DESC').return.all()

You can also tell RediSearch to preload the sorting index to improve performance when you sort. This doesn't work with all of the types that you can sort by, but it's still pretty useful. To preload the index, mark the field in the Schema with the sortable property:

const albumSchema = new Schema(Album, {
  artist: { type: 'string' },
  title: { type: 'text', sortable: true },
  year: { type: 'number', sortable: true },
  genres: { type: 'string[]' },
  outOfPublication: { type: 'boolean' }
})

If your schema is for a JSON data structure (the default), you can mark number, date, and text fields as sortable. You can also mark string and boolean fields as sortable, but this will have no effect and will generate a warning.

If your schema is for a Hash, you can mark string, number, boolean, date, and text fields as sortable.

Fields of the types point and string[] are never sortable.

If this seems like a confusing flowchart to parse, don't worry. If you call .sortBy on a field in the Schema that's not marked as sortable and it could be, Redis OM will log a warning to let you know.

Advanced Stuff

This is a bit of a catch-all for some of the more advanced stuff you can do with Redis OM.

Schema Options

Additional field options can be set depending on the field type. These correspond to the Field Options available when creating a RediSearch full-text index. Other than the separator option, these only affect how content is indexed and searched.

schema type RediSearch type indexed sortable normalized stemming phonetic weight separator caseSensitive
string TAG yes HASH Only HASH Only - - - yes yes
number NUMERIC yes yes - - - - - -
boolean TAG yes HASH Only - - - - - -
string[] TAG yes HASH Only HASH Only - - - yes yes
number[] NUMERIC yes yes - - - - - -
date NUMERIC yes yes - - - - -
point GEO yes - - - - - -
text TEXT yes yes yes yes yes yes - -
  • indexed: true | false, whether this field is indexed by RediSearch (default true)
  • sortable: true | false, whether to create an additional index to optimize sorting (default false)
  • normalized: true | false, whether to apply normalization for sorting (default true)
  • matcher: string defining phonetic matcher which can be one of: 'dm:en' for English, 'dm:fr' for French, 'dm:pt' for Portugese, 'dm:es' for Spanish (default none)
  • stemming: true | false, whether word-stemming is applied to text fields (default true)
  • weight: number, the importance weighting to use when ranking results (default 1)
  • separator: string, the character to delimit multiple tags (default '|')
  • caseSensitive: true | false, whether original letter casing is kept for search (default false)

Example showing additional options:

const commentSchema = new Schema(Comment, {
  name: { type: 'text', stemming: false, matcher: 'dm:en' },
  email: { type: 'string', normalized: false, },
  posted: { type: 'date', sortable: true },
  title: { type: 'text', weight: 2 },
  comment: { type: 'text', weight: 1 },
  approved: { type: 'boolean', indexed: false },
  iphash: { type: 'string', caseSensitive: true },
  notes: { type: 'string', indexed: false },
})

There are several other options available when defining a schema for your entity. Check them out in the detailed documentation for the Schema class.

Documentation

This README is pretty extensive, but if you want to check out every last corner of Redis OM for Node.js, take a look at the complete API documentation.

Troubleshooting

I'll eventually have a FAQ full of answered questions, but since this is a new library, nobody has asked anything yet, frequently or otherwise. So, if you run into a problem, open an issue. Even cooler, dive into the code and send a pull request. If you just want to ping somebody, hit me up on the Redis Discord server.

Contributing

Contributions are always appreciated. I take PayPal and Bitcoin. Just kidding, I would sincerely appreciate your help in making this software better. Here are a couple of ways to help:

  • Bug reports: This is a new project. You're gonna find them. Open an issue and I'll look into it. Or hunt down the problem and send me a pull request.
  • Documentation: You can improve the life of a lot of developers by fixing typos, grammar, and bad jokes. Or by just pointing out where a little more detail would help. Again, open an issue or send a pull request.

redis-om-node's People

Contributors

abwinkler999 avatar antonio592021 avatar berviantoleo avatar brekim avatar captaincodeman avatar chayim avatar cm-ayf avatar colemilne54 avatar danipatko avatar didas-git avatar garrett-green avatar guyroyse avatar hostedposted avatar lionardo avatar lucemans avatar neeraj-ghodla avatar omgmanu avatar pogiii avatar slinkypotato avatar vladimirchuprazov avatar vsarang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

redis-om-node's Issues

Add enum as a new type

I know the library is in active development right now but it would be really nice to have it ASAP. Thanks for your code :)

Add JSON serialization to Entity

Implement a generic .toJSON method on the Entity class so that JSON.Serialize will return a JSON string containing the attributes specified in the Schema. Currently, it returns the internal implementation of the Entity.

Fetch by ids

Hello, how can I fetch entities by ids?

sth like

const albumsEntities = await albumRepository.fetch([albumId1,  albumId2 ...])

// or 

const albumsEntities = await albumRepository.fetch(albumId1,  albumId2 ...)

matchExact methods break redis-parser when common search term automatically ignored by redis-search

As noted on the README.md Full Text Search, "Full-text search is pretty clever. It understands that certain words (like a, an, or the) are common and ignores them." If the resulting search term reduces the term-count to one or fewer, redis-parser will throw an error.

Example: await repository.search().where("MyData").matchExact("to Home").count();

redis-search will remove "to", because it is common, leaving just "Home." This only leaves one search term when redis-parser is expecting 2 or more.

UnhandledPromiseRejectionWarning: ReplyError: Syntax error at offset 9 near to
    at parseError (/<test>/node_modules/redis-parser/lib/parser.js:179:12)
    at parseType (/<test>/node_modules/redis-parser/lib/parser.js:302:14)
    at emitUnhandledRejectionWarning (internal/process/promises.js:168:15)
    at processPromiseRejections (internal/process/promises.js:247:11)
    at processTicksAndRejections (internal/process/task_queues.js:96:32)
(node:25411) ReplyError: Syntax error at offset 9 near to
    at parseError (/<test>/node_modules/redis-parser/lib/parser.js:179:12)
    at parseType (/<test>/node_modules/redis-parser/lib/parser.js:302:14)
(node:25411) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
    at emitDeprecationWarning (internal/process/promises.js:180:11)
    at processPromiseRejections (internal/process/promises.js:249:13)
    at processTicksAndRejections (internal/process/task_queues.js:96:32)

This appears to affect all phrase methods in full text search. These methods should probably default to .match() if the filtered search terms are fewer than two. Thanks.

EDIT: I guess, if the desired result, in my example, is the phrase "to Home", just providing results for "Home" would not be valid. So maybe there's a way to force redis-search to not ignore in these cases.

redis-parser: 3.0.0
redis-om-node: 0.1.3

Add ability to save from Entity

Right now you have to pass in an Entity to the Repository. It would be great to be able to call .save right on the Entity. This pairs nicely with issue #1.

Define own entityId

Awesome package this is! ❤

Is there a way to define an own entityId?

I see there is a methode generateId()
that is automatically called on Schema construction.

I would like to pass an existing generated id from an auth service or something similar.

Add number[] as a new type

Currently blocked, as RediSearch does not allow arrays of numbers for TAG fields. Add this as soon as it's an option.

Unique value for field

const schema = new Schema(MyEntity, {
  token: { type: 'string' },
  status: { type: 'boolean' },
  date_expire: { type: 'date' },
});

how can i specifie that token field is unique ?

Easier assignment of initial Entity data

Currently, you must assign each entity property manually:

    let album = repository.createEntity()
    album.artist = "Mushroomhead"

However, it would be nice to create this initial data with an existing object (like a request body). One potential solution is an optional arg to the createEntity method:

    let data = { artist:  "Mushroomhead" }
    let album = repository.createEntity(data)

Remove set of items

Hi!

I noticed that .remove() method uses node-redis .unlink() under the hood, but currently can remove only one key.

Redis UNLINK can accept many keys to remove. Just checked the node-redis .unlink() and it accepted array of keys and removed them properly. Would be nice to have this in redis-om-node!

For example:

repository.remove('id1');

repository.remove(['id1', 'id2', ...]);

Possibly related to #65

Add support for Schema type array of objects

Currently the Schema only supports the following types: number, string, boolean, string[]
Redis also supports keys having an array of objects.

However, redis-om doesn't work with that, it stores it inside redis as shown in the image below.

image

Add weighted text searches

Right now, all text searches are considered equal. But RediSearch allows for weighted searches. Take advantage of that during schema creation by adding a weight option to the schema for text fields.

ReplyError: ERR unknown command `FT.CREATE`

Getting the following error when trying to create an index for using search/count functionality:

ReplyError: ERR unknown command `FT.CREATE`, with args beginning with: `Album:index`, `ON`, `JSON`, `PREFIX`, `1`, `Album:`, `SCHEMA`, `$.artist`, `AS`, `artist`, `TAG`, `SEPARATOR`, `|`, `$.title`

When trying to await repository.createIndex() with the following code. Client is able to open/connect as responds to ['PING']. Trying to see if missing anything obvious with creating a repository and creating an index with json and search module enabled.

docker-compose.yml (copied from https://github.com/redis/redis-om-node/blob/main/docker/docker-compose.yml):

version: '3.9'
services:
  redis:
    image: 'redislabs/redismod:preview'
    ports:
      - '6379:6379'
    volumes:
      - ./data:/data
    entrypoint: >
      redis-server
        --loadmodule /usr/lib/redis/modules/redisai.so
          ONNX redisai_onnxruntime/redisai_onnxruntime.so
          TF redisai_tensorflow/redisai_tensorflow.so
          TFLITE redisai_tflite/redisai_tflite.so
          TORCH redisai_torch/redisai_torch.so   
        --loadmodule /usr/lib/redis/modules/redisearch.so
        --loadmodule /usr/lib/redis/modules/redisgraph.so
        --loadmodule /usr/lib/redis/modules/redistimeseries.so
        --loadmodule /usr/lib/redis/modules/rejson.so
        --loadmodule /usr/lib/redis/modules/redisbloom.so
        --loadmodule /usr/lib/redis/modules/redisgears.so
    deploy:
      replicas: 1
      restart_policy:
        condition: on-failure

module:

const { Client, Entity, Repository, Schema } = require('redis-om');

class Album extends Entity {}

const schema = new Schema(
  Album,
  {
    artist: { type: 'string' },
    title: { type: 'string' },
    year: { type: 'number' },
    genres: { type: 'array' },
    outOfPublication: { type: 'boolean' },
  },
  {
    dataStructure: 'JSON',
  },
);

(async function() {
  const client = new Client();
  await client.open();
  console.log(await client.execute(['PING'])); // PONG
  const repository = new Repository(schema, client);
  await repository.createIndex();
})();

TypeError: Cannot read properties of undefined (reading 'replace')

Where was the mistake made?

// redis.js

export async function searchCars(q) {
    await connect();
    const repository = new Repository(schema, client);
    console.log(repository)
    let cars = await repository.search()
        .where('make').eq(q)
        .or('model').eq(q)
        .or('description').matches(q)
        .return.all();

    return cars;
}

// search.js

export default async function handler(req, res) {
  const q = req.query.q;
  const cars = await searchCars(q);
  res.status(200).json({ cars });
}

Implement implicit transaction and locking in the context of redis-om for master-detail relations

I am requesting a new feature to implement implicit transaction and locking in the context of redis-om for master-detail relations. Please see the code snippet below that illustrates the desired behaviour with enhanced schema definitions.

let albumSchema = new Schema(Album, {
  artist: { type: 'string' },
  title: { type: 'text' },
  year: { type: 'number' },
  genres: { type: 'string[]' },
  outOfPublication: { type: 'boolean' },
  studioId: { entity: 'Studio' } // 1-to-1 relation
})

let studioSchema = new Schema(Studio, {
  name: { type: 'string' },
  city: { type: 'string' },
  state: { type: 'string' },
  location: { type: 'point' },
  established: { type: 'date' },
  albums: { collection: 'Album', via: 'studioId' } // 1-to-m relation
})

let albumRepository = client.fetchRepository(albumSchema)
let studioRepository = client.fetchRepository(studioSchema)

// 1. 1-to-1 relation data persistence
let album = albumRepository.createEntity({
  artist: "Mushroomhead",
  title: "The Righteous & The Butterfly",
  year: 2014,
  genres: [ 'metal' ],
  outOfPublication: true
})

let studio = {
  name: "Bad Racket Recording Studio",
  city: "Cleveland",
  state: "Ohio",
  location: { longitude: -81.6764187, latitude: 41.5080462 },
  established: new Date('2010-12-27')
}

// Implement implicit transaction and locking
const { id, studioId } = albumRepository.save(album, { entity: 'Studio', data: studio })

// 2. 1-to-m relation data persistence
let albums = [{
  artist: "Mushroomhead",
  title: "Beautiful Stories for Ugly Children",
  year: 2010,
  genres: [ 'metal' ],
  outOfPublication: true
}, {
  artist: "Mushroomhead",
  title: "The Righteous & The Butterfly",
  year: 2014,
  genres: [ 'metal' ],
  outOfPublication: true
}]


let studio = studioRepository.createEntity({
  name: "Bad Racket Recording Studio",
  city: "Cleveland",
  state: "Ohio",
  location: { longitude: -81.6764187, latitude: 41.5080462 },
  established: new Date('2010-12-27')
})

// Implement implicit transaction and locking
const { id, albumIds } = studioRepository.save(studio, { collection: 'Album', data: albums })

Save JSON's array into JSON

Hi there, I am not really experienced with redis or schemas and stuff and this is what I wanted to use:

Schema:

{
ID: string,
warns: [
{ reason: "Something", id: "1" },
{ reason: "something else", id: "2" }
]
}

How can I do this? I am really confused. Thanks!

Implement missing `contain()` and `contains()` search functions

First issue. First of, awesome job 👍🏼

Currently the WhereString class does not have an implementation of the abstract contain and contains functions declared in the WhereField class. Implementation of the missing "contain(s)" WhereField search functions in WhereString is required to return partial text matching.

I'm happy to create a PR for this if that's ok?

Improved error handling when opening the Client

When opening the client, it will throw an error if an existing client is open.

this.validateShimClosed();

It may be better if this error is removed or just logged as a warning. OR the client should implement a public boolean property like is isOpen so the end user can do a check at runtime. Currently, it seems impossible to know if the client is open or not.

Unique Cache Keys

This is a bit of a contrived example but given the following schema how would you suggest to maintain unique entities?

class BlogPost extends Entity {}

const schema = new Schema(BlogPost, {
  id: { type: 'number' },
  subject: { type: 'string' },
  body: { type: 'string' }
});

If my server is handling two concurrent requests and both have a cache miss initially, then ideally I only want to save the blog posts to the cache for one of the requests. But given that the IdStrategy is currently using ulid, I don't see how you'd do something like this.

Normally I'd have posted this question to the GitHub discussions but I noticed that wasn't enabled yet.

How does Repository#save work?

It is stated in the docs that You also use .save to update an existing entity, but how does it know if there is an existing entity?

Handle error on reconnect to Redis

Node Redis can automatically reconnect to Redis if the connection is dropped. However, it raises an error when this happens. Redis OM is not currently handling that error and will throw an exception if the connection is dropped.

Found with help from @MrScopes.

Transactions & relations

First of all, let me thank you for this cool library :)
Now, I'm really considering using Redis Enterprise as a primary DB in the future projects!

So, my question arises from this section in the readme https://github.com/redis/redis-om-node#-embedding-your-own-logic-into-entities
We can fetch "relations" in our entities, but I'm more interested in updating those relations as a part of a transaction. I can see that currently, saving an entity requires you to call repository.save(entity), but what about related entries? What if I need to update an entity, some sub-entity and commit or discard all of the changes?

It would be cool to have something like

getCurrentClient().startTransaction(() => {
  repository1.save(entity);
  repository2.save(subentity);
}).then(...).catch(...)

help/bug: save seems to override not merge

Not sure if I'm just not understanding something or if I'm hitting a bug but any time I fetch a GuildSettings, update it and save I lose all the existing data.

import { Entity, Schema } from 'redis-om';
import { client } from '../common/redis.js';

export interface GuildSettings {
    'auditLog.enabled': boolean;
    'auditLog.channel': string;
    'auditLog.channels': string[];
    'moderation.enabled': boolean;
    'moderation.reporting.enabled': boolean;
    'moderation.reporting.channel': string;
}

export class GuildSettings extends Entity {
    /**
     * Deletes the guild settings from the database
     */
    async delete() {
        await guildSettingsRepository.remove(this.entityId);
    }

    /**
     * Persists the guild settings to the database
     */
    async save() {
        await guildSettingsRepository.save(this);
    }
};

export const guildSettingsSchema = new Schema(GuildSettings, {
    'auditLog.enabled': { type: 'boolean' },
    'auditLog.channel': { type: 'string' },
    'auditLog.channels': { type: 'string[]' },
    'moderation.enabled': { type: 'boolean' },
    'moderation.reporting.enabled': { type: 'boolean' },
    'moderation.reporting.channel': { type: 'string' }
}, {
    dataStructure: 'JSON'
});

export const guildSettingsRepository = client.fetchRepository(guildSettingsSchema);

await guildSettingsRepository.createIndex();
// Get the guild settings
const guildSettings = await guildSettingsRepository.fetch(interaction.guild.id);

// Update moderation settings
guildSettings['moderation.enabled'] = enabled ?? guildSettings['auditLog.enabled'];
guildSettings['moderation.reporting.enabled'] = reportingEnabled ?? guildSettings['moderation.reporting.enabled'];
guildSettings['moderation.reporting.channel'] = reportingChannel?.id ?? guildSettings['moderation.reporting.channel'];

// Save update to database
await guildSettings.save();

If auditLog.enabled was set as true before it'll now be missing in redis.

Clustering Support

Right now, the Redis connection string does not allow connecting to Redis clusters. Add that support.

Use reflection to do object mapping for a more declarative API.

Currently using the schema in ts looks like this. This feels like 2x the code necessary since you are declaring the schema twice.
With reflection the API would be more declarative and make this code much smaller and easier to understand.

Without reflection:

interface Album {
  artist: string;
  title: string;
  year: number;
  genres: string[];
  outOfPublication: boolean;
}

class Album extends Entity {}

let schema = new Schema(Album, {
  artist: { type: 'string' },
  title: { type: 'string' },
  year: { type: 'number' },
  genres: { type: 'array' },
  outOfPublication: { type: 'boolean' }
})

With reflection:

@Entity()
class Album {
  @Property()
  artist: string;
  @Property()
  title: string;
  @Property()
  year: string;
  @Property()
  genres: string[];
  @Property()
  outOfPublication: boolean;
}

let schema = new Schema(Album);

This is just an example but demonstrates how declarative and powerful reflection can make this api.

.createIndex() required 1 parameters but got none?

Hi there, the documentation doesn't mention anything needed to be added to .createIndex() function and this is my code:

import RedisClient from '@node-redis/client/dist/lib/client';
import { Client } from 'redis-om'

let redisClient: Client;
(async () => {

    /* pulls the Redis URL from .env */
    const url = process.env.REDIS_URL

    /* create and open the Redis OM Client */
    const rc = new Client()



    const redisClient2 = await rc.open(url)

    await redisClient2.createIndex()
    
    redisClient = redisClient2

})();

export default redisClient

The error I get:
Screenshot 2022-04-24 at 9 59 15 AM

Add support for GEO fields

RediSearch supports GEO fields. Add a geo type to the Schema so this feature can be more easily used.

Find all not working

const client = new Client();
await client.open('redis://localhost:6379');

let tokenAPiRepository = client.fetchRepository(tokenSchema);
return await tokenAPiRepository.search().return.all();

i get this error:
ERR unknown command `FT.SEARCH`, with args beginning with: `TokenApi:index`, `*`, `LIMIT`, `0`, `10`,

any idea to fixe that ?

point fields can be set to invalid coordinates

Right now, points can be set to coordinates that are invalid or that don't makes sense. Latitude on a globe goes from -90 to +90 and longitude from -180 to +180 but Redis OM will allow you to set it beyond these limits. This results in indexing errors in RediSearch and those documents never being returned in search results. Redis OM needs to generate an error if you try to set a point to an invalid value or if you try to search by it with an invalid value.

Add Sort capability

RediSearch has lots of sorting capabilities. Develop features around that.

Vector Similarity Search

I've been experimenting with the Vector similarity feature and have some uses for it (with both text and images) so plan on adding support when I have chance.

Early thoughts are to have a new field type of vector that would accept one of two similarity objects containing the parameters for each type (FLAT vs NHSW)

The actual JavaScript object datatype would be an Uint8Array or Float32Array and the lib would handle encoding for Redis.

It's likely that some additional search options would also be required.

Filed as EntityI ID

There is a way to set Fields as Entity ID?

import { Inject, Injectable } from '@nestjs/common'
import { Client, Entity, EntityData, Schema } from 'redis-om'
import { JsonRepository } from 'redis-om/dist/repository/repository'

class Game extends Entity {}

const schema = new Schema(
  Game,
  {
    id: { type: 'string' },
    name: { type: 'string' },
    actualRound: { type: 'number' },
  },
  // It will be great to have sth like this:
  // { idField: 'id' }
)

@Injectable()
export class GamesRepository extends JsonRepository<Game> {
  constructor(@Inject('STORAGE_CONNECTION') client: Client) {
    super(schema, client)
  }

  protected async writeEntity(key: string, data: EntityData): Promise<void> {
    const { id } = data
    await this.client.jsonset(`Game:${id}`, data)
  }
}

For now, I just override the writeEntity method.

Btw. I love your project, GJ <3

RedisError: The query to RediSearch had a syntax error: "Syntax error at offset 9 near messagesPerWindow".

RedisError: The query to RediSearch had a syntax error: "Syntax error at offset 9 near messagesPerWindow".
This is often the result of using a stop word in the query. Either change the query to not use a stop word or change the stop words in the schema definition. You can check the RediSearch source for the default stop words at: https://github.com/RediSearch/RediSearch/blob/master/src/stopwords.h.

I'm getting the error above when searching for an entitiy where levels.messagesPerWindow is greater than or equal to 1.

I looked through the stop words and I'm not sure what part of this is the issue. The only thing I can think of is that a is in the stop words list and I use it in messages but that seems unlikely.

This is the code I'm using.

import autoBind from 'auto-bind';
import { Entity, EntityData, Schema } from 'redis-om';
import { client } from '../../../common/redis.js';

export interface Member {
    id: string | null;
    guildId: string | null;
    'levels.xp': number | null;
    'levels.level': number | null;
    'levels.messagesPerWindow': number | null;
    'stats.messageCount': number | null;
    'stats.reactionCount': number | null;
    'stats.voiceMinutes': number | null;
}

export class Member extends Entity {
    constructor(schema: Schema<any>, id: string, data?: EntityData | undefined) {
        super(schema, id, data);
        autoBind(this);
    }

    /**
     * Deletes the member from the database
     */
    async delete() {
        await memberRepository.remove(this.entityId);
    }

    /**
     * Persists the member to the database
     */
    async save() {
        await memberRepository.save(this);
    }
};

export const memberSchema = new Schema(Member, {
    id: { type: 'number' },
    guildId: { type: 'number' },
    'levels.xp': { type: 'number' },
    'levels.level': { type: 'number' },
    'levels.messagesPerWindow': { type: 'number' },
    'stats.messageCount': { type: 'number' },
    'stats.reactionCount': { type: 'number' },
    'stats.voiceMinutes': { type: 'number' }
}, {
    dataStructure: 'HASH'
});

export const memberRepository = client.fetchRepository(memberSchema);

await memberRepository.createIndex();

The following is what throws the error.
If I try and do a search without the where clause it returns all members without any issue.

const members = await memberRepository.search().where('levels.messagesPerWindow').greaterThan(1).return.all();

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.