datanoise / mongo.cr Goto Github PK

View Code? Open in Web Editor NEW

97.0 7.0 35.0 108 KB

Crystal binding for MongoDB C Driver

License: MIT License

Crystal 100.00%

mongo.cr's Introduction

mongo.cr

This library provides binding for MongoDB C Driver. The goal is to provide a driver to access MongoDB.

Status

Beta

Requirements

Crystal language version 0.20 and higher.
libmongoc version 1.1.0
libbson verion 1.1.0

On Mac OSX use homebrew to install the required libraries:

$ brew install mongo-c

On Linux you need to install libmongoc-1.1-0 and libbson-1.1-0 from your package manager or from source:

wget https://github.com/mongodb/mongo-c-driver/releases/download/1.1.0/mongo-c-driver-1.1.0.tar.gz
tar -zxvf mongo-c-driver-1.1.0.tar.gz && cd mongo-c-driver-1.1.0/
./configure --prefix=/usr --libdir=/usr/lib64
make
sudo make install

Installation

Add this to your application's shard.yml:

mongo:
  github: datanoise/mongo.cr
  branch: master

Usage

require "mongo"

client = Mongo::Client.new "mongodb://<user>:<password>@<host>:<port>/<db_name>"
db = client["db_name"]

collection = db["collection_name"]
collection.insert({ "name" => "James Bond", "age" => 37 })

collection.find({ "age" => { "$gt" => 30 } }) do |doc|
  puts typeof(doc)    # => BSON
  puts doc
end

License

MIT clause - see LICENSE for more details.

mongo.cr's People

Stargazers

Watchers

mongo.cr's Issues

Mongo.cr and the event loop

Does mongo.cr use the asynchronous event loop provided by Crystal, or will it block?

Would be nice to have a folder for some examples?

Hey, I was just wondering if anyone could possibly write an examples directory for this, so that we can actually see how to use this properly. There's seems to be a lot of little details getting in the way of using this effectively without a lot of mongoc lib knowledge. Also having to dig around in the codebase and guess what to do with a lot of the stuff is a bit of a nightmare.

Unable to build on Debian stretch

Environment: a Debian stretch up to date, with libmongoc-dev package

> crystal --version
Crystal 0.22.0 [3c71228] (2017-04-20) LLVM 3.5.0
> llvm-config-3.9 --version
3.9.1
> cat /etc/{issue,debian_version}
Debian GNU/Linux 9 \n \l

9.0
> dpkg -l libmongoc-dev
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                            Version                      Architecture                 Description
+++-===============================================-============================-============================-===================================================================================================
ii  libmongoc-dev                                   1.4.2-1+b1                   amd64                        MongoDB C client library - dev files

Code: an attempt to connect to a local Mongo database.

> crystal init app try_mongo
      create  try_mongo/.gitignore
      create  try_mongo/LICENSE
      create  try_mongo/README.md
      create  try_mongo/.travis.yml
      create  try_mongo/shard.yml
      create  try_mongo/src/try_mongo.cr
      create  try_mongo/src/try_mongo/version.cr
      create  try_mongo/spec/spec_helper.cr
      create  try_mongo/spec/try_mongo_spec.cr
Initialized empty Git repository in /path/to/try_mongo/.git/

Adding mongo.cr

> cd try_mongo
try_mongo> cat >> shard.yml 
dependencies:
  mongo:
    github: datanoise/mongo.cr
    branch: master
^D

try_mongo> shards install
Updating https://github.com/datanoise/mongo.cr.git
Installing mongo (master)

Add basic code to try db connect

try_mongo> cat > src/try_mongo.cr 
require "mongo"

client = Mongo::Client.new "mongodb://localhost"
db = client["test"]
collection = db["mongo_try"]
 
puts collection.count
^D

Attempt to build...

> crystal deps build
Dependencies are satisfied
Building: try_mongo
Error target try_mongo failed to compile:
/usr/bin/ld: /opt/crystal/embedded/lib/../lib/libgc.a(os_dep.o): undefined reference to symbol '_end'
//usr/lib/x86_64-linux-gnu/libyajl.so.2: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
Error: execution of command failed with code: 1: `cc -o "try_mongo/bin/try_mongo" "${@}"  -rdynamic  -lmongoc-1.0 -lbson-1.0 -lpcre -lgc -lpthread /opt/crystal/src/ext/libcrystal.a -levent -lrt -ldl -L/usr/lib -L/usr/local/lib`

Date Range Query

When trying to execute the following query

{ $and: [ { "created_at": { $gte: ISODate("2020-05-01T00:00:00.000+0000") } }, { "created_at": { $lt: ISODate("2020-06-01T00:00:00.000+0000") } } ] }

I've tried

collection.find{ $and: [ { "created_at": { $gte: ISODate("2020-05-01T00:00:00.000+0000") } }, { "created_at": { $lt: ISODate("2020-06-01T00:00:00.000+0000") } } ] }.each do |doc|
docs << doc
p doc
end

This generates the following error

Error: $global_variables are not supported, use @@class_variables instead

I've tried quoting the globals, e.g..

collection.find{ "$and": [ { "created_at": { "$gte": ISODate("2020-05-01T00:00:00.000+0000") } }, { "created_at": { "$lt": ISODate("2020-06-01T00:00:00.000+0000") } } ] }.each do |doc|
docs << doc
p doc
end

This generates

Error: unexpected token: :

Any suggestions?

Many thanks!

How to find all documents from collection

In ruby mongo driver to find all documents in collection is

collection.find

However, in this driver expect to have some arguments pass to it. Which argument should I pass to find method in order to get the ideal result ?

Structs can't have finalizers because they are not tracked by the GC

> crystal spec
Error in line 1: while requiring "./spec/bulk_operation_spec.cr"

in spec/bulk_operation_spec.cr:1: while requiring "../src/mongo"

require "../src/mongo"
^

in src/mongo.cr:2: while requiring "./bson"

require "./bson"
^

in src/bson.cr:3: while requiring "./bson/*"

require "./bson/*"
^

in src/bson/value.cr:14: structs can't have finalizers because they are not tracked by the GC

    def finalize
        ^~~~~~~~

From https://github.com/crystal-lang/crystal/blob/master/CHANGELOG.md#0204-06-01-2017

(breaking change) Defining a finalize method on a struct now gives a compile error

Because of memory leaks, see crystal-lang/crystal#3840

lib mongoc error: No such file there

Maybe someone could help me with this, I installed everything but it is not working anyway.

Error:

/home/daniel/.cache/crystal/crystal-run-database.tmp: error while loading shared libraries: libmongoc-1.0.so.0: cannot open shared object file: No such file or directory

Undefined method [] for Nil:Class during compilation

Getting a problem when trying to create an instance of a collection, I can't get past the compilation as I think the compiler is trying to account for the client instance possibly being 'Nil' and there being no method handlers for '[]' on the object which is very odd, as I have validation expressions outside the object. If someone could give a hand on why this is happening, that'd be great.

Implementation:

 class MxMongo                                                                                       
 -»»»# Initialise everything we need                                                                 
 -»»»def initialize(host : String, username : String, password : String, port : String)              
 -»»»-»»»@client = Nil                                                                               
 -»»»-»»»@database = "aggregation"                                                                   
 -»»»-»»»self.connect(host, username, password, port)                                                
 -»»»end                                                                                             
 -»»»                                                                                                
 -»»»# Connect to the database                                                                       
 -»»»def connect(host : String, username : String, password : String, port : String)                 
 -»»»-»»»@client = Mongo::Client.new "mongodb://#{username}:#{password}@#{host}:#{port}/#{@database}"
 -»»»-»»»return @client                                                                              
 -»»»end                                                                                             
                                                                                                     
 -»»»# Get an instance of the database                                                               
 -»»»def get_instance                                                                                
 -»»»-»»»if @client.not_nil! && db.not_nil!                                                          
 -»»»-»»»-»»»return @client                                                                          
 -»»»-»»»else                                                                                        
 -»»»-»»»-»»»return Nil                                                                              
 -»»»-»»»end                                                                                         
 -»»»end                                                                                             
                                                                                                     
 -»»»def get_collection(collection : String)                                                         
 -»»»-»»»if [email protected]? && !collection.nil?                                                        
 -»»»-»»»-»»»client = @client                                                                        
 -»»»-»»»-»»»return client.not_nil![collection] if collection.not_nil!                               
 -»»»-»»»else                                                                                        
 -»»»-»»»-»»»return Nil                                                                              
 -»»»-»»»end                                                                                         
 -»»»end                                                                                             
 end

Error I recieve:

                                                                                                                      
test = mongo_client.get_collection("aggregation")                                                                     
                    ^~~~~~~~~~~~~~                                                                                    
                                                                                                                      
in classes/init_mongo.cr:27: undefined method '[]' for Nil:Class (compile-time type is (Mongo::Client | Nil:Class))   
                                                                                                                      
   return client.not_nil![collection] if collection.not_nil!                                                          
                         ^                                                                                            
                                                                                                                      
================================================================================                                      
                                                                                                                      
Nil:Class trace:                                                                                                      
                                                                                                                      
  /usr/lib/crystal/object.cr:156                                                                                      
                                                                                                                      
      def not_nil!                                                                                                    
          ^~~~~~~~                                                                                                    
                                                                                                                      
  /usr/lib/crystal/object.cr:157                                                                                      
                                                                                                                      
        self                                                                                                          
        ^~~~

Apologies for the formatting, copied from vim

Not working with libmongoc 1.3 or 1.5(latest release)

Invalid memory access (signal 11) at address 0xa0
[4496293] *CallStack::print_backtrace:Int32 +117
[4468520] __crystal_sigfault_handler +56
[4875304] sigfault_handler +40
[139785657689056] ???
[139785662203494] ???
[139785662203690] mongoc_client_new +58
[4756622] *Mongo::Client#initialize:Nil +30
[4756561] *Mongo::Client::new:Mongo::Client +81
[4450987] ???

Separate the BSON lib

I think it would be best if it lived in another repo and then added to the deps block in Projectfile for this project.

Add support for Change Streams

Doc: https://docs.mongodb.com/manual/changeStreams/

Change streams allow applications to access real-time data changes without the complexity and risk of tailing the oplog.
Applications can use change streams to subscribe to all data changes on a collection and immediately react to them.

Note: I think this needs to resolve #29 first, to be able to do other things while waiting for a Change Event.

Error when trying to retrieve value by key from BSON object

Below error when trying to retrieve value by key from a BSON object

Error in src/api.cr:8: instantiating 'Mongo::Collection#find(Hash(String, Hash(String, Int32)))'

collection.find({"age" => {"$gt" => 30}}) do |doc|
           ^~~~

in lib/mongo/src/mongo/collection.cr:133: instantiating 'Mongo::Cursor#each()'

    find(query, fields, flags, skip, limit, batch_size, prefs).each do |doc|
                                                               ^~~~

in lib/mongo/src/mongo/collection.cr:133: instantiating 'Mongo::Cursor#each()'

    find(query, fields, flags, skip, limit, batch_size, prefs).each do |doc|
                                                               ^~~~

in src/api.cr:8: instantiating 'Mongo::Collection#find(Hash(String, Hash(String, Int32)))'

collection.find({"age" => {"$gt" => 30}}) do |doc|
           ^~~~

in src/api.cr:10: instantiating 'BSON#[](String)'

  puts doc["age"]
          ^

in lib/mongo/src/bson.cr:127: instantiating 'fetch(String)'

    fetch(key) { raise IndexError.new }
    ^~~~~

in lib/mongo/src/bson.cr:116: instantiating 'BSON::Value#value()'

      Value.new(value).value
                       ^~~~~

in lib/mongo/src/bson/value.cr:42: undefined constant Time::Kind::Utc

        Time.new(spec, Time::Kind::Utc)

[request] upgrade mongo-c driver version

hi, i hope somebody can upgrade mongo-c driver that higher than 1.1.0 , i always fail when trying to make mongo-c driver version 1.1.0

src/mongoc/mongoc-stream-tls.c:134:5: error: dereferencing pointer to incomplete type "Bio {aka struct bio_st}

Spec fails with libmongoc 1.4.2 (Debian stretch)

Environment: a Debian stretch up to date, with libmongoc-dev package

> crystal --version
Crystal 0.22.0 [3c71228] (2017-04-20) LLVM 3.5.0
> llvm-config-3.9 --version
3.9.1
> cat /etc/{issue,debian_version}
Debian GNU/Linux 9 \n \l

9.0
> dpkg -l libmongoc-dev
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                            Version                      Architecture                 Description
+++-===============================================-============================-============================-===================================================================================================
ii  libmongoc-dev                                   1.4.2-1+b1                   amd64                        MongoDB C client library - dev files

From mongo.cr master branch.

> git clone https://github.com/datanoise/mongo.cr
Cloning into 'mongo.cr'...
remote: Counting objects: 355, done.
remote: Total 355 (delta 0), reused 0 (delta 0), pack-reused 354
Receiving objects: 100% (355/355), 78.30 KiB | 0 bytes/s, done.
Resolving deltas: 100% (203/203), done.
Checking connectivity... done.
> crystal spec
FF......................................................................

Failures:

  1) Mongo::Uri should be able to create new uri
     Failure/Error: host.port.should eq(27017)

       Expected: 27017
            got: 0

     # spec/uri_spec.cr:10

  2) Mongo::Uri should be able to create new uri with host and port
     Failure/Error: host.port.should eq(27017)

       Expected: 27017
            got: 0

     # spec/uri_spec.cr:18

Finished in 590.46 milliseconds
72 examples, 2 failures, 0 errors, 0 pending

Failed examples:

crystal spec spec/uri_spec.cr:5 # Mongo::Uri should be able to create new uri
crystal spec spec/uri_spec.cr:13 # Mongo::Uri should be able to create new uri with host and port

Can't convert query result to array

So I inserted following data into the database:

{
    "name": "Group #1",
    "users": [
        "daniel",
        "bob"
    ]
}

But if I want to get that array back, the Result is an BSON::Code and I can't do anything with it. How could that be fixed and is there any workaround? @datanoise

Error in bson.cr

Hello.
I try to run example from README.md and stuck on following error:

 
Error in ./src/limnal/controller.cr:1: while requiring "./mongo"

require "./mongo"
^

in ./libs/mongo/mongo.cr:2: while requiring "./bson"

require "./bson"
^

Syntax error in ./libs/mongo/bson.cr:63: expecting token ')', not 'as'

    LibBSON.bson_free(cstr as Void*)
                           ^

My code:

require "./mongo"

client = Mongo::Client.new "mongodb://<here I use mlab hosted database>"
db = client["limnal"]

collection = db["thread"]
collection.insert({"name" => "James Bond", "age" => 37})

collection.find({"age" => {"$gt" => 30}}) do |doc|
  puts doc
end

Please help. thanx

Unhandled exception: File not open for writing (IO::Error) from usr/share/crystal/src/crystal/system/unix/file_descriptor.cr:23:11 in 'unbuffered_write'

By running the homepage example in a containerized environment I'm getting the following error:

Unhandled exception: File not open for writing (IO::Error)
  from usr/share/crystal/src/crystal/system/unix/file_descriptor.cr:23:11 in 'unbuffered_write'
  from usr/share/crystal/src/io/buffered.cr:205:5 in 'flush'
  from usr/share/crystal/src/logger.cr:175:7 in 'write'
  from usr/share/crystal/src/logger.cr:155:5 in 'log'
  from src/lib/mongo/src/mongo.cr:35:18 in 'log'
  from src/lib/mongo/src/mongo.cr:39:5 in '->'
  from mongoc_log
  from ???
  from ???
  from ???
  from ???
  from ???
  from ???
  from ???
  from mongoc_cursor_next
  from src/lib/mongo/src/mongo/cursor.cr:47:8 in 'next'

The containerized environment that I'm using is ubuntu:18.04 image with the following packages installed: libmongoc-dev, libbson-dev.
Is there some kind of setup that I'm missing?

Compiling on fedora

Hi,

I'm trying to build this package on fedora, since I have trouble using it:
First I install

dnf install libevent-devel pcre-devel gc-devel libmongo-client-devel

While downloading the code via git, I can not build this library crystal build src/mongo.cr, running it lead me to a

/usr/lib/gcc/x86_64-redhat-linux/5.3.1/../../../../lib64/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'

cc compilation crashes

How does one execute a command using the 'client.command()' method?

I'm trying to use the command() method to query the connection status of the database, but the compiler seems to keep complaining about the fact the query should be of type BSON, but I'm unsure how to actually create a BSON object, I tried making use of the from_json method on the BSON class, but couldn't get that to work. This is what I have so far:

def live?                                                                 
-»»»begin                                                                 
-»»»-»»»if @client.command("aggregation", {connectionStatus: 1})
-»»»-»»»-»»»return true                                                   
-»»»-»»»else                                                              
-»»»-»»»-»»»return false                                                  
-»»»-»»»end                                                               
-»»»rescue                                                                
-»»»-»»»return false                                                      
-»»»end                                                                   
end

When I try to compile this I get:

Error in mxmongo.cr:14: instantiating 'MxMongo#live?()'                                                                                                           
                                                                                                                                                                  
if mongo_client.live?                                                                                                                                             
                ^~~~~                                                                                                                                             
                                                                                                                                                                  
in classes/mx_mongo.cr:25: instantiating '(Mongo::Client | Nil:Class)#command(String, NamedTuple(connectionStatus: Int32))'                                       
                                                                                                                                                                  
   if @client.command("aggregation", {"connectionStatus": 1})                                                                                                     
              ^~~~~~~                                                                                                                                             
                                                                                                                                                                  
in lib/mongo/src/mongo/client.cr:40: instantiating 'command(String, NamedTuple(connectionStatus: Int32), BSON, LibMongoC::QueryFlags, Int32, Int32, Int32, Nil)'  
                                                                                                                                                                  
  def command(db_name, query, fields = BSON.new, flags = LibMongoC::QueryFlags::NONE,                                                                             
  ^                                                                                                                                                               
                                                                                                                                                                  
in lib/mongo/src/mongo/client.cr:43: argument 'query' of 'LibMongoC#client_command' must be Pointer(LibBSON::BSONHandle), not NamedTuple(connectionStatus: Int32) 
                                                                                                                                                                  
                                        query, fields, prefs)                                                                                                     
                                        ^~~~~

Insert a document with array of hash

When I try to insert a doc with an array of tuple :

og = [{"property"=>"og:title","content"=>"Open Graph protocol"},{"property"=>"og:type","content"=>"website"}]

bookmarkCollection.insert({
  "link" => "https://angular.io",
  "og" => og,
  "tags" => [firstTag, secondTag]
})

I've following error :

no overload matches 'BSON#[]=' with types String, (BSON | Mongo::Cursor | Nil)

 - BSON#[]=(key, value : Int64)
 - BSON#[]=(key, value : Binary)
 - BSON#[]=(key, value : Bool)
 - BSON#[]=(key, value : Float64 | Float32)
 - BSON#[]=(key, value : MinKey)
 - BSON#[]=(key, value : MaxKey)
 - BSON#[]=(key, value : Nil)
 - BSON#[]=(key, value : ObjectId)
 - BSON#[]=(key, value : String)
 - BSON#[]=(key, value : Symbol)
 - BSON#[]=(key, value : Time)
 - BSON#[]=(key, value : Timestamp)
 - BSON#[]=(key, value : Code)
 - BSON#[]=(key, value : BSON)
 - BSON#[]=(key, value : Regex)
Couldn't find overloads for these types:
 - BSON#[]=(key : String, value : Mongo::Cursor)

        bson[i.to_s] = item

how to work with BSON arrays?

@datanoise I need to construct a field within a document that contains an array of embedded documents. How can I do this? It looks like the provided BSON library does not support BsonArray?

Is this project dead?

Sure, there was a last commit some time ago, but there are so old issues and pull requets, also it isn't working with the current crystal version.

You solved on error, than you have another one. No end there.

@datanoise

Please update to current Crystal version.

How to find document by Object ID?

What I want: Find a Document by the Document ID.

My Code and tries:

Collection.update({ "_id" => { "$oid" => ANYVALIDID } }, { "$set" => { "title" => "Hello World" } })
# Not working

id = BSON::ObjectId.new(doc["id"].to_s)
Coolection.update({ "_id" => id }, { "title" => "Hello World" })
# Not working. Error:
# Index out of bounds (IndexError)
# 0x5d9b1f: *BSON#[]<String>:(BSON | BSON::Code | BSON::MaxKey | BSON::MinKey | BSON::ObjectId | # BSON::Symbol | BSON::Timestamp | Bool | Float64 | Int32 | Int64 | Regex | String | Time | Nil) at /app/lib/mongo/src/bson.cr 127:18

Any Ideas
@datanoise

Getting the _id of an inserted document

Since Collection#insert returns nil, any idea how to find out what _id mongodb auto-added to the new document?

Is there any way to list all collection entries ?

Hello,

I want to list all entries of my collection.

Is there any way to do this ?

Regards,

update to work with Crystal 0.24.1

There are a few minor syntax changes that prevent mongo.cr from compiling with the new version of Crystal. I fixed a few of them manually, however the last one is a legit bug that I have not been able to figure out, involving how the port number is parsed in certain circumstances. The spec failure I was unable to fix is this one:

crystal spec spec/uri_spec.cr:21 # Mongo::Uri should be able to create new uri with host and port

I'll post a PR if I do get it working

get id of inserted document

In the comments for the collection insert method, it says: "you can retrieve the generated _id from the last_error method", however when I call the last_error method I don't see where to get the _id.

https://github.com/datanoise/mongo.cr/blob/master/src/mongo/collection.cr#L149

undefined method 'to_bson' for JSON::Any (did you mean 'to_json'?)

in lib/mongo/src/mongo/collection.cr:150: instantiating 'insert(JSON::Any, LibMongoC::InsertFlags, Nil)'

def insert(document, flags = LibMongoC::InsertFlags::NONE, write_concern = nil)
^

in lib/mongo/src/mongo/collection.cr:151: undefined method 'to_bson' for JSON::Any (did you mean 'to_json'?)

unless LibMongoC.collection_insert(self, flags, document.to_bson, write_concern, out error)