Comments (5)
marcstein
commented on May 27, 2024
1
Ahhhh. Excellent!
Thanks so much!
Marc
Marc Stein CEO
underwrite.ai
P. (617) 545-7171 M. (857) 222-3291
E. ***@***.***
W. https://www.underwrite.ai
<https://www.linkedin.com/in/marc-stein-b83b765>
…On Sun, May 2, 2021 at 3:30 PM Julien Elbaz ***@***.***> wrote:
Ah I see, so if I understand correctly this is not a long-running process.
You can close the underlying cryomongo client by calling
Moongoon.client.close before the program exits.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#11 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGEJZMIW2X6EZTGMB3E5GLTLWR6BANCNFSM437UYXBQ>
.
from cryomongo.
elbywan
commented on May 27, 2024
Hi @marcstein,
I'm using mongoon (very cool tool, btw).
Thanks!
The issue that I'm seeing is that each write to the Mongodb both opens files on the operating system (adding to the ulimit) and also keeps opening new connections to the Mongo server. Once I get to 1500 connections, the Mongo server maxes out and throws an error.
This is strange. Are you creating mongo clients every time you write to the DB? I am unable to reproduce, the following code snippet works and use a single connection to the database (well actually two to be honest, one for commands and one for monitoring).
client = Mongo::Client.new
database = client["driver_test"]
collection = database["test"]
{Mongo::Commands::Drop, Mongo::Commands::Create}.each do |command|
database.command(command, name: "test")
rescue e : Mongo::Error::Command
# ignore the server error, drop will fail if the collection has not been created before.
end
100_000.times do |i|
collection.insert_one({a: 1})
puts collection.count_documents(max_time_ms: 6000).to_json
Fiber.yield
end2021-05-02T20:44:15.480+0200 I NETWORK [listener] connection accepted from 127.0.0.1:61385 #1 (1 connection now open)
2021-05-02T20:44:15.480+0200 I NETWORK [conn1] received client metadata from 127.0.0.1:61385 conn1: { application: { name: "cryomongo" }, driver: { name: "cryomongo", version: "0.3.8" }, os: { type: "Darwin" } }
2021-05-02T20:44:15.481+0200 I NETWORK [listener] connection accepted from 127.0.0.1:61386 #2 (2 connections now open)
2021-05-02T20:44:15.481+0200 I NETWORK [conn2] received client metadata from 127.0.0.1:61386 conn2: { application: { name: "cryomongo" }, driver: { name: "cryomongo", version: "0.3.8" }, os: { type: "Darwin" } }
2021-05-02T20:44:15.481+0200 I COMMAND [conn2] CMD: drop driver_test.test
2021-05-02T20:44:15.481+0200 I STORAGE [conn2] Finishing collection drop for driver_test.test (f187bad4-09a5-48a8-ae5c-229d2499d68f).
2021-05-02T20:44:15.538+0200 I STORAGE [conn2] createCollection: driver_test.test with generated UUID: 2573bd52-9da5-4ce8-ae28-2bb3f5204aa7 and options: {}
2021-05-02T20:44:15.668+0200 I INDEX [conn2] index build: done building index _id_ on ns driver_test.test
2021-05-02T20:44:15.668+0200 I COMMAND [conn2] command driver_test.test appName: "cryomongo" command: create { create: "test", $db: "driver_test", lsid: { id: UUID("2ad27d2b-0c1c-4179-9218-1504034f38d3") } } numYields:0 reslen:38 locks:{ ParallelBatchWriterMode: { acquireCount: { r: 1 } }, ReplicationStateTransition: { acquireCount: { w: 1 } }, Global: { acquireCount: { w: 1 } }, Database: { acquireCount: { w: 1 } }, Collection: { acquireCount: { r: 1, W: 1 } }, Mutex: { acquireCount: { r: 2 } } } flowControl:{ acquireCount: 1, timeAcquiringMicros: 1 } storage:{} protocol:op_msg 130ms
2021-05-02T20:44:15.669+0200 I SHARDING [conn2] Marking collection driver_test.test as collection version: <unsharded>
from cryomongo.
elbywan
commented on May 27, 2024
Also here is the moongoon version:
Moongoon.connect("mongodb://localhost:27017", database_name: "test_db")
class MyModel < Moongoon::Collection
collection "models"
property a : Int32?
property b : Int32?
end
MyModel.clear
100_000.times do
model = MyModel.new(a: 1, b: 2).insert
puts MyModel.count
Fiber.yield
end
from cryomongo.
marcstein
commented on May 27, 2024
Thanks for the immediate response!
Here's the code that I'm running
class Cc_v3_test < Moongoon::Collection
collection "cc_v3_test"
property uuid : String
property model : String
property vector : String
property client : String
property requested_amount : String
property lead_price : String
property score : String
property paycheck_amount : String
property years_with_bank_account : String
property net_monthly_income : String
property created_at : String
property xml : String
property decision : String
property clarity_id : String
property timestamps
end
begin
Moongoon.connect("mongodb+srv://
***@***.***", database_name:
"underwrite")
cc_v3_test = Cc_v3_test.new(uuid: "#{uuid}", model: "challenger2",
vector: "#{vector}", client: "#{client}", requested_amount:
"#{requested_amount}", lead_price: "#{lead_price}", score: "#{score}",
paycheck_amount: "#{paycheck_amount}", years_with_bank_account:
"#{years_with_bank_account}", net_monthly_income: "#{net_monthly_income}",
created_at: "#{Time.utc}", xml: "#{body.gsub('"', '\'')}", decision:
"#{decision}", clarity_id: "#{clarity_id}")
cc_v3_test.insert
rescue ex
puts ex.message
end
end
This application runs as a service. What am I doing wrong?
Thanks!
Marc
Marc Stein CEO
underwrite.ai
P. (617) 545-7171 M. (857) 222-3291
E. ***@***.***
W. https://www.underwrite.ai
<https://www.linkedin.com/in/marc-stein-b83b765>
…On Sun, May 2, 2021 at 2:45 PM Julien Elbaz ***@***.***> wrote:
Hi @marcstein <https://github.com/marcstein>,
I'm using mongoon (very cool tool, btw).
Thanks! ❤️
The issue that I'm seeing is that each write to the Mongodb both opens
files on the operating system (adding to the ulimit) and also keeps opening
new connections to the Mongo server. Once I get to 1500 connections, the
Mongo server maxes out and throws an error.
This is strange. Are you creating mongo clients every time? I am unable to
reproduce, the following code snippet works and use a single connection to
the database (well actually two to be honest, one for commands and one for
monitoring).
client = Mongo::Client.new
database = client["driver_test"]
collection = database["test"]
{Mongo::Commands::Drop, Mongo::Commands::Create}.each do |command|
database.command(command, name: "test")
rescue e : Mongo::Error::Command
# ignore the server error, drop will fail if the collection has not been created before.
end
100_000.times do |i|
collection.insert_one({a: 1})
puts collection.count_documents(max_time_ms: 6000).to_json
Fiber.yield
end
2021-05-02T20:44:15.480+0200 I NETWORK [listener] connection accepted from 127.0.0.1:61385 #1 (1 connection now open)
2021-05-02T20:44:15.480+0200 I NETWORK [conn1] received client metadata from 127.0.0.1:61385 conn1: { application: { name: "cryomongo" }, driver: { name: "cryomongo", version: "0.3.8" }, os: { type: "Darwin" } }
2021-05-02T20:44:15.481+0200 I NETWORK [listener] connection accepted from 127.0.0.1:61386 #2 (2 connections now open)
2021-05-02T20:44:15.481+0200 I NETWORK [conn2] received client metadata from 127.0.0.1:61386 conn2: { application: { name: "cryomongo" }, driver: { name: "cryomongo", version: "0.3.8" }, os: { type: "Darwin" } }
2021-05-02T20:44:15.481+0200 I COMMAND [conn2] CMD: drop driver_test.test
2021-05-02T20:44:15.481+0200 I STORAGE [conn2] Finishing collection drop for driver_test.test (f187bad4-09a5-48a8-ae5c-229d2499d68f).
2021-05-02T20:44:15.538+0200 I STORAGE [conn2] createCollection: driver_test.test with generated UUID: 2573bd52-9da5-4ce8-ae28-2bb3f5204aa7 and options: {}
2021-05-02T20:44:15.668+0200 I INDEX [conn2] index build: done building index _id_ on ns driver_test.test
2021-05-02T20:44:15.668+0200 I COMMAND [conn2] command driver_test.test appName: "cryomongo" command: create { create: "test", $db: "driver_test", lsid: { id: UUID("2ad27d2b-0c1c-4179-9218-1504034f38d3") } } numYields:0 reslen:38 locks:{ ParallelBatchWriterMode: { acquireCount: { r: 1 } }, ReplicationStateTransition: { acquireCount: { w: 1 } }, Global: { acquireCount: { w: 1 } }, Database: { acquireCount: { w: 1 } }, Collection: { acquireCount: { r: 1, W: 1 } }, Mutex: { acquireCount: { r: 2 } } } flowControl:{ acquireCount: 1, timeAcquiringMicros: 1 } storage:{} protocol:op_msg 130ms
2021-05-02T20:44:15.669+0200 I SHARDING [conn2] Marking collection driver_test.test as collection version: <unsharded>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#11 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGEJZNWKJ7Z74UOEW2UCCTTLWMU5ANCNFSM437UYXBQ>
.
from cryomongo.
elbywan
commented on May 27, 2024
Ah I see, so if I understand correctly this is not a long-running process.
You can close the underlying cryomongo client by calling Moongoon.client.close before the program exits.
from cryomongo.
Related Issues (9)
- No SRV records found when querying url HOT 1
- Timeout error connecting to CosmosDB Mongo API HOT 2
- Potentially show new errInfo data in validation errors HOT 2
- SASL SCRAM-SHA-256 authentication failure on mongo 4.2.7 HOT 4
- Increasing number of Mongo connections over time HOT 3
- Bug in macro generation code HOT 1
- Default database HOT 2
- connect_timeout option HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cryomongo.