Git Product home page Git Product logo

mongodb-tools's Introduction

mongodb-tools - Utilities for working with MongoDB

  • collection-stats.py - Display statistics about the collections in all databases
  • index-stats.py - Displays statistics about the indexes in all databases
  • redundant-indexes.py - Finds indexes that may be redundant

Blogs And Articles

Dependencies

python-virtualenv
python-dev

Installation

git clone https://github.com/jwilder/mongodb-tools
cd mongodb-tools
./setup.sh
source virtualenv/bin/activate

Test Data Setup

In on terminal run:

$ ./run-mongo.sh


In another terminal run:

$ python examples/testdata.py

collection-stats.py

 $ ./collection-stats.py

 Checking DB: examples2.system.indexes
 Checking DB: examples2.things
 Checking DB: examples1.system.indexes
 Checking DB: examples1.address
 Checking DB: examples1.typeless_address
 Checking DB: examples1.user
 Checking DB: examples1.typeless_user


 +----------------------------+--------+--------+---------+--------------+---------+------------+
 |         Collection         | Count  | % Size | DB Size | Avg Obj Size | Indexes | Index Size |
 +----------------------------+--------+--------+---------+--------------+---------+------------+
 | examples1.address          |      2 |   0.0% | 184.00b |       92.00b |    2    |     15.97K |
 | examples1.system.indexes   |      9 |   0.0% | 912.00b |      101.33b |    0    |      0.00b |
 | examples1.typeless_address |      2 |   0.0% | 216.00b |      108.00b |    1    |      7.98K |
 | examples2.system.indexes   |      2 |   0.0% | 164.00b |       82.00b |    0    |      0.00b |
 | examples1.typeless_user    | 101879 |  26.7% |  10.10M |      104.00b |    3    |      8.18M |
 | examples1.user             | 101879 |  36.0% |  13.60M |      140.00b |    3    |     15.20M |
 | examples2.things           | 100000 |  37.3% |  14.11M |      148.00b |    2    |      5.67M |
 +----------------------------+--------+--------+---------+--------------+---------+------------+
 Total Documents: 303773
 Total Data Size: 37.82M
 Total Index Size: 29.08M
 RAM Headroom: 2.87G
 RAM Used: 2.74G (61.6%)
 Available RAM Headroom: 1.10G

index-stats.py

$ ./index-stats.py


Checking DB: examples2.system.indexes
Checking DB: examples2.things
Checking DB: examples1.system.indexes
Checking DB: examples1.address
Checking DB: examples1.typeless_address
Checking DB: examples1.user
Checking DB: examples1.typeless_user

Index Overview
+----------------------------+------------------------+--------+------------+
|         Collection         |         Index          | % Size | Index Size |
+----------------------------+------------------------+--------+------------+
| examples1.address          | _id_                   |   0.0% |      7.98K |
| examples1.address          | _types_1               |   0.0% |      7.98K |
| examples1.typeless_address | _id_                   |   0.0% |      7.98K |
| examples1.typeless_user    | _id_                   |  10.9% |      3.17M |
| examples1.typeless_user    | address_id_1           |  10.9% |      3.17M |
| examples1.typeless_user    | typeless_address_ref_1 |   6.4% |      1.85M |
| examples1.user             | _id_                   |  10.9% |      3.17M |
| examples1.user             | _types_1_address_id_1  |  13.2% |      3.84M |
| examples1.user             | _types_1_address_ref_1 |  28.2% |      8.20M |
| examples2.things           | _id_                   |  10.7% |      3.11M |
| examples2.things           | _types_1               |   8.8% |      2.56M |
+----------------------------+------------------------+--------+------------+

Top 5 Largest Indexes
+-------------------------+------------------------+--------+------------+
|        Collection       |         Index          | % Size | Index Size |
+-------------------------+------------------------+--------+------------+
| examples1.user          | _types_1_address_ref_1 |  28.2% |      8.20M |
| examples1.user          | _types_1_address_id_1  |  13.2% |      3.84M |
| examples1.typeless_user | _id_                   |  10.9% |      3.17M |
| examples2.things        | _id_                   |  10.7% |      3.11M |
| examples2.things        | _types_1               |   8.8% |      2.56M |
+-------------------------+------------------------+--------+------------+

Total Documents: 303773
Total Data Size: 37.82M
Total Index Size: 29.08M
RAM Headroom: 2.87G
RAM Used: 2.73G (61.4%)
Available RAM Headroom: 1.11G

mongodb-tools's People

Contributors

0xroch avatar esquireofoz avatar jwilder avatar nheudecker avatar sheavner avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mongodb-tools's Issues

Fails with PrettyTable 0.6.0

requirements.txt doesn't specify a PrettyTable version. Current is 0.6.0. Running fails with:

Traceback (most recent call last):
  File "./index_stats.py", line 120, in <module>
    main()
  File "./index_stats.py", line 70, in main
    x.set_field_align("Collection", "l")
  File "/home/joeshaw/mongodb-tools/virtualenv/lib/python2.6/site-packages/prettytable.py", line 163, in __getattr__
    raise AttributeError(name)
AttributeError: set_field_align

Explicitly specifying 0.5.0 works. Didn't look into it beyond this.

allow to use SSL

Using pymongo >= 3.0 would be possible interact with a mongo db > 3.0 that run with SSL enabled.

here a patch for that.

WiredTiger not supported?

Just upgraded our cluster to wired tiger. Was interested to see what the savings were, but it seems these tools don't work with WT? Get very minimal output (just the totals, which are also incorrect).

Any plans to support WT?

not authorized for query on <database>.system.namespaces

I'm getting this error when I try to run collection_stats.py with against my database. The user I'm specifying has rights to the db ( "readWrite", "dbAdmin", "dbOwner", "userAdmin", "dbAdminAnyDatabase"). Mongo is version 2.4.5

If I specify -d admin, then I'm getting the stats for the admin database without problem.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.