Git Product home page Git Product logo

pentaho-mongodb-delete-plugin's Introduction

Pentaho MongoDB Delete Plugin

Build Status

The Kettle Plugin to provides kettle steps for delete document inside a Mongo DB collection.

Building

The Pentaho MongoDB Delete Plugin is built with Apache Maven for dependency management. To build from source code.

$ git clone git://github.com/maasdi/pentaho-mongodb-delete-plugin.git
$ cd pentaho-mongodb-delete-plugin
$ mvn package

Maven will compile and package all you need and will create new folder 'pentaho-mongodb-delete-plugin' under target directory.

Installation

Install from Pentaho Marketplace

  1. From PDI, Select menu Help - Marketplace
  2. Search MongoDB Delete Plugin, click MongoDB Delete Plugin row from search result
  3. To install, click 'Install this plugin' button

Manual Installation

  1. Download binary here then extract the zip file OR If you build from source code go to folder target/pentaho-mongodb-delete-plugin
  2. Copy folder pentaho-mongodb-delete-plugin and all contents
  3. Paste to folder ${KETTLE_INSTALATION_DIRECTORY}/plugins/steps/
  4. If your PDI still running, please restart and you should see new steps MongoDB Delete under Big Data category

alt text

Documentation

Please go to wiki page for the documentation.

Issue

Do you have issue while using this plugin? Please report here

Sample

Here are the samples how to use:

  1. delete-all.ktr
  2. delete-by-incoming-row.ktr
  3. delete-by-json-query.ktr
  4. delete-by-json-query-exec-each-row.ktr

License

Licensed under the Apache License, Version 2.0.

pentaho-mongodb-delete-plugin's People

Contributors

maasdi avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar

Forkers

czlovewyw guoyiwu hkollera yydo

pentaho-mongodb-delete-plugin's Issues

不兼容mongo3.x

目前pentaho已经升级到8.0版了，mongo使用3.x驱动
这个项目已经不支持访问3.x认证的数据库了

要尽快修复啊，建议修改成使用mongourl的形式
这样随着版本变迁不需要改动很多，这里面大部分代码是从别的项目拷贝的，实在没有必要

Mongo connection errors when connecting to mongo database with SSL enabled

Hi there,

Im getting connection errors when trying to connect to mongo database with SSL enabled.Can you please let me know if current plugin supports connection to mongo with ssl enabled.

Error:
2019/07/12 10:29:23 - RemoveOldVersion (salesbyday).0 - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : An error occurred during write: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=ndbuat-shard-00-00-xxxxx.azure.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}, {address=ndbuat-shard-00-02-xxxxx.azure.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}, {address=ndbuat-shard-00-01-xxxxx.azure.mongodb.net:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}]
2019/07/12 10:29:23 - RemoveOldVersion (salesbyday).0 - Retrying write to mongodb in 10 seconds...

Thanks,
Hiran

Unable to connect with mongo

Hi Guys,
i'm trying to remove all record from my mongo collection. but mongo delete plugin giving me connection error. Same connection credential working in mongo input and output step.Difference between mongo input and delete plungin is mongo delete plungin option don't have option authenticated database.
we got following error
Invalid database name format. Database name is either empty or it contains spaces.
detail of error is here can anybody help to solve this

org.pentaho.di.core.exception.KettleException:
java.lang.IllegalArgumentException: Invalid database name format. Database name is either empty or it contains spaces.
Invalid database name format. Database name is either empty or it contains spaces.

at com.github.maasdi.mongo.wrapper.NoAuthMongoClientWrapper.getDatabaseNames(NoAuthMongoClientWrapper.java:374)
at com.github.maasdi.di.trans.steps.mongodbdelete.MongoDbDeleteDialog.setupDBNames(MongoDbDeleteDialog.java:974)
at com.github.maasdi.di.trans.steps.mongodbdelete.MongoDbDeleteDialog.access$1500(MongoDbDeleteDialog.java:83)
at com.github.maasdi.di.trans.steps.mongodbdelete.MongoDbDeleteDialog$10.widgetSelected(MongoDbDeleteDialog.java:470)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at com.github.maasdi.di.trans.steps.mongodbdelete.MongoDbDeleteDialog.open(MongoDbDeleteDialog.java:911)
at org.pentaho.di.ui.spoon.delegates.SpoonStepsDelegate.editStep(SpoonStepsDelegate.java:120)
at org.pentaho.di.ui.spoon.Spoon.editStep(Spoon.java:8949)
at org.pentaho.di.ui.spoon.trans.TransGraph.editStep(TransGraph.java:3291)
at org.pentaho.di.ui.spoon.trans.TransGraph.mouseDoubleClick(TransGraph.java:785)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.trans.steps.mongodbinput.MongoDbInputDialog.open(MongoDbInputDialog.java:1091)
at org.pentaho.di.ui.spoon.delegates.SpoonStepsDelegate.editStep(SpoonStepsDelegate.java:120)
at org.pentaho.di.ui.spoon.Spoon.editStep(Spoon.java:8949)
at org.pentaho.di.ui.spoon.trans.TransGraph.editStep(TransGraph.java:3291)
at org.pentaho.di.ui.spoon.trans.TransGraph.mouseDoubleClick(TransGraph.java:785)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.trans.steps.mongodbinput.MongoDbInputDialog.open(MongoDbInputDialog.java:1091)
at org.pentaho.di.ui.spoon.delegates.SpoonStepsDelegate.editStep(SpoonStepsDelegate.java:120)
at org.pentaho.di.ui.spoon.Spoon.editStep(Spoon.java:8949)
at org.pentaho.di.ui.spoon.trans.TransGraph.editStep(TransGraph.java:3291)
at org.pentaho.di.ui.spoon.trans.TransGraph.mouseDoubleClick(TransGraph.java:785)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1375)
at org.pentaho.di.ui.spoon.Spoon.waitForDispose(Spoon.java:8104)
at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:9466)
at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:701)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.pentaho.commons.launcher.Launcher.main(Launcher.java:92)

Caused by: java.lang.IllegalArgumentException: Invalid database name format. Database name is either empty or it contains spaces.
at com.mongodb.DB.(DB.java:67)
at com.mongodb.DBApiLayer.(DBApiLayer.java:58)
at com.mongodb.Mongo.getDB(Mongo.java:344)
at com.mongodb.DBPort$SaslAuthenticator.getDatabase(DBPort.java:928)
at com.mongodb.DBPort$SaslAuthenticator.sendSaslStart(DBPort.java:935)
at com.mongodb.DBPort$SaslAuthenticator.authenticate(DBPort.java:898)
at com.mongodb.DBPort.authenticate(DBPort.java:432)
at com.mongodb.DBPort.checkAuth(DBPort.java:443)
at com.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:293)
at com.mongodb.DBTCPConnector.call(DBTCPConnector.java:273)
at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:84)
at com.mongodb.DB.command(DB.java:320)
at com.mongodb.DB.command(DB.java:299)
at com.mongodb.DB.command(DB.java:374)
at com.mongodb.Mongo.getDatabaseNames(Mongo.java:373)
at com.github.maasdi.mongo.wrapper.NoAuthMongoClientWrapper.getDatabaseNames(NoAuthMongoClientWrapper.java:369)
... 51 more

Deleting with $exists: false condition removes rows that have data

Hi, I've encountered the following problem:

Ex:
I have two rows in the collection for one Company, one that has the fields _id, CompanyId, CompanyName and another that only has _id and CompanyId.

Having the delete criteria: { "CompanyName" : { "$exists" : false }} it deletes both rows, even the one with CompanyName data. For more details see attach.

Can you please help me with this issue?
Thank you!
Capture

Problem with plugin installation on DI server

Hi,

I am running Enterprise Version of PDI 5.4. I installed plugin on my local machine and on server following instruction for manual installation. When I create a transformation and run it locally everything is perfect. When I try to run it on remote server I have exception:
There was an error passing the exported transformation to the remote server:
org.pentaho.di.core.exception.KettleMissingPluginsException:
Missing plugins found while loading a transformation

Step : MongoDbDelete
at org.pentaho.di.trans.TransMeta.loadXML(TransMeta.java:2882)
at org.pentaho.di.trans.TransMeta.(TransMeta.java:2718)
at org.pentaho.di.trans.TransMeta.(TransMeta.java:2670)
at org.pentaho.di.trans.TransMeta.(TransMeta.java:2647)
at org.pentaho.di.trans.TransMeta.(TransMeta.java:2627)
at org.pentaho.di.trans.TransMeta.(TransMeta.java:2608)
at org.pentaho.di.www.RegisterPackageServlet.generateBody(RegisterPackageServlet.java:100)
at org.pentaho.di.www.BodyHttpServlet.doGet(BodyHttpServlet.java:73)
at org.pentaho.di.www.CarteServlet.doGet(CarteServlet.java:72)
at com.pentaho.pdi.www.RoleBoundCarteServlet.doGet(SourceFile:65)
at org.pentaho.di.www.CarteServlet.doPost(CarteServlet.java:61)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)

thank you,
Irina

Delete records using "IN " condition in mongo db delete step pentaho

Hi I have a requirement where i have to delete all the records from a collection based on a date filter conditions.
ex. Delete from colection where date not in ('date1','date2').
i used the following condition which works fine, but these dates are dynamic. and when i try to pass as a variable it doesnt work as expected, It truncates all the data. Can someone please help me out.

Working on hardcode: {'as_of_dt': {$nin :[{$date :"2018-05-22T04:00:00.000Z"},{$date :"2018-05-24T04:00:00.000Z"},
{$date :"2018-05-23T04:00:00.000Z"},{$date :"2018-04-30T04:00:00.000Z"},{$date :"2018-03-31T04:00:00.000Z"}] }}

Doesnt work : {'as_of_dt': {$nin :['${delete_date}']}}
{'as_of_dt': {$nin :[${delete_date}]}}
delete date is the variable assigned to the values coming up dynamically

Bugs , on PEntaho ver 7.

Doesnt Work anymore with authentication method mongo 3. using mechanism database auth.
Any update ??

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.