barzan / dbseer Goto Github PK

README.md references a sc directory with Python scripts, but no sc directory exists in repository.

Hi,

My JAVA_HOME is set to /usr/lib/jvm/java-1.7.0-openjdk-amd64 because it seemed that java-9-oracle is not supported with your program. I tried to build the dbseer_frontend with both intellij and manually but non of these worked.

Btw, could you please provide a step-by-step video tutorial of installing the program?.

Here is the output of the ant command:
~/dbseer/dbseer_front_end$ ant -f dbseer_front_end.xml
Buildfile: /home/mooh/dbseer/dbseer_front_end/dbseer_front_end.xml

init:

clean.module.dbseer_front_end:
[delete] Deleting directory /home/mooh/dbseer/dbseer_front_end/out/production/dbseer_front_end

clean:

compile.module.dbseer_front_end.production:
[mkdir] Created dir: /home/mooh/dbseer/dbseer_front_end/out/production/dbseer_front_end
[javac] /home/mooh/dbseer/dbseer_front_end/dbseer_front_end.xml:259: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 148 source files to /home/mooh/dbseer/dbseer_front_end/out/production/dbseer_front_end
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/comp/clustering/IncrementalDBSCAN.java:19: error: package com.esotericsoftware.minlog does not exist
[javac] import com.esotericsoftware.minlog.Log;
[javac] ^
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/comp/process/live/LiveLogProcessor.java:27: error: package dbseer.middleware.constant does not exist
[javac] import dbseer.middleware.constant.MiddlewareConstants;
[javac] ^
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/gui/DBSeerGUI.java:19: error: package com.esotericsoftware.minlog does not exist
[javac] import com.esotericsoftware.minlog.Log;
[javac] ^
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/gui/panel/DBSeerMiddlewarePanel.java:33: error: package dbseer.middleware.event does not exist
[javac] import dbseer.middleware.event.MiddlewareClientEvent;
[javac] ^
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/middleware/MiddlewareClientRunner.java:19: error: package com.esotericsoftware.minlog does not exist
[javac] import com.esotericsoftware.minlog.Log;
[javac] ^
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/middleware/MiddlewareClientRunner.java:21: error: package dbseer.middleware.client does not exist
[javac] import dbseer.middleware.client.MiddlewareClient;
[javac] ^
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/middleware/MiddlewareClientRunner.java:41: error: cannot find symbol
[javac] private MiddlewareClient client;
[javac] ^
[javac] symbol: class MiddlewareClient
[javac] location: class MiddlewareClientRunner
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/comp/process/live/LiveLogProcessor.java:86: error: cannot find symbol
[javac] servers = serverStr.split(MiddlewareConstants.SERVER_STRING_DELIMITER);
[javac] ^
[javac] symbol: variable MiddlewareConstants
[javac] location: class LiveLogProcessor
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/gui/panel/DBSeerMiddlewarePanel.java:685: error: cannot find symbol
[javac] MiddlewareClientEvent event = (MiddlewareClientEvent) arg;
[javac] ^
[javac] symbol: class MiddlewareClientEvent
[javac] location: class DBSeerMiddlewarePanel
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/gui/panel/DBSeerMiddlewarePanel.java:685: error: cannot find symbol
[javac] MiddlewareClientEvent event = (MiddlewareClientEvent) arg;
[javac] ^
[javac] symbol: class MiddlewareClientEvent
[javac] location: class DBSeerMiddlewarePanel
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/gui/panel/DBSeerMiddlewarePanel.java:686: error: cannot find symbol
[javac] if (event.event == MiddlewareClientEvent.IS_MONITORING)
[javac] ^
[javac] symbol: variable MiddlewareClientEvent
[javac] location: class DBSeerMiddlewarePanel
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/gui/panel/DBSeerMiddlewarePanel.java:710: error: cannot find symbol
[javac] else if (event.event == MiddlewareClientEvent.IS_NOT_MONITORING)
[javac] ^
[javac] symbol: variable MiddlewareClientEvent
[javac] location: class DBSeerMiddlewarePanel
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/gui/panel/DBSeerMiddlewarePanel.java:719: error: cannot find symbol
[javac] else if (event.event == MiddlewareClientEvent.ERROR)
[javac] ^
[javac] symbol: variable MiddlewareClientEvent
[javac] location: class DBSeerMiddlewarePanel
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/middleware/MiddlewareClientRunner.java:55: error: cannot find symbol
[javac] client = new MiddlewareClient(host, id, password, port, dir);
[javac] ^
[javac] symbol: class MiddlewareClient
[javac] location: class MiddlewareClientRunner
[javac] /home/mooh/dbseer/dbseer_front_end/src/dbseer/middleware/MiddlewareClientRunner.java:57: error: cannot find symbol
[javac] client.setLogLevel(Log.LEVEL_NONE);
[javac] ^
[javac] symbol: variable Log
[javac] location: class MiddlewareClientRunner
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.
[javac] 15 errors

BUILD FAILED
/home/mooh/dbseer/dbseer_front_end/dbseer_front_end.xml:259: Compile failed; see the compiler error output for details.

Total time: 6 seconds

I am attempting to run all of DBSeer, MaxScale, and DBSeer GUI on the same Linux VM machine (CentOS 7.4).

While attempting to start DBSeer UI, I get the following exception. Any ideas?

[arunm@vm-arunm dbseer_front_end_jar]$ java -Xmx2048M -jar dbseer_front_end.jar ../../../dbseer.ini
Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
	at dbseer.gui.DBSeerGUI$2.run(DBSeerGUI.java:510)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

I noticed something interesting. Earlier, I had successfully run DBSeer GUI on a Windows machine. At that time,

$ pwd
/cygdrive/c/src/dbseer/dbseer_front_end/out/artifacts/dbseer_front_end_jar

$ ls -l
total 13025
-rwxrwxr-x+ 1 a00476658 Domain Users 13334531 Oct  9 16:46 dbseer_front_end.jar
-rwxrwxr-x+ 1 a00476658 Domain Users      179 Oct 17 11:31 settings.xml

Please notice that there is a "settings.xml" file there. When I compile the java source code on Linux, I see the jar file, but no "settings.xml" file. Is that a reason why GUI is failing to start?

So I created a "settings.xml" file by hand, and the exception moved down a few more lines. I think that the exception is because of the fact that UI windows are not opening on Linux. Do you have any idea why? My current theory is that "settings.xml" file is created whenever the user sets dbseer path etc. the first time using UI. Because UI is not opening, "settings.xml" file is not created.

I need some advise using Dbseer middleware.

I have successfully installed, and can run the Dbseer UI on a Windows VM.

What are some best practices for running the middleware? Should it run on Linux or Windows? Should it run on the same box that runs the MySQL dbms?

Hi, thanks for the great effort on DBSeer! I am currently running DBSeer on AWS instance using the DBSeer Docker image, however, I couldn't start the UI correctly based the "Docker Usage Guide for DBSeer.pdf".

Here are my steps to fix this:

1. enable the X11 forwarding on aws instance
2. install and start Xming on my local Windows laptop
3. ssh to the aws instance and pull the DBSeer image
4. echo $DISPLAY to check if it is the current_ip:11.0
5. use DISPLAY_NUMBER=$(echo $DISPLAY | cut -d. -f1 | cut -d: -f2) to set the DISPLAY_NUMBER, which is 11 here
6. use socat TCP4:localhost:60${DISPLAY_NUMBER} UNIX-LISTEN:/tmp/.X11-unix/X${DISPLAY_NUMBER} & to set the DISPALY inside and outside the docker correctly
7. use the docker run -e DISPLAY -v /var/lib/postgresql/11/main/:/dbseer/dataset -v /tmp/.X11-unix/:/tmp/.X11-unix -v /root/.Xauthority:/root/.Xauthority --net=host --rm dongyoungy/dbseer /run_dbseer to start the UI, the .Xauthority should be mounted to the /root/ not /developer/ since the default user to the image is root.

Now I can start the UI.

reference: https://blog.yadutaf.fr/2017/09/10/running-a-graphical-app-in-a-docker-container-on-a-remote-server/

the environment
java 1.8
octave 4.0.1
i tried to install dbseer_front_end step by step according to insall.md, but some errors occured .
i am not able to find the reason and really need your help.....
i executed the following commands:

1. i changed relative numbers from 6 to 8 in dbseer_front_end.xml
   for example:
   <property name="compiler.args.dbseer_front_end" value="-encoding UTF-8 -source 1.6 ${compiler.args} "/>
   to
   <property name="compiler.args.dbseer_front_end" value="-encoding UTF-8 -source 1.8 ${compiler.args} "/>
2. lauch the dbseer gui
   root$ java -Xmx2048M -jar out/artifacts/dbseer_front_end_jar/dbseer_front_end.jar ./dbseer.ini
   some errors occured:
   `Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
   at dbseer.gui.DBSeerGUI$2.run(DBSeerGUI.java:487)
   at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
   at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
   at java.awt.EventQueue.access$500(EventQueue.java:97)
   at java.awt.EventQueue$3.run(EventQueue.java:709)
   at java.awt.EventQueue$3.run(EventQueue.java:703)
   at java.security.AccessController.doPrivileged(Native Method)
   at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
   at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
   at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
   at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
   at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
   at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
   at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
   at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

thanks a lot !

Thanks for the great effort on DBSeer!
I have implemented the GUI front-end in my MAC and middleware with MaxScale (dbseerroute plugin) in centos. I use MYSQL 5.6. I have successfully used the GUI to connect the middleware. Then I conduct stress testing using OLTP-Bench replaying TPC-C workload through the port that dbseerroute listens to. I have got the results below, meaning that dbseerroute is collecting the log:
1636890641:::server1:::127.0.0.1:::0::: UPDATE WAREHOUSE SET W_YTD = W_YTD + 3857.9599609375 WHERE W_ID = 1 @@@ SELECT W_STREET_1, W_STREET_2, W_CITY, W_STATE, W_ZIP, W_NAME FROM WAREHOUSE WHERE W_ID = 1@@@ UPDATE DISTRICT SET D_YTD = D_YTD + 3857.9599609375 WHERE D_W_ID = 1 AND D_ID = 8@@@ SELECT D_STREET_1, D_STREET_2, D_CITY, D_STATE, D_ZIP, D_NAME FROM DISTRICT WHERE D_W_ID = 1 AND D_ID = 8@@@ SELECT C_FIRST, C_MIDDLE, C_LAST, C_STREET_1, C_STREET_2, C_CITY, C_STATE, C_ZIP, C_PHONE, C_CREDIT, C_CREDIT_LIM, C_DISCOUNT, C_BALANCE, C_YTD_PAYMENT, C_PAYMENT_CNT, C_SINCE FROM CUSTOMER WHERE C_W_ID = 1 AND C_D_ID = 8 AND C_ID = 704@@@ UPDATE CUSTOMER SET C_BALANCE = -40980.8515625, C_YTD_PAYMENT = 40980.76171875, C_PAYMENT_CNT = 15 WHERE C_W_ID = 1 AND C_D_ID = 8 AND C_ID = 704@@@ INSERT INTO HISTORY (H_C_D_ID, H_C_W_ID, H_C_ID, H_D_ID, H_W_ID, H_DATE, H_AMOUNT, H_DATA) VALUES (8,1,704,8,1,'2021-11-14 19:50:41',3857.9599609375,'oofgcaeqr ajfdvorhe')
However, every time I start the stress testing, I get the error 5 in the GUI with no other hint. I use octave and below is the dbseer.ini. I will appreciate it if you could give me some suggestions with the situations! Thanks for your precious time!

[dbseer]
; set a statistical package for DBSeer. DBSeer currently supports Matlab (R2007b and greater) and Octave (4.0.0 and higher).
; set 'matlab' for MATLAB, 'octave' for Octave. Default is 'matlab'.
stat_package=octave
; specify the type of target database. Currently only support 'mysql'.
database=mysql
; specify the type of target OS. Currently only support 'linux'.
os=linux
; specify the number of transactions required to start clustering.
; this is also the minumum number of transactions that DBSeer requires to create a dataset from monitoring data.
dbscan_init_pts=1000

; section for mysql-related configurations
[mysql]
; set a delimiter for transaction log. Must match the log_delimiter value in performancelogroute in MaxScale. (default: :::)
log_delimiter=:::
; set a query delimiter for transaction log. Must match the query_delimiter value in performancelogroute in MaxScale. (default: @@@)
query_delimiter=@@@
~

I tried to run the dbseer GUI from within windows as follows:

$ java -Xmx2048M -jar ./out/artifacts/dbseer_front_end_jar/dbseer_front_end.jar ./dbseer.ini

And got an error window that displays: The binary 'octave' for Octave not found. Program will be terminated.

The dbseer.ini file contains a line:

stat_package=octave

And I have downloaded octave (and other required sub-packages) on Windows.