Comments (16)
ocafebabe
commented on September 6, 2024
4
MariaDB please!
from replicadb.
osalvador
commented on September 6, 2024
No, not yet.
I'll add more databases based on the GitHub votes.
from replicadb.
aaronps
commented on September 6, 2024
My vote: mysql+1
I actually went to read the code to find that mysql is not supported, the:
java.lang.NullPointerException: null
at org.replicadb.ReplicaDB.main(ReplicaDB.java:61) [ReplicaDB-0.8.9.jar:0.8.9]
is not useful. Note that I did check in advance that the mysql connector was NOT on the lib directory within the docker image and mounted it myself.
At the end it is my own fault for not reading the README.md introduction properly where it states clearly what is actually supported.
from replicadb.
KM8Oz
commented on September 6, 2024
mysql please .......
from replicadb.
KM8Oz
commented on September 6, 2024
java.lang.NullPointerException: null
from replicadb.
osalvador
commented on September 6, 2024
We will start working on MySQL support from next month
from replicadb.
Peter-YAN
commented on September 6, 2024
Is mysql supported in v0.10?
Got error after run replicadb:
2021-06-23 18:31:16,301 DEBUG SqlManager:255 User specified connection params. Using properties specific API for making connection.
2021-06-23 18:31:17,154 ERROR ReplicaDB:128 Got exception running ReplicaDB:
java.sql.SQLSyntaxErrorException: Access denied for user 'root'@'192.168.1.%' to database 'wind'
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120) ~[mysql-connector-java-8.0.21.jar:8.0.21]
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97) ~[mysql-connector-java-8.0.21.jar:8.0.21]
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) ~[mysql-connector-java-8.0.21.jar:8.0.21]
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:836) ~[mysql-connector-java-8.0.21.jar:8.0.21]
at com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:456) ~[mysql-connector-java-8.0.21.jar:8.0.21]
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:246) ~[mysql-connector-java-8.0.21.jar:8.0.21]
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:197) ~[mysql-connector-java-8.0.21.jar:8.0.21]
at java.sql.DriverManager.getConnection(DriverManager.java:664) ~[?:1.8.0_242]
at java.sql.DriverManager.getConnection(DriverManager.java:208) ~[?:1.8.0_242]
at org.replicadb.manager.SqlManager.makeSinkConnection(SqlManager.java:267) ~[ReplicaDB-0.10.0.jar:0.10.0]
at org.replicadb.manager.SqlManager.getConnection(SqlManager.java:105) ~[ReplicaDB-0.10.0.jar:0.10.0]
at org.replicadb.manager.SqlManager.truncateTable(SqlManager.java:387) ~[ReplicaDB-0.10.0.jar:0.10.0]
at org.replicadb.manager.SqlManager.preSinkTasks(SqlManager.java:508) ~[ReplicaDB-0.10.0.jar:0.10.0]
at org.replicadb.ReplicaDB.processReplica(ReplicaDB.java:88) [ReplicaDB-0.10.0.jar:0.10.0]
at org.replicadb.ReplicaDB.main(ReplicaDB.java:42) [ReplicaDB-0.10.0.jar:0.10.0]
2021-06-23 18:31:17,172 INFO ReplicaDB:49 Total process time: 1135ms
from replicadb.
osalvador
commented on September 6, 2024
Hi, v0.10 is still in beta, but MySQL is supported as source.
I think your error is about permissions for the root user?
from replicadb.
Peter-YAN
commented on September 6, 2024
Hi, v0.10 is still in beta, but MySQL is supported as source.
I think your error is about permissions for the
rootuser?
^_^ yes, sloved it after update root permissions. Thanks.
from replicadb.
Peter-YAN
commented on September 6, 2024
But i use incremental mode, i got the following error.
_2021-06-24 09:19:49,787 INFO MySQLManager:174 Creating staging table with this command: CREATE TABLE public.ASHAREAGENCYrepdb4310 AS (SELECT * FROM ASHAREAGENCY WHERE 1 = 0 )
2021-06-24 09:19:49,822 ERROR ReplicaDB:128 Got exception running ReplicaDB:
java.sql.SQLSyntaxErrorException: INSERT, CREATE command denied to user 'root'@'client.nfs.fid.sxzq.com' for table 'ASHAREAGENCYrepdb4310'
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120) ~[mysql-connector-java-8.0.21.jar:8.0.21]
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97) ~[mysql-connector-java-8.0.21.jar:8.0.21]
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) ~[mysql-connector-java-8.0.21.jar:8.0.21]
at com.mysql.cj.jdbc.StatementImpl.executeUpdateInternal(StatementImpl.java:1335) ~[mysql-connector-java-8.0.21.jar:8.0.21]
at com.mysql.cj.jdbc.StatementImpl.executeLargeUpdate(StatementImpl.java:2108) ~[mysql-connector-java-8.0.21.jar:8.0.21]
at com.mysql.cj.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1245) ~[mysql-connector-java-8.0.21.jar:8.0.21]
at org.replicadb.manager.MySQLManager.createStagingTable(MySQLManager.java:175) ~[ReplicaDB-0.10.0.jar:0.10.0]
at org.replicadb.manager.SqlManager.preSinkTasks(SqlManager.java:498) ~[ReplicaDB-0.10.0.jar:0.10.0]
at org.replicadb.ReplicaDB.processReplica(ReplicaDB.java:88) [ReplicaDB-0.10.0.jar:0.10.0]
at org.replicadb.ReplicaDB.main(ReplicaDB.java:42) [ReplicaDB-0.10.0.jar:0.10.0]
2021-06-24 09:19:49,831 INFO SqlManager:475 Dropping staging table with this command: DROP TABLE public.ASHAREAGENCYrepdb4310
2021-06-24 09:19:49,833 ERROR ReplicaDB:146 java.sql.SQLSyntaxErrorException: DROP command denied to user 'root'@'client.nfs.fid.sxzq.com' for table 'ASHAREAGENCYrepdb4310'_
According to log, the reason the db permission, but all permission are grant to root.
mysql> show grants for 'root';
+------------------------------------------------+
| Grants for root@% |
+------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' |
| GRANT ALL PRIVILEGES ON `wind`.* TO 'root'@'%' |
+------------------------------------------------+
2 rows in set (0.00 sec)
from replicadb.
Peter-YAN
commented on September 6, 2024
^_^，
after change conf, set sink.staging.schema=test.
2021-06-24 09:53:29,380 INFO MySQLManager:174 Creating staging table with this command: CREATE TABLE test.ASHAREAGENCYrepdb5210 AS (SELECT * FROM ASHAREAGENCY WHERE 1 = 0 )
2021-06-24 09:53:29,391 INFO SqlManager:386 Truncating sink table with this command: TRUNCATE TABLE test.ASHAREAGENCYrepdb5210
2021-06-24 09:53:29,399 INFO ReplicaTask:36 Starting TaskId-0
2021-06-24 09:53:29,399 DEBUG ManagerFactory:47 Trying with scheme: jdbc:mysql:
2021-06-24 09:53:29,400 DEBUG ManagerFactory:47 Trying with scheme: jdbc:mysql:
2021-06-24 09:53:29,400 DEBUG SqlManager:212 No connection parameters specified. Using regular API for making connection.
2021-06-24 09:53:29,418 DEBUG SqlManager:255 User specified connection params. Using properties specific API for making connection.
2021-06-24 09:53:29,523 DEBUG SqlManager:131 TaskId-0: Using fetchSize for next query: 5000
2021-06-24 09:53:29,523 INFO SqlManager:141 TaskId-0: Executing SQL statement: SELECT * FROM ASHAREAGENCY
2021-06-24 09:53:29,523 INFO SqlManager:148 TaskId-0: With args:
2021-06-24 09:53:29,888 WARN ConnManager:188 Options source-columns and sink-columns are null, getting from Source ResultSetMetaData: OBJECT_ID,S_INFO_WINDCODE,S_AGENCY_NAME,S_RELATION_TYPCODE,S_BUSINESS_TYPCODE,S_AGENCY_NAMEID,BEGINDATE,ENDDATE,SEQUENCE,ANN_DATE,OPDATE,OPMODE
2021-06-24 09:53:29,891 INFO MySQLManager:153 Loading data with this command: LOAD DATA LOCAL INFILE 'dummy' INTO TABLE test.ASHAREAGENCYrepdb5210 CHARACTER SET UTF8 FIELDS TERMINATED BY X'1F' (OBJECT_ID,S_INFO_WINDCODE,S_AGENCY_NAME,S_RELATION_TYPCODE,S_BUSINESS_TYPCODE,S_AGENCY_NAMEID,BEGINDATE,ENDDATE,SEQUENCE,ANN_DATE,OPDATE,OPMODE)
2021-06-24 09:55:00,048 ERROR ReplicaDB:128 Got exception running ReplicaDB:
java.lang.UnsupportedOperationException: MySQL does not yet support data insertion
at org.replicadb.manager.MySQLManager.mergeStagingTable(MySQLManager.java:182) ~[ReplicaDB-0.10.0.jar:0.10.0]
at org.replicadb.manager.SqlManager.postSinkTasks(SqlManager.java:521) ~[ReplicaDB-0.10.0.jar:0.10.0]
at org.replicadb.ReplicaDB.processReplica(ReplicaDB.java:120) [ReplicaDB-0.10.0.jar:0.10.0]
at org.replicadb.ReplicaDB.main(ReplicaDB.java:42) [ReplicaDB-0.10.0.jar:0.10.0]
2021-06-24 09:55:00,058 INFO SqlManager:475 Dropping staging table with this command: DROP TABLE test.ASHAREAGENCYrepdb5210
2021-06-24 09:55:00,080 INFO ReplicaDB:49 Total process time: 91621ms
Is MySQL just support complete mode?
Could I sync multi tables once?
from replicadb.
osalvador
commented on September 6, 2024
Hi @Peter-YAN ,
The latest release now has the full support for MySQL and MariaDB, you can download and try it.
To replicate multiple tables at once, see this comment: #27 (comment)
from replicadb.
Peter-YAN
commented on September 6, 2024
Hi @Peter-YAN ,
The latest release now has the full support for MySQL and MariaDB, you can download and try it.
To replicate multiple tables at once, see this comment: #27 (comment)
Hi @osalvador ,
I have tried version 0.10.2, the incremental mode is not supported. ReplicaDB configuration wizard is not updated yet.
the error:
E,S_AGENCY_NAMEID,BEGINDATE,ENDDATE,SEQUENCE,ANN_DATE,OPDATE,OPMODE
2021-06-25 09:38:19,242 INFO MySQLManager:166 Loading data with this command: LOAD DATA LOCAL INFILE 'dummy' INTO TABLE test.ASHAREAGENCYrepdb4410 CHARACTER SET UTF8 FIELDS TERMINATED BY X'1F' (OBJECT_ID,S_INFO_WINDCODE,S_AGENCY_NAME,S_RELATION_TYPCODE,S_BUSINESS_TYPCODE,S_AGENCY_NAMEID,BEGINDATE,ENDDATE,SEQUENCE,ANN_DATE,OPDATE,OPMODE)
2021-06-25 09:38:19,250 ERROR ReplicaTask:72 ERROR in TaskId-0 inserting data to sink table: (conn=63) The receiver is not a wrapper and does not implement the interface
2021-06-25 09:38:19,251 ERROR ReplicaDB:128 Got exception running ReplicaDB:
java.util.concurrent.ExecutionException: java.sql.SQLSyntaxErrorException: (conn=63) The receiver is not a wrapper and does not implement the interface
at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_242]
at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_242]
at org.replicadb.ReplicaDB.processReplica(ReplicaDB.java:109) [ReplicaDB-0.10.2.jar:0.10.2]
at org.replicadb.ReplicaDB.main(ReplicaDB.java:42) [ReplicaDB-0.10.2.jar:0.10.2]
Caused by: java.sql.SQLSyntaxErrorException: (conn=63) The receiver is not a wrapper and does not implement the interface
at org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.createException(ExceptionFactory.java:62) ~[mariadb-java-client-2.7.3.jar:?]
at org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.create(ExceptionFactory.java:181) ~[mariadb-java-client-2.7.3.jar:?]
at org.mariadb.jdbc.MariaDbStatement.unwrap(MariaDbStatement.java:1489) ~[mariadb-java-client-2.7.3.jar:?]
at org.replicadb.manager.MySQLManager.insertDataToTable(MySQLManager.java:59) ~[ReplicaDB-0.10.2.jar:0.10.2]
at org.replicadb.ReplicaTask.call(ReplicaTask.java:70) ~[ReplicaDB-0.10.2.jar:0.10.2]
at org.replicadb.ReplicaTask.call(ReplicaTask.java:14) ~[ReplicaDB-0.10.2.jar:0.10.2]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_242]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_242]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_242]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_242]
2021-06-25 09:38:19,261 INFO SqlManager:475 Dropping staging table with this command: DROP TABLE test.ASHAREAGENCYrepdb4410
config:
from replicadb.
osalvador
commented on September 6, 2024
Hi @Peter-YAN ,
There was a problem having the MySQL and MariaDB driver in the same classpath. Since MariaDB driver has better performance than MySQL driver, I'll use the first one for both databases.
The latest release (0.10.5) now has integration test for MySQL as source and sink and test have been passed, you can download and try it. Sorry for the inconvenience.
Regards!
from replicadb.
Peter-YAN
commented on September 6, 2024
Hi @Peter-YAN ,
There was a problem having the MySQL and MariaDB driver in the same classpath. Since MariaDB driver has better performance than MySQL driver, I'll use the first one for both databases.
The latest release (0.10.5) now has integration test for MySQL as source and sink and test have been passed, you can download and try it. Sorry for the inconvenience.
Regards!
Thanks.
Hi @Peter-YAN ,
There was a problem having the MySQL and MariaDB driver in the same classpath. Since MariaDB driver has better performance than MySQL driver, I'll use the first one for both databases.
The latest release (0.10.5) now has integration test for MySQL as source and sink and test have been passed, you can download and try it. Sorry for the inconvenience.
Regards!
Hi @osalvador Thanks. great, succeed to run replicadb in incremental mode.
But i don't kown the detail about mysql merge operation.
could you tell me the process steps?
Is it only use primary key to decide update or ignore??
Or all the feilds are compaired to decide ignoring or updating?
Regards.
from replicadb.
osalvador
commented on September 6, 2024
Hi @Peter-YAN ,
Here is the documentation about incremental mode.
The merge statement (ON DUPLICATE KEY UPDATE in MySQL) only use the primary keys to decide if the row is updated because it has been modified in the source database, or inserted because it is a new row, but the row is never ignored.
Regards!
from replicadb.
Related Issues (20)
- set up multiple tables HOT 2
- Error when trying to copy colums with special characters HOT 1
- docker: no matching manifest for linux/arm64/v8 in the manifest list entries. HOT 2
- The connection string contains an empty host '[]'. HOT 2
- How to give where condition in "source.where" HOT 1
- The connection string contains an empty host '[]'. HOT 2
- Is replicadb supports real time or point in time sync data replication? HOT 1
- please share the conf file for mssql to mssql
- Complete data is not copied(total rows are around 4,50,000)
- How many rows of data can be copied ? HOT 15
- / FS getting full HOT 10
- Password as a Variable HOT 1
- Is complete load mandatory or can we use Incremental directly?
- Max transaction size limitation with Galera Cluster HOT 1
- mysql - mssql HOT 3
- MSSQL specify number of jobs - Error
- Rounding decimal values during syncronization
- PostgreSQL Sequence Reset
- DB2 To SQL Server
- Incorrect BLOB data copied from MariaDB (LONGBLOB) to PostgreSQL (BYTEA) HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from replicadb.