Git Product home page Git Product logo

online-migration's Introduction

Note: I've reset the merge from Gaëtan Denaisse as it brakes some of my usage. I'll review it more carrefully and merge later. I'll also make it compatible with last mysql-utilities

online-migration

This is a script to keep track of database schema. It helps to check and migrate to a new version of the schema and rollback if necessary.

It uses pt-online-schema-change as backend (http://www.percona.com/doc/percona-toolkit/2.2/pt-online-schema-change.html) and Oracle MySQL utilities (http://dev.mysql.com/doc/workbench/en/mysql-utilities.html).

Requirements:

  • pt-online-schema-change
  • mysql-utilities (1.3.3)
  • mysql-connector-python
  • mysql client
  • perl-DBI (dependency of pt-online-schema-change)
  • perl-DBD-MySQL (dependency of pt-online-schema-chane)

Puppet:

A new type (mysql_schema) and its provider as been added to the project

Extra:

Currently rollback the addition of foreign keys doesn't work because foreign keys are not copied with mysqldbcopy see http://bugs.mysql.com/bug.php?id=63783

A bug in pt-online-schema-change fails the example with the world database, see https://bugs.launchpad.net/percona-toolkit/+bug/1207186

Copyrights:

GPLv2 all rights reserved to Frédéric Descamps <[email protected]>

online-migration's People

Contributors

lefred avatar mpeeters avatar

Stargazers

Hugefiver avatar  avatar weitao avatar  avatar Evan Elias avatar Juanito Fatas avatar  avatar Balaji Subramaniam avatar Andres Montalban avatar  avatar Ricardo Boni avatar  avatar Giuseppe Paterno' avatar Xavier Krantz avatar Marcos Sader avatar Francisco Caballero avatar  avatar Maarten Engelen avatar Johannes Wachter avatar Jan Brauer avatar  avatar Balazs Pocze avatar  avatar Panagiotis PJ Papadomitsos avatar Martin Loy avatar  avatar Daniel Beckham avatar

Watchers

 avatar Gaëtan Denaisse avatar  avatar  avatar James Cloos avatar Balazs Pocze avatar Cédric P. avatar  avatar  avatar

online-migration's Issues

Quoting backquotes for pt-osc

pt-online-schema-change h=127.0.0.1,P=3306,u=root,p=,D="world",t=City --alter="ALTER TABLE world.City DROP INDEX pol_idx, DROP COLUMN PolutionLevel;

"
-bash: world: command not found
-bash: City: command not found

Rollback the migration when creation fails

When for any reason (SQL syntax for example) the migration creation fails, all the steps already done for that migration should be removed.

Ex:

  [root@devel ~]# ./online-migration.py create world sql1.sql "tables creations"
  ERROR : ("Query failed. 1064 (42000): You have an error in your SQL syntax; check the manual that
  corresponds to your MySQL server version for the right syntax to use near ') ENGINE=InnoDB 
   DEFAULT CHARSET=latin1' at line 1", 0) !

On the filesystem, the files xxxx-down.mig and xxxx-up.mig should be removed (xxxx-up.meta is not created when it fails).

The database record with status "running" should be removed too:

mysql> select * from migration_sys;
+----+-------+---------+---------------------+---------------------+---------+
| id | db    | version | start_date          | apply_date          | status  |
+----+-------+---------+---------------------+---------------------+---------+
|  1 | world |       0 | 2014-08-01 21:34:17 | 2014-08-01 21:34:17 | ok      |
|  2 | world |       1 | 2014-08-01 22:57:41 | 2014-08-01 22:57:41 | running |
+----+-------+---------+---------------------+---------------------+---------+

handle error when init and db not existing yet

When init for a database that doesn't exist yet is called, an python staktrace is returned but the folder and the file 0000-up.mig are created.
An entry with satus "ok" is also created in online_migration.migration_sys which is bad !

Cheksum errors when GTID used

If GTID are used, the schema checksum is different as it includes the GTID:

[root@production world]# diff 0000.schema_tmp 0000.schema_tmp.dev
3c3
< SET @@GLOBAL.GTID_PURGED = 'e6045074-4734-11e4-9885-0800272e951c:1-211';


SET @@GLOBAL.GTID_PURGED = '1b6a42a0-4736-11e4-988c-0800272e951c:1-262';

Stored procedure versionning

Would it be possible to add store procedures support to the versionning tool ?
With schema changes comes SP changes in many cases.

I would be glad to help, to do this.

exit if pt-online-schema-change is not found

If pt-online-schema-change is not found, we should stop (until we specify we won't use it, if this new feature
gets implemented).

We avoid this and let bad status:

ERROR : Problem while running :
    pt-online-schema-change h=127.0.0.1,P=3306,u=root,p=,D="world",t=City --alter="add PolutionLevel int;" 
    --execute >>online_migration.log 2>&1

add configuration file for connections

Let's add cfg file to deal with the connections. The file will be /etc/online-migration.cfg and its content will be like:

 [world]
 user=root
 password=fred
 host=127.0.0.1
 port=3340

 [sysbench]
 user=test
 password=test
 host=127.0.0.1
 port=3306 

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.