See http://sia.tech - Basically you can rent disk space on other user's computers. With 6-times redundancy.

API docs are here: https://github.com/NebulousLabs/Sia/tree/master/doc

Transform this https://github.com/dizda/CloudBackupBundle/blob/master/DependencyInjection/DizdaCloudBackupExtension.php

Into something easier to read https://github.com/nbobtc/BitcoindBundle/blob/master/DependencyInjection/BitcoindExtension.php

now KnpLabs / KnpGaufretteBundle uses stable releases of gaufrette (~0.1.7)

and dizda / CloudBackupBundle need 0.2

Has the bundle a restore option integrated?

Thanks ;)

As requested by @jongotlin in #110 (comment)

@dizda Are you fine with tagging 3.5?

@dizda hello.
I try to create backup of my MySQL database, but I have some problem, when I run command:

php app/console --env=prod dizda:backup:start

show error:

php app/console --env=prod dizda:backup:start
[2015-10-18 11:56:22] app.CRITICAL: [dizda-backup] Unexpected exception. {"exception":"[object] (RuntimeException(code: 0): Unknown suffix ''' used for variable 'port' (value ''3306'')\r\nmysqldump: Error while setting value ''3306'' to 'port'\r\n at vendor\\dizda\\cloud-backup-bundle\\Dizda\\CloudBackupBundle\\Database\\BaseDatabase.php:45)"}
Something went terribly wrong. We could not create a backup. Read your log files to see what caused this error.

It seems what problem in suffix '''. How I can fix it?

Hi @dizda !

Thanks for your awesome work on this bundle.

I thought there would be a "restore" command line.
Because if there is a backup, that would be great to have something to restore.

Is it something simple to implement ?

Cheers,

It is possible to change the composer.json to have it on symfony 2.3 ?

Hello,

What shall i put in the "cloud_storages: gaufrette: service_name:" section ?

i always got :

• Dumping MySQL database... OK
  • Compressing archive... OK
  • Temporary files have been cleared.

and i don't get where stored zipped file is.
Thanks

Hi,

is it possible to run a backup from an action in a symfony controller?

I'm trying to implement a backup service which will be called via rest from a management system.
This service should start a backup process immediately.

I tried to install CloudBackupBundle but the required version of knplabs/gaufrette (0.2) does not exist ! The following error occurs during installation :

• Installation request for dizda/cloud-backup-bundle dev-master -> satisfiable by dizda/cloud-backup-bundle[dev-master].
  • dizda/cloud-backup-bundle dev-master requires knplabs/gaufrette 0.2.*@dev -> no matching package found.

I'm using CloudBackupBundle to backup my database and upload it to Dropbox every hour with cron. Everything was working just fine since yesterday. Now I'm receiving mails like this :

Cannot extract login CSRF token.( code: 268699650; file: /Users/clement/Projets/draftquest/draftquest/vendor/hakre/dropbox-uploader/DropboxUploader.php; line: 260)

The same thing happens with dizda:backup:start :

- Dumping MySQL database... OK
- Compressing archive...  OK
- Uploading to Dropbox... 


  [Exception]                       
  Cannot extract login CSRF token.  



dizda:backup:start [-F|--folders] [processor]

My dropbox credentials didn't change and I can still use them from the dropbox website. My dropbox is far from full. I tried to composer update but it didn't solved the problem. Any ideas ?

Look up the discussion we had there #30 with @lexxpavlov

I propose to mark the old Dropbox client as @deprecated, then remove it for 4.0.0 release.

https://github.com/dizda/CloudBackupBundle/blob/master/Client/DropboxClient.php
https://github.com/dizda/CloudBackupBundle/blob/master/DependencyInjection/Configuration.php#L52

What do you think guys?

I just installed this bundle, I see the command being generated as:

"mysqldump --defaults-extra-file="/Users/user/Sites/app/var/cache/prod/backup/mysql/mysql.cnf" db_dev  > '/Users/user/Sites/app/var/cache/prod/backup/mysql/db_dev.sql'"

But this file does not exist (I cleared the cache after install for both dev and prod).

prod/backup/mysql/mysql.cnf

It just backs up an empty file with no errors.

Hello,

Right now, the bundle support just one gaufrette filesystem at a time.
Example :

dizda_cloud_backup:
    processor:
        type: tar
    cloud_storages:
        gaufrette:
            service_name: amazon_backup_filesystem  # Gaufrette filesystem service name

Is it possible to have more filesystems ? I would have for instance a local and a distant gaufrette filesystem like :

dizda_cloud_backup:
    processor:
        type: tar
    cloud_storages:
        gaufrette:
            service_name: 
                - local_backup_filesystem
                - amazon_backup_filesystem

So i have enabled upload to dropbox

dropbox_sdk:
    remote_path:    appv2-db
    access_token:   ---secret---

When i try to start backup, i got this error

[2016-05-03 14:11:43] app.CRITICAL: [dizda-backup] Unexpected exception. {"exception":"[object] (Dizda\\CloudBackupBundle\\Exception\\UploadException(code: 0): Invalid path \"directory/app/cache/prod/backup/../backup_compressed/app_2016-05-03_14-11-43.tar\". at /var/www/api.app.dev/vendor/dizda/cloud-backup-bundle/Dizda/CloudBackupBundle/Client/DropboxSdkClient.php:38)"} []

I check there is no folder backup or backup_compressed in prod cache folder.
How come?

Nevermind, after i debug i found the problem was because the config must be using /appv2-db.
Will be greate if the library showing proper error message.

I'm trying to configure CloudBackupBundle to work in my developpement environnement (MAMP Pro). But when I try php app/console dizda:backup:start I get the following error :

[RuntimeException]
Warning: Using a password on the command line interface can be insecure.
mysqldump: Got error: 2002: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2) when trying to connect

Other console commands using mysql like php app/console doctrine:schema:update are working fine. Am I doing something wrong here ?

Here is my config.yml

dizda_cloud_backup:
    # By default backup files will have your servers hostname as prefix
    # such as: hostname_2014-01-01_21-08-39.tar
    output_file_prefix: ~
    timeout: 300
    error_notification:
        from: [email protected]
        to: [ [email protected] ]
    processor:
        type: tar # Required: tar|zip|7z
    folders: [ web/uploads ]
    cloud_storages:
        # Dropbox account credentials (use parameters in config.yml and store real values in prameters.yml)
        dropbox:
            user:     [email protected] # Required
            password: password  # Required
            remote_path: "/" # Not required, default "/", but you can use path like "/Accounts/backups/"

    databases:
        mysql:
            db_host:     "%database_host%"
            db_port:     "%database_port%"
            database:   "%database_name%"
            db_user:     "%database_user%"
            db_password: "%database_password%"

And my parameters.yml :

parameters:
    database_driver: pdo_mysql
    database_host: localhost
    database_port: null
    database_name: draftquest
    database_user: root
    database_password: pass

Hello, @dizda

When I run command php app/console --env=prod dizda:backup:start zip from documentation: https://github.com/dizda/CloudBackupBundle/blame/master/README.md#L212
I have an error(RuntimeException) message: Too many arguments

Is something wrong with this command or we have an error in documentation? Thank you.

Hello,

I have two problems that I would like to share and to know if someone else have same issues:

• I have the backup command written in crontab and it is executed everyday. After the execution of the command I always receive following email:

Curl error: (#52) Empty reply from server( code: 268697857; file: /var/www/vhosts/my.domain.com/my_symfony_project/vendor/hakre/dropbox-uploader/DropboxUploader.php; line: 241)

• Second issue is that I have configured in config.yml two email directions to receive backup notifications and only one of them receive this error.

  Someone else has similar problems?

Hello,
I get this error

Error: [DropboxUploader] #268699650 Cannot extract login CSRF token.

Can you help me?

I don't know why or if it is related to the bundle or my machine.

On prod machine:

• Version 1.6.3 runs fine
• dev-master stalls at compressing archive

On local machine:

• Version 1.6.3 runs fine
• dev-master runs fine

I will investigate in this further.

It could be fun to try to support Symfony 3.0. Is this possible for the next major version?

This is up for grabs for anybody. It would be nice to have an adapter to Flysystem.

Can I set:

    cloud_storages:
        dropbox:
            user:     %username%
            password:  %password%

from db and not from yml file?

Thanks.
v.

Note this issue related to PR #88 and #75.

I am working on an app that makes use of Dizda_cloud_backup to be able to make backups of the DB when needed.

Then the same app also has a bundle written for it that allows the entire app to be upgraded when needed. But upgrades can go wrong at times, so it is written to make a backup first of the DB and the app files itself using dizda. To be able to also backup the app files I have to tell dizda about its location.

This is where the getter comes in as it can be used to access the ProcessorManager and the addFolder settter of the ProcessorManager is then used to add the said folder.

And a command that will be called from a cron job will also be added to the app. This command will make a backup using dizda (of only the DBs) and then email it to a specific user. Here access to the ProcessManager is also needed to be able to call getArchivePath to include it in the email.

Hi,
I'm wondering some issues trying backup to my dropbox account.

php app/console dizda:backup:start

[Exception]
Login unsuccessful.

The credentials seems to work fine logging from dropbox.com.

Thank you,
Cesare

Awesome lib, would be really useful to ignore tables from the dump (can be provided via config). I found a hacky solution like so:

# database/MySQL.php

    protected function getCommand()
    {
        return sprintf('mysqldump %s %s %s > %s',
            $this->auth,
            $this->database,
            '--ignore-table=database.table', // Add this (optional, if empty/blank then is ignored)
            $this->dataPath.$this->fileName);
    }

I have a database which is quite large, I only want to backup the important tables (some of the other crap is useful to have - but not required for backup (it's a temporary holding table)).

Perhaps just have a config reference named "ignore tables" which holds an array of tables to ignore.

also, could we have an option to name the outputted (copied) tar file (instead of using the computer name - "Macbook Pro" - on my pc), or even name the first part the name of the databases exported? (database_name-2015-10-22-10-30-20.tar) - rather than (Macbook-Pro-2015-10-22-10-30-20.tar)

http://stackoverflow.com/questions/425158/skip-certain-tables-with-mysqldump

Thanks!

It is great that so many people are using this bundle and that the development rushes forward. I would like to have a Upgrade.md or a Changelog.md that highlights the important changes and tells a user what steps to take to upgrade the bundle.

Why not integrate something like Gaufrette that also is available as Symfony2 bundle instead of developing every cloud storage solution again ? That would mean to have Amazon S3, FTP, MongoGrid, Rackspace and many other backup locations at once.

Here is a list of available adapters:

https://github.com/KnpLabs/Gaufrette/tree/master/src/Gaufrette/Adapter

The error:

The annotation "@void" in method Dizda\CloudBackupBundle\Splitter\BaseSplitter::executeSplit() was never imported. Did you maybe forget to add a "use" statement for this annotation?

i corrected it (line 81-83) by removing one oh the * in line 81 (from /** to /*) it's preventing to consider it like an annotation.

Maybe its should be an option to update the bundle?

Hello, thanks for the bundle, it's good job.

I haven't find anywhere in the doc if there is an option for conditional folder backup.

there is the folder option in the config :
folders: [ web/uploads , other/folder ]

It's working fine, but I don't alway want to save all my images as this is a big folder specialy with Capifony
Is there a way to do this ? Maybe as a command line parameter ?

Thx.

Hi,

It seems it's not possibile to do a backup if not running the command from a shell.

Extracting the code within the console command would allow to create a service and use it anywhere in our application.

Thanks

➜  composer require dizda/cloud-backup-bundle           
Please provide a version constraint for the dizda/cloud-backup-bundle requirement: dev-master
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Installation request for dizda/cloud-backup-bundle dev-master -> satisfiable by dizda/cloud-backup-bundle[dev-master].
    - dizda/cloud-backup-bundle dev-master requires knplabs/knp-gaufrette-bundle ~0.1.7 -> no matching package found.
  Problem 2
    - dizda/cloud-backup-bundle dev-master requires knplabs/knp-gaufrette-bundle ~0.1.7 -> no matching package found.
    - godisco/backend 2.3.x-dev requires dizda/cloud-backup-bundle dev-master -> satisfiable by dizda/cloud-backup-bundle[dev-master].
    - Installation request for godisco/backend 2.3.x-dev -> satisfiable by godisco/backend[2.3.x-dev].

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion> for more details.

Read <http://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.

Installation failed, reverting ./composer.json to its original content.

• That's really weird.. I have knplabs/knp-gaufrette-bundle installed with dev-master version

Will it be available any time soon? What can I do to help?

From this commit, backup does not work anymore for me.

It throws a RuntimeException after tar execution ($database-compression()).

Indeed, this executes the command "tar --exclude=site_2013_07_11-11_36_48.tar -czf (...)app/cache/dev/db/site_2013_07_11-11_36_48.tar -C (...)/app/cache/dev/db/ . 2>/dev/null"

I tried to display $process->getExitCode() after executing ->run(), it returns "1" !

I also tried to type the command manually and the exit code is 0 (using echo $? to check the exit code).

To fix that I just rollback commit 6036b6a (18 days ago)

+ $process->run();

•    $process->run(function ($type, $data) {
  

•        if (Process::ERR == $type) {
  

•            throw new \RuntimeException($data);
  

•        }
  

•    });
  

•    if (!$process->isSuccessful()) {
  

•        throw new \RuntimeException($process->getErrorOutput());
  

•    }
  

Hi,

i follwed the install instructions. At first try I get:

$ php app/console dizda:backup:start -vvv
- Dumping MySQL database...


  [RuntimeException]
  Unknown suffix ''' used for variable 'port' (value ''3306'')
  mysqldump: Error while setting value ''3306'' to 'port'



Exception trace:
 () at G:\Projekte\PHP\dawei\vendor\dizda\cloud-backup-bundle\Dizda\CloudBackupBundle\Databases\BaseDatabase.php:46
 Dizda\CloudBackupBundle\Databases\BaseDatabase->execute() at G:\Projekte\PHP\dawei\vendor\dizda\cloud-backup-bundle\Dizda\CloudBackupBundle\Database
\MySQL.php:57
 Dizda\CloudBackupBundle\Databases\MySQL->dump() at G:\Projekte\PHP\dawei\vendor\dizda\cloud-backup-bundle\Dizda\CloudBackupBundle\Command\BackupComm
nd.php:87
 Dizda\CloudBackupBundle\Command\BackupCommand->execute() at G:\Projekte\PHP\dawei\vendor\symfony\symfony\src\Symfony\Component\Console\Command\Comma
d.php:257
 Symfony\Component\Console\Command\Command->run() at G:\Projekte\PHP\dawei\vendor\symfony\symfony\src\Symfony\Component\Console\Application.php:882
 Symfony\Component\Console\Application->doRunCommand() at G:\Projekte\PHP\dawei\vendor\symfony\symfony\src\Symfony\Component\Console\Application.php:
95
 Symfony\Component\Console\Application->doRun() at G:\Projekte\PHP\dawei\vendor\symfony\symfony\src\Symfony\Bundle\FrameworkBundle\Console\Applicatio
.php:96
 Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at G:\Projekte\PHP\dawei\vendor\symfony\symfony\src\Symfony\Component\Console\Applicatio
.php:126
 Symfony\Component\Console\Application->run() at G:\Projekte\PHP\dawei\app\console:27


dizda:backup:start [-F|--folders] [processor]

My settings:

dizda_cloud_backup:
    output_file_prefix: backup
    timeout: 300
    processor:
        type: tar # Required: tar|zip|7z
        options:
            compression_ratio: 6
            password: %backup_archive_password%
            # Split into many files of `split_size` bytes
            split:
                enable: false # Default false
                split_size: 1000 # Make each zip files no larger than "split_size" in bytes
                storages: [ Dropbox ] # Which cloud storages will upload split files
    folders: [ uploads  ]
    cloud_storages:
        # Dropbox account credentials (use parameters in config.yml and store real values in prameters.yml)
        dropbox:
            user:        %backup_dropbox_user%
            password:    %backup_dropbox_password%
            remote_path: %backup_dropbox_remote_path%
    databases:
        mysql:
            all_databases: false # Only required when no database is set
            database:      %database_name%
            db_host:       %database_host%
            db_port:       %database_port%
            db_user:       %database_user%
            db_password:   %database_password%

parameters:
    database_driver: pdo_mysql
    database_host: 127.0.0.1
    database_port: 3306
    database_name: develop
    database_user: secret
    database_password: secret

    backup_archive_password: secret
    backup_dropbox_user: secret
    backup_dropbox_password: secret
    backup_dropbox_remote_path: /

Maybe its couse Iam using windows?

https://github.com/dropbox/dropbox-sdk-php

[Symfony\Component\Process\Exception\ProcessTimedOutException]
The process "tar c -C /var/www/site.com/releases/20140929123936/app/cache/dev/backup/ . | gzip -6 > /var/www/site.com/releases/20140929123936/app/cache/dev/backup/../backup_compressed/hostname_2014-09-29_13-42-37.tar" exceeded the timeout of 60 seconds.

Hi, can we use this package without a symfony installation?

Thanks

This is a suggestion to which events we should dispatch and exceptions we should throw. Please work with me and give me feedback.

What events to we got?

An event describes a thing that has happened. The event should be broad casted and not modified (you cant modify a radio borad cast). Nor should priority matter.

So, looking at our bundle. What does it? What happens?
The only place we do stuff is in the Managers. We have 3 managers and one "manager manager" (BackupManager).

• DatabaseManager steps throw each DatabaseInterface and dumps it.
• ProcessorManager does 3 things. It copies folders, compresses files and removes backup files.
• ClientManager steps throw each ClientInterface and upload every file.
• BackupManager handles Exceptions had initiate the next step in the process.

(We should extract the filesystem actions from the ProcessorManager)

I suggest events accordingly:

With these new events a user could easily:

• Verify that the upload is completed (UploadEvent)
• Copy in more files to the zip (FilesystemEvent)
• Calculate hashes for the compressed files ( ProcessorEvent)
• Notify other services that the backup has happened (BackupEvent)
• etc etc.

What exceptions might occur?

I suggest we should create these exception classes and then make sure that our managers do not throw any other exceptions.

• DatabaseException
• ProcessorException
• FilesystemException
• UploadException

All of the above should implement BackupException.

Centralized logging.

With these events we could centralize the logging into an event listener. We could also make sure to catch and logg all exceptions from the BackupManager

Hi Nyholm! I was created an issue with this feature request :)

https://github.com/bachvtuan/Backup-To-Google-Drive

Do you think adding zipsplit as option a good idea? For Dropbox at least will help a lot to bypass single size limit.

If yes let me know to do it and PR.

Thanks a lot.

Hi,

I am using your bundle successfully for a few month now but I only used it on my debug server.
But I do not know why I can't do a backup on my production server as I have this exception occuring.
My environment is the same on the debug and production server (Debian 7), using php 5.6.24 and Symfony 2.8 LTS

Do you know how can I fix this ? :(
Please do not hesitate to ask me more details as it is my very first issue on a public repo.
Thanks for everything,

Harold

The logs are :

[2017-02-20 10:58:14] app.CRITICAL: [dizda-backup] Unexpected exception. {"exception":"[object] (UnexpectedValueException(code: 0): RecursiveDirectoryIterator::__construct(/var/www/html/api.hipok.com/app/..//dev): failed to open dir: No such file or directory at /var/www/html/api.hipok.com/vendor/symfony/symfony/src/Symfony/Component/Filesystem/Filesystem.php:435)"} 
Something went terribly wrong. We could not create a backup. Read your log files to see what caused this error.

The configuration :

    # By default backup files will have your servers hostname as prefix
    # such as: hostname_2014-01-01_21-08-39.tar
    output_file_prefix: api_hipok
    timeout: 300
    processor:
        type: tar # Required: tar|zip|7z
        options:
            compression_ratio: 6
            password: ******
            # Split into many files of `split_size` bytes
            split:
                enable: false # Default false
                split_size: 1000 # Make each zip files no larger than "split_size" in bytes
                storages: [ Dropbox ] # Which cloud storages will upload split files
    folders: [ web ]
    cloud_storages:
        # Using dropbox via official API. You need to add "dropbox/dropbox-sdk": "1.1.*" in your composer.json file
        dropbox_sdk:
            remote_path: /api.hipok.com # Required. Path to upload files (where the root '/' will be application folder)
            access_token: MYACCESSTOKEN # Required. Access token provided by DropBox to authenticate your application. You can follow instructions at https://www.dropbox.com/developers/core/start/php

    databases:
        mysql:
            all_databases: false # Only required when no database is set
            database: '%database_name%'          # Required if all_databases is false
            db_host: '%database_host%'   # This, and following is not required and if not specified, the bundle will take ORM configuration in parameters.yml
            db_port: '%database_port%'           # Default 3306
            db_user: '%database_user%'
            db_password: '%database_password%'```

Hello, thanks for your work.

It may be great to check if mysqldump is available and put an error message if needed.

Currently, without mysqldump (on windows) callable from the command line, here are the logs :

[2016-11-28 14:55:05] app.INFO: [dizda-backup] Dumping MySQL database [] []
[2016-11-28 14:55:32] app.INFO: [dizda-backup] Dumping MySQL database [] []
[2016-11-28 14:55:39] app.CRITICAL: [dizda-backup] Unexpected exception. []
[2016-11-28 14:55:39] app.INFO: [dizda-backup] Cleaning up after us. [] []

I get this error:

Cannot extract 't'! (form action is 'https://dl-web.dropbox.com/upload')

Any idea?
v.

hello this script just work in symfony ?

Would be great if you set tags after merging/pushing.

1.0.0
1.0.1

and so on..

0.0.X+1 = BugFix
0.X+1.0 = Feature
X+1.0.0 = Code Break

Hi,
This is my conf :

config.xml
dizda_cloud_backup:
# By default backup files will have your servers hostname as prefix
# such as: hostname_2014-01-01_21-08-39.tar
output_file_prefix: hostname
timeout: 300
processor:
type: tar # Required: tar|zip|7z
options:
compression_ratio: 6
password: qwerty
# Split into many files of split_size bytes
split:
enable: false # Default false
split_size: 1000 # Make each zip files no larger than "split_size" in bytes
storages: [ Gaufrette ] # Which cloud storages will upload split files
folders: [ web/bdd ]
cloud_storages:
# Local storage definition
local:
path: C:\wamp\www\trunkRefonte
gaufrette:
service_name: # Gaufrette filesystem(s) service name
- foo_filesystem
databases:
mysql: # When no parameters is specified under mysql, the bundle taking those from parameters.yml

knp_gaufrette:
adapters:
foo:
local:
directory: 'C:\wamp\www\trunkRefonte'
filesystems:
bar:
adapter: foo
alias: foo_filesystem

I would like to backup my bdd in localhost, but i think my problem is in adapters/foo/local/directory or cloud_storages/local/path. I do not know which path I ought to put in this section.

Could you help me ?

Thanks