If preserve-paths are set, then when running Composer installs or updates inside Docker containers, the allocated CPU and memory get saturated, and eventually, the process dies with a SIGKILL.

This happens even on my iMac Pro with 10 threads, 32 GB memory, and 4 GB swap allowed for Docker's use, and even with PHP's memory_limit set to -1.

If preserve-paths are not set, then Composer installs and updates proceed as they should inside Docker containers. They also proceed as they should with Laravel and Symfony apps. But something preserve-paths does seems not to play nicely with Docker.

Here's the preserve-paths setting I'm using:

"preserve-paths": [
      ".env",
      ".gitignore",
      ".htaccess",
      "robots.txt",
      "web/modules/custom",
      "web/themes/custom",
      "web/libraries",
      "web/sites/<obfuscated>"
    ],

The total file size of everything in preserve-paths doesn't seem to matter. The same thing happens with this, specifying files of negligible size:

"preserve-paths": [
      ".env",
      ".gitignore",
      ".htaccess",
      "robots.txt"
    ],

I had wondered whether an interaction with cweagans/composer-patches might be at play, but the same thing happens if I leave out patches.

Completely removing xdebug from PHP in the compiled Docker container does not solve the issue, either, and neither does setting the environment variable COMPOSER_MEMORY_LIMIT to -1 (just to make sure the unlimited memory setting from PHP CLI was actually taking).

The problem does not happen outside Docker containers, even on machines with fewer resources than what I'm allowing Docker on my iMac Pro.

I am encountering the array as null error when first running composer install:

Steps to Reproduce

git clone [email protected]:devshop-packages/devshop-control-template.git`
cd devshop-control-template
composer install
...
 [ErrorException]                                     
  Trying to access array offset on value of type null  
                                                       

A second call to composer install works. To reproduce again, remove vendor.

rm -rf vendor composer.lock
composer install
...
 [ErrorException]                                     
  Trying to access array offset on value of type null  
                                                       

Fix

The fix has already be committed, just doesn't have a tag:
4c5f62f

Workaround

composer require drupal-composer/preserve-paths:dev-master

Or if you don't commit composer.lock, require the hash exactly:

composer require drupal-composer/preserve-paths:dev-master#4c5f62fe97f48e38c5e80cf2f9be8666f683e454

Solution

git tag 0.1.6

Full Output

$ composer i -v
> pre-install-cmd: DrupalProject\composer\ScriptHandler::checkComposerVersion
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Dependency resolution completed in 0.000 seconds
Analyzed 186 packages to resolve dependencies
Analyzed 397 rules to resolve dependencies
Package operations: 125 installs, 0 updates, 0 removals
Installs: cweagans/composer-patches:1.6.7, drupal-composer/preserve-paths:0.1.5, composer/installers:v1.9.0, oomphinc/composer-installers-extender:v1.1.2, composer/semver:1.5.1, symfony/polyfill-ctype:v1.18.1, symfony/yaml:v3.4.43, symfony/process:v3.4.43, sensiolabs/ansi-to-html:v1.2.0, psr/http-message:1.0.1, php-http/promise:1.1.0, php-http/httplug:v1.1.0, symfony/polyfill-php72:v1.18.1, paragonie/random_compat:v9.99.99, symfony/polyfill-php70:v1.18.1, symfony/polyfill-intl-normalizer:v1.18.1, symfony/polyfill-intl-idn:v1.18.1, ralouphie/getallheaders:3.0.3, guzzlehttp/psr7:1.6.1, guzzlehttp/promises:v1.3.1, guzzlehttp/guzzle:6.5.5, php-http/guzzle6-adapter:v1.1.1, psr/cache:1.0.1, php-http/discovery:1.9.1, symfony/polyfill-php80:v1.18.1, symfony/deprecation-contracts:v2.1.3, symfony/options-resolver:v5.1.3, php-http/message-factory:v1.0.2, clue/stream-filter:v1.4.1, php-http/message:1.8.0, php-http/client-common:1.10.0, php-http/cache-plugin:1.7.1, knplabs/github-api:v2.15.0, symfony/event-dispatcher:v2.8.52, cpliakas/git-wrapper:1.7.0, devshop/devmaster:dev-component/composer/devmaster 090fce8, drupal/drupal:7.72.0, drupal/admin_menu:3.0.0-rc6, drupal/adminrole:1.1.0, drupal/aegir_ansible:dev-1.x 3fd9ce3, drupal/aegir_cloud:dev-1.x 09d2453, drupal/aegir_config:1.0.0-beta1, drupal/sshkey:2.0.0, drupal/aegir_ssh:1.0.0, drupal/betterlogin:1.5.0, drupal/bootstrap:3.26.0, drupal/token:1.7.0, drupal/cas:1.7.0, drupal/cas_attributes:1.0.0-rc3, drupal/chosen:2.1.0, drupal/composer_autoloader:1.3.0, drupal/ctools:1.15.0, drupal/devel:1.7.0, drupal/devshop_stats:dev-1.x ea80b5f, drupal/features:2.11.0, drupal/views:3.24.0, drupal/entity:1.9.0, drupal/views_bulk_operations:3.6.0, drupal/hosting:dev-4.x 337f676, drupal/hosting_filemanager:dev-1.x c488ea3, drupal/hosting_git:3.181.0, drupal/hosting_https:3.182.0, drupal/hosting_certificate:3.182.0, drupal/hosting_logs:3.181.0, drupal/hosting_site_backup_manager:3.180.0, drupal/statsd:1.1.0, drupal/hosting_statsd:1.0.0-beta1, drupal/hosting_tasks_extra:3.180.0, drupal/jquery_update:2.7.0, drupal/module_filter:2.2.0, drupal/libraries:2.5.0, drupal/navbar:1.7.0, drupal/openidadmin:1.0.0, drupal/overlay_paths:1.3.0, drupal/r4032login:1.8.0, drupal/timeago:2.3.0, guzzle/guzzle:v3.9.3, npm-asset/ansi-regex:5.0.0, npm-asset/sprintf-js:1.0.3, npm-asset/argparse:1.0.10, npm-asset/underscore:1.10.2, npm-asset/backbone:1.4.0, npm-asset/chosen:0.0.2, npm-asset/color-name:1.1.4, npm-asset/color-convert:2.0.1, npm-asset/emoji-regex:8.0.0, npm-asset/entities:2.0.3, npm-asset/esutils:2.0.3, npm-asset/is-fullwidth-code-point:3.0.0, npm-asset/uc.micro:1.0.6, npm-asset/linkify-it:2.2.0, npm-asset/p-try:2.2.0, npm-asset/p-limit:2.3.0, npm-asset/p-locate:4.1.0, npm-asset/locate-path:5.0.0, npm-asset/mdurl:1.0.1, npm-asset/minimist:1.2.5, npm-asset/decamelize:1.2.0, npm-asset/camelcase:5.3.1, npm-asset/yargs-parser:18.1.3, npm-asset/y18n:4.0.0, npm-asset/which-module:2.0.0, npm-asset/strip-ansi:6.0.0, npm-asset/string-width:4.2.0, npm-asset/set-blocking:2.0.0, npm-asset/require-main-filename:2.0.0, npm-asset/require-directory:2.1.1, npm-asset/get-caller-file:2.0.5, npm-asset/path-exists:4.0.0, npm-asset/find-up:4.1.0, npm-asset/types--color-name:1.1.1, npm-asset/ansi-styles:4.2.1, npm-asset/wrap-ansi:6.2.0, npm-asset/cliui:6.0.0, npm-asset/yargs:15.4.1, npm-asset/requirejs:2.3.6, npm-asset/mkdirp:0.5.5, npm-asset/markdown-it:10.0.0, npm-asset/lodash:4.17.19, npm-asset/file:0.2.2, npm-asset/doctrine:3.0.0, npm-asset/modernizr:3.11.3, symfony/filesystem:v3.4.43, toin0u/digitalocean-v2:2.3.0, webflo/drupal-finder:1.2.0
  - Installing cweagans/composer-patches (1.6.7): Loading from cache
 Extracting archiveNo patches found for cweagans/composer-patches.
Gathering patches for root package.
Gathering patches for dependencies. This might take a minute.
Found 1 patches for cweagans/composer-patches.
  - Installing drupal-composer/preserve-paths (0.1.5): Loading from cache
 Extracting archiveNo patches found for drupal-composer/preserve-paths.

                                                       
  [ErrorException]                                     
  Trying to access array offset on value of type null  
                                                       

Exception trace:
 () at /home/jon/Projects/devshop/src/DevShop/Templates/DevShopControlTemplate/vendor/drupal-composer/preserve-paths/src/PluginWrapper.php:88
 Composer\Util\ErrorHandler::handle() at /home/jon/Projects/devshop/src/DevShop/Templates/DevShopControlTemplate/vendor/drupal-composer/preserve-paths/src/PluginWrapper.php:88
 DrupalComposer\PreservePaths\PluginWrapper->postPackage() at /home/jon/Projects/devshop/src/DevShop/Templates/DevShopControlTemplate/vendor/drupal-composer/preserve-paths/src/Plugin.php:69
 DrupalComposer\PreservePaths\Plugin->postPackage() at n/a:n/a
 call_user_func() at /usr/share/php/Composer/EventDispatcher/EventDispatcher.php:164
 Composer\EventDispatcher\EventDispatcher->doDispatch() at /usr/share/php/Composer/EventDispatcher/EventDispatcher.php:116
 Composer\EventDispatcher\EventDispatcher->dispatchPackageEvent() at /usr/share/php/Composer/Installer.php:636
 Composer\Installer->doInstall() at /usr/share/php/Composer/Installer.php:232
 Composer\Installer->run() at /usr/share/php/Composer/Command/InstallCommand.php:122
 Composer\Command\InstallCommand->execute() at /usr/share/php/Symfony/Component/Console/Command/Command.php:245
 Symfony\Component\Console\Command\Command->run() at /usr/share/php/Symfony/Component/Console/Application.php:835
 Symfony\Component\Console\Application->doRunCommand() at /usr/share/php/Symfony/Component/Console/Application.php:185
 Symfony\Component\Console\Application->doRun() at /usr/share/php/Composer/Console/Application.php:281
 Composer\Console\Application->doRun() at /usr/share/php/Symfony/Component/Console/Application.php:117
 Symfony\Component\Console\Application->run() at /usr/share/php/Composer/Console/Application.php:113
 Composer\Console\Application->run() at /usr/bin/composer:62

install [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--] [<packages>]...

chmod() returns error if files are owned by another user than one triggering composer command. That makes all file disappear (moved to composer cache folder) and never restored.

Maybe we should test if we can perform chmod() operation and fail early with information what have happened.

Ref.

Unit tests were ran using Mac OS Catalina:

$ vendor/bin/phpunit
PHPUnit 8.5.9 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.2.33 with Xdebug 2.9.8
Configuration: /path-to-repos/preserve-paths/phpunit.xml.dist

F...                                                                4 / 4 (100%)

Time: 9.86 seconds, Memory: 6.00 MB

There was 1 failure:

1) DrupalComposer\PreservePaths\Tests\FixturesTest::testFixtures with data set #0 ('example', array(array('update', 'drupal/drupal')), array('web/index.php', 'web/sites/all/modules/contrib...module'))
Failed asserting that file "web/sites/all/modules/contrib/views/views.module" exists.

/path-to-repos/preserve-paths/tests/FixturesTest.php:74

FAILURES!
Tests: 4, Assertions: 50, Failures: 1.

I think Travis CI didn't catch it on the 0.1.6 release because the composer self-update only updated Composer to the latest 1.x release, so that might need another look as well.

The last version has been for more than a year old and there are 11 commits to be included.

Please, could you do a new tag release?

I am attempting to use this package to install WordPress along with plugins and themes using only Composer.
The problem is, if I preserve wp-content as a whole, the files under plugins and themes that are included in WP core do not get installed at all.

What would really solve this issue for me is the ability to also add negated paths so that composer will still install those but not touch any other plugins/themes.
Something like this:

"preserve-paths": [
  "wp-content",
  "!wp-content/plugins/akismet",
  "!wp-content/themes/twentyseventeen"
]

Could this feature be added?

When attempting to preserve sites/default in Drupal, \Composer\Util\Filesystem->rename() does not actually rename the directory but passes control to ->copyThenRemove(), which then attempts to do what its name implies.

So if there are any protected files in the paths being preserved, PathPreserver->preserve() fails to complete and a RuntimeException is thrown that a file could not be deleted.

I really would appreciate Composer 2 support.

I'm using latest version of preserve-path i.e 0.1.6.

I have following code for my D7 project:

 "installer-paths": {
      "dist/": [ "type:drupal-core" ],
      "dist/sites/all/libraries/{$name}": [ "type:drupal-library"],
      "dist/sites/all/modules/custom/{$name}": [ "type:drupal-custom-module" ],
      "dist/sites/all/modules/contrib/{$name}": [ "type:drupal-module" ],
      "dist/sites/all/themes/shared/{$name}": [ "type:drupal-custom-theme" ],
      "dist/sites/all/themes/contrib/{$name}": [ "type:drupal-theme" ]
    },
    "preserve-paths": [
      "dist/sites/myproject.com
    ]

There are already custom modules in dist/sites/myproject.com/modules/custom/module_name location which needs to be there everytime but whenever I do composer update or install it removes "dist" directory and again setup. It is not preserving the path which I have define to preserve.

  [TypeError]
  Composer\Util\Filesystem::isAbsolutePath(): Argument #1 ($path) must be of type string, null given, called in .../test2/vendor/drupal-composer/preserve-paths/src/PluginWrapper.php on line 190

Since composer 2.5.6 or 2.5.7 (works fine on 2.5.5 but haven't checked 2.5.6 - happens always on 2.5.7) - this error occurs due to metapackage encountered.

It is similar to the issue in #42 - and the reproduction case in #42 will fully replicate this instance.

So as of 2.5.7 at least of composer v2 this plugin no longer works at all.

When I am installing Drupal 7 alongside a number of modules from a freshly cloned repo it appears that the modules folder seems to get backed up in a partially completed state, and then later on restored. This overwrites part of the modules. The modules folder ends up with a bunch of empty folders and missing modules.

The console output indicates that some folders are being overwritten:

[...]
  - Installing drush/drush (8.4.8): Extracting archive
  - Installing symfony/filesystem (v3.4.47): Extracting archive
Files of installed package were overwritten with preserved path web/sites/all/modules/contrib!
Files of installed package were overwritten with preserved path web/sites/all/modules/contrib!

This happens only with Composer 2, so possibly this is caused by the parallel installation of the packages. Possibly some modules get installed before Drupal core itself is installed? I tried setting the COMPOSER_MAX_PARALLEL_HTTP=1 environment variable but this doesn't help. It only throttles the downloading of the packages, not the installation.

I am not using composer-patches. It happens with the latest version of Composer (2.0.12).

Here are the relevant parts of the composer.json file that is causing the problem:

{
    "repositories": [
        {
            "type": "composer",
            "url": "https://packages.drupal.org/7"
        }
    ],
    "require": {
        "php": "^7.4",
        "ext-curl": "*",
        "ext-gd": "*",
        "ext-json": "*",
        "ext-openssl": "*",
        "ext-pdo": "*",
        "ext-xml": "*",
        "composer/installers": "^1.2",
        "composer/semver": "^1.4",
        "drupal-composer/preserve-paths": "^0.1",
        "drupal/admin_menu": "3.0-rc6",
        "drupal/bootstrap": "^3.26",
        "drupal/composer_autoloader": "^1.0",
        "drupal/ctools": "1.15",
        "drupal/drupal": "^7.80",
        "drupal/ds": "2.16",
        "drupal/email": "1.3",
        "drupal/entity": "1.9",
        "drupal/entity_token": "1.9",
        "drupal/features": "2.11",
        "drupal/field_collection": "1.0-beta13",
        "drupal/field_group": "1.6",
        "drupal/googleanalytics": "2.6",
        "drupal/isotope": "2.0",
        "drupal/jquery_update": "2.7",
        "drupal/l10n_update": "1.2",
        "drupal/libraries": "2.5",
        "drupal/link": "1.7",
        "drupal/menu_block": "2.8",
        "drupal/module_filter": "2.2",
        "drupal/nodeblock": "1.7",
        "drupal/pathauto": "1.0-rc1",
        "drupal/strongarm": "2.0",
        "drupal/token": "1.7",
        "drupal/transliteration": "3.2",
        "drupal/video_embed_field": "2.0-beta11",
        "drupal/views": "3.22",
        "drupal/webform": "4.22",
        "drupal/weight": "3.1",
        "drupal/wysiwyg": "2.0",
        "drupal/xmlsitemap": "2.6",
        "drupal/youtube": "1.7",
        "drush/drush": "^8.0",
        "symfony/filesystem": "~2.7|^3",
        "webflo/drupal-finder": "^1.0.0"
    },
    "conflict": {
        "drupal/core": "8.*"
    },
    "config": {
        "sort-packages": true
    },
    "autoload": {
        "classmap": [
            "scripts/composer/ScriptHandler.php"
        ]
    },
    "scripts": {
        "pre-install-cmd": [
            "DrupalProject\\composer\\ScriptHandler::checkComposerVersion"
        ],
        "pre-update-cmd": [
            "DrupalProject\\composer\\ScriptHandler::checkComposerVersion"
        ],
        "post-install-cmd": [
            "DrupalProject\\composer\\ScriptHandler::createRequiredFiles"
        ],
        "post-update-cmd": [
            "DrupalProject\\composer\\ScriptHandler::createRequiredFiles"
        ],
        "post-create-project-cmd": [
            "DrupalProject\\composer\\ScriptHandler::removeInternalFiles"
        ]
    },
    "extra": {
        "installer-paths": {
            "web/": ["type:drupal-core"],
            "web/profiles/{$name}/": ["type:drupal-profile"],
            "web/sites/all/drush/{$name}/": ["type:drupal-drush"],
            "web/sites/all/libraries/{$name}/": ["type:drupal-library"],
            "web/sites/all/modules/contrib/{$name}/": ["type:drupal-module"],
            "web/sites/all/themes/{$name}/": ["type:drupal-theme"]
        },
        "preserve-paths": [
            "web/sites/all/modules/contrib",
            "web/sites/all/modules/features",
            "web/sites/all/themes/contrib",
            "web/sites/default/drushrc.php",
            "web/sites/default/files",
            "web/sites/default/settings.php"
        ]
    }
}

It may be possible to

• retrieve all currently installed package install paths
• log installed package paths

to autodetect path overrides.

I've isolated it to a reproducible test case. If you toggle the required package drupal/drupal from 7.38 to 7.36 (or any other version) and composer update it places the sites folder from the drupal core download at the path: "ROOT/sites". In other words the directory that comes with drupal that is usually the ROOT/sites directory can now be found at "ROOT/sites/sites". I'm very happy to help or PR but I'm not yet sure how your plugin works. I assume it should merge (preferring preserved directories) or discard the directory? I'm available on IRC or hangouts at michaelfavia. Thanks for the useful plugin. Best, -mf

Relevant section of my composer.json:

  "extra": {
    "custom-installer": {
      "drupal-module": "docroot/sites/all/modules/contrib/{$name}/",
      "drupal-theme": "docroot/sites/all/themes/contrib/{$name}/",
      "drupal-library": "docroot/sites/all/libraries/{$name}/",
      "drupal-drush": "docroot/sites/all/drush/{$name}/",
      "drupal-profile": "docroot/profiles/{$name}/",
      "drupal-core": "docroot/"
    },
    "preserve-paths": [
      "docroot/sites/all/modules",
      "docroot/sites/all/modules",
      "docroot/sites/all/themes",
      "docroot/sites/all/libraries",
      "docroot/sites/all/drush",
      "docroot/sites/all/vendor",
      "docroot/sites/default/settings.php",
      "docroot/sites/default/files"
    ],

It's easy to misconfigure preserve paths.

If you preserve a path that, perhaps, receives a single file like a .gitkeep (for Drupal 7 for example the sites/all/themes folder would receive this), then you can hit issues where you lose the data.

• Package drupal/drupal installs (web/)
• Preserve of (web/sites/all/themes/) occurs
• Install creates web/sites/all/themes/.htaccess
• Package drupal/date_api installs (metapackage with no install-path so it calculates the root: /)
• Preserve of (web/sites/all/themes/) occurs again because it exists again
• End of install, restore the original, but then we overwrite again with just a .htaccess file

I think part of the issue is metapackages, as those cause a preserved path to be encountered twice. So packages like date_api, migrate_ui, and field_object can all cause secondary backups if the backup path overlaps.

Other part of the issue is the overlapping configuration - if you do have a preserve path which has its folder or some contents existing in some packages, even if you don't want them and did want to just overwrite it, then all it takes is a metapackage and it will mess up.

Perhaps the fix is metapackage exclusion, but I worry there is some cases still where paths overlap (drupal/drupal for example overlaps everything - there's nothing to say there would be a drupal/parent-module package that installs to modules/contrib/parent and then a drupal/parent-module-child-module package that installs to modules/contrib/parent/modules/child... and this would also cause the issue above if you configured an overlapping path such as modules/contrib/parent/modules)

Perhaps also need to count the number of times preserve is hit on a path and only preserve once, and then at the end once all packages affecting it are installed it is only then restored?

I ran into a problem on shared hosting server (which means limited privileges like e.g. can't restart MySQL as suggested on some previous issues).

The hosting is forcing php 7.4 by default, while 7.2 is still available.

Can't upgrade Drupal 7 websites with Drush 8 and PHP 7.4 or even run simple drush status it gives:

"continue" targeting switch is equivalent to "break". Did you mean to use "continue 2"? userprotect.module:627                                                                           [warning]
"continue 2" targeting switch is equivalent to "break 2". Did you mean to use "continue 3"? userprotect.module:647                                                                       [warning]
Deprecated function: Array and string offset access syntax with curly braces is deprecated in include_once() (line 20 of /users/webesite.com/htdocs/includes/file.phar.inc).          [error]

But if you do whereis php you get:

php: /usr/bin/php7.0 /usr/bin/php7.3 /usr/bin/php /usr/bin/php7.2 /usr/bin/php5.6 /usr/bin/php7.4 /usr/lib/php /etc/php /usr/include/php /usr/share/php7.3-json /usr/share/php5.6-sybase /usr/share/php7.4-pspell /usr/share/php7.0-mbstring /usr/share/php7.2-gd /usr/share/php7.3-soap /usr/share/php7.4-sqlite3 /usr/share/php7.4-soap /usr/share/php7.3-mysql /usr/share/php7.0-sqlite3 /usr/share/php7.0-soap /usr/share/php5.6-bcmath /usr/share/php7.1-xml /usr/share/php7.2-sybase /usr/share/php7.0-imap /usr/share/php7.4-gd /usr/share/php5.6-xmlrpc /usr/share/php7.2-soap /usr/share/php7.4-mbstring /usr/share/php7.0-readline /usr/share/php5.6-zip /usr/share/php5.6-intl /usr/share/php7.1-common /usr/share/php7.3-gd /usr/share/php7.4-bcmath /usr/share/php5.6-soap /usr/share/php7.3-common /usr/share/php5.6-mcrypt /usr/share/php7.0-sybase /usr/share/php7.2-xmlrpc /usr/share/php7.4-xml /usr/share/php7.2-json /usr/share/php7.0-mysql /usr/share/php5.6-gd /usr/share/php7.2-imap /usr/share/php7.4-common /usr/share/php7.2-mbstring /usr/share/php7.2-common /usr/share/php /usr/share/php7.3-xmlrpc /usr/share/php7.2-pspell /usr/share/php7.3-zip /usr/share/php7.0-intl /usr/share/php7.3-mbstring /usr/share/php7.3-readline /usr/share/php7.2-opcache /usr/share/php5.6-opcache /usr/share/php7.3-opcache /usr/share/php7.0-common /usr/share/php5.6-common /usr/share/php7.4-imap /usr/share/php7.0-gd /usr/share/php7.2-mysql /usr/share/php7.4-mysql /usr/share/php5.6-sqlite3 /usr/share/php7.0-mcrypt /usr/share/php7.2-intl /usr/share/php7.3-xml /usr/share/php7.4-curl /usr/share/php7.0-zip /usr/share/php5.6-mysql /usr/share/php7.4-opcache /usr/share/php5.6-mbstring /usr/share/php7.3-bcmath /usr/share/php7.4-json /usr/share/php7.4-xmlrpc /usr/share/php7.2-bcmath /usr/share/php7.4-intl /usr/share/php7.2-xml /usr/share/php7.0-pspell /usr/share/php7.0-opcache /usr/share/php7.4-sybase /usr/share/php7.2-sqlite3 /usr/share/php5.6-curl /usr/share/php7.0-json /usr/share/php5.6-pspell /usr/share/php7.2-zip /usr/share/php7.3-pspell /usr/share/php7.0-xmlrpc /usr/share/php7.2-curl /usr/share/php7.3-sybase /usr/share/php5.6-imap /usr/share/php7.3-intl /usr/share/php5.6-json /usr/share/php7.4-zip /usr/share/php7.0-xml /usr/share/php7.0-bcmath /usr/share/php7.3-imap /usr/share/php7.3-sqlite3 /usr/share/php7.2-readline /usr/share/php7.0-curl /usr/share/php7.3-curl /usr/share/php5.6-xml /usr/share/php5.6-readline /usr/share/php7.4-readline /usr/share/man/man1/php.1.gz

And as you can see we still have /usr/bin/php7.2, so if we make an alias or run something like this:

/usr/bin/php7.2 ~/composer/vendor/drush/drush/drush.php status we get working Drush

Drupal version                  :  7.67
Site URI                        :  http://default
Database driver                 :  mysql
Database hostname               :  localhost
Database port                   :
Database username               :  asd
Database name                   :  asd
Database                        :  Connected
Drupal bootstrap                :  Successful
Drupal user                     :
Default theme                   :  asd
Administration theme            :  adminimal
PHP configuration               :  /etc/php/7.2/cli/php.ini
PHP OS                          :  Linux
Drush script                    :  /users/asd.com/composer/vendor/drush/drush/drush.php
Drush version                   :  8.1.12
Drush temp directory            :  /tmp
Drush configuration             :
Drush alias files               :
Install profile                 :  standard
Drupal root                     :  /users/asd.com/htdocs
Drupal Settings File            :  sites/default/settings.php
Site path                       :  sites/default
File directory path             :  sites/default/files
Private file directory path     :  sites/default/files/prv
Temporary file directory path   :  ../tmp

But if we run same command to update the website /usr/bin/php7.2 ~/composer/vendor/drush/drush/drush.php pm-update we get error pm-updatestatus failed.

A recent change in composer - composer/composer#11455 - now breaks preserve-paths when it deals with metapackages.

Error: Composer\Util\Filesystem::isAbsolutePath(): Argument #1 ($path) must be of type string, null given, called in .../vendor/drupal-composer/preserve-paths/src/PluginWrapper.php on line 190

See also laminas/laminas-component-installer#69

Environment

dev:

• Device: macOS Big Sur
• OS version: v11.4
• Docker: version 20.10.11, build dea9396
• Composer: version 2.2.1
• drupal-composer/preserve-paths: 0.1.6

prd:

• alpine:3.14
• php8.0.13
• no sudo-er / user: nobody

Describe the problem:

If vendor/ folder does not exist in the repo, the command docker build -t app . fails with the error:

  [RuntimeException]                                                                                                 
  /.composer/cache/preserve-paths/518d646133f3bc0cdce8020fedc7a2cfdb3ed202 does not exist and could not be created.

Steps to reproduce:

1. Delete in local dev environment the vendor file
2. run: docker build -t app .
3. You get the RuntimeException error

Expected Results:

• No runtime exception. Other packages are not bind on build with the vendor
• The vendor folder must not be committed or built in advance into an automated ci/cd environment.

or, I miss something?

Hi, I'm trying to use with Windows, however I get a memory size exhausted error. It looks as though it gets stuck in a constant loop because it doesn't ever reach the break condition. If you replace the line below (line 234 of PathPreserver.php):

if ($folder === '.' || $folder === '/') {

with:

if ($folder === '.' || $folder === '/' || $folder === 'C:\') {

It solves the issue.

It'd be great if this could be updated!

Thanks,
Charlotte.

When I install a Drupal modules I get:

Deprecation Notice: The callback derhasi\Composer\Plugin_composer_tmp1::prePackage declared at phar:///usr/local/bin/composer/src/Composer/Plugin/PluginManager.php(195) : eval()'d code accepts a Composer\Script\PackageEvent but pre-package-install events use a Composer\Installer\PackageEvent instance. Please adjust your type hint accordingly, see https://getcomposer.org/doc/articles/scripts.md#event-classes in phar:///usr/local/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php:317

An example of command I am using to do the install: $composer require drupal/geofield:7.x

I'm trying the following: https://www.drupal.org/node/2471553

I'm getting an Allowed memory size error when running composer update

Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocat
e 36 bytes) in D:\web\UUtrecht\DrupalComposer\vendor\derhasi\composer-preserve-p
aths\src\PathPreserver.php on line 240

The method file_exists in the file PathPreserver causes an infinitive loop.

static public function file_exists($path) 
    // Get all parent directories.
    $folders = array();
    $reset_perms = array();
    $folder = $path;
    while ($folder = dirname($folder)) {
      //      It always handling the same folder
      if ($folder === '.' || $folder === '/') {
        break;
      }
      elseif ($folder === '') {
        continue;
      }
      $folders[] = $folder;
//      I called a break in order to fixed this. 
//      if  ($folders[count($folders) - 1] === $folder) {
//        break;
//      }
    }

This is what I’ve just posted in our local dev-list. Has anyone else dealt with the issue and is there some solution available?

A note about developing Drupal 7 Composer projects locally - there’s a high risk you’re losing your uncommitted changes in the custom codebase if performing lando rebuild or composer * tasks because of how "drupal-composer/preserve-paths" plugin works. Sometimes it fails to bring back the custom code to the webroot and only way to resolve this is git reset --hard. That of course means all the code changes will be gone. So to reduce that risk please stash or stage your precious work before rebuilding!

Hi, setting up Drupal 7 via composer with PHP 7.4 currently fails with the following error:

$ composer -v create-project drupal-composer/drupal-project:7.x-dev some-dir --no-interaction
Installing drupal-composer/drupal-project (7.x-dev 267d7fe14c74af8c04fc84c7f483434eca35ef90)
  - Installing drupal-composer/drupal-project (7.x-dev 267d7fe): Cloning 267d7fe14c74af8c04fc84c7f483434eca35ef90 from cache
Created project in some-dir
> pre-update-cmd: DrupalProject\composer\ScriptHandler::checkComposerVersion
Loading composer repositories with package information
Updating dependencies (including require-dev)
Dependency resolution completed in 0.255 seconds
Analyzed 6353 packages to resolve dependencies
Analyzed 403588 rules to resolve dependencies
Package operations: 29 installs, 0 updates, 0 removals
Installs: composer/installers:v1.7.0, cweagans/composer-patches:1.6.7, drupal-composer/preserve-paths:0.1.5, composer/semver:1.5.0, drupal/drupal:7.67.0, drupal/composer_autoloader:1.3.0, symfony/polyfill-ctype:v1.13.1, webmozart/assert:1.6.0, webmozart/path-util:2.3.0, webflo/drupal-finder:1.2.0, pear/console_table:v1.3.1, symfony/finder:v3.4.36, symfony/event-dispatcher:v3.4.36, psr/log:1.1.2, symfony/debug:v4.4.1, symfony/polyfill-mbstring:v1.13.1, symfony/console:v3.4.36, symfony/var-dumper:v3.4.36, symfony/yaml:v3.4.36, dflydev/dot-access-data:v1.1.0, consolidation/output-formatters:3.5.0, consolidation/annotated-command:2.12.0, jakub-onderka/php-console-color:v0.2, jakub-onderka/php-console-highlighter:v0.4, dnoegel/php-xdg-base-dir:0.1, nikic/php-parser:v4.3.0, psy/psysh:v0.9.11, drush/drush:8.3.2, symfony/filesystem:v3.4.36
  - Installing composer/installers (v1.7.0): Downloading (100%)
 Extracting archive  - Installing cweagans/composer-patches (1.6.7): Downloading (100%)
 Extracting archiveNo patches found for cweagans/composer-patches.
Gathering patches for root package.
Gathering patches for dependencies. This might take a minute.
Found 1 patches for cweagans/composer-patches.
  - Installing drupal-composer/preserve-paths (0.1.5): Downloading (100%)
 Extracting archiveNo patches found for drupal-composer/preserve-paths.


  [ErrorException]
  Trying to access array offset on value of type null


Exception trace:
 () at /var/www/html/some-dir/vendor/drupal-composer/preserve-paths/src/PluginWrapper.php:88
 Composer\Util\ErrorHandler::handle() at /var/www/html/some-dir/vendor/drupal-composer/preserve-paths/src/PluginWrapper.php:88
 DrupalComposer\PreservePaths\PluginWrapper->postPackage() at /var/www/html/some-dir/vendor/drupal-composer/preserve-paths/src/Plugin.php:69
 DrupalComposer\PreservePaths\Plugin->postPackage() at n/a:n/a
 call_user_func() at phar:///usr/local/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php:176
 Composer\EventDispatcher\EventDispatcher->doDispatch() at phar:///usr/local/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php:116
 Composer\EventDispatcher\EventDispatcher->dispatchPackageEvent() at phar:///usr/local/bin/composer/src/Composer/Installer.php:622
 Composer\Installer->doInstall() at phar:///usr/local/bin/composer/src/Composer/Installer.php:229
 Composer\Installer->run() at phar:///usr/local/bin/composer/src/Composer/Command/CreateProjectCommand.php:195
 Composer\Command\CreateProjectCommand->installProject() at phar:///usr/local/bin/composer/src/Composer/Command/CreateProjectCommand.php:145
 Composer\Command\CreateProjectCommand->execute() at phar:///usr/local/bin/composer/vendor/symfony/console/Command/Command.php:245
 Symfony\Component\Console\Command\Command->run() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:835
 Symfony\Component\Console\Application->doRunCommand() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:185
 Symfony\Component\Console\Application->doRun() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:267
 Composer\Console\Application->doRun() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:117
 Symfony\Component\Console\Application->run() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:106
 Composer\Console\Application->run() at phar:///usr/local/bin/composer/bin/composer:61
 require() at /usr/local/bin/composer:24

create-project [-s|--stability STABILITY] [--prefer-source] [--prefer-dist] [--repository REPOSITORY] [--repository-url REPOSITORY-URL] [--dev] [--no-dev] [--no-custom-installers] [--no-scripts] [--no-progress] [--no-secure-http] [--keep-vcs] [--remove-vcs] [--no-install] [--ignore-platform-reqs] [--] [<package>] [<directory>] [<version>]

It doesn't seem to work. I am looking for something like:

"web/sites/default/settings*.php"

When adding or deleting patches, paths are not preserved.

This is the test composer.json I am using.

{
  "name": "example",
  "description": "Example config for a D7 site using composer based build process.",
  "repositories": [{
    "type": "composer",
    "url": "https://packagist.drupal-composer.org"
  }],
  "require": {
    "composer/installers": "~1.0",
    "cweagans/composer-patches": "~1.0",
    "derhasi/composer-preserve-paths": "0.1.*",
    "drupal/drupal": "7.43",
    "drupal/views": "~7.0",
    "drupal/ctools": "~7.0",
    "drupal/memcache": "~7.0",
    "drupal/panels": "~7.0"
  },
  "conflict": {
    "drupal/core": "8.*"
  },
  "scripts": {
    "post-create-project-cmd": [
      "rm README.md LICENSE .travis.yml phpunit.xml.dist"
    ]
  },
  "config": {
    "vendor-dir": "vendor"
  },
  "minimum-stability": "dev",
  "prefer-stable": true,
  "extra": {
    "_readme": [
      "This is an example comment!"
    ],
    "installer-paths": {
      "docroot/": ["type:drupal-core"],
      "docroot/sites/all/modules/contrib/{$name}/": ["type:drupal-module"],
      "docroot/sites/all/themes/contrib/{$name}/": ["type:drupal-theme"],
      "docroot/sites/all/libraries/{$name}/": ["type:drupal-library"],
      "docroot/sites/all/drush/{$name}/": ["type:drupal-drush"],
      "docroot/profiles/{$name}/": ["type:drupal-profile"]
    },
    "patches": {
      "drupal/drupal": {
        "Node access grants should be statically cached": "https://www.drupal.org/files/issues/node_access_grants-static-cache-11.patch",
        "Invalid image style URLs should return 404, not 403.": "https://www.drupal.org/files/issues/drupal-image-style-not-found-2211429-4.patch",
        "Save and restore css/js/head with block cache.": "https://www.drupal.org/files/issues/d7-block-cache-1460766-34.patch",
        "Ignore front end vendor folders to improve directory search performance": "https://www.drupal.org/files/issues/ignore_frontend_folders-2329453-101-7.x.patch"
      }
    },
    "preserve-paths": [
      "docroot/robots.txt",
      "docroot/.htaccess",
      "docroot/sites/all/modules/contrib",
      "docroot/sites/all/themes/contrib",
      "docroot/sites/all/libraries",
      "docroot/sites/all/drush",
      "docroot/sites/default/settings.php",
      "docroot/sites/default/files"
    ]
  }
}

After installing, I am making changes in robots.txt which I want to preserve.
Adding or updating any packages works fine and preserves the changes.

Now If I add/remove a patch and do composer update, all the changes are gone.

I did a short debugging, and here is what I think is happening:
For cleanly applying patches composer-patches plugin deletes package files (in this case docroot) by responding to ScriptEvents::PRE_INSTALL_CMD event which happens to be before the ScriptEvents::PRE_PACKAGE_INSTALL / ScriptEvents::PRE_PACKAGE_UPDATE events in event loop - where composer-preserve-paths plugin backs up the files. So when composer-preserve-paths gets chance to backup files, they are already removed, so nothing to backup and hence restore.

We shoudl use Symfony Coding Standards in the project. Therefore using phpcs to check for that

Package throws a runtime exception with --no-cache option set when running composer install or similar:

$ composer --no-dev --no-cache -o install
Gathering patches for root package.
Loading composer repositories with package information
Installing dependencies from lock file
Package operations: 0 installs, 1 update, 0 removals

                                                                                                           
   [RuntimeException]                                                                                          
   /dev/null/preserve-paths/e1ecf99583a81f8bf0a10be879439cc0dc06d423 does not exist and could not be created.

Alternative on Windows, check against

strpos($folder, ':')

to ?

Now that drupal-composer/drupal-scaffold mainly got replaced by drupal/core-composer-scaffold preserving paths doesn't work on scaffolding files anymore.

Steps to reproduce

1. $ composer create-project drupal-composer/drupal-project:8.x-dev some-dir -n
2. $ cd some-dir
3. $ composer require drupal-composer/preserve-paths
4. Add to extra:

   "preserve-paths": [
         "web/sites/example.settings.local.php"
   ]
   

5. $ echo '$config[\'environment_indicator.indicator\'][\'bg_color\'] = \'#0066ff\';' >> web/sites/example.settings.local.php
6. $ composer install -n

Expected result:

• changes to example.settings.local.php being preserved

Actual result:

• changes to example.settings.local.php overwritten