civicrm / civicrm-setup Goto Github PK

Background: We aim to allow installing CiviCRM on Drupal 8 via composer/git without requiring the admin to run GenCode.

The file templates/CRM/common/version.tpl is generated by GenCode and then used once... by GenCode. If you grep universe, there's nothing else that references CRM/common/version.tpl.

Task:
If the file isn't used, then do we really need to remove it? Perhaps we can keep it around (in tarballs) on the off-chance that some unpublished legacy process relies on it, but omit from the D8 builds?

The CLI install steps are a little shorter on Drupal/Backdrop than on WordPress.

In all three cases, you start out by creating a new/empty CMS site and extracting the source-code for your CiviCRM build. Then run cv core:install to setup Civi.

The difference:

• On Drupal/WordPress, cv core:install will auto-activate the CMS extension (FlushDrupal.civi-setup.php or FlushBackdrop.civi-setup.php). This is akin to running drush en civicrm -y. automatically.
• On WordPress, you have to manually activate -- eg run wp plugin activate civicrm..

To make the admin experience more consistent (and to save a step), the FlushWordPress.civi-setup.php should also activate the plugin.

It's been in civicrm-core for a while.

CiviCRM has integrations with drush and wp-cli which can perform installation, e.g. drush civicrm-install or wp civicrm install. These serve a similar purpose as cv core:install.

At the moment, there isn't a huge functional difference; however, civicrm-setup is positioned better to bring improvements like:

• Automatically enabling official extensions during installation
• Creating a pure CMS-first bootstrap, wherein civicrm.settings.php is optional
• Installing the latest SQL schema / translations / example-data (without needing setup.sh / regen.sh / GenCode)

There are a few ways we could reconcile the projects and keep the features at parity:

1. Update civicrm.drush.inc and wp-cli/civicrm.php to use the civicrm-setup API ($setup->checkRequirements(), $setup->installDatabase(), etal). (Pro: Preserve compatibility with existing sysadmin scripts. drush/wp users don't need to download cv.)
2. Consolidate on cv core:install; phase out other CLI scripts. (Pro: Documentation can be consolidated. Use the same commands and test-scripts for all environments.)

I'm pretty neutral on which approach is better. (Come to think of it, they're not mutually exclusive...)

Port https://github.com/civicrm/civicrm-core/pull/7322/files to a plugin

Background: When installing on Drupal 8, settings such as DSN and locale should be autodetected.
Task:

• Copy plugins/init/Drupal.civi-setup.php or plugins/init/WordPress.civi-setup.php
• Create Drupal8.civi-setup.php
• Adapt for D8. For source material on how to detect things like DSN, look at [email protected] in civicrm.install / drush.civicrm.inc.

Background: The installation process blends together (a) values autodetected by the plugins with (b) values supplied by the installer.

Issue: The signature of Civi\Setup::init($modelValues) is misleading (as indicated by the existence of #14) -- it implies that any fields can be passed to init(...), but it really only works for a couple fields (cms, setupPath). If you try to set other values (db, cmsDb, components, et al), then the autodetection logic may override them.

The more general way to to manipulate the model is to let the autodetection run first:

Civi\Setup::init(['cms' => ..., 'setupPath' => ..., 'srcPath' => ...]);

And then get a reference to the model object:

$model = Civi\Setup::instance()->getModel();

And update its properties:

// Update some properties of the model
$model->db = ['server'=>'localhost:3306', 'username'=>'admin', 'password'=>'s3cr3t', 'database'=>'mydb'];
$model->cmsDb = ['server'=>'localhost:3306', 'username'=>'admin', 'password'=>'s3cr3t', 'database'=>'mydb'];

// Or, if you prefer arrays...
$model->setValues([
  'db' => ['server'=>'localhost:3306', 'username'=>'admin', 'password'=>'s3cr3t', 'database'=>'mydb'],
  'cmsDb' => ['server'=>'localhost:3306', 'username'=>'admin', 'password'=>'s3cr3t', 'database'=>'mydb'],
]);

See also: #14 (comment)

The main test coverage for this library comes from CoreLifecycleTest. That test creates empty sites running Drupal, WordPress, and Backdrop and attempts installation.

To create sites, it uses civibuild with the drupal-empty, backdrop-empty, and wp-empty configurations -- which all create single-site builds.

We should also have test-coverage for multi-site builds.

Perhaps a decent way to get there... start by adding multisite configurations for civibuild (e.g civibuild create drupal-empty-ms..., civibuild create backdrop-empty-ms..., civibuild create wp-empty-ms... would create a small multisite network)?

The library civicrm-setup is largely a a reorganization of the older civicrm-core/install folder, which is the web-based installer used on Drupal/WordPress/Backdrop. However, the Joomla integration always had a separate installer.

A good next-step for supporting Joomla would be introducing CLI support. Some activities involved:

• Ensure that the new CMS-first CLI bootstrap works on Joomla.
• Add a Joomla.civi-setup.php plugin. Perhaps use WordPress.civi-setup.php as an example.
• There will be an extra requirement (that's not part of the WordPress.civi-setup.php example): during the civi.setup.installFiles event, you need to create an extra copy of civicrm.settings.php which is slightly different. (One copy for backend/admin; one copy for frontend/user.)

When we have installer logic that works on Joomla CLI, then it should be easier to update Joomla web UI to also use civicrm-setup.

The current version of civicrm-setup still relies on the same SQL files as the old installer -- which means:

• All the SQL files need to be pre-generated and pre-translated -- so, eg, developers need to understand th workflow of both setup.sh and regen.sh.
• Updates for the schema and l10n data cannot be simply dropped in to place. Generally, people have to download a carefully build tarball.
• We need more bits of infrastructure providing the above.
• You can only get sample data on en_US.

To simplify the workflow:

• Rewrite plugins/installDatabase/InstallSchema.civi-setup.php...
• Don't read specific SQL files.
• Instead, use CRM/Core/CodeGen/*, xml/schema/*, and l10n/* to produce the SQL on-demand.

Eventually, this should allow us to remove regen.sh and remove the various civicrm_data.xx_XX.mysql files.

The CLI-based install works on Drupal/Backdrop, but the web-based doesn't.

Why: The old civicrm/install (on Drupal/Backdrop) follows a slightly different protocol than civicrm-setup. Compare these protocols:

• Old (Drupal/Backdrop): Navigate to sites/all/modules/civicrm/install/index.php?installType=XYZ. This script displays a web UI, installs Civi, boots the CMS, and activates the module. However, this means that the installer is responsible for booting the CMS.
• Old (WP): Activate the civicrm.php plugin. Show a link for the installer (page=civicrm-installer). In this case, the CMS does its normal booting/lifecycle, and the install UI is just a regular page.

civicrm-setup is closer to the WP protocol for two reasons: (1) it's better -- it reduces our responsibility for bootstrap and ensures that all CMS services are available throughout installation and (2) it's convenient -- our initial go-live scenario was for WP.

How:

civicrm.module needs to fail more gracefully. In particular: if you activate civicrm.module without a pre-existing civicrm.settings.php or civicrm_*, the site should still work, and it should display a status message with a link to the install screen.

At time of writing, my local D7 build produces random errors if you activate civicrm.module without having a civicrm.settings.php file.

The list of available languages currently follows this dataflow:

1. The file xml/templates/languages.tpl contains a prototype of a SQL statement to insert a list of languages in a table.
2. GenCode calls CRM/Core/CodeGen/I18n.php, which does regex-foo to extract the list of languages. It writes the list to install/langs.php.
3. The plugin AvailableLangs.civi-setup.php reads the list of from install/langs.php.

Complaints with this arrangement:

1. (Primary complaint) Using civicrm-setup to install requires that you previously ran GenCode (or copied the files)
2. Using civicrm-setup to install requires that you have the old install/ folder.
3. It is aesthetically displeasing to read data by running regex against a SQL statement.

Possible solutions:

1. Commit install/langs.php to git (like we do with DAOs). Sounds easy - and fixes the primary complaint (#1).
2. Copy the static list of languages into AvailableLangs.civi-setup.php. Sounds easy - and eventually fixes everything (if we manage to deprecate the install/ folder). But in the interim, adding a language requires an extra step. (Patch both civicrm-core and civicrm-setup.)
3. Update civicrm-core so that the canonical list of languages is not in a SQL file. (Put it in something easy to parse -- like .json or .csv or .php.) Update both CRM/Core/CodeGen/I18n.php and AvailableLangs.civi-setup.php to read from this file. (This is prettiest, but a little more work/coordination.)