pivotal-cf / docs-mysql Goto Github PK

View Code? Open in Web Editor NEW

3.0 76.0 25.0 21.8 MB

Docs for MySQL for Pivotal Cloud Foundry

HTML 100.00%

docs-mysql's Introduction

docs-mysql

Docs for VMware SQL with MySQL for Tanzu Application Service

Which branch to use?

Note: Provide instructions in your PRs to indicate which branches you want Docs to apply your commits to.

Branch name	Use for…
master	Do not use.
3.1	in progress branch for next release
3.0	Current version
2.10	Supported version
2.9	No longer supported.

Note: Ignore branches 2.8 and earlier; none are supported or in use.

Where is the book repository?

The private book repo associated with this content repo is docs-book-mysql.

Partials

Cross-product partials for VMware SQL with MySQL for Tanzu Application Service are single sourced from the Services Partials repo.

Previously, these partials were sourced from the v018.x branch of the On Demand Service Broker SDK content repository.

Style Guide

This is a word list for terminology and word usage specific to the VMware SQL with MySQL for Tanzu Application Service docs.

Word	Explanation
adbr plugin	This is the short form of the ApplicationDataBackupRestore plugin. Use the short form in headings and after the first use in body text.
ApplicationDataBackupRestore (adbr) plugin	First use in body text for the adbr plugin.
CA certificate	A primary(?) certificate that is signed by a certificate authority.
certificate	generic word, there are TLS certificates, internal certificates, CA certificates
highly available cluster	No need to capitalize. Don't mix with "high availability cluster" or Galera. Galera happens to be the technology right now, but highly available cluster is the topology. You can abbreviate to HA cluster after spelling out on first use. Don't use "HAC".
internal certificate	These are certificates managed by VMware SQL with MySQL for Tanzu Application Service. There are many of these. They get rotated with upgrades.
leader-follower	It seems that we always hyphenate it. But we don't capitalize it.
MySQL database cluster node	Only applies to Galera clusters. Spell out like this at first use, then opt for "node" on the rest of the page.
node	See above. The old proxy page used: MySQL node, server node, MySQL servers, cluster back-ends, back-end database cluster node, database node VM, proxy instance, proxy. But, only two things are referred to here, nodes and proxies.
proxy	Use proxy consistently instead of alternating between "proxy instance" and "proxy".
TLS certificates	Certificates needed if you use TLS; there are two or three of these. One is the TLS CA certificate.
TLS CA certificate	The CA certificated used for TLS. You can add your own or use the one that Ops Manager provides.
Multi-Site Replication	This is the name of the feature and plan in the UI in v2.7.2 and later
multi-site replication service key	When referring to a service key that is only used for nodes in different foundations or data centers to communicate, use this term at least on first use. If you are referring to a "normal" service key, like one that is used for TLS, just use `service key`

Steps for local development

$ git clone [email protected]:pivotal-cf/docs-layout-repo
$ git clone [email protected]:pivotal-cf/docs-mysql
$ cd docs-mysql && git checkout <branch> && cd -
$ git clone [email protected]:pivotal-cf/docs-book-mysql
$ cd docs-book-mysql
$ bundle install
$ bundle exec bookbinder watch
$ open http://127.0.0.1:XXXX/p-mysql/<branch>

docs-mysql's People

Contributors

Stargazers

Watchers

docs-mysql's Issues

Mention new users and single schema for v2

Somewhere we should talk about:

How each app binding creates a new mysql user, and you cannot specify the name
How the single schema is visible to all users
How you cannot create new or named schemas outside of the default of service_instance_db
How the app binding users are removed when the service binding is removed

Testing

mysql-example-app does not push

We just tried deploying the example app in order to test that we have configured the tile correctly. We tried both on an air-gapped PCF and also on PWS and got the same error on cf push both times. It looks like something to do with compiling the mysql gem. See below for the push output on PWS.

➜  mysql-example-app  cf push mysql-example-app -c 'bundle exec ruby main.rb -p $PORT'
Pushing app mysql-example-app to org <org> / space <development> as <user>...
Getting app info...
Creating app with these attributes...
+ name:       mysql-example-app
  path:       /private/tmp/app/mysql-example-app
+ command:    bundle exec ruby main.rb -p $PORT
  routes:
+   mysql-example-app.cfapps.io

Creating app mysql-example-app...
Mapping routes...
Comparing local files to remote cache...
Packaging files to upload...
Uploading files...
 94.89 KiB / 94.89 KiB [===================================================================================================================================] 100.00% 1s

Waiting for API to complete processing files...

Staging app and tracing logs...
   Downloading dotnet_core_buildpack_beta...
   Downloading dotnet_core_buildpack...
   Downloading nodejs_buildpack...
   Downloading go_buildpack...
   Downloading python_buildpack...
   Downloaded nodejs_buildpack
   Downloading php_buildpack...
   Downloaded dotnet_core_buildpack_beta
   Downloading binary_buildpack...
   Downloaded python_buildpack
   Downloading staticfile_buildpack...
   Downloaded dotnet_core_buildpack
   Downloaded go_buildpack
   Downloading java_buildpack...
   Downloaded binary_buildpack
   Downloading ruby_buildpack...
   Downloaded php_buildpack
   Downloaded java_buildpack
   Downloaded staticfile_buildpack
   Downloaded ruby_buildpack
   Cell 76dba435-3477-41a6-bea1-9bd134040065 creating container for instance a549b1d8-7d10-4a6f-ba05-146417b43d6d
   Cell 76dba435-3477-41a6-bea1-9bd134040065 successfully created container for instance a549b1d8-7d10-4a6f-ba05-146417b43d6d
   Downloading app package...
   Downloaded app package (626.6K)
   -----> Ruby Buildpack version 1.7.41
   -----> Supplying Ruby
   -----> Installing bundler 1.17.3
          Copy [/tmp/buildpacks/c159075287b1d8f79d1972f92fd69671/dependencies/1b585b3a526373724c47492bbb271365/bundler-1.17.3-any-stack-b7502506.tgz]
   -----> Installing bundler 2.0.2
          Copy [/tmp/buildpacks/c159075287b1d8f79d1972f92fd69671/dependencies/5639c1c8e58c5952de84770f9dccb8c8/bundler-2.0.2-any-stack-8f6de4eb.tgz]
          **WARNING** You have not declared a Ruby version in your Gemfile.
          Defaulting to 2.4.6
          See http://docs.cloudfoundry.org/buildpacks/ruby/index.html#runtime for more information.
   -----> Installing ruby 2.4.6
          Copy [/tmp/buildpacks/c159075287b1d8f79d1972f92fd69671/dependencies/588d0d2dfceadf2cd4d517cd31f8f2f7/ruby-2.4.6-linux-x64-cflinuxfs3-9761fb0d.tgz]
   -----> Update rubygems from 2.6.14.4 to 3.0.4
   -----> Installing rubygems 3.0.4
          Copy [/tmp/buildpacks/c159075287b1d8f79d1972f92fd69671/dependencies/d21dd8fe1d3818b5d0d2d4ba70bfd2f4/rubygems-3.0.4-any-stack-8292d0fb.tgz]
          **WARNING** You have the `.bundle/config` file checked into your repository
          It contains local state like the location of the installed bundle
          as well as configured git local gems, and other settings that should
          not be shared between multiple checkouts of a single repo. Please
          remove the `.bundle/` folder from your repo and add it to your `.gitignore` file.
   -----> Installing dependencies using bundler 2.0.2
          Running: bundle install --without development:test --jobs=4 --retry=4 --path /tmp/contents830015225/deps/0/vendor_bundle --binstubs /tmp/contents830015225/deps/0/binstubs --deployment
          Using bundler 2.0.2
          Installing mysql2 0.3.11 with native extensions
          Installing rack 1.5.2
          Installing tilt 1.4.1
          Installing rack-protection 1.5.0
          Installing sinatra 1.4.3
          Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

          current directory:
          /tmp/contents830015225/deps/0/vendor_bundle/ruby/2.4.0/gems/mysql2-0.3.11/ext/mysql2
          /tmp/contents830015225/deps/0/ruby/bin/ruby -I
          /tmp/contents830015225/deps/0/ruby/lib/ruby/site_ruby/2.4.0 -r
          ./siteconf20190712-81-1crspg4.rb extconf.rb
          checking for rb_thread_blocking_region()... no
          checking for rb_wait_for_single_fd()... yes
          checking for mysql.h... yes
          checking for errmsg.h... yes
          checking for mysqld_error.h... yes
          creating Makefile

          current directory:
          /tmp/contents830015225/deps/0/vendor_bundle/ruby/2.4.0/gems/mysql2-0.3.11/ext/mysql2
          make "DESTDIR=" clean

          current directory:
          /tmp/contents830015225/deps/0/vendor_bundle/ruby/2.4.0/gems/mysql2-0.3.11/ext/mysql2
          make "DESTDIR="
          compiling client.c
          In file included from client.c:1:0:
          ./mysql2_ext.h:19:10: fatal error: mysql_com.h: No such file or directory
           #include <mysql_com.h>
                    ^~~~~~~~~~~~~
          **ERROR** Unable to install gems: exit status 5
          compilation terminated.
          Makefile:242: recipe for target 'client.o' failed
          make: *** [client.o] Error 1

          make failed, exit code 2

          Gem files will remain installed in
          /tmp/contents830015225/deps/0/vendor_bundle/ruby/2.4.0/gems/mysql2-0.3.11 for
          inspection.
          Results logged to
          /tmp/contents830015225/deps/0/vendor_bundle/ruby/2.4.0/extensions/x86_64-linux/2.4.0-static/mysql2-0.3.11/gem_make.out

          An error occurred while installing mysql2 (0.3.11), and Bundler cannot continue.
          Make sure that `gem install mysql2 -v '0.3.11' --source 'http://rubygems.org/'`
          succeeds before bundling.

          In Gemfile:
            mysql2
   Failed to compile droplet: Failed to run all supply scripts: exit status 15
   Exit status 223
   Cell 76dba435-3477-41a6-bea1-9bd134040065 stopping instance a549b1d8-7d10-4a6f-ba05-146417b43d6d
   Cell 76dba435-3477-41a6-bea1-9bd134040065 destroying container for instance a549b1d8-7d10-4a6f-ba05-146417b43d6d
Error staging application: App staging failed in the buildpack compile phase
FAILED

[Code: 1055, SQL State: 42000] (conn=3494)

Step1: as below in dev enviornment database instance in pcf(Pivotal cloud foundry)

when i am using mysql 5.7.24-26-log vesrion in Pcf (Pivotal cloud foundry) the below error i am getting.

Error:
[Code: 1055, SQL State: 42000] (conn=3494) Expression #4 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'service_instance_db.qs.id_queue_summary' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by

Step2:

I need to upgrade maria db in dev database instance in Pcf. how to do.

mariadb 5.7.24-26 to 10.1.18-MariaDB ---->pivotal cloud foundry

Can you please help me on this ,on your quick response.

Advance thanks for your support

Missing configuration step

There is no mention of Errands configuration in this document that should be there.

mysql product api

Dears,

I use the below JSON to create a new plan mysql :

{
         "properties": {
           ".properties.plan_collection": {
             "value": [
               {
                 "name": {"value": "microplan"},
                 "description": {"value": "Micro Service Plan (10mb)"},
                 "max_storage_mb": {"value": "10"},
                 "max_user_connections": {"value": "10"},
                 "private": {"value": "true"}
               }
             ]
           }
         }
}

But, it always throws back this error :

{"errors":{".properties.plan_collection":["Record 1: Name can't be blank","Record 1: Name must only contain lowercase letters, numbers, hyphen(-), or underscore(_)","Record 1: Description can't be blank","Record 1: Max Storage Mb can't be blank","Record 1: Max User Connections can't be blank"]}}

the curl I used is :

curl -k "https://opsmanager.domain.com/api/v0/staged/products/p-mysql-50e42f2a0862baa9827a/properties" -H "Authorization: bearer $accesstoken" -H "Content-Type: application/json" -X PUT -d @microplan.json

What is wrong with my JSON ? I follow this : https://docs.pivotal.io/pivotalcf/2-0/opsman-api/#updating-a-collection-property

The Using TLS section doesn't tell you how to figure out if you're already using TLS

I went through the steps in the "Using TLS" guide here. However, I'm not sure how I can verify that my app is actually using TLS to talk to the DB. Is there some command I can run to figure that out?

It would also be nice to know that at the beginning of the guide so I could clearly see TLS go from not set up to set up.

Region in the backups config : v2.7

The region field is not marked required in the "Backups" configuration options. The field when left blank brings up a
Please review the errors below Region '' is not valid for the endpoint, type: BlobstoreVerifier
Shouldn't be an optional field ?

Second test

mysql-example-app.tgz can not be unzip

Incorrect referenced credentials to log into Credhub

document section Find the CredHub Credentials references the PAS tiles UAA->Services Credhub Credentials. But when one uses these credentials this error results

> --client-name="services_credhub_client" \
> --client-secret="<my secret>"
The provided credentials are incorrect. Please validate your input and retry your request.```

instead the **BOSH Commandline Credentials** from Ops Director tiles credentials should be used. Note we are targeting BOSH's Credhub instance located on the Ops Director as directions note.

Missing step - Resource Config

There is no instruction in this document to configure Resource Config. The user does not have any clue on what needs to be done here. Or, if there is any need to configure a load balancer.

test

Rabbit?

Line 54 references RabbitMQ for PCF. Pretty sure this should be MySQL.

Contents of tile should not be listed as compatible

Hi there! I just noticed that the tile documentation lists three things as compatible - Xenial stemcells, Percona Server, and golang:

docs-mysql/release-notes.html.md.erb

Lines 170 to 176 in ead440a

 <tr> 

 <td>Percona Server</td> 

 <td>5.7.23-23</td> 

 </tr> 

 <tr> 

 <td>golang</td> 

 <td>1.10.3</td>

But only Xenial is a component you can choose to use with Pivotal MySQL (as opposed to Trusty stemcells). The others are components of the tile; they're not choices for a customer to make.

These, instead, should be listed in the release notes as, "updated version of Percona Server to X"

/cc @jzlwang

Restore single SI doc doesn't tell user how to find encryption passphrase

docs-mysql/restoring-single-instance.html.md.erb

Lines 40 to 43 in 2dc0006

 1. Decrypt and expand the backup artifact into the MariaDB data directory. For example: 

 <pre class="terminal"> 

 $ gpg --decrypt mysql-backup.tar.gpg | tar -C /mysql -xvf - 

 </pre>

This step requires a passphrase. There should be a link to the restoring section of the backups docs; getting the passphrase is covered in step 1.

Troubleshooting persistent disk full could include deleting old mysql-binlogs

As an operator, in order to recover instances with full persistent disks, it would be helpful for the troubleshooting guide to suggest deleting an old binlog from /var/vcap/store/mysql/data to free up space.

background

One of our customer's mysql (2.5, leader-follower) instances had their persistent disk fill to 99%.

In order to upgrade the instance to a larger plan, we had to bosh ssh onto the VM and purge audit logs, as described in the troubleshooting documentation. Unfortunately, that didn't clear off enough room.

We received a suggestion from support (case num 196363) to delete an old binlog on both the leader & follower from /var/vcap/store/mysql/data (e.g., mysql-bin.000003, etc), as they were taking up ~513MB each.

This was a great suggestion and deleting just one gave the instance enough room to successfully terminate && allowed the instance to upgrade to a larger plan.

Although in an ideal world an instance could be upgraded to a larger plan without operator involvement, in this world it would have been helpful to have that included in the documentation. Or perhaps in a community knowledge-base article?

Thanks for your time and consideration!

max_user_connections on restarting an application

I'm using a spring boot application with on demand mysql database provided by pcf.

When ever I'm restarting my spring boot application. I'm get following errors.

Looks like there is issue in pcf or spring.io to release the connections even when an application is stopped and being restarted.

Please let me know if I'm missing anything.

Following are the major error logs
o.s.c.s.r.PooledDataSourceCreator : Found Tomcat JDBC connection pool on the classpath. Using it for DataSource connection pooling.
[ main] o.a.tomcat.jdbc.pool.ConnectionPool : maxIdle is larger than maxActive, setting maxIdle to: 30

[OUT] java.sql.SQLSyntaxErrorException: Could not connect: User 'ZZZZ' has exceeded the 'max_user_connections' resource (current value: 40)

Some customer confusion due to PCF 2.x not being mentioned here

docs-mysql/release-notes.html.md.erb

Line 66 in e4c7647

* Can only be installed in an Ops Manager v1.12 or later environment.

There has been some customer confusion due to PCF 2.x not being mentioned here. For consistency with the v1 Release Notes, maybe this should be something like:

MySQL for PCF v2.2 requires PCF v1.12 or later or PCF v2.0 or later.

Enable TLS on mysql 2.4.x doesn't have example with BOSH DNS hostnames

The Enable TLS section of the documentation uses static IPs for hostnames when enabling TLS. However, mysql 2.4.x+ uses BOSH DNS for name resolution, so instead of "hostname": "10.1.16.3", you'll see "hostname": "q-n2s3y1.q-g13347.bosh".

Or in the case of a leader-follower topology, you'll see:

  "hostname": "q-n2s3y1.q-g13347.bosh",
  "hostnames": [
    "q-m14289n2s0.q-g13347.bosh",
    "q-m14288n2s0.q-g13347.bosh"
  ],

I am confused on a few fronts:

how should I enable TLS on these service instances using bosh dns? I'm assuming it's `cf update-service db-leader-follower -c '{"enable_tls": ["q-n2s3y1.q-g13347.bosh"]}'
how should I enable TLS on service instances using bosh DNS that are also leader-follower? In the above configuration, it's a leader-follower topology. What should my update-service string look like?
Is it possible to DISABLE TLS after a service-instance has been enabled?

Thanks for your time

Audit logging docs doesn't state what setting we choose

docs-mysql/install-config.html.md.erb

Lines 504 to 506 in 060c223

 <tr><td><strong>Enable Server Activity Logging</strong></td> 

 <td>Select this checkbox to record who connects to the servers and what queries are processed using the Percona Audit Log Plugin. 

 For more information, see the <a href="https://www.percona.com/doc/percona-server/5.7/management/audit_log_plugin.html">Percona Documentation</a>.

When describing the ability to enable or disable audit logging, we don't actually say what we've set it to.

There is no Upgrade doc

There is no upgrade doc which describes the steps and more importantly documents if there is downtime expected when doing an upgrade.

v2.8 Data at Rest - Broken Link

The link provided in the 'More Information' column for 'Encrypted Data at Rest' seems to be broken, and results in a 'Page not Found' error page.
Here's is the link in question:

<td><a href="https://docs.pivotal.io/partners/vormetric-transparent-encryption/">Thales Vormetric plugin</a></td>

Did this doc page get moved/renamed? Will be good to have a valid link here that provides helpful info on how to achieve Data at Rest encryption for MySQL DB instances provisioned from the Marketplace.

If backups can only be restored to single-node plans, docs should make larger mention of that

As an operator hardening their backup/recovery process, it's important to know that backups can only be restored to a single-node plan:

Note: You can only restore to a single-node plan. Therefore, in this procedure the new service instance you create for restore uses a single-node plan. If you are restoring to a leader-follower plan, you can update the plan on the new service instance after you run the restore utility. This is included in the instructions in Run the Restore Utility below.

This is significant to us because currently all the plans in our foundations are leader-follower. Maybe it's in the install doc and I missed it, but it would be helpful to know this when configuring plans

(Thanks for your time!)

rabbitmq screenshot for mysql docs

Screenshots on the docs show rabbitmq not mysql

	<tr>
	<td>Percona Server</td>
	<td>5.7.23-23</td>
	</tr>
	<tr>
	<td>golang</td>
	<td>1.10.3</td>

	1. Decrypt and expand the backup artifact into the MariaDB data directory. For example:
	<pre class="terminal">
	$ gpg --decrypt mysql-backup.tar.gpg \| tar -C /mysql -xvf -
	</pre>

	<tr><td><strong>Enable Server Activity Logging</strong></td>
	<td>Select this checkbox to record who connects to the servers and what queries are processed using the Percona Audit Log Plugin.
	For more information, see the <a href="https://www.percona.com/doc/percona-server/5.7/management/audit_log_plugin.html">Percona Documentation</a>.