sfdo-tooling / cumulusci Goto Github PK

Create a repository with a very basic Force.com project to test the build and automation scripts.

When creating a GitHub Release, upload the unpackaged/pre and unpackaged/post bundles from the package as individual zip attachments to the Release.

Also, pass the zip url to mrbelvedere to configure the bundles for installation.

Question: Does storing zips from every build potentially create issues with limits in GitHub?

Currently, we use a loop with a hard coded count of 5 iterations to try to install a newly created beta package. Typically, this is enough time for a new package to be copied between pods and available for installation.

However, a better approach is to use time rather than number of iterations. Loop through trying to install the package and at the end of each loop where the installation failed, find if we have been trying for longer than the timeout value. If so, stop the loop and fail otherwise keep trying.

Within the buildPackageXmlFromDir macro definition EmailTemplates and Letterheads are not mapped

Title says it all.. Screenshot attached.

#77 is Awesome!!! Thank you.

We have multiple patterns that are used in test class names. Would like to have the namematch property allow a comma separated list of patterns to find the requisite classes.

Currently, you have to manually create the unmanaged package in a new DE org before you can deploy unmanaged code into it. I incorrectly assumed you can't create the unmanaged package via the MDAPI because we got INVALID_CROSS_REFERENCE_ID errors when trying to run deployCI against a new DE org. It turns out that the only reason that happens is that the uninstall target which runs at the start of the build tries to retrieve metadata from the package which is when the error occurs.

I think the solution is to do an empty deployment at the start of the build using a package.xml file with the <fullName> element set which should create the unmanaged package in the target org.

See https://www.codeship.io/projects/40064/builds/2553086

The app references a tab that is not listed in package.xml.

Our company has a bitbucket license , can I use this with bitbucket instead of github ? #Question

The timeout property for the InstalledPackage based macros is hard coded. Has resulted in REQUEST_RUNNING_TOO_LONG error when installing dependent packages.

Three issues discovered in this build:
https://codeship.io/projects/35613/builds/2388196

1. There should be release note content since there hasn't been a production release and one merged pull request has content
2. We need to handle a graceful SystemExit exception if there is no content without thinking it is an error and warrants a traceback.
3. The failure of the release_notes.py execution should have marked the build as a failure

How can we handle the following scenario when an organization has >1 concurrent build for their Codeship account:

1. User A pushes feature/User-A-Code branch on repo my_package
2. Codeship starts build of feature/User-A-Code to the my_package.feature org
3. User B pushes feature/User-B-Code branch on repo my_package
4. Codeship starts a concurrent build of feature/User-B-Code to the my_package.feature org
5. feature/User-B-Code build fails due to the my_package.feature org is still deploying feature/User-A-Code branch

This file will contain generic targets that can be used by all projects.

Each project that wants to use this dev process will have a build.xml file that will include project-specific info. It will also import (https://stackoverflow.com/questions/14203337/ant-import-vs-include-tasks) the build.xml from CumulusCI.

The retrievePackagedToSrc target can be used to retrieve all metadata from the package into the src directory.

The deployDevOrg target should do pretty much everything the deployCI build does except it should not run all tests. It should also have its own pre and post hook targets so projects can override the configuration of developer orgs.

When editing a file in vi, a swap file is stored. The updatePackageXml target processes the swap file. For example, if you have vi open on MyObject.object, you will also have a .MyObject.object.swp file. You wind up with MyObject and .MyObject listed in the package.xml. The package xml generation should ignore these files.

When you run the uninstall target in a repository which has Workflows, the following error occurs:

Error: Cannot delete a workflow object; Workflow Rules and Actions must be deleted individually

Need to figure out a way to selectively exclude single file based metadata from the destructiveChanges.xml generation.

Ant generates a lot of temporary files that it needs for processing. We could let the user define where to create them, so no dev mistakenly commits one of them to the repo if he sees it inside the project.

Low priority, since it's unlikely this happens.

When I deploy a github project into a new dev org, I would like for all profiles to have full access to all incoming metadata. When I recently ran through the org-init process, I don't think field security was assigned, because I'm now having to go back into the profile config and enable all tabs, as well as enable read/write on all fields in the packaged code.

For one potential user, the following metadata types need to be supported by the build scripts:

ConnectedApp
CustomObjectTranslation
Profile
ReportType
Translations

Some of the ant macros spit out a lot of unnecessary output to STDOUT. Ant provides no means to turn off that output, but since codeship.sh calls the ant tasks it can filter the STDOUT from them to remove the unnecessary lines.

We need to make sure that we're showing the build status to date and that we can correctly capture the exit status of the ant command.

Selenium build appears to fail if the property key and value are separated like: ' = ' instead of run together like: '='.

Failed:

cumulusci.package.name.managed = rC Bios

Passed:

cumulusci.package.name.managed=rC Bios

Failure traceback:

deployUnpackagedPost:

BUILD SUCCESSFUL
Total time: 5 minutes 28 seconds
Requirement already up-to-date: selenium in /home/rof/.virtualenv/lib/python2.7/site-packages
Cleaning up...
Requirement already up-to-date: requests in /home/rof/.virtualenv/lib/python2.7/site-packages
Cleaning up...
Starting browser
Loaded package listing page
Traceback (most recent call last):
File "../CumulusCI/ci/package_upload.py", line 232, in <module>
uploader.build_package(build_name)
File "../CumulusCI/ci/package_upload.py", line 132, in build_package
driver.find_element_by_xpath("//th[contains(@class,'dataCell')]/a[text()='%s']" % self.package).click()
File "/home/rof/.virtualenv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 230, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "/home/rof/.virtualenv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 662, in find_element
{'using': by, 'value': value})['value']
File "/home/rof/.virtualenv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "/home/rof/.virtualenv/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: u'Unable to locate element: {"method":"xpath","selector":"//th[contains(@class,\'dataCell\')]/a[text()=\'cumulusci.package.name.managed = rC Bios\']"}' ; Stacktrace: 
at FirefoxDriver.prototype.findElementInternal_ (file:///tmp/tmpKpv9s8/extensions/[email protected]/components/driver-component.js:9618:7)
at fxdriver.Timer.prototype.setTimeout/<.notify (file:///tmp/tmpKpv9s8/extensions/[email protected]/components/driver-component.js:548:5) 

Related to #44

Once the test repository exists, configure a Codeship project to test all commits of the CumulusCI repository and start using CumulusCI on CumulusCI (sort of) to create a more stable release process.

Our dev process is based on the premise that the project's code will work the same in developer orgs and in the packaging org. We should document the places where the namespace is and isn't automatically resolved, and describe the workarounds for the latter.

The documentation currently in the repository is from an earlier version of the process which didn't include things like automated builds of beta packages.

When a deploy to the packaging org fails for whatever the reason may be, the selenium script to upload a new package still kicks in. I can image you don't want a new (beta) upload if a deploy fails.

Since CumulusCI is designed to support the same package code being deployed into both unmanaged and managed packages, there is a challenge of how to support the @deprecated annotation which is only valid when deployed to the packaging org.

The solution is likely to use a commented annotation and have a build target remove the comment (i.e. from //@deprecated to @deprecated) before deploying to the packaging org via deployCIPackageOrg. This way, developers working on unmanaged code would still be able to deploy the code in the repo to their orgs making the development experience with tools like the Force.com IDE a lot more pleasant. Deployments to the packaging org shouldn't be happening from the Force.com IDE but instead via the deployCIPackageOrg target so dynamically modifying the code only for managed deployments seems reasonable.

https://docs.readthedocs.org/en/latest/getting_started.html#write-your-docs

Being able to specify both parallel testing as well as include/exclude values would be great!

cumulusci.test.parallel=true
cumulusci.test.namematch.include=%Test,rC_%Tests_%
cumulusci.test.namematch.exclude=rC_DeveloperLocalTests_%

Additionally, being able to override this with an environment variable would be great! That we our feature branch builds would use the fast tests, and the master builds would use the full test suite.

export CUMULUSCI_TEST_NAMEMATCH_INCLUDE=FastTests_%

I believe it was because I didn't have all the necessary environment variables set up that the build failed. The log was not very explanatory:

Implement support for CircleCI as an easy to setup automation alternative to Jenkins for running the process automation.

Currently, some of the python scripts hard code the branch prefixes. These should get passed as environment variables to override or set to the default values feature/ master beta/ and release/

We have the packagePassword attribute in the installPackage macro, it's not being used.

Would be nice to add the password to the version.properties file, if security isn't a concern. Otherwise build.properties, perhaps?

Advantages:

• We could make the dependencies be Maven dependencies. Like:
  • AntUnit (new dependency - to test ant script)
  • Ant Contrib (already a dependency)
• We could create an archetype for new projects (existing projects could be added the necessary files through a specific target)

Calling Ant from Maven should not be a problem, since we can use the Maven AntRun plugin (https://maven.apache.org/plugins/maven-antrun-plugin/). We can also use Maven from Ant with https://maven.apache.org/ant-tasks/.

In case the deployUnpackagedPost is necessary for tests to pass, the runAllTests should run after deployUnpackagedPost.

When updating a required managed package, check for an unpackaged/NAMESPACE/pre and unpackaged/NAMESPACE/post. If subdirectories exist under either, run deployMetadataBundles for the directory to deploy the bundles either before or after updating the package.

Currently, package_upload.py uses an OAuth token to log into the packaging org. With cloud CI platforms like Codeship.io and CircleCI, you already have to configure the packaging org's username, password, and security token so it makes more sense to just use those in selenium rather than requiring the configuration of OAuth.

If we add a new property in cumulusci.properties, every implementing project currently would have to add the property to their project's cumulusci.properties file.

A better approach is to use a default value for all properties if they are not set in cumulusci.properties.

These are the ant-related scripts that are not package-specific.

Finish the integration with Codeship.io to provide the automation layer as an easier to setup alternative to Jenkins.

The following hooks should be added to avoid the need to override the base targets:

preDeployCI - Runs at the start of the deployCI target
preDeployCIPackageOrg - Runs at the start of the deployCIPackageOrg target
preDeployManaged - Runs at the start of the deployManaged target
preDeployManagedBeta - Runs at the start of the deployManagedBeta target
preDeployWithoutTest - Runs at the start of the deployWithoutTest target
postDeployCI - Runs at the end of the deployCI target
postDeployCIPackageOrg - Runs at the end of the deployCIPackageOrg target
postDeployManaged - Runs at the end of the deployManaged target
postDeployManagedBeta - Runs at the end of the deployManagedBeta target
postDeployWithoutTest - Runs at the end of the deployWithoutTest target

Example use case: A project has static resource bundle source files stored outside the src directory and wants to zip them up before any CI deployment. They want to create the static resource zips differently for unmanaged vs managed deployments.

In this use case, they would currently have to override some of the main targets. With these hooks in place, they could just override the hook while keeping the core functionality of the main targets.

The process for handling patch releases was considered but has not yet been implemented. Since NPSP 3.0 is deployed to everyone at once and we isolate all work in feature branches, we can easily cut new releases and push out for any issues that come up. Thus, we've never needed to do patch upgrades.

However, many projects face customer demands that require use of the patch release process. The solution is to create a branch to track the patch changes and ensure any fixes which need to be included in master get included. This probably means adding new build targets and automation scripts to support the process, however much of the existing process should be able to be leveraged for patch releases.