cnenning / go-artifactory-scm-plugin Goto Github PK

Dear cnenning,

I have found a small label bug.
The label value from Artifactory always includes a slash at the end.

Can you fix it please?

Thank you
Lasim

Add another text field to specify a filename pattern to be able to download just a few files.

gocd server version:17.11
setting Proxy URL
But found the plugin does not through proxy.
We see go-server.log and confirm our environment variables is through proxy
In plugin-artifactory-pkg.log we see it does not through proxy

When a timeout happens during checkout operation go logs a message like this:
Interaction with plugin with id 'artifactory-scm' implementing 'scm' extension failed while requesting for 'checkout'. Reason: [The plugin sent a response that could not be understood by Go. Plugin returned with code '500' and the following response: 'java.net.SocketTimeoutException: Read timed out']
Afterwards pipelines show weird behavior.

go API "plugin settings configuration" should be supported

https://developer.go.cd/current/writing_go_plugins/plugin_settings/version_1_0/plugin_settings_configuration.html

There are date/time parsing related errors all over logs 😢 . That's not a big issue just annoying.

But it happens that api request latest-revisions-since fails which causes too old versions to be deployed. That is an issue!

SCM URLs must end with a trailing slash, the plugin should check that.

I'm trying to use go-artifactory-scm-plugin.
It's all working for the latest version but how do I set a pipeline to use a specific version e.g. I have version folders 1.1.10, 1.1.11 and 1.1.12. I want to use version 1.1.10. I've set the environment variable VERSION=1.1.10 in GOCD but the pipeline always pulls the latest version (1.1.12 in this case).

What do I need to do to pull a specific version?

Thanks

It would allow for greater flexibility in custom pipeline scripts if just selected version number would be available instead of downloaded files.

An idea is to add a checkbox to material definiton to let plugin just write a file version.txt instead of downloading files.

Could be used e.g. for docker images.

It is missleading that message of "check connection" button claims it found oldest version. Would be nice if it would contain newest version instead.

There are cases where some directories are present in artifactory which shall be ignored by go. This could be achived with another optional Regex.

I observed NPEs like following. I usually happens when a new material is created and stops after artifacts have been downloaded via that material. Url is stripped but correct.

.... ArtifactoryScmPlugin:77 - ERROR!
    request name: latest-revision
    request body: {"scm-configuration":{
                               "version_only":{"value":""},
                               "password":{"value":""},
                               "version_regex":{"value":""},
                               "dummy_id":{"value":""},
                               "pattern":{"value":"...."},
                               "url":{"value":"...."},
                               "username":{"value":""}
                           },"scm-data":{},"flyweight-folder":"...."}
java.lang.NullPointerException
    at com.github.cnenning.artiscm.AbstractArtifactoryPlugin.buildRevisionJson(AbstractArtifactoryPlugin.java:433)
    at com.github.cnenning.artiscm.ArtifactoryScmPlugin.handleLatestRevision(ArtifactoryScmPlugin.java:172)
    at com.github.cnenning.artiscm.ArtifactoryScmPlugin.handleApiRequest(ArtifactoryScmPlugin.java:64)
    at com.github.cnenning.artiscm.AbstractArtifactoryPlugin.handle(AbstractArtifactoryPlugin.java:193)
    at com.thoughtworks.go.plugin.infra.DefaultPluginManager$2.execute(DefaultPluginManager.java:187)
    at com.thoughtworks.go.plugin.infra.DefaultPluginManager$2.execute(DefaultPluginManager.java:182)
    at com.thoughtworks.go.plugin.infra.FelixGoPluginOSGiFramework.executeActionOnTheService(FelixGoPluginOSGiFramework.java:315)
    .....

Artifactory might be protected by password. go should be able to provide it.

... that would make it easier to identify go-requests in artifactory log. Currently plugins show ow with a generic java http client user-agent.

Package Materials are useful when you just want to deploy latest version instead of choosing a version.

As package-plugin does not download files but just creates variables containing names found in artifactory it should be easy to support directories as well.

That would allow more flexibility in own pipeline scripts.

For go installations with many pipelines there are occurring exceptions like:
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool

To avoid those the connection pool size should be configurable (like timeouts are).

Hi,

After plugin installation I don't see any configuration which is related to artifactory. List of materials still has the same structure, without artifactory scm. Also I didn't manage to find any global configurations that are related to this plugin.
Could you please advise the steps how to configure plugin or I am missing something?

This is a continuation of Issue 21.

Sometimes the plugin doesn't pull anything when a pipeline runs (I don't see any of the logging in the pipeline logs). This may be due to the SocketTimeout I mention later.

Sometimes it pulls old versions of the artifact. This is an ongoing issue and related to issue 21.

I've done all the things we discussed in issue 21. Increased the socket timeout to 5000. Restarted the GO server. I've deleted and re-added the material. It still fetches an old version of the artifact.

I've looked in the GOCD and plugin logs. Here are some interesting lines:

go-server.log
2018-03-21 16:18:47,924 INFO [94@MessageListener for MaterialUpdateListener] MaterialRepository:857 - Saving revisions for material [[dummy_id=donation, url=http://artifactory.justgiving.com:80/artifactory/int-Files/justgiving/helm/]] after removing the following duplicates [1.0.11, 1.0.12, 1.0.13, 1.0.14, 1.0.15, 1.0.16, 1.0.17, 1.0.18, 1.0.19, 1.0.20, 1.0.21, 1.0.22, 1.0.23, 1.0.24, 1.0.25, 1.0.26, 1.0.27, 1.0.28, 1.0.29, 1.0.30, 1.0.31, 1.0.32]

It's version 1.0.32 that I need. Why does it say they are duplicates (I know this is GOCD code, not your plugin).

plugin-artifactory-scm.log
2018-03-21 13:03:11,331 ERROR [91@MessageListener for MaterialUpdateListener] ArtifactoryScmPlugin:77 - ERROR! request name: latest-revisions-since request body: {"scm-configuration":{"password":{"value":"xxxxx"},"dummy_id":{"value":"recurring-channels"},"url":{"value":"http://artifactory.justgi ving.com:80/artifactory/int-Files/justgiving/helm/"},"username":{"value":"xxxxx"}},"scm-data":{},"flyweight-folder":"/mnt/go-server/pipelines/flyweight/47ed6d68-dfe4-45d 3-96a5-1c3aec72d3c9","previous-revision":{"revision":"1.0.32","timestamp":"2018-03-14T14:18:00.000Z","data":{}}} java.net.SocketTimeoutException: Read timed out

I was having the above errors quite a bit but haven't seen them since changing the plugin config settings to:
2018-03-21 13:45:04,805 INFO [Thread-75] ArtifactoryScmPlugin:52 - setting connectTimeout: 240 2018-03-21 13:45:04,805 INFO [Thread-75] ArtifactoryScmPlugin:52 - setting socketTimeout: 5000 2018-03-21 13:45:04,806 INFO [Thread-75] ArtifactoryScmPlugin:52 - setting connectionRequestTimeout: 240 2018-03-21 13:45:04,806 INFO [Thread-75] ArtifactoryScmPlugin:52 - setting ConnPoolSize: 5

Any help much appreciated.

This happens due to changes for #4 and changes in go 16.11. To avoid it and still get the benefit of #4 I'm going to introduce a dummy id field.

Currently it is not possible to do just that.

Chages for #3 should be applied when getting directory listings, too.

I have artifactory structure like this:

/1.0.9/someFile.gzip
/1.0.10/someFile.gzip
...
/1.0.24/someFile.gzip

Some pipelines pull correctly the latest version, 1.0.24, but others seem to pull randomly from previous versions. I've checked they don't specify a version regex. I can't understand how to get the pipelines all pulling the latest version consistently.