nickgrealy / aws-sqs Goto Github PK

Done

• Remove dependency on GitHub specific Events
• Remove "required" validation on "AccessKey" and "SecretKey" fields (can be implicit role on machine)
• Rename packages to avoid conflicts with SCM configuration
• REBRAND: from scm-sqs to aws-sqs

Todo

• Clean up redundant classes from SCM plugin
• Update documentation

Longer Term

• Use EnvironmentContributor (to workaround SECURITY-170)
• Migrate to gradle
• Review unit tests

Hi Nick,

Thank you for the great work with the aws-sqs plugin!

We are trying to use multiple CodeCommit repos with a single SQS queue. Currently, when the plugin receives any sort of notification from CodeCommit, it triggers the build of ALL jobs that matches the branch (of the SNS message), but without consideration of the repo that generated it (eventSourceARN).

For example, if I have two repos ("FOO" and "BAR") and a job for each that tracks the "master" branch, a notification of "FOO/master" being updated will trigger both jobs, which is obviously not correct.

Is it possible to extend this plugin to use the eventSourceARN to filter out which repo to trigger? Without this we will have to create N SQS queues for N repos, which can be a lot, and also makes the Jenkins configuration of them very cumbersome.

For your reference here is a sample SNS message that shows where eventSourceARN is located.

{
  "Records": [
    {
      "awsRegion": "us-east-1",
      "codecommit": {
        "references": [
          {
            "commit": "eedd7213b474f914db1c25c3313d9e7be6ba71d0",
            "ref": "refs/heads/feature/my_new_feature"
          },
          {
            "commit": "eedd7213b474f914db1c25c3313d9e7be6ba71d0",
            "ref": "refs/tags/v110"
          }
        ]
      },
      "customData": "",
      "eventId": "464817b2-e391-4cf9-bf52-bd0ed1ad6900",
      "eventName": "TriggerEventTest",
      "eventPartNumber": 1,
      "eventSource": "aws:codecommit",
      "eventSourceARN": "arn:aws:codecommit:us-east-1:{aws-account-id}:{codecommit-repo-name}",
      "eventTime": "2016-10-11T13:52:59.745+0000",
      "eventTotalParts": 1,
      "eventTriggerConfigId": "464817b2-e391-4cf9-bf52-bd0ed1ad6900",
      "eventTriggerName": "notifications",
      "eventVersion": "1.0",
      "userIdentityARN": "{aws-user-arn}"
    }
  ]
}

Thank you for your help!

SQS is sending a single message, but my job is being triggered multiple times. I believe this is due to multiple listeners being registered:

https://github.com/jenkinsci/aws-sqs-plugin/blob/2690f662d05e92e863d0bc1527b16770f38fda50/src/main/java/io/relution/jenkins/awssqs/threading/SQSQueueMonitorImpl.java#L193-L195

This seems to be born out with info from the Jenkins log:

0001CA Received 1 message(s) from MY-QUEUE-NAME (sqs.us-west-2.amazonaws.com) {7e088060-ab00-45b7-b05b-00a956e57c43}
Jul 06, 2017 5:30:05 AM INFO io.relution.jenkins.awssqs.logging.Log write
0001CA Message received...
Jul 06, 2017 5:30:05 AM INFO io.relution.jenkins.awssqs.logging.Log write
0001CA Triggering job with 4 parameters...
Jul 06, 2017 5:30:05 AM INFO io.relution.jenkins.awssqs.logging.Log write
0001CA Triggering job [COMPLETED]
Jul 06, 2017 5:30:05 AM INFO io.relution.jenkins.awssqs.logging.Log write
0001CA Message received...
Jul 06, 2017 5:30:05 AM INFO io.relution.jenkins.awssqs.logging.Log write
0001CA Triggering job with 4 parameters...
Jul 06, 2017 5:30:05 AM INFO io.relution.jenkins.awssqs.logging.Log write
0001CA Triggering job [COMPLETED]
Jul 06, 2017 5:30:05 AM INFO io.relution.jenkins.awssqs.logging.Log write
0001CA Send delete request for 1 message(s) to MY-QUEUE-NAME (sqs.us-west-2.amazonaws.com) {7e088060-ab00-45b7-b05b-00a956e57c43}
Jul 06, 2017 5:30:05 AM INFO io.relution.jenkins.awssqs.logging.Log write
0001CA Deleted 1 message(s) (0 failed) from MY-QUEUE-NAME (sqs.us-west-2.amazonaws.com) {7e088060-ab00-45b7-b05b-00a956e57c43}

I'm not able to see why multiple listeners were registered or how to prevent this from happening in the future. Any suggestions/pointers?

Thanks awesome plugin.

The README.md use case is great for AWS Code-commit integration, but it would be great to include a simpler use case and the task parameter details as well/instead. For example the plugin itself doesn't require any AWS Code-Commit, SNS etc.

Are there any other magic parameters (eg for the message Attributes)?

A bit cheeky to raise this as if an issue rather than fork, apologies :)