Good afternoon, the message in CloudTrail appears incomplete, does anyone know how to retrieve the complete encoded message?

eg.:
"errorMessage": "You are not authorized to perform this operation. Encoded authorization failure message: yy00r41dxhsGHv2wBQ6iAaslG-_TOIqTkRzNZxyp9HZH9guY5aCjlsPBxxM8myj_Y2pLcK0jhv7-dysstqTNhwrY1EZjM1y3biI8lVziop7LkyGx6WTkA8Zcae-f5s3grzQCKwzTsANzE5v4UeuikcACczPrAxXRRloVxVvJokwXwHytymJ3UNOA_FG_1XfYvUtv9kgqE6-Yx6mrx7N5EeT0nX_e1iiNnSdSKNA9VmHITSwNaHdkAtoR_etVANZOO4wD78j5_PddNQemoiG0H9q4053r1xbHITwH-uV2HVOqzeSz-sv9Fk3GoRzQJ-w149RgOxRN3t4HX3jcOUMRfP6vOYmFaIo3llx_UZGtRpl_AtV6HvLm17kxNiK-Kg1ER8wZGVU2xRFAZhWgHstBz1ia0ZigEE9PKXZbn0RUBXM_dQkJsHgL3BhZCs27EtxWSMNkxPcOPy8_5Go2NYJ_pYOFUI9pnIZCUEz1pYRsW7ylK9QTxuG5iJpbjFGjX_SBnNA9u_hheKi8SmQ3lO6izedi_Tl5zJRTKp9B7FXrMXfbWYPupMylzVP9pSCih_fs2ZHWNPpa70U4g8IecipPm-UtYdM8svUmX0uDy-zAVgeAjFIWp52JWMfgWLvqUtGfFcz54xokwwezzDjikmVM75-GqSF4fLovXqQFC82k3wMqpblHvFFXSgXIas5pItLOK-k67b-NAQDA0BQMirjaWOzy271k7TlWp2twfMrOITcXCjYbMa35eCvSIry_bR-dYi6qSDyskAOi053KARVw1o0kZrSU8om7qG1Ppt_8rbjyn_op5lDZ6XzD63LHSgNv18rMlOMLt00rMSa4Bjt0du39rVcEv6uYl4Q4O0nmdQDk1fgLiC6IqOa1GPPvHRNHlTqeRvu-Z85JrbREHJN-ZhV8irX9qrGrnBMd...",

The artifacts deployed to Maven repository com are fat JARs , 50MB+, instead of ~90kb.

This is not only a size issue, but inside the JARs are complete versions of all the dependencies in the tree, such as Jackson (old one), a version that conflict with my project's version (newer).
These bundled dependencies cannot be excluded by a pom's exclusion, as they are not a dependency once they are baked into the JAR.

At this point the only workaround for me is to compile on my own and deploy to a private maven repo.

In the screenshot of event history (taken from the CloudTrail web console) , the name of the bucket affected by a change is reflected under the column: Resource name . How can I retrieve this same value using the aws-cloudtrail-processing-library . The library returns the name of the bucket where CloudTrail saves the log files and not the affected bucket. Also, even after downloading the logs from the bucket, I do not see this information.

Do I need to use the the LookupAttribute as stated in the documentation for CloudTrail API ?

When a cloudtrail is setup, it not only sends notification messages to the SQS queue but also validation messages like this one.

{
"Type" : "Notification",
"MessageId" : "e8ec0cb4-e3cc-5533-8e72-5f0cb6428101",
"TopicArn" : "arn:aws:sns:us-east-2:69443:ionAudit",
"Message" : "CloudTrail validation message.",
"Timestamp" : "2018-07-05T17:20:14.111Z",
"SignatureVersion" : "1",
"Signature" : "63Bsy+S5euNtWQx3eebA9wFZN+mLJbw8cD3ArhI+d8GzVJ2DDtqklDR5ktKpqrxtcyl3fU6QxGuy7BlVYCO0HHn3oASMAtYDc5UxQa0YMYomhE2vMsAyG11fksqJ/Pgr0jd/DpBT01Ue8cvW15/JuKzXj50pZLWP1oKcJ4S2qpn02tFwNMr6CwqP0/DVZs5Td/xT6n9LMm4+IoPGT2L1rIwCKOe/C2l3MKGkPUPgNnT+dADN9jxOevsaPc9aXJueZQLGkJOtlNJKvnYYLcQB+p4A==",
"SigningCertURL" : "https://sns.us-east-2.amazonaws.com/Sd002d285f9598aa1d9b.pem",
"UnsubscribeURL" : "https://sns.us-east-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-2:3:Sudit:bc854278-4a1d-413d-8a78-1206c8e4fe57"
}

The lib throws an exception when trying to parse these messages.

2018-07-05 16:58:23,765 ERROR pool-1-thread-1 cloudtrail.processinglibrary.impl.DefaultExceptionHandler:32 - Failed to parse sqs message.
com.amazonaws.services.cloudtrail.processinglibrary.exceptions.ProcessingLibraryException: Failed to parse sqs message.
at com.amazonaws.services.cloudtrail.processinglibrary.utils.LibraryUtils.handleException(LibraryUtils.java:170)
at com.amazonaws.services.cloudtrail.processinglibrary.manager.SqsManager.parseMessage(SqsManager.java:165)
at com.amazonaws.services.cloudtrail.processinglibrary.reader.EventReader.getSources(EventReader.java:113)
at com.amazonaws.services.cloudtrail.processinglibrary.AWSCloudTrailProcessingExecutor$ScheduledJob.run(AWSCloudTrailProcessingExecutor.java:175)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: SourceSerializerChain expect JSON content
at com.amazonaws.services.cloudtrail.processinglibrary.serializer.SourceSerializerChain.getCloudTrailSource(SourceSerializerChain.java:98)
at com.amazonaws.services.cloudtrail.processinglibrary.serializer.SourceSerializerChain.getSource(SourceSerializerChain.java:73)
at com.amazonaws.services.cloudtrail.processinglibrary.manager.SqsManager.parseMessage(SqsManager.java:158)
... 9 more
Caused by: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'CloudTrail': was expecting ('true', 'false' or 'null')
at [Source: (String)"CloudTrail validation message."; line: 1, column: 11]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:673)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken(ReaderBasedJsonParser.java:2835)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1889)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:747)
at com.fasterxml.jackson.databind.ObjectMapper._readTreeAndClose(ObjectMapper.java:4030)
at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:2539)
at com.amazonaws.services.cloudtrail.processinglibrary.utils.SNSMessageBodyExtractor.getMessageBody(SNSMessageBodyExtractor.java:37)
at com.amazonaws.services.cloudtrail.processinglibrary.serializer.CloudTrailSourceSerializer.getSource(CloudTrailSourceSerializer.java:59)
at com.amazonaws.services.cloudtrail.processinglibrary.serializer.SourceSerializerChain.getCloudTrailSource(SourceSerializerChain.java:96)

Seems like the lib should toss these messages. Or is there something I can setup so these validation messages don't go to my queue?

Hi! I'm currently trying to use the DefaultEventSerializer.

In the AbstractEventSerializer:
Line 495-496 are causing an IllegalArgumentException:
private UUID convertToUUID(String str) {
return UUID.fromString(str);
}

Context:
While I am processing CloudTrail logs; sometimes (maybe like 1-2 events every hour) an event has an "requestID" value which isn't a proper UUID, or so java complains.

Line 157-158 are the lines leading up to the use of that function.
158: eventData.add(key, this.convertToUUID(this.jsonParser.nextTextValue()));

Since this value isn't of any significance to me, it isn't any real issue, but the exception is however present when used in production.

Hello

I tried to figure out what was the diff. between 1.1.0 and 1.1.1 to better understand the release notes of 1.1.1, and found that no code was changed/added, nor any dependencies were updated, only minor docs change.

Were PRs/commits not pushed?

The following line block of code will result in aws-cloudtrail-processing-library spamming the log output with event names every time an event is processed:

Builder factory = new Builder(this, configuration_).withExceptionHandler(this); cloudTrailExecutor_ = factory.build();

I believe the root cause is that the DefaultEventsProcessor is always registered for callbacks irrespective of whether or not an EventProcessor is defined.

https://github.com/aws/aws-cloudtrail-processing-library/blob/master/src/main/java/com/amazonaws/services/cloudtrail/processinglibrary/impl/DefaultEventsProcessor.java#L34

I have been working on getting CloudTrail setup so that there is a central account where an S3 bucket lies for the CloudTrail logs to be written to and a central SQS queue subscribed to SQS Topics in other accounts. This setup has been working successfully with the exception of some messages resulting in a JsonMappingException. The exact messages it:

'''
JsonMappingException: Can not deserialize instance of java.util.ArrayList out of VALUE_STRING token
'''

This exception is occurring in the library when attempting to parse the SQS Message to determine its source. I reviewed one of the messages and found that the s3ObjectKey value is in fact a String rather than a List, which explains the exception. The library should support s3ObjectKeys being Strings if sqsMessages generates Strings for the value. The message itself does not appear to be a CloudTrail message but it would seem that the library should support parsing the messages successfully, and reject based on the fact that its not a cloud trail message, or give the user of the library the ability to reject the message.

Below is the relevant parts an example message causing the issue:

'''
"Type" : "Notification",
"MessageId" : "",
"TopicArn" : "arn:aws:sns:us-west-2:222222222222:MyTopic",
"Subject" : "[AWS Config:us-west-2] Configuration History Delivery Completed for Account 222222222222",
"Message" : "{"s3ObjectKey":"us-west-2/AWSLogs/222222222222/Config/us-west-2/2015/11/17/ConfigHistory/222222222222_Config_us-west-2_ConfigHistory_AWS::EC2::Subnet_20151117T164715Z_20151117T222216Z_1.json.gz","s3Bucket":"MyBucket","notificationCreationTime":"2015-11-17T22:38:57.729Z","messageType":"ConfigurationHistoryDeliveryCompleted","recordVersion":"1.1"}",
"Timestamp" : "2015-11-17T22:38:57.744Z",
'''

Hi,
I tried to run the sample application, SampleApp. However, it constantly returns:
pollQueue is false , and latency is 202832 milliseconds.
This seems to be from the SampleProgressReporter, and the executor never seems to go past the pollQueue phase. Any ideas on how to fix this? Thanks.

Best,
Michael Qiu