From looking at the code, it doesn't seem like there is any way to configure sqs and disable or limit logging when sending to sqs. So if I'm sending thousands of messages per few seconds, that generates a lot of logs and hurts performance

amazon-sqs-java-messaging-lib does not currently support the JMSPriority header, it is ignored.

Below is our current deployment environment:

Wildfly 8.2
Camel 2.14
Camel JMS (which in turn uses Spring JMS)
JMS Providers HornetQ and ActiveMQ
Currently we rely on our JMS providers to support

JMSPriority
MessageSelectors
transacted

We've been facing high CPU utilization after putting in SQS JMS based implementation on production.

The root cause turned out to SQS connectivity issues and the use of CacheLevel: CACHE_CONSUMER and a Fixed BackOff. Turns out CACHE_CONSUMER is too aggressive and even with an ExponentialBackOff, it will try to retry quite aggressively. After changing the CacheLevel to CACHE_CONNECTION, the CPU utilization is stable in our test environments.

We are facing another issue though. To simulate SQS connectivity issues, we kill the network connection and monitor using VisualVM. The no. of threads created for SQSSessionCallbackScheduler are quite high and they never exit.

See screenshot below:

After resuming network connectivity, the threads don't seem to go down and new threads are created after this point.

We are using Spring DMLC with the below configuration:

ConcurrentConsumers: 10
MaxConcurrentConsumers: 25
CacheLevel: CACHE_CONNECTION (1)
AcknowledgeMode: CLIENT
BackOff: Exponential ( InitialInterval: 5000, Multiplier: 1.5)
IdleConsumerLimit: 1
IdleTaskExecutionLimit: 5
ReceiveTimeout: 1000

Any help?

amazon-sqs-java-messaging-lib does not currently support MessageSelectors. Attempting to create a Message Selector results in the following error:

javax.jms.JMSException : SQSSession does not support MessageSelector. This should be null. [Proxied because : Original exception not deserilizable, ClassNotFoundException]
    at com.amazon.sqs.javamessaging.SQSSession.createConsumer(SQSSession.java:539)
    at org.springframework.jms.core.JmsTemplate.createConsumer(JmsTemplate.java:993)
    at org.springframework.jms.core.JmsTemplate.doReceive(JmsTemplate.java:722)
    at org.springframework.jms.core.JmsTemplate$10.doInJms(JmsTemplate.java:706)
    at org.springframework.jms.core.JmsTemplate$10.doInJms(JmsTemplate.java:703)
    at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:466)

Below is our current deployment environment:

• Wildfly 8.2
• Camel 2.14
• Camel JMS (which in turn uses Spring JMS)
• JMS Providers HornetQ and ActiveMQ

Currently we rely on our JMS providers to support

• JMSPriority
• MessageSelectors
• transacted

while the a queue message consumer more than 100000,the consumer is error cause by Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:953) ~[na:1.7.0_75]
at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1725) ~[na:1.7.0_75]
at sun.security.ssl.HandshakeOutStream.flush(HandshakeOutStream.java:122) ~[na:1.7.0_75]
at sun.security.ssl.ClientHandshaker.serverHelloDone(ClientHandshaker.java:970) ~[na:1.7.0_75]
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:341) ~[na:1.7.0_75]
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:901) ~[na:1.7.0_75]
at sun.security.ssl.Handshaker.process_record(Handshaker.java:837) ~[na:1.7.0_75]
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1023) ~[na:1.7.0_75]
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1332) ~[na:1.7.0_75]
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1359) ~[na:1.7.0_75]
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1343) ~[na:1.7.0_75]
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543) ~[httpclient-4.5.jar:4.5]
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409) ~[httpclient-4.5.jar:4.5]
at com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.connectSocket(SdkTLSSocketFactory.java:128) ~[aws-java-sdk-core-1.10.32.jar:na]
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177) ~[httpclient-4.5.jar:4.5]
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304) ~[httpclient-4.5.jar:4.5]
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611) ~[httpclient-4.5.jar:4.5]
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446) ~[httpclient-4.5.jar:4.5]
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882) ~[httpclient-4.5.jar:4.5]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[httpclient-4.5.jar:4.5]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) ~[httpclient-4.5.jar:4.5]
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:749) ~[aws-java-sdk-core-1.10.32.jar:na]
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:505) ~[aws-java-sdk-core-1.10.32.jar:na]
... 33 common frames omitted
Caused by: java.io.EOFException: SSL peer shut down incorrectly
at sun.security.ssl.InputRecord.read(InputRecord.java:482) ~[na:1.7.0_75]
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:934) ~[na:1.7.0_75]
... 55 common frames omitted

my EC2 AppServer deploy app by java version 1.7.0_75-b13, i guess the error is not error cause by java version.

I've attempted to configure this library with Wildfly 8.2.0 using its generic JMS support. The generic JMS glue code makes some calls to the following methods which raise "method not supported" exceptions:

com.amazon.sqs.javamessaging.SQSSession#setMessageListener
com.amazon.sqs.javamessaging.SQSConnection#createConnectionConsumer(javax.jms.Queue, java.lang.String, javax.jms.ServerSessionPool, int)

Without support for MDB, this library does not support enough of the JMS feature set to consider using this library over our regular JMS brokers.

What is the significance of this project being owned by the awslabs Github org rather the main aws org?

Is there an official statement on how projects hosted in these two different orgs differ, e.g. in terms of the level of Amazon support, maintenance and future development?

It would be useful for this to be clarified to inform Amazon customer decisions on whether 'awslabs' projects are suitable for use in production applications, and what if any risks apply when using the projects.

Thanks.

At the moment, the maximum value for the delaySeconds field is 900 seconds (15 minutes). I was wondering, why can't we increased it to 1 hour at least?

Many thanks,
Ciprian.

Hello team,
How can set the Q url if the q is exists on different AWS account.Means my server and q are sitting in the different AWS account.
Eg:
How can access this q using url (eg: https://sqs.ap-southeast-2.amazonaws.com/123456/qname)

Thanks
Sidath

` @Autowired
JmsTemplate template;

public void send(String message) {

	template.send(SQSQueue.OSAMA_TEST, new MessageCreator() {

		public Message createMessage(Session session) throws JMSException {
			Message createMessage = session.createTextMessage("kokokokokokokooooooooooooo");
			createMessage.setStringProperty(SQSMessagingClientConstants.JMSX_GROUP_ID, "messageGroup1");

			createMessage.setStringProperty(SQSMessagingClientConstants.JMS_SQS_DEDUPLICATION_ID,
					"1" + System.currentTimeMillis());

			return createMessage;
		}
	});

}

`

com.amazonaws.services.sqs.model.AmazonSQSException: The request must contain the parameter MessageGroupId. (Service: AmazonSQS; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 00412118-2709-5bdd-84d1-7338fdc5d8e0) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667) at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513) at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:1792) at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:1768) at com.amazonaws.services.sqs.AmazonSQSClient.executeSendMessageBatch(AmazonSQSClient.java:1659) at com.amazonaws.services.sqs.AmazonSQSClient.sendMessageBatch(AmazonSQSClient.java:1635) at com.amazonaws.services.sqs.buffered.SendQueueBuffer$SendMessageBatchTask.process(SendQueueBuffer.java:512) at com.amazonaws.services.sqs.buffered.SendQueueBuffer$OutboundBatchTask.run(SendQueueBuffer.java:443) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

The behavior of negative acknowledgment is to change the visibility timeout of a received message to 0. Where the value of NACK_TIMEOUT is not configurable while creating the SQS Factory for JMS.

https://github.com/awslabs/amazon-sqs-java-messaging-lib/blob/master/src/main/java/com/amazon/sqs/javamessaging/acknowledge/NegativeAcknowledger.java#L99

When a message is received, and the processing fails (Listener method throws an error), the message is immediately received again. In most of the cases, the message can be processed with a certain delay.

Is it possible to configure it to not change the visibility timeout, so it respects the Queue's default Receive Timeout configuration?

PS - I originally asked on Stackoverflow

If any of the message attributes have a custom data type set, this results in their data type coming in with the sqs message as "SimpleType.CustomType"
Thus if you select "String" and enter "Color", the data type is "String.Color"
If you select "Number.int" and add "age", you get "Number.int.age".

The getObjectValue method on the SQSMessage class is parsing for exact match of data type, where it should be looking for startsWith.

The result is that parsing of the message attributes throws a JMS Exception, and the message bounces back to the queue instead of being processed.

amazon-sqs-java-messaging-lib does not currently support transacted SQS Session. Attempting to create a transacted SQS Session results in the following error:

javax.jms.JMSException : SQSSession does not support transacted [Proxied because : Original exception not deserilizable, ClassNotFoundException]
    at com.amazon.sqs.javamessaging.SQSConnection.createSession(SQSConnection.java:173)
    at org.springframework.jms.support.JmsAccessor.createSession(JmsAccessor.java:196)
    at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:457)
    ... 110 more

Below is our current deployment environment:

Wildfly 8.2
Camel 2.14
Camel JMS (which in turn uses Spring JMS)
JMS Providers HornetQ and ActiveMQ
Currently we rely on our JMS providers to support

JMSPriority
MessageSelectors
transacted

The code examples appear to reference the Spring configuration file, but the actual contents do not seem to appear anywhere.

It might also be nice if the code examples included in the documentation were checked into the repo and were compilable.

AWS SDK 2.x has been out for almost a year. Our organization is interested in upgrading to it, but unfortunately, we rely on this library, which at this time only supports AWS SDK 1.x.

I would like to request an upgrade to enable compatibility with AWS SDK 2.x. I imagine this will probably be a separate repository, like the other 2.x libraries.

if the concrete implementation doesn't explicitly throw a JMSException, it does not need to be declared as part of the method signature. for example, SQSMessageConsumer.setMessageListener does not need to declare this exception.

i have a desire to use this implementation and i'm ok w/ using the SQS* classes directly (i'm wrapping them in other code). not having to deal w/ these exceptions when they aren't actually thrown would be helpful and help simplify the underlying code.

there also seems to be inconsistency in how RuntimeException subclasses are handled. in some cases, they are re-wrapped and re-thrown as JMSExceptions (bleh). in other cases, they are not.

We are seeing a issue where our jms listener threads end up blocking each other when the SQSSession#recover throws a Exception. The exception occurs in the recover process where it attempts to change the visibility timeout for a message. Since we do not have the permission SQS:ChangeMessageVisibility enabled on our queue, the request to change the message's visibility time out throws a AmazonServiceException - Access Denied. This results in a CPU spike that does not go away and continually grows if the same issue is encountered. From reading the SQS docs, my understanding is that the reason the visibility time out message is invoke during the recover process is to allow the message to be place immediately back into the queue. In instead of having this exception thrown when the queue does not have the SQS:ChangeMessageVisibility permission, is it possible to catch and log the response instead of having the error bubble up? Or, somewhere in the documentation, there should be a note in regards to having this permission enabled for SQSSession#recover to work properly.

tainer-1] ERROR - AmazonServiceException: changeMessageVisibilityBatch. RequestId: 20151ac2-2260-5a13-a37e-48a62a47d705
HTTPStatusCode: 403 AmazonErrorCode: AccessDenied
com.amazonaws.services.sqs.model.AmazonSQSException: Access to the resource https://sqs.us-east-1.amazonaws.com/070420408495/PX-NON-CLIENT-EXTRACT-POC1 is denied. (Service: AmazonSQS; Status Code: 403; Error Code: AccessDenied; Request ID: 20151ac2-2260-5a13-a37e-48a62a47d705)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
	at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:1792)
	at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:1768)
	at com.amazonaws.services.sqs.AmazonSQSClient.executeChangeMessageVisibilityBatch(AmazonSQSClient.java:600)
	at com.amazonaws.services.sqs.AmazonSQSClient.changeMessageVisibilityBatch(AmazonSQSClient.java:576)
	at com.amazon.sqs.javamessaging.AmazonSQSMessagingClientWrapper.changeMessageVisibilityBatch(AmazonSQSMessagingClientWrapper.java:377)
	at com.amazon.sqs.javamessaging.acknowledge.NegativeAcknowledger.action(NegativeAcknowledger.java:90)
	at com.amazon.sqs.javamessaging.acknowledge.BulkSQSOperation.bulkAction(BulkSQSOperation.java:61)
	at com.amazon.sqs.javamessaging.SQSSession.recover(SQSSession.java:485)
	at org.springframework.jms.listener.AbstractMessageListenerContainer.rollbackOnExceptionIfNecessary(AbstractMessageListenerContainer.java:804)
	at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:658)
	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:319)
	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1166)
	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1158)
	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1055)
	at java.lang.Thread.run(Thread.java:745)
10/20/2017 09:19:20:313 ageListenerContainer [tainer-1] ERROR - Application exception overridden by rollback exception

I'd like to delay the delivery of certain messages, like the DelaySeconds message attribute (or message timers) but I wasn't sure how to do it with JMS. A simple message attribute with that as the name?

After a couple of days and thousands of successfully processed messages our JMS message listener encountered a server-side AWS error. While attempting to recover it blocked all of the threads which required the application to be restarted.

2018-07-26 10:06:49,172 WARN  -> (handler-6) [DefaultMessageListenerContainer] Setup of JMS message listener invoker failed for destination 'MY_EXAMPLE_QUEUE.fifo' - trying to recover. Cause: AmazonServiceException: deleteMessage. RequestId: e4796bb3-1242-5fc0-9277-3860f7f7dbf8
HTTPStatusCode: 500 AmazonErrorCode: InternalError
2018-07-26 10:06:49,172 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:06:49,172 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:06:55,309 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:06:55,310 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:00,833 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:00,833 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:18,390 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:18,390 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:18,411 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:18,411 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:22,651 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:22,651 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:28,260 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:28,260 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:44,331 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:44,331 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:50,191 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:50,192 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:50,211 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:50,211 INFO  -> (handler-6) [SQSMessageConsumer] Shutting down ConsumerPrefetch executor
2018-07-26 10:07:50,234 INFO  -> (handler-6) [SQSSession] Shutting down SessionCallBackScheduler executor
2018-07-26 10:07:50,240 INFO  -> (handler-6) [DefaultMessageListenerContainer] Successfully refreshed JMS Connection

https://stackoverflow.com/questions/53727538/aws-sdk-exception-port-already-in-use

The issue has been completely described on the link.

I think this case needs to be handled. All i see print stack traces in the console and ultimately halting my paralled execution. It goes for a while and then the execution resumes.

We can use multi-consumers in a Session, but only one SQSSessionCallbackScheduler.
Why not callback and ack in the prefetch thread?

This library works fine with aws sdk 1.9.6 but stopped working after upgrade 1.10.x or later.

Error:
java.lang.NoSuchMethodError: com.amazonaws.AmazonWebServiceRequest.copyPrivateRequestParameters()Ljava/util/Map;
at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2278)
at com.amazonaws.services.sqs.AmazonSQSClient.getQueueUrl(AmazonSQSClient.java:516)
at com.amazon.sqs.javamessaging.AmazonSQSMessagingClientWrapper.getQueueUrl(AmazonSQSMessagingClientWrapper.java:260)
at com.amazon.sqs.javamessaging.AmazonSQSMessagingClientWrapper.getQueueUrl(AmazonSQSMessagingClientWrapper.java:230)
at com.amazon.sqs.javamessaging.SQSSession.createQueue(SQSSession.java:576)

On the aws documentation :

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-send-message-with-attributes.html

it mentioned about adding message attribute of type Number

final Map<String, MessageAttributeValue> messageAttributes = new HashMap<>();
messageAttributes.put("AccurateWeight", new MessageAttributeValue()
        .withDataType("Number")
        .withStringValue("230.000000000000000001"));

But this client library

		<dependency>
			<groupId>com.amazonaws</groupId>
			<artifactId>amazon-sqs-java-messaging-lib</artifactId>
			<version>1.0.1 - 1.0.4</version>
		</dependency>

It does not accept Number.

https://github.com/awslabs/amazon-sqs-java-messaging-lib/blob/master/src/main/java/com/amazon/sqs/javamessaging/message/SQSMessage.java#L1193

This is causing messages to be lost, also the exception is swallowed which is difficult to detect.

Is this a bug on the library? Or a documentation bug?

To Reproduce:

WORKING :
aws sqs send-message --queue-url <url> --message-body "a" --message-attributes file://msg_attr.json
{
    "City": {
      "DataType": "String",
      "StringValue": "Any City"
    },
    "Population": {
      "DataType": "Number.double",
      "StringValue": "1250800.0"
    }
}

NOT WORKING 
aws sqs send-message --queue-url <url> --message-body "a" --message-attributes file://msg_attr.json
{
    "City": {
      "DataType": "String",
      "StringValue": "Any City"
    },
    "Population": {
      "DataType": "Number",
      "StringValue": "1250800"
    }
}
    @JmsListener(destination = "sampleQ", containerFactory = "sampleJmsListenerContainerFactory")
    public void stageReceiveMessage(String message)  {
        try {
            System.out.println(message);
            System.out.println(message);
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }

As per the SQS FAQs http://aws.amazon.com/sqs/faqs/

Q: How many times will I receive each message?

Amazon SQS is engineered to provide “at least once” delivery of all messages in its queues. 
Although most of the time each message will be delivered to your application exactly once, 
you should design your system so that processing a message more than once does 
not create any errors or inconsistencies. 

Does the amazon-sqs-java-messaging-lib do anything to address this or is it still the responsibility of the application to deal with this?

In the normal SQS SDK, you can set the maximum number of messages to retrieve. However I don't see an equivalent way to do that with this library. I know you can set it to prefetch messages, but that seems to only affect the start of the session, then after that it looks like it receives 1 or 2 at a time. Is there a way to do this that I'm not seeing? It would be nice to do this since I know I'll be processing many thousands of messages so I may as well request a bunch at once to cut down on API requests.

My usecase involves setting up many worker threads, each with their own JMS queue and consumer reading from a single SQS queue, so I can process many messages in parallel. I could be doing something wrong so if there's a way to do this I'm all ears.

To enable publishing to SQS FIFI queues thise properties must be supported.

Class com.amazonaws.services.sqs.model.MessageSystemAttributeName has them defined but they are ignored by SQSMessageProducer and other classes.

If you add this code:
final String groupId = message.getStringProperty(MessageSystemAttributeName.MessageGroupId.toString());
final String deduplicationId = message.getStringProperty(MessageSystemAttributeName.MessageDeduplicationId.toString());
if (groupId != null) {sendMessageRequest.setMessageGroupId(groupId);}
if (deduplicationId != null) {sendMessageRequest.setMessageDeduplicationId(deduplicationId);}

TO THE:

SQSMessageProducer.sendInternal(Queue queue, Message message)
right before
String messageId = amazonSQSClient.sendMessage(sendMessageRequest).getMessageId();

And will send your JMS message with the following spring properties:
jmsMessage.setStringProperty(MessageSystemAttributeName.MessageDeduplicationId.toString(), myDeduplicationId);
jmsMessage.setStringProperty(MessageGroupId.MessageDeduplicationId.toString(), myMessageGroupId);

It will work.

Hi, Thank you for providing such an awesome JMS interface for SQS. Is there any plan for supporting Message Selectors while creating consumers from Session. If so then will greatly appreciate any tentative time lines for the availability for the same.

SQS message attribute custom type is optional free text.

com.amazon.sqs.javamessaging.message.SQSMessage.JMSMessagePropertyValue.getObjectValue only allows

• String without custom type: STRING.equals(type)
• Number with a hard-coded list of customer types: int, Boolean, byte, double, float, long, short.

So, the following message attribute types:

• String.foo
• Number
• Number.integer or Number.boolean
• Binary
• Binary.foo
  will cause an JMSException(type + " is not a supported JMS property type")

com.amazon.sqs.javamessaging.SQSMessageConsumerPrefetch.processReceivedMessages catches JMSException and does not acknowledge a message with one of the types above without other exception being raised or anything logged.

How I think types should be treated:

1. String / binary custom types might be disregarded.
2. Number missing / unknown type should default to something (long?)
3. At least an error should be logged if the message cannot be acknowledged due to JMSException.

This issue still seem to be occurring, despite two similar issues were closed.

Below is my POM dependencies with JDK 1.7. Help please!!

<build>
    <plugins>
        <plugin>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>1.7</source>
                <target>1.7</target>
            </configuration>
        </plugin>
    </plugins>
</build>

<dependencies>

    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk-simpleworkflow</artifactId>
        <version>1.11.78</version>
    </dependency>

    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk-stepfunctions</artifactId>
        <version>1.11.86</version>
    </dependency>
    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>amazon-sqs-java-messaging-lib</artifactId>
        <version>1.0.4</version>
        <type>jar</type>
    </dependency>
    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>amazon-sqs-java-extended-client-lib</artifactId>
        <version>1.0.1</version>
        <type>jar</type>
    </dependency>

    <!-- https://mvnrepository.com/artifact/commons-logging/commons-logging -->
    <dependency>
        <groupId>commons-logging</groupId>
        <artifactId>commons-logging</artifactId>
        <version>1.1.3</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->
    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpclient</artifactId>
        <!--<version>4.3.2</version>-->
        <version>4.5.2</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpcore -->
    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpcore</artifactId>
        <version>4.4.4</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-annotations -->
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-annotations</artifactId>
        <version>2.6.0</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-core -->
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-core</artifactId>
        <version>2.6.7</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind -->
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.6.7.1</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/joda-time/joda-time -->
    <dependency>
        <groupId>joda-time</groupId>
        <artifactId>joda-time</artifactId>
        <version>2.8.1</version>
    </dependency>

</dependencies>

Hi all,

I have the requirement to set up the SNS(Region: us-east-2) on my project. For this purpose first, I have upgraded the SDK version to 1.11.63. But existing Sqs start giving the below errors.

Problem:
1. Region US-EAST-2 support is not in SDK 1.9.6, so need to upgrade.
2. If upgraded to SDK 1.11.X, then getting below Errors:

Error:1
java.lang.NoSuchMethodError: com.amazonaws.services.sqs.AmazonSQSClient.beforeClientExecution(Lcom/amazonaws/AmazonWebServiceRequest;)Lcom/amazonaws/AmazonWebServiceRequest

Error:2
import com.amazonaws.util.json.JSONObject: missing in this SDK version and lots of code written on this in my app on JSONObject class.

Please, can anyone suggest me the fix for the issue without requiring many changes in my code?

It would be nice if we could use the extended SQS client that supports storing large messages in S3 with this JMS wrapper. I've submitted a pull request for this. Could you please take a look and incorporate this in the library?

Hello team,
How can set the Q url if the q is exists on different AWS account
for both receiving and sending.

SQSConnectionFactory connectionFactory = new SQSConnectionFactory(
new ProviderConfiguration(),
AmazonSQSClientBuilder.standard()
.withCredentials( new DefaultAWSCredentialsProviderChain())
);

@Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
    DefaultJmsListenerContainerFactory factory =
            new DefaultJmsListenerContainerFactory();
    factory.setConnectionFactory(this.connectionFactory);
    factory.setDestinationResolver(new DynamicDestinationResolver());
    factory.setConcurrency("3-10");
    return factory;
}

and my listener is looks like

@JmsListener(destination = "${jms.outboundQueueName}")
public void onReceive(String message) {
try {
log.info("onReceive request and send to : "+message);
} catch (Exception e) {
log.warn("sendErrorMessage" + e.getMessage());
}
}

Eg:
How can access this q using url (eg: https://sqs.ap-southeast-2.amazonaws.com/123456/qname)

Thanks
Sidath

The com.amazon.sqs.javamessaging.acknowledge.AcknowledgeMode has a private originalAcknowledgeMode field. Enum members are global singletons, so having a mutable private field in this case makes no sense.

Demonstration of the issue:

final AcknowledgeMode mode1 = AcknowledgeMode.ACK_RANGE.withOriginalAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
final AcknowledgeMode mode2 = AcknowledgeMode.ACK_RANGE.withOriginalAcknowledgeMode(Session.DUPS_OK_ACKNOWLEDGE);
assert Session.CLIENT_ACKNOWLEDGE == mode1.getOriginalAcknowledgeMode();  // Fails!

If UNORDERED_ACKNOWLEDGE is used for the session acknowledge mode, and I wish to retry messages 'x' times with multiple subscribers (using Spring JMS) what happens to the unacknowledged messages? As far as I can tell the unAckMessages in the UnorderedAcknowledger just keeps on growing.

If I use CLIENT_ACKNOWLEDGE and don't ack the message as I wish to retry is there anyway to remove it from the list of unacknowledged messages, so it is not implicitly acknowledged (deleted) when a good message is processed and ack'd?

Hi awslabs !

I was wondering if it would make sense to set JMSTimestamp to SentTimestamp when converting from SQS message to JMS message

Thanks for your feedback if any

The SQS JMS implementation always prefetches at least one message before it invokes message listener. This prevents EC2 from autoscaling based on the number of messages in the SQS queue especially when execution of the message takes as long as 2 hours. Is there anyway we can optionally prefetch or do not prefetch at all? Unless I am missing something, there is no way I could extend the behavior of SQSMessageConsumerPrefetch class to make it usable for us. It will be nice if the prefetch is made optional.

Hello,

This issues is referenced at: #18 but the suggested solution there, don't solve the problem I am facing.

It seems there is a problem with acknowledge and SQSSession.UNORDERED_ACKNOWLEDGE.

I was running the SyncMessageReceiverUnorderedAcknowledge.java program, exactly as written on: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/jmsclient.html#jmsclient-ackmode

When the below code is reached:

// Acknowledge the message if asked
if (acknowledge) message.acknowledge();

I get the following exception:

Exception in thread "main" java.lang.NoSuchMethodError: com.amazonaws.services.sqs.AmazonSQS.deleteMessage(Lcom/amazonaws/services/sqs/model/DeleteMessageRequest;)V
at com.amazon.sqs.javamessaging.AmazonSQSMessagingClientWrapper.deleteMessage(AmazonSQSMessagingClientWrapper.java:127)
at com.amazon.sqs.javamessaging.acknowledge.UnorderedAcknowledger.acknowledge(UnorderedAcknowledger.java:42)
at com.amazon.sqs.javamessaging.message.SQSMessage.acknowledge(SQSMessage.java:883)
at sample.sqs.SyncMessageReceiverUnorderedAcknowledge.receiveMessage(SyncMessageReceiverUnorderedAcknowledge.java:116)
at sample.sqs.SyncMessageReceiverUnorderedAcknowledge.main(SyncMessageReceiverUnorderedAcknowledge.java:67)

I am running with the following gradle dependencies:
compile("com.amazonaws:aws-java-sdk-sqs:1.11.13")
compile("com.amazonaws:amazon-sqs-java-messaging-lib:1.0.0")

I debugged the code, and all the aws amazon java classes looks perfect.

In addition, I created a new program that will run isolated deleteMessage.
With deleteMessage(DeleteMessageRequest deleteMessageRequest) - I get the same exception.
With DeleteMessageBatchResult deleteMessageBatch(DeleteMessageBatchRequest deleteMessageBatchRequest) - It works!

I changed gradle dependencies:
compile("com.amazonaws:aws-java-sdk-sqs:1.9.6")

And now it works perfect.

Opening pom.xml of this project, I can see:
<properties> <aws-java-sdk.version>1.9.6</aws-java-sdk.version> </properties>

When do you plan to change it to the latest one (e.g. 1.11.13)? or one of the latest?

I am using aws jms library with spring, versions are shown below.

        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>amazon-sqs-java-messaging-lib</artifactId>
            <version>1.0.0</version>
            <type>jar</type>
        </dependency>


        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-jms</artifactId>
            <version>4.1.4-RELEASE</version>
        </dependency>

In my spring application context, the definition of my listener is as shown below (simplified).

    <bean id="credentialsProviderBean" class="com.amazonaws.auth.DefaultAWSCredentialsProviderChain"/>

    <bean id="clientConfiguration" class="com.amazonaws.ClientConfiguration">
        <!-- Connect via a Proxy -->
        <property name="proxyHost" value="${png.http.proxy.host}"/>
        <property name="proxyPort" value="${png.http.proxy.port}" />
        <property name="maxConnections" value="${png.sqs.connections.consumer.pool.size}"/>
    </bean>

    <bean id="connectionFactoryBuilder" class="com.amazon.sqs.javamessaging.SQSConnectionFactory$Builder">
        <property name="regionName" value="${png.aws.region}"/>
        <property name="awsCredentialsProvider" ref="credentialsProviderBean"/>
        <property name="clientConfiguration" ref="clientConfiguration"/>
    </bean>

    <bean id="ConnectionFactory" class="com.amazon.sqs.javamessaging.SQSConnectionFactory"
          factory-bean="connectionFactoryBuilder"
          factory-method="build" />

<bean id="processorListener" class="com.intuit.mobile.png.notification.processor.ProcessorListener" init-method="init"/> 

    <jms:listener-container container-type="default" connection-factory="ConnectionFactory" acknowledge="auto"
                            concurrency="${png.sqs.connections.consumer.pool.size}">
        <jms:listener destination="${png.processor.queue.normal}" ref="processorListener" method="onMessage" />
    </jms:listener-container>

We are noticing multiple delivery of messages to consumers running on the same instance, separated by milliseconds apart. I understand that the contract is at-least once and there are cases where duplicate delivery can occur...but can that happen in such short intervals.

Basically what we are seeing is message duplication happening at really short intervals....where one message gets multiplied into 2 or 3 sometimes.

Some questions that comes to mind is..

• do you support only once delivery guarantees of JMS?
• can the issue described earlier (message duplication) happen so frequently with AWS JMS library?
• do you recommend the use of spring jms libraries with aws jms? Is this a tested pattern?
• Do you account for visibility timeouts and making sure that the message gets returned to the pool if it waits too long in the cache etc.? What is the cutoff time to call onMessage...ie. if the message lives in the cache for 25 seconds and my visibility timeout is 30 seconds, do you still invoke onMessage() when only 5 seconds is left for the visisbility timeout expiration? Can you share some details around this implementation so that we can handle the border conditions?

We can set DelaySeconds on individual messages sent to standard sqs queues
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-send-message-with-timer.html

How to set this parameter using this library

I tried to apply the IAM role to the ECS task role and the spring application was unable to pick up the IAM permissions. I had to apply the IAM role to the EC2 instances in the ECS cluster in order for the app to pick up the IAM permissions.

When a running a spring application in a docker container on ECS, I would like to place the IAM role on the ECS task definition instead of the EC2 instance(s).

Is there any way to make this implementation less aggressive (i.e. for Long Polling)? We are using the async MessageListener interface and it becomes more aggressive over time. After about 18-hours we're seeing 1+million empty messages API calls and growing.

We had a bunch of messages in an SQS queue where the messageAttribute data types were not recognized by the code with SQSMessage.
private static Object getObjectValue(String value, String type) throws JMSException
Now, I will agree that a bad value type is partially to blame. I would expect something better from the library code, all that was happening is that the message would get rejected, and returned to the queue and then re-read, infinite loop.

Perhaps default to String? or skip the bad value type? Something other than throwing a JMSException.

I have multiple threads running concurrently. And since one message's execution might be dependent on another, after checking state of previous message's execution state, I want the message to remain invisible till the previous message execution is completed. Would it be possible to set visibility timeout for the message when using amazon sqs - jms integration?

I noticed that this library depends on aws-java-sdk-sqs version 1.9.6 which conflicts with aws-java core version 1.10.73 we would like to use. Can this dependency be upgraded to 1.10.73 since 1.9.6 seems quite old now? BTW, these 2 versions are incompatible as I found out the hard way

java.lang.NoSuchMethodError: com.amazonaws.AmazonWebServiceRequest.copyPrivateRequestParameters()Ljava/util/Map;

TextMessage's that have JMSCorrelationID set when received have null correlationId.

CorrelationID is useful for request/response patterns.

SQS Messages may have attributes on them with DataType values of 'String', 'Number', and 'Binary', as documented on https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-message-attributes.html

Currently, the library throws a JMSException if it receives a Number or Binary and only handles String and a handful of 'Custom' types.

A library that purports to join JMS and SQS should be able to, at the bare minimum of functionality, consume perfectly normal SQS messages that meet the documentation for them.

Currently using 1.0.4 and I'm interested in getting the SentTimestamp property from SQSTextMessage, but unfortunately only ApproximateReceiveCount is mapped across as "JMSXDeliveryCount" at SQSMessage.java:125

Attributes that I can see coming from com.amazonaws.services.sqs.model.Message

Is this something that can be added?

When I change the target in the pom.xml to 1.6 I get the following error.

Caused by: org.apache.maven.plugin.compiler.CompilationFailureException: Compilation failure
An unknown compilation problem occurred

at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:909)
at org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:129)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:132)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 19 more

We are using com.amazonaws:aws-java-sdk:1.9.6 & com.amazonaws:amazon-sqs-java-messaging-lib:1.0.3 libraries for consuming the messages from SQS. We use Spring JMSTemplate for consume the messages with CLIENT_ACKNOWLEDGE mode.

We noticed an issue where multiple consumers picked the same message & also the message moved to the dead queue on sqs. When we inquired with AWS SQS support team, Below is the response (in quotes) we received. Can you let us know if any configuration is missing & what kind of logging needs to be enabled to debug this issue ?

"After looking at logs, below is why the message was returned 5 times within 1 second.

For each ReceiveMessage call, I see a ChangeMessageVisibilityBatch call with visibility timeout set to 0. This call was putting the message back in available state right away, allowing other consumers to pick the message. Below is the caller of ChangeMessageVisibilityBatch call.

UserAgent: aws-sdk-java/1.11.18 Linux/3.10.0-693.11.6.el7.x86_64 Java_HotSpot(TM)_64-Bit_Server_VM/25.131-b11/1.8.0_131 /SQS Java Messaging Client v1.0

Look like, it is the same code making both calls (ReceiveMessage and ChangeMessageVisibilityBatch). In order to understand why these calls were made in the first place, I request you to enable debug logging in the code. For any other issues with the library, I recommend opening an issue in the github repo. Our team that manages the library will address them.
"