Comments (12)
The issue was resolved when the dlq was removed from elasticmq.conf. I created a new file like this https://github.com/roribio/alpine-sqs/blob/master/opt/elasticmq.conf, deleted the dlq queues, and mounted it to /opt/custom. Solved issue where no message was coming back.
from alpine-sqs.
I have two ideas:
Since sqs-insight is retrieving messages continuously, your Javascript client is coming up against the defaultVisibilityTimeout of 10 seconds, which is supposed to prevent two clients accessing the same message concurrently. To test if that's the case, I would exec into the container and stop sqs-insight:
$ supervisorctl stop insight
FWIW: I had the exact same issue and this is what was causing the problem
from alpine-sqs.
Hi, have you figured this out yet?
I haven't worked with this for a while but, when I was actively developing with it, I could push and pop messages without problems.
That said, I'm happy to try to help you debug this. Could you provide me with the configuration you are using to set this up?
from alpine-sqs.
hi, I didnt use it anymore, I switched to goaws, I was using the image with the default config, so pushing and reading from the default queue which is already there, but for some reason the receiveMessage was always returning empty.
from alpine-sqs.
thanks a lot guys
from alpine-sqs.
@roribio I have the same issues. My Example (with NodeJS)
Configuration
docker-compose.yml
version: '3'
services:
sqs:
ports:
- "9324:9324"
- "9325:9325"
image: roribio16/alpine-sqs:latest
volumes:
- "$PWD/config:/opt/custom"
config/sqs-insight.conf
{
"port": 9325,
"rememberMessages": 100,
"endpoints": [
{
"key": "fake-key",
"secretKey": "fake-secret",
"region": "us-east-1",
"url": "http://localhost:9324/queue/default"
}
],
"dynamicEndpoints": [
{
"key": "fake-key",
"secretKey": "fake-secret",
"region": "us-east-1",
"url": "http://localhost:9324",
"visibility": 1
}
]
}
config/sqs-init.sh
#!/bin/sh
mkdir -p /opt/config
# First, copy default configs:
cp /opt/*.conf /opt/config/
# Secondly, copy custom configs:
cp /opt/custom/*.conf /opt/config/
# Now copy sqs-insight config to correct location:
cp /opt/config/sqs-insight.conf /opt/sqs-insight/config/config_local.json
sleep 1
exit 0
config/elasticmq.conf
include classpath("application.conf")
node-address {
protocol = http
host = "*"
port = 9324
context-path = ""
}
rest-sqs {
enabled = true
bind-port = 9324
bind-hostname = "0.0.0.0"
// Possible values: relaxed, strict
sqs-limits = strict
}
queues {
default {
defaultVisibilityTimeout = 10 seconds
delay = 0 seconds
receiveMessageWait = 0 seconds
},
dev-action {
defaultVisibilityTimeout = 10 seconds
delay = 0 seconds
receiveMessageWait = 0 seconds
}
}
Code
Examples based on the docs.
send.js
var queueURL = "http://localhost:9324/queue/dev-action";
var AWS = require('aws-sdk');
AWS.config.update({ region: 'us-east-1' });
var sqs = new AWS.SQS({ apiVersion: '2012-11-05' });
var params = {
DelaySeconds: 10,
MessageAttributes: {
"TitleTitle": {
DataType: "String",
StringValue: "The Whistler"
},
"Author": {
DataType: "String",
StringValue: "John Grisham"
},
"WeeksOn": {
DataType: "Number",
StringValue: "6"
}
},
MessageBody: "Information about current NY Times fiction bestseller for week of 12/11/2016.",
QueueUrl: queueURL
};
sqs.sendMessage(params, function (err, data) {
if (err) {
console.log("Error", err);
} else {
console.log("Success", data.MessageId);
}
});
When I run this script, I get a message in my dev-action
queue:
So that is good !
But now I want to consume it....
receive.js
var queueURL = "http://localhost:9324/queue/dev-action";
// Load the AWS SDK for Node.js
var AWS = require('aws-sdk');
// Set the region
AWS.config.update({region: 'REGION'});
// Create an SQS service object
var sqs = new AWS.SQS({apiVersion: '2012-11-05'});
var params = {
AttributeNames: [
"SentTimestamp"
],
MaxNumberOfMessages: 1,
MessageAttributeNames: [
"Title"
],
QueueUrl: queueURL,
VisibilityTimeout: 20,
WaitTimeSeconds: 0
};
sqs.receiveMessage(params, function(err, data) {
if (err) {
console.log("Receive Error", err);
} else if (data.Messages) {
var deleteParams = {
QueueUrl: queueURL,
ReceiptHandle: data.Messages[0].ReceiptHandle
};
sqs.deleteMessage(deleteParams, function(err, data) {
if (err) {
console.log("Delete Error", err);
} else {
console.log("Message Deleted", data);
}
});
}
});
What I get as (raw) response is (Above script doesn't print anything)
<ReceiveMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
<ReceiveMessageResult>
</ReceiveMessageResult>
<ResponseMetadata>
<RequestId>00000000-0000-0000-0000-000000000000</RequestId>
</ResponseMetadata>
</ReceiveMessageResponse>
Any ideas/suggestions ?
from alpine-sqs.
Hi,
Thank you for your detailed report 💯
I'm re-opening the issue and I'll investigate further.
Thanks!
from alpine-sqs.
I'm testing with these two commands and I can't reproduce your error:
$ aws --endpoint-url http://localhost:9324 sqs send-message --queue-url http://localhost:9324/queue/default --message-body "Hello, World."
$ aws --endpoint-url http://localhost:9324 sqs receive-message --queue-url http://localhost:9324/queue/default
I have two ideas:
- Since sqs-insight is retrieving messages continuously, your Javascript client is coming up against the
defaultVisibilityTimeout
of 10 seconds, which is supposed to prevent two clients accessing the same message concurrently. To test if that's the case, I wouldexec
into the container and stopsqs-insight
:
$ supervisorctl stop insight
However, I could not reproduce that scenario with the commands I tested with.
- There's a mismatch between the queue's
defaultVisibilityTimeout = 10 seconds
and your client'sVisibilityTimeout: 20
. That could be the source of your grief.
Let me know what you find.
Cheers!
P.S.: I accidentally closed the issue 🤷♂️
from alpine-sqs.
Closing since there's been no activity in two weeks.
from alpine-sqs.
I am having the same issue, and stopping insight
seems to fix it. But I waisted hours before I found this thread.
So wouldn't changing the default value for defaultVisibilityTimeout
to 0 seconds
in opt/elasticmq.conf
fix this issue entirely. And then if anyone wants they can set the timeout in their requests or override it in a mounted volume. It seems strange that the default behavior for the service would be "buggy" in this way.
from alpine-sqs.
insight was also the issue for me, i wen around supervisord and used this image in docker-compose with the following parameter
entrypoint: sh -c "/opt/sqs-init.sh && /opt/jdk/bin/java -Dconfig.file=/opt/config/elasticmq.conf -jar /opt/elasticmq-server.jar"
from alpine-sqs.
Stopping the insights fixed the issue for me too. But the sole purpose of the local stack for me is to have queue insights.
Is there a work-around for this to have both insights and have the both aws cli
and aws sdk
retrieving the messages.
Just as a note, I'm using this project to debug Laravel's SQS Queues locally.
from alpine-sqs.
Related Issues (20)
- Docs for using with AWSSDK HOT 4
- Provide an image elasticmq-native based
- Delete message, creates another one HOT 3
- CPU over 100% when queues are not empty HOT 10
- about init queue
- support FIFO q? HOT 7
- Dead Letter Queue fails
- bad ui experience when there are too many queues... HOT 1
- How to support replica and cluster
- Registering new queues with the UI
- FIFO queue deduplication ID doesn't work
- Unable to locate credentials. You can configure credentials by running "aws configure" HOT 2
- roribio16/alpine-sqs:latest not working wie apple silicon HOT 6
- Queue fails to deliver messages HOT 1
- Insight Startup Issue with extra queues HOT 1
- Queue is not working when launching
- Value default for parameter MessageGroupId is invalid HOT 2
- Change credentials
- Support for JSON protocol HOT 5
- .
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from alpine-sqs.