Git Product home page Git Product logo

Comments (12)

sc6 avatar sc6 commented on May 27, 2024 4

The issue was resolved when the dlq was removed from elasticmq.conf. I created a new file like this https://github.com/roribio/alpine-sqs/blob/master/opt/elasticmq.conf, deleted the dlq queues, and mounted it to /opt/custom. Solved issue where no message was coming back.

from alpine-sqs.

kwarcu avatar kwarcu commented on May 27, 2024 2

I have two ideas:

Since sqs-insight is retrieving messages continuously, your Javascript client is coming up against the defaultVisibilityTimeout of 10 seconds, which is supposed to prevent two clients accessing the same message concurrently. To test if that's the case, I would exec into the container and stop sqs-insight:
$ supervisorctl stop insight

FWIW: I had the exact same issue and this is what was causing the problem

from alpine-sqs.

roribio avatar roribio commented on May 27, 2024

Hi, have you figured this out yet?

I haven't worked with this for a while but, when I was actively developing with it, I could push and pop messages without problems.

That said, I'm happy to try to help you debug this. Could you provide me with the configuration you are using to set this up?

from alpine-sqs.

vikkio88 avatar vikkio88 commented on May 27, 2024

hi, I didnt use it anymore, I switched to goaws, I was using the image with the default config, so pushing and reading from the default queue which is already there, but for some reason the receiveMessage was always returning empty.

from alpine-sqs.

vikkio88 avatar vikkio88 commented on May 27, 2024

thanks a lot guys

from alpine-sqs.

SegersIan avatar SegersIan commented on May 27, 2024

@roribio I have the same issues. My Example (with NodeJS)

Configuration

docker-compose.yml

version: '3'
services:
  sqs:
    ports:
    - "9324:9324"
    - "9325:9325"
    image: roribio16/alpine-sqs:latest
    volumes:
    - "$PWD/config:/opt/custom"

config/sqs-insight.conf

{
    "port": 9325,
    "rememberMessages": 100,

    "endpoints": [
        {
            "key": "fake-key",
            "secretKey": "fake-secret",
            "region": "us-east-1",
                "url": "http://localhost:9324/queue/default"
        }
    ],

    "dynamicEndpoints": [
        {
           "key": "fake-key",
           "secretKey": "fake-secret",
           "region": "us-east-1",
           "url": "http://localhost:9324",
           "visibility": 1
        }
    ]
}

config/sqs-init.sh

#!/bin/sh

mkdir -p /opt/config

# First, copy default configs:
cp /opt/*.conf /opt/config/

# Secondly, copy custom configs:
cp /opt/custom/*.conf /opt/config/

# Now copy sqs-insight config to correct location:
cp /opt/config/sqs-insight.conf /opt/sqs-insight/config/config_local.json

sleep 1
exit 0

config/elasticmq.conf

include classpath("application.conf")

node-address {
    protocol = http
    host = "*"
    port = 9324
    context-path = ""
}

rest-sqs {
    enabled = true
    bind-port = 9324
    bind-hostname = "0.0.0.0"
    // Possible values: relaxed, strict
    sqs-limits = strict
}

queues {
    default {
        defaultVisibilityTimeout = 10 seconds
        delay = 0 seconds
        receiveMessageWait = 0 seconds
    },
    dev-action {
        defaultVisibilityTimeout = 10 seconds
        delay = 0 seconds
        receiveMessageWait = 0 seconds
    }
}

Code

Examples based on the docs.
send.js

var queueURL = "http://localhost:9324/queue/dev-action";
var AWS = require('aws-sdk');
AWS.config.update({ region: 'us-east-1' });
var sqs = new AWS.SQS({ apiVersion: '2012-11-05' });


var params = {
    DelaySeconds: 10,
    MessageAttributes: {
        "TitleTitle": {
            DataType: "String",
            StringValue: "The Whistler"
        },
        "Author": {
            DataType: "String",
            StringValue: "John Grisham"
        },
        "WeeksOn": {
            DataType: "Number",
            StringValue: "6"
        }
    },
    MessageBody: "Information about current NY Times fiction bestseller for week of 12/11/2016.",
    QueueUrl: queueURL
};

sqs.sendMessage(params, function (err, data) {
    if (err) {
        console.log("Error", err);
    } else {
        console.log("Success", data.MessageId);
    }
});

When I run this script, I get a message in my dev-action queue:
screenshot 2018-11-07 at 23 01 42
So that is good !

But now I want to consume it....
receive.js

var queueURL = "http://localhost:9324/queue/dev-action";

// Load the AWS SDK for Node.js
var AWS = require('aws-sdk');
// Set the region
AWS.config.update({region: 'REGION'});

// Create an SQS service object
var sqs = new AWS.SQS({apiVersion: '2012-11-05'});


var params = {
    AttributeNames: [
        "SentTimestamp"
    ],
    MaxNumberOfMessages: 1,
    MessageAttributeNames: [
        "Title"
    ],
    QueueUrl: queueURL,
    VisibilityTimeout: 20,
    WaitTimeSeconds: 0
};

sqs.receiveMessage(params, function(err, data) {
    if (err) {
        console.log("Receive Error", err);
    } else if (data.Messages) {
        var deleteParams = {
            QueueUrl: queueURL,
            ReceiptHandle: data.Messages[0].ReceiptHandle
        };
        sqs.deleteMessage(deleteParams, function(err, data) {
            if (err) {
                console.log("Delete Error", err);
            } else {
                console.log("Message Deleted", data);
            }
        });
    }
});

What I get as (raw) response is (Above script doesn't print anything)

<ReceiveMessageResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">
              <ReceiveMessageResult>
                
              </ReceiveMessageResult>
              <ResponseMetadata>
                <RequestId>00000000-0000-0000-0000-000000000000</RequestId>
              </ResponseMetadata>
            </ReceiveMessageResponse>

Any ideas/suggestions ?

from alpine-sqs.

roribio avatar roribio commented on May 27, 2024

Hi,

Thank you for your detailed report 💯

I'm re-opening the issue and I'll investigate further.

Thanks!

from alpine-sqs.

roribio avatar roribio commented on May 27, 2024

I'm testing with these two commands and I can't reproduce your error:

$ aws --endpoint-url http://localhost:9324 sqs send-message --queue-url http://localhost:9324/queue/default --message-body "Hello, World."

$ aws --endpoint-url http://localhost:9324 sqs receive-message --queue-url http://localhost:9324/queue/default

I have two ideas:

  1. Since sqs-insight is retrieving messages continuously, your Javascript client is coming up against the defaultVisibilityTimeout of 10 seconds, which is supposed to prevent two clients accessing the same message concurrently. To test if that's the case, I would exec into the container and stop sqs-insight:

$ supervisorctl stop insight

However, I could not reproduce that scenario with the commands I tested with.

  1. There's a mismatch between the queue's defaultVisibilityTimeout = 10 seconds and your client's VisibilityTimeout: 20. That could be the source of your grief.

Let me know what you find.

Cheers!

P.S.: I accidentally closed the issue 🤷‍♂️

from alpine-sqs.

roribio avatar roribio commented on May 27, 2024

Closing since there's been no activity in two weeks.

from alpine-sqs.

maldimirov avatar maldimirov commented on May 27, 2024

I am having the same issue, and stopping insight seems to fix it. But I waisted hours before I found this thread.

So wouldn't changing the default value for defaultVisibilityTimeout to 0 seconds in opt/elasticmq.conf fix this issue entirely. And then if anyone wants they can set the timeout in their requests or override it in a mounted volume. It seems strange that the default behavior for the service would be "buggy" in this way.

from alpine-sqs.

ruslantalpa avatar ruslantalpa commented on May 27, 2024

insight was also the issue for me, i wen around supervisord and used this image in docker-compose with the following parameter

entrypoint: sh -c "/opt/sqs-init.sh && /opt/jdk/bin/java -Dconfig.file=/opt/config/elasticmq.conf -jar /opt/elasticmq-server.jar"

from alpine-sqs.

Olcod avatar Olcod commented on May 27, 2024

@roribio

Stopping the insights fixed the issue for me too. But the sole purpose of the local stack for me is to have queue insights.

Is there a work-around for this to have both insights and have the both aws cli and aws sdk retrieving the messages.

Just as a note, I'm using this project to debug Laravel's SQS Queues locally.

from alpine-sqs.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.