pruvonet / squiss-ts Goto Github PK

Expected Behavior

Setup a documentation site for proper documentation

Current Behavior

Long and unreadable documentation in README.md file

Possible Solution

Setup a slate documentation site

I am unable to delete message, at all. I am calling simply:

poller.on('message', (msg)=>{
  msg.del();
});

And my program crashes, recieving:
(node:15264) UnhandledPromiseRejectionWarning: AWS.SimpleQueueService.BatchEntryIdsNotDistinct: Id 9f9954d8-ac18-4892-b59b-a7ced5ddb382 repeated.

Expected Behavior

If I receive messages with the correct attributes (__ SQS_S3__, SQS_GZIP) and with unwrapSns: true then it should parse my message back to its original form.

Current Behavior

If I receive a message in the form gzip or s3 then the message is not parsed properly.

Possible Solution

Incorrect path in the constructor of the class Message, must be added to if (opts.unwrapSns) - this.attributes = attributeUtils_1.parseMessageAttributes(unwrapped.MessageAttributes);
And in parseAttributeValue replace with this

const type = unparsedAttribute.DataType || unparsedAttribute.Type;
const stringValue = unparsedAttribute.StringValue || unparsedAttribute.Value;

Your Environment

• Version used: 4.1.1
• Node version: 10.19.0

The devDependency @types/node was updated from 12.12.8 to 12.12.9.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@types/node is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

As explained at https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-temporary-queues.html
And implemented here https://github.com/awslabs/amazon-sqs-java-temporary-queues-client
The temp/virtual queues pattern allows for 2 way scalable communication using SQS queues

CVE-2015-9251 - Medium Severity Vulnerability

Vulnerable Library - jquery-1.7.2.min.js

JavaScript library for DOM operations

Library home page: https://cdnjs.cloudflare.com/ajax/libs/jquery/1.7.2/jquery.min.js

Path to dependency file: squiss-ts/node_modules/jmespath/index.html

Path to vulnerable library: squiss-ts/node_modules/jmespath/index.html

Dependency Hierarchy:

• ❌ jquery-1.7.2.min.js (Vulnerable Library)

Found in HEAD commit: 9b4ec2c6366c49ad22ffe58a91f08185fe51b78d

Found in base branch: master

Vulnerability Details

jQuery before 3.0.0 is vulnerable to Cross-site Scripting (XSS) attacks when a cross-domain Ajax request is performed without the dataType option, causing text/javascript responses to be executed.

Publish Date: 2018-01-18

URL: CVE-2015-9251

CVSS 3 Score Details (6.1)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: Required
  • Scope: Changed
• Impact Metrics:
  • Confidentiality Impact: Low
  • Integrity Impact: Low
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://nvd.nist.gov/vuln/detail/CVE-2015-9251

Release Date: 2018-01-18

Fix Resolution: jQuery - v3.0.0

Step up your Open Source Security Game with WhiteSource here

As far as I understand it, activePollIntervalMs should delay the poller by a specified number of ms before picking up another message? If so, I don't seem to be able to get it to work. As soon as one message is processed, the next message is picked up with no delay, even though I have set:

activePollIntervalMs: 5000,

Any ideas?

Expected Behavior

squiss-ts should run without any runtime warnings

Current Behavior

Usage of gzipUtils will cause warning messages to be logged due to new Buffer() being called. https://github.com/PruvoNet/squiss-ts/blob/master/src/gzipUtils.ts

See https://nodejs.org/docs/latest-v10.x/api/buffer.html#buffer_new_buffer_string_encoding

N.B. Also called in tests at https://github.com/PruvoNet/squiss-ts/blob/master/src/test/src/Message.spec.ts#L37 & https://github.com/PruvoNet/squiss-ts/blob/master/src/test/src/Message.spec.ts#L41

Possible Solution

Replace Buffer constructor with Buffer.from()

Steps to Reproduce (for bugs)

Run tests and check logs. See https://travis-ci.com/PruvoNet/squiss-ts/jobs/257131231#L250

Context

No issues with usability, just a warning message in logs

Your Environment

• Version used: latest
• Node version: >10

This package still uses iltorb, which is no longer needed in modern versions of Node:

npm WARN deprecated [email protected]: The zlib module provides APIs for brotli compression/decompression starting with Node.js v10.16.0, please use it over iltorb

CVE-2012-6708 - Medium Severity Vulnerability

Vulnerable Library - jquery-1.7.2.min.js

JavaScript library for DOM operations

Library home page: https://cdnjs.cloudflare.com/ajax/libs/jquery/1.7.2/jquery.min.js

Path to dependency file: squiss-ts/node_modules/jmespath/index.html

Path to vulnerable library: squiss-ts/node_modules/jmespath/index.html

Dependency Hierarchy:

• ❌ jquery-1.7.2.min.js (Vulnerable Library)

Found in HEAD commit: 9b4ec2c6366c49ad22ffe58a91f08185fe51b78d

Found in base branch: master

Vulnerability Details

jQuery before 1.9.0 is vulnerable to Cross-site Scripting (XSS) attacks. The jQuery(strInput) function does not differentiate selectors from HTML in a reliable fashion. In vulnerable versions, jQuery determined whether the input was HTML by looking for the '<' character anywhere in the string, giving attackers more flexibility when attempting to construct a malicious payload. In fixed versions, jQuery only deems the input to be HTML if it explicitly starts with the '<' character, limiting exploitability only to attackers who can control the beginning of a string, which is far less common.

Publish Date: 2018-01-18

URL: CVE-2012-6708

CVSS 3 Score Details (6.1)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: Required
  • Scope: Changed
• Impact Metrics:
  • Confidentiality Impact: Low
  • Integrity Impact: Low
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://nvd.nist.gov/vuln/detail/CVE-2012-6708

Release Date: 2018-01-18

Fix Resolution: jQuery - v1.9.0

Step up your Open Source Security Game with WhiteSource here

The dependency aws-sdk was updated from 2.562.0 to 2.563.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

aws-sdk is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Your Greenkeeper Bot 🌴

Timeout extender takes into account the advancedCallMs only for the first time it schedules an extension action. After that, that param is not being taken under consideration, causing race condition between following renew times and message visibility end time.

Are there any plans to support aws-sdk version 3 of the AWS SDK? One of its perks is the modular architecture, it would reduce significantly the huge size of the dependency.

Expected Behavior

Current Behavior

Possible Solution

Steps to Reproduce (for bugs)

Context

Your Environment

• Version used:
• Node version:

It is a very usual scenario where you have messages in your dead letter queue and you want to resbumit them back to your queue (while maybe modifying the message)

CVE-2020-11022 - Medium Severity Vulnerability

Vulnerable Library - jquery-1.7.2.min.js

JavaScript library for DOM operations

Library home page: https://cdnjs.cloudflare.com/ajax/libs/jquery/1.7.2/jquery.min.js

Path to dependency file: squiss-ts/node_modules/jmespath/index.html

Path to vulnerable library: squiss-ts/node_modules/jmespath/index.html

Dependency Hierarchy:

• ❌ jquery-1.7.2.min.js (Vulnerable Library)

Found in HEAD commit: 9b4ec2c6366c49ad22ffe58a91f08185fe51b78d

Found in base branch: master

Vulnerability Details

In jQuery versions greater than or equal to 1.2 and before 3.5.0, passing HTML from untrusted sources - even after sanitizing it - to one of jQuery's DOM manipulation methods (i.e. .html(), .append(), and others) may execute untrusted code. This problem is patched in jQuery 3.5.0.

Publish Date: 2020-04-29

URL: CVE-2020-11022

CVSS 3 Score Details (6.1)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: Required
  • Scope: Changed
• Impact Metrics:
  • Confidentiality Impact: Low
  • Integrity Impact: Low
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://blog.jquery.com/2020/04/10/jquery-3-5-0-released/

Release Date: 2020-04-29

Fix Resolution: jQuery - 3.5.0

Step up your Open Source Security Game with WhiteSource here

CVE-2020-11023 - Medium Severity Vulnerability

Vulnerable Library - jquery-1.7.2.min.js

JavaScript library for DOM operations

Library home page: https://cdnjs.cloudflare.com/ajax/libs/jquery/1.7.2/jquery.min.js

Path to dependency file: squiss-ts/node_modules/jmespath/index.html

Path to vulnerable library: squiss-ts/node_modules/jmespath/index.html

Dependency Hierarchy:

• ❌ jquery-1.7.2.min.js (Vulnerable Library)

Found in HEAD commit: 9b4ec2c6366c49ad22ffe58a91f08185fe51b78d

Found in base branch: master

Vulnerability Details

In jQuery versions greater than or equal to 1.0.3 and before 3.5.0, passing HTML containing elements from untrusted sources - even after sanitizing it - to one of jQuery's DOM manipulation methods (i.e. .html(), .append(), and others) may execute untrusted code. This problem is patched in jQuery 3.5.0.

Publish Date: 2020-04-29

URL: CVE-2020-11023

CVSS 3 Score Details (6.1)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: Required
  • Scope: Changed
• Impact Metrics:
  • Confidentiality Impact: Low
  • Integrity Impact: Low
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-11023

Release Date: 2020-04-29

Fix Resolution: jquery - 3.5.0

Step up your Open Source Security Game with WhiteSource here

CVE-2020-7656 - Medium Severity Vulnerability

Vulnerable Library - jquery-1.7.2.min.js

JavaScript library for DOM operations

Library home page: https://cdnjs.cloudflare.com/ajax/libs/jquery/1.7.2/jquery.min.js

Path to dependency file: squiss-ts/node_modules/jmespath/index.html

Path to vulnerable library: squiss-ts/node_modules/jmespath/index.html

Dependency Hierarchy:

• ❌ jquery-1.7.2.min.js (Vulnerable Library)

Found in HEAD commit: 9b4ec2c6366c49ad22ffe58a91f08185fe51b78d

Found in base branch: master

Vulnerability Details

jquery prior to 1.9.0 allows Cross-site Scripting attacks via the load method. The load method fails to recognize and remove "<script>" HTML tags that contain a whitespace character, i.e: "</script >", which results in the enclosed script logic to be executed.

Publish Date: 2020-05-19

URL: CVE-2020-7656

CVSS 3 Score Details (6.1)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: Required
  • Scope: Changed
• Impact Metrics:
  • Confidentiality Impact: Low
  • Integrity Impact: Low
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: rails/jquery-rails@8f601cb

Release Date: 2020-05-19

Fix Resolution: jquery-rails - 2.2.0

Step up your Open Source Security Game with WhiteSource here

Expected Behavior

When stopping the queue using stop() with timeout, the promise need to be resolved only once.

Current Behavior

When stopping the queue using stop() with timeout, the promise might get resolved twice.

Possible Solution

Save the state of the resolve, and not perform it twice.

Steps to Reproduce (for bugs)

1. Call stop() with timeout while there are messages being handled.
2. Wait for the timeout to pass, and the promise will be resolved,
3. Mark all messages as handled.
4. Promise will be resolved again

The devDependency @types/node was updated from 12.12.1 to 12.12.2.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@types/node is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

CVE-2020-7598 - Medium Severity Vulnerability

Vulnerable Library - minimist-0.0.8.tgz

parse argument options

Library home page: https://registry.npmjs.org/minimist/-/minimist-0.0.8.tgz

Path to dependency file: squiss-ts/package.json

Path to vulnerable library: squiss-ts/node_modules/minimist/package.json

Dependency Hierarchy:

• mocha-6.2.2.tgz (Root Library)
  • mkdirp-0.5.1.tgz
    • ❌ minimist-0.0.8.tgz (Vulnerable Library)

Found in HEAD commit: 9b4ec2c6366c49ad22ffe58a91f08185fe51b78d

Found in base branch: master

Vulnerability Details

minimist before 1.2.2 could be tricked into adding or modifying properties of Object.prototype using a "constructor" or "proto" payload.

Publish Date: 2020-03-11

URL: CVE-2020-7598

CVSS 3 Score Details (5.6)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: High
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: Low
  • Integrity Impact: Low
  • Availability Impact: Low

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://github.com/substack/minimist/commit/63e7ed05aa4b1889ec2f3b196426db4500cbda94

Release Date: 2020-03-11

Fix Resolution: minimist - 0.2.1,1.2.3

Step up your Open Source Security Game with WhiteSource here

It will be a cool feature, to allow the library user, to decide that he wants to rate-limit the handling of the messages across many microservices, instead of just using the concurrency control alone.
Example use case - a message queue to send emails with SMTP global hourly rate-limiting enforced.
We can leverage the following library for the task: https://github.com/dex4er/js-sliding-window-rate-limiter

The dependency aws-sdk was updated from 2.570.0 to 2.571.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

aws-sdk is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Your Greenkeeper Bot 🌴

It is not allowed o send extra parameters in the Entries value of the request

Your issue may already be reported!
Please search on the issue tracker before creating one.

Expected Behavior

If the call to deleteMessageBatch rejects, then that rejected Promise should be handled properly and not bubble up.

Current Behavior

An UnhandledPromiseRejectionWarning is logged to console, for example:
(node:58881) UnhandledPromiseRejectionWarning: AWS.SimpleQueueService.NonExistentQueue: The specified queue does not exist for this wsdl version.

Possible Solution

The cause of this bug is that a rejected Promise is returned from https://github.com/PruvoNet/squiss-ts/blob/master/src/Squiss.ts#L352, however that rejected Promise is not caught at https://github.com/PruvoNet/squiss-ts/blob/master/src/Squiss.ts#L471 which is where the call to _deleteMessages occurs.

Proposed solutions:

1. Return resolved promise from catch block in _deleteMessages, the error has already been emitted so no need to reject as well?
2. Add .catch() handler in _deleteXMessages which suppresses the error

Steps to Reproduce (for bugs)

Can be replicated via the unit tests. For visibility set https://github.com/PruvoNet/squiss-ts/blob/master/src/test/src/index.spec.ts#L725 to be the only test to run:
it.only('emits error when delete call fails', () => {
Warnings can then be seen in the console

[me@mymachine squiss-ts (master)]$ npm test

> [email protected] test /Users/me/GitHub/squiss-ts
> npm run lint && npm run mocha


> [email protected] lint /Users/me/GitHub/squiss-ts
> tslint -c tslint.json 'src/**/*.ts' 'test/**/*.ts'


> [email protected] mocha /Users/me/GitHub/squiss-ts
> mocha --opts src/test/mocha.opts



  index
    Failures
(node:70693) UnhandledPromiseRejectionWarning: Error: test
    at Object.promise (/Users/me/GitHub/squiss-ts/src/test/src/index.spec.ts:731:41)
    at getQueueUrl.then (/Users/me/GitHub/squiss-ts/src/Squiss.ts:348:16)
    at <anonymous>
(node:70693) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 3)
(node:70693) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
      ✓ emits error when delete call fails (72ms)


  1 passing (84ms)

Context

We are currently testing out squiss-ts by testing it in different scenarios before implementing in production code. A real world example which would cause an error like this would be a network blip.

Your Environment

squiss-ts: 4.0.6
Node: 8.16.1

Hi,

Looking at using this library, its awesome!

I have a use-case:

We have an sqs-processor that involves booting up Puppeteer and google lighthouse to audit webpages. The processing time varies wildly, and sometimes the program hangs, hence I have been using (in my own manual implementation) a timer that repeatedly extends the message visibility up to a certain limit. With this library, I can now use the autoExtendTimeout feature which is awesome!

However, there's a problem; when the message reaches it's hard time limit (i.e. noExtensionsAfterSecs) I need to capture this event, so that I can shut down the Puppeteer browser and lighthouse processes - i.e. actually stop the message processing. As far as I can tell, when the noExtensionsAfterSecs limit is reached, the message is simply returned to the queue, i.e. effectively calling release() on it?

It would be awesome if I could capture this event with a listener so I can do as I please on a timeout event.

I've been looking at poller.on('handled')... to see if there is anything I can grab from there, but it doesn't look like the handled event is fired in this case.

CVE-2020-7608 - Medium Severity Vulnerability

Vulnerable Library - yargs-parser-13.1.1.tgz

the mighty option parser used by yargs

Library home page: https://registry.npmjs.org/yargs-parser/-/yargs-parser-13.1.1.tgz

Path to dependency file: squiss-ts/package.json

Path to vulnerable library: squiss-ts/node_modules/yargs-parser/package.json

Dependency Hierarchy:

• mocha-6.2.2.tgz (Root Library)
  • ❌ yargs-parser-13.1.1.tgz (Vulnerable Library)

Found in HEAD commit: 9b4ec2c6366c49ad22ffe58a91f08185fe51b78d

Found in base branch: master

Vulnerability Details

yargs-parser could be tricked into adding or modifying properties of Object.prototype using a "proto" payload.

Publish Date: 2020-03-16

URL: CVE-2020-7608

CVSS 3 Score Details (5.3)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Local
  • Attack Complexity: Low
  • Privileges Required: Low
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: Low
  • Integrity Impact: Low
  • Availability Impact: Low

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: yargs/yargs-parser@63810ca

Release Date: 2020-06-05

Fix Resolution: 5.0.1;13.1.2;15.0.1;18.1.1

Step up your Open Source Security Game with WhiteSource here

The dependency aws-sdk was updated from 2.580.0 to 2.581.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

aws-sdk is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Your Greenkeeper Bot 🌴

☝️ Important announcement: Greenkeeper will be saying goodbye 👋 and passing the torch to Snyk on June 3rd, 2020! Find out how to migrate to Snyk and more at greenkeeper.io

The dependency aws-sdk was updated from 2.634.0 to 2.635.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

aws-sdk is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Your Greenkeeper Bot 🌴

The devDependency @types/node was updated from 12.12.6 to 12.12.7.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@types/node is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Helps lowering message sizes to reduce costs of api calls

Is your feature request related to a problem? Please describe.
We currently use sqs-consumer which is Apache 2.0 licensed. We really liked some of the features in squiss which was ISC and were super excited to find this updated TypeScriptified fork. Unfortunately due to legal restrictions out of our control, we aren't able to use or contribute to GPL libraries, which means sadly we can't get involved with this library.

Describe the solution you'd like
I'd love for the repo to be relicensed as MIT, ISC or Apache 2.0. Appreciate it might be an impossible request but doesn't hurt to ask!

Describe alternatives you've considered
As mentioned above:

• sqs-consumer - doesn't have concurrency utilization
• squiss - unmaintained

Additional context
Before we noticed the license, we actually prototyped a solution using squiss-ts and it worked fantastically. So even if you are unable to change the license, I just wanted to say great effort on the library :)

Need to fire event when S3 events occur such as upload, download, and delete.
This will be useful if you want to monitor how many messages are getting sent that way

Currently we are using our own typing of the linked-list library. This should be removed once wooorm/linked-list#12 is fixed

Same behaviour like https://github.com/awslabs/amazon-sqs-java-extended-client-lib

When a message is received and failed to get parsed, there is an un-handled promise rejection error.