Add CODE_OF_CONDUCT.md

There is a moderate severity security issue with one of your dependencies (i.e., the path to the vuln serverless-localhost > rmdir > node.extend.

It looks like the easiest fix is to just stop using rmdir since it doesn't do much and hasn't been updated in quite a while. Let me know if you'd like me to submit a PR.

More info: https://npmjs.com/advisories/781

Hello,
First of all, thanks for this lib and the serverless-dynamodb-local plugin!

I'm using this library through the serverless-dynamodb-local and when using it with yarn, I have to install the dynamodb jar everytime I install a new package (since yarn removes the bin folder that this lib creates to download the jar binary).

I have been able to make the changes that will use a better directory for this when using it with serverless (a .dynamodb dir int the service folder, same as .serverless one).

Most of the changes are in the serverless plugin, for this lib I just need to change the following piece of code in https://github.com/99xt/dynamodb-localhost/blob/master/dynamodb/starter.js

var starter = {
    start: function (options, config) {
        /* Dynamodb local documentation http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html */
        var additionalArgs = [],
            port = options.port || config.start.port,
-           db_dir = utils.absPath(config.setup.install_path),
+           db_dir = options.install_path || utils.absPath(config.setup.install_path),
            jar = config.setup.jar;

Then, in the serverless plugin I would change in https://github.com/99xt/serverless-dynamodb-local/blob/v1/index.js:

    constructor(serverless, options) {
        this.serverless = serverless;
        this.service = serverless.service;
        this.serverlessLog = serverless.cli.log.bind(serverless.cli);
        this.config = this.service.custom && this.service.custom.dynamodb || {};
-       this.options = options;
+       this.options = _.merge({
+               localPath: path.join(serverless.config.servicePath, '.dynamodb')
+           },
+           options
+       );

    startHandler() {
        const config = this.config;
        const options = _.merge({
-                sharedDb: this.options.sharedDb || true
+                sharedDb: this.options.sharedDb || true,
+                install_path: this.options.localPath
            },
            config && config.start,
            this.options
        );

        // otherwise endHandler will be mis-informed
        this.options = options;
        if (!options.noStart) {
          dynamodbLocal.start(options);
        }
        return BbPromise.resolve()
        .then(() => options.migrate && this.migrateHandler())
        .then(() => options.seed && this.seedHandler());
    }

Let me know what you think! I have the changes ready so if you are ok, I can create the two PRs.

We have found in our use of DynamoDB local that the response from real AWS DynamoDB and Local DynamoDB are different when you attempt to delete data which does not exist. When using DynamoDB local the callback for <DynamoDbInstance>.deleteItem is called with a non-null value for the data parameter, however, for real AWS DynamoDB the callback is called with null for the data parameter.

We use dynamodb-localhost through serverless-dynamodb-local. We use [email protected] which pulls in [email protected]

Here is a representation of how we are exuting the deleteItem command:

var params = {
  Key: invalidKey,        // key that does not exist 
  TableName: "validTable" // table exists
 };
 dynamodb.deleteItem(params, function(err, data) {
 // data here should be null, which is the correct AWS behaviour
   if (err) console.log(err, err.stack);
   else     console.log(data); 
 });

When the data parameter is not null, dynamodb local responds with the following:

{"ConsumedCapacity":{"TableName":"validTable","CapacityUnits":1},"LOG_ID":"cqg6"}

Current implementation does not cater for redirect, causing error when downloading dynamodb_local

/Users/anggra/Documents/Work/web/testing/node_modules/dynamodb-localhost/dynamodb/installer.js:23
                throw new Error('Error getting DynamoDb local latest tar.gz location ' + response.headers.location + ': ' + response.statusCode);
                ^

Error: Error getting DynamoDb local latest tar.gz location http://103.1.138.149/s3-us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_latest.tar.gz: 302
    at ClientRequest.<anonymous> (/Users/anggra/Documents/Work/web/testing/node_modules/dynamodb-localhost/dynamodb/installer.js:23:23)
    at Object.onceWrapper (events.js:291:19)
    at emitOne (events.js:96:13)
    at ClientRequest.emit (events.js:189:7)
    at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:522:21)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:99:23)
    at Socket.socketOnData (_http_client.js:411:20)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:189:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at Socket.Readable.push (_stream_readable.js:134:10)
    at TCP.onread (net.js:551:20)

I've created a fix at https://github.com/anggras/dynamodb-localhost/tree/handle-redirect
Please let me know if it is acceptable

Hey 👋

My first contribution here - seems to be a regression of #37

latest tar.gz was updated yesterday

<ContentsContents>
  <Key>dynamodb_local_latest.tar.gz</Key>
  <LastModified>2023-02-02T18:02:28.000Z</LastModified>
  <ETag>"265a5d3d71889672d30e2bce91e83387-6"</ETag>
  <Size>42739735</Size>
  <StorageClass>STANDARD</StorageClass>
</Contents>

But starting dynamodb fails with

Started downloading dynamodb-local from http://s3-us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_latest.tar.gz into .../.dynamodb. Process may take few minutes.
✖ Uncaught exception
Environment: linux, node 14.17.0, framework 3.24.1 (local), plugin 6.2.2, SDK 4.3.2
Credentials: Local, "..." profile
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
Error: Error getting DynamoDb local latest tar.gz location undefined: 403
    at ClientRequest.<anonymous> (/.../node_modules/dynamodb-localhost/dynamodb/installer.js:29:15)

Unsure who owns the s3 bucket, but it is not reachable programmatically, but reachable through browser download https://s3-us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_latest.tar.gz

Checking if we are missing any headers

I keep getting this error for some reason. Doesn't give too much detail about what exactly happened. Any ideas?

DynamoDB Local failed to start with code 1
{ Error: connect ECONNREFUSED 127.0.0.1:8000
    at Object.exports._errnoException (util.js:1012:11)
    at exports._exceptionWithHostPort (util.js:1035:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1080:14)
  message: 'connect ECONNREFUSED 127.0.0.1:8000',
  code: 'NetworkingError',
  errno: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 8000,
  region: 'us-west-2',
  hostname: 'localhost',
  retryable: true,
  time: 2016-09-13T03:26:05.804Z }

Looks like http://s3-us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_latest.tar.gz was just released today but is returning a 403. I can see the entry:

<Contents>
  <Key>dynamodb_local_latest.tar.gz</Key>
  <LastModified>2019-02-06T12:51:06.000Z</LastModified>
  <ETag>"619b49fdfd32827c7b3c69873458100f-3"</ETag>
  <Size>17464086</Size>
  <StorageClass>STANDARD</StorageClass>
</Contents>

Issue

Using this library under Windows (cmd) does not spawn the DynamoDB java process.

Possible solution

1. Passing in the shell: true option into child_process.spawn would execute the command in a shell.
2. Replace child_process.spawn with a cross platform compatible shell library such as shelljs.

The recent critical fixes to the download url have not been published to npm. Please bump the version to make that happen.

Actual Behaviour

When I run
sls dynamodb install

it fails with the errors below:
Error: Error getting DynamoDb local latest tar.gz location undefined: 403

The AWS page and the corresponding S3 buckets that provide the local versions of DynamoDB seem to be working under https only.

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.DownloadingAndRunning.html

It appears that serverless-dynamodb-local used to download the tar.gz file over http and that worked for a long time. But it appears that the security on the s3 bucket for local DynamoDbs have been tightened and only https is supported. This happened in early March 2023.

A fix to this would involve modifying the download link to https protocol and also ensuring that https works in installer.js

Expected Behaviour

Expect the latest local DynamoDB tar.gz file to download and be unzipped locally.

Steps to reproduce it

Run
sls dynamodb install

LogCat for the issue

Provide logs for the crash here

Screenshots of the issue

Where-ever possible attach a screenshot of the issue.

Would you like to work on the issue?

YES

installing new DyanamoDB does not add the local path in the path.

I have to assign localpath to "bin" in order to install the database.

[ '-Djava.library.path=/Users/aaalsubaie/Projects/propman-server/node_modules/dynamodb-localhost/dynamodb//bin/DynamoDBLocal_lib', '-jar', 'DynamoDBLocal.jar', '-port', 8000 ]

this might be the source of the problem:
db_dir = utils.absPath(config.setup.install_path)

The latest update of tar version in package.json is breaking the 'sls dynamodb install' functionality in your other repo - serverless-dynamodb-local. Change it back to 2.0.0 fixes the issue.

Behind a firewall, http could be disabled in some usecases... it would be nice to be able to support downloading via https as well

As simple as this command is, this is truly required on any setup that requires testing.

Ideally in my mocha tests, I would write in the global_config.js:

/* eslint-disable */
const dynamodbLocal = require('dynamodb-localhost');

// setup the database
before(async function () {
  await dynamodbLocal.install();
  await dynamodbLocal.start({ port: 8000 });
});

// teardown
after(async function() {
  await dynamodbLocal.stop();
});

// run before each test
beforeEach(() => {});

// run after each test
afterEach(() => {});

Obviously with other test files next to it, who just make a dynamodb connection to the local test database.
This setup, the serverless plugin, or the original dynamodb-local, none of them support the option to run this in the background, as such, these are all useless when attempting to run a CI/CD job on. I can't tell a pipeline to "press ctrl-c" when you're done.

Since I bought a MacBook with the M1 chip, all my projects using dynamodb-localhost (indirectly through serverless-dynamodb-local) are a pain, since the Java version of DynamoDB does not work on Apple Silicon.
There are several issues in dependent repos about this problem. StackOverflow's best answer is to install a separate Java environment within Rosetta, or even a separate entire brew installation, but this is really a hassle and comes with some issues.

I'm not sure if anyone is working on providing the missing (SQLite) library for arm64, but in the meantime, I was wondering if this project would be interested in offering support for running DynamoDB through docker, since that actually works fine on M1 macs?

It would save a ton of hassle for users who are today stuck with having to install Java under Rosetta 2 and then somehow get the rest of their environment running on native Apple architecture to interact with DynamoDB in a Java installation in an emulated i386 shell?

Basically the only difference is that instead of calling java -jar DynamoDB.jar <options> you call docker run -d -p 8000:8000 amazon/dynamodb-local -jar DynamoDB.jar <options>, and I was thinking this could either be a new option within the options passed to start(), or a separate startDocker(), or possibly controlled through an environment variable.

I'd be happy to give a PR a shot if interesting.

Replace dynamodb handling core functionality with dynamodb-localhost library
https://github.com/99xt/dynamodb-localhost

If you're trying to install behind a (corp) proxy, the http client doesn't attempt to make the connection via the users HTTP(S)_PROXY envvar.

I have a patch ready if this would be useful.

Only caveat is that the URL in the HTTP_PROXY envvar begins with a protocol e.g. HTTP_PROXY=http://localhost:3128. HTTP_PROXY=localhost:3128 wont work, due to limitations in Node's url parsing function.

mocha is listed as a dependency which is wrong. I haven't checked the others but I assume that this is a bug.

Hi! 👋

Firstly, thanks for your work on this project! 🙂

Today I used patch-package to patch [email protected] for the project I'm working on.

Here is the diff that solved my problem:

diff --git a/node_modules/dynamodb-localhost/dynamodb/config.json b/node_modules/dynamodb-localhost/dynamodb/config.json
index 857d66f..207799a 100644
--- a/node_modules/dynamodb-localhost/dynamodb/config.json
+++ b/node_modules/dynamodb-localhost/dynamodb/config.json
@@ -1,6 +1,6 @@
 {
     "setup": {
-        "download_url": "http://dynamodb-local.s3-website-us-west-2.amazonaws.com/dynamodb_local_latest.tar.gz",
+        "download_url": "https://dynamodb-local.s3-website-us-west-2.amazonaws.com/dynamodb_local_latest.tar.gz",
         "install_path": "/bin",
         "jar": "DynamoDBLocal.jar"
     },
diff --git a/node_modules/dynamodb-localhost/dynamodb/installer.js b/node_modules/dynamodb-localhost/dynamodb/installer.js
index b4c9450..3db2802 100644
--- a/node_modules/dynamodb-localhost/dynamodb/installer.js
+++ b/node_modules/dynamodb-localhost/dynamodb/installer.js
@@ -3,17 +3,17 @@
 var tar = require('tar'),
     zlib = require('zlib'),
     path = require('path'),
-    http = require('http'),
+    https = require('https'),
     fs = require('fs'),
     ProgressBar = require('progress'),
     utils = require('./utils');
 
 var download = function (downloadUrl, installPath, callback) {
-    http.get(downloadUrl, function (response) {
+    https.get(downloadUrl, function (response) {
             if (302 != response.statusCode) {
                 callback(new Error('Error getting DynamoDb local latest tar.gz location: ' + response.statusCode));
             }
-            http.get(response.headers.location, function (redirectResponse) {
+            https.get(response.headers.location, function (redirectResponse) {
                     var len = parseInt(redirectResponse.headers['content-length'], 10),
                         bar = new ProgressBar('Downloading dynamodb-local [:bar] :percent :etas', {
                             complete: '=',

This issue body was partially generated by patch-package.

A recent vulnerability in the "tar" package was found.

dynamodb-localhost > tar

Allows an "Arbitrary File Overwrite"

https://nodesecurity.io/advisories/803

Running the following test:

describe('DynamodbService', () => {
  let instance: UserPersistenceService;

  beforeEach(() => {
    instance = new UserPersistenceService();
    dynamodb.install();
    dynamodb.start({
      port: 8000
    });
  });

  it('can be instantiated', () => {

    expect(instance instanceof UserPersistenceService).toBeTruthy();
  });


  afterEach(() => {
    dynamodb.stop({
      port: 8000
    })
  });
});

produces the following output in tests results:

 PASS  packages/datagator/src/lib/dynamodb/dynamodb.service.spec.ts (6.001s)
  ● Console

    console.log node_modules/dynamodb-localhost/dynamodb/installer.js:12
      Started downloading dynamodb-local from http://s3-us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_latest.tar.gz into /home/developer/src/microgamma/node_modules/dynamodb-localhost/dynamodb/bin. Process may take few minutes.
    console.log node_modules/dynamodb-localhost/index.js:30
      Dynamodb Local Started, Visit: http://localhost:8000/shell

I expect the output to be clean and to still be able to get debugging info if wanted using DEBUG=dynamodb-localhost:* environment variable

Replace dynamodb migration handling functionality with dynamodb-migrations library
https://github.com/99xt/dynamodb-migrations

Hi,

Can you replace the file libsqlite4java-osx.dylib by this file libsqlite4java-osx-aarch64-1.0.392.dylib on install on Mac M1 to ensure compatibility ?

https://search.maven.org/artifact/io.github.ganadist.sqlite4java/libsqlite4java-osx-aarch64/1.0.392/dylib

Thank you in advance

Great work guys!

It would be awesome if you would allow us to pass the convertEmptyValues option to the local dynamodb.
Currently whenever you try to migrate a json object with an empty string as a value, you'll get:

One or more parameter values were invalid: An AttributeValue may not contain an empty string

exception.

DynamoDB Local Web Shell seems to be deprecated.

DynamoDB Local Web Shell was deprecated with version 1.16.X and is not available any longer from 1.17.X to latest.

See here: https://stackoverflow.com/questions/70535330/dynamodb-local-web-shell-does-not-load

It worked on DynamoDB Local Web Shell version 1.16.0, but not on 1.17.0.
So I think it is necessary to correct the message displayed when starting DynamoDB Local.

I have created a pull request, so please check it.

At amazon/dynamodb-local:1.16.0

$ docker run -p 8000:8000 amazon/dynamodb-local:1.16.0

At amazon/dynamodb-local:1.17.0

$ docker run -p 8000:8000 amazon/dynamodb-local:1.17.0

During installation step, download is failing, where is it downloading from?

(using serverless-dynamodb-local)

Full stack trace:

sls dynamodb install
[Error: Error getting DynamoDb local latest tar.gz location: 404]
/Users/csilk/dev/project/node_modules/dynamodb-localhost/dynamodb/installer.js:43
                    throw Error("Error in downloading Dynamodb local " + err);
                    ^

Error: Error in downloading Dynamodb local Error: connect ECONNREFUSED 127.0.0.1:80
    at Error (native)
    at ClientRequest.<anonymous> (/Users/csilk/dev/project/node_modules/dynamodb-localhost/dynamodb/installer.js:43:27)
    at emitOne (events.js:90:13)
    at ClientRequest.emit (events.js:182:7)
    at Socket.socketErrorListener (_http_client.js:262:9)
    at emitOne (events.js:90:13)
    at Socket.emit (events.js:182:7)
    at emitErrorNT (net.js:1246:8)
    at nextTickCallbackWith2Args (node.js:475:9)
    at process._tickDomainCallback (node.js:430:17)
$ sls dynamodb install
[Error: Error getting DynamoDb local latest tar.gz location: 404]
/Users/csilk/dev/project/node_modules/dynamodb-localhost/dynamodb/installer.js:43
                    throw Error("Error in downloading Dynamodb local " + err);
                    ^

Error: Error in downloading Dynamodb local Error: connect ECONNREFUSED 127.0.0.1:80
    at Error (native)
    at ClientRequest.<anonymous> (/Users/csilk/dev/project/node_modules/dynamodb-localhost/dynamodb/installer.js:43:27)
    at emitOne (events.js:90:13)
    at ClientRequest.emit (events.js:182:7)
    at Socket.socketErrorListener (_http_client.js:262:9)
    at emitOne (events.js:90:13)
    at Socket.emit (events.js:182:7)
    at emitErrorNT (net.js:1246:8)
    at nextTickCallbackWith2Args (node.js:475:9)
    at process._tickDomainCallback (node.js:430:17)