Git Product home page Git Product logo

parse-server-fs-adapter's Introduction

Parse Server FS Adapter

Build Status Snyk Badge Coverage auto-release

Node Version

npm latest version


The Parse Server File System Storage Adapter.


Installation

npm install --save @parse/fs-files-adapter

Configuration

Parse Server Config

{
  "appId": 'my_app_id',
  "masterKey": 'my_master_key',
  // other options
  // ...
  "filesAdapter": {
    "module": "@parse/fs-files-adapter",
    "options": {
      "filesSubDirectory": "my/files/folder", // Optional, defaults to `./files`
      "encryptionKey": "someKey" // Optional, but mandatory if you want to encrypt files
    } 
  }
}

In-Code Instance

var FSFilesAdapter = require('@parse/fs-files-adapter');

var fsAdapter = new FSFilesAdapter({
  "filesSubDirectory": "my/files/folder", // optional, defaults to ./files
  "encryptionKey": "someKey" //optional, but mandatory if you want to encrypt files
});

var api = new ParseServer({
	appId: 'my_app',
	masterKey: 'master_key',
	filesAdapter: fsAdapter
})

Rotating Encryption Key

Periodically you may want to rotate your encryptionKey for security reasons. When this is the case, you can start up a development parse-server that has the same configuration as your production server. In the development server, initialize the file adapter with the new key and use the examples below.

Note that the examples below to rotate keys are are not optimized for performance. Is it therefore not recommended to rotate a large number of files using the code below in a production environment; instead use dedicated resources for that.

Encrypting Previously Unencrypted Files

var FSFilesAdapter = require('@parse/fs-files-adapter');

var fsAdapter = new FSFilesAdapter({
  "filesSubDirectory": "my/files/folder", // Optional, defaults to `./files`
  "encryptionKey": "newKey" // Use the new key
});

var api = new ParseServer({
	appId: 'my_app',
	masterKey: 'master_key',
	filesAdapter: fsAdapter
});

const { rotated, notRotated } =  await api.filesAdapter.rotateEncryptionKey();
console.log('Files rotated to newKey: ' + rotated);
console.log('Files that couldn\'t be rotated to newKey: ' + notRotated);

After successfully rotating your key, you should change the encryptionKey to newKey on your production server and then restart the server.

Encrypting Previously Encrypted Files

To encrypt files with a new key that were previously encrypted with a different key, the same process applies as above, but you pass in your oldKey to rotateEncryptionKey().

const {rotated, notRotated} =  await api.filesAdapter.rotateEncryptionKey({oldKey: oldKey});
console.log('Files rotated to newKey: ' + rotated);
console.log('Files that couldn\'t be rotated to newKey: ' + notRotated);

You can also only rotate a select list of files that were previously encrypted with oldKey and you want to encrypt with newKey. This is useful if for some reason there errors and some of the files werent rotated and returned in notRotated. The same process as above, but pass in your oldKey along with the array of fileNames to rotateEncryptionKey().

const { rotated, notRotated } =  await api.filesAdapter.rotateEncryptionKey({ oldKey: oldKey, fileNames: ["fileName1.png","fileName2.png"] });
console.log('Files rotated to newKey: ' + rotated);
console.log('Files that couldn\'t be rotated to newKey: ' + notRotated);

Other Considerations

  • Multiple Instances of Parse Server

    When using the adapter across multiple Parse Server instances it's important to establish "centralization" of your file storage (this is the same premise as the other file adapters, you are sending/receiving files through a dedicated link). You can accomplish this at the file storage level by Samba mounting (or any other type of mounting) your storage to each of your parse-server instances, e.g if you are using parse-server via docker (volume mount your SMB drive to - /Volumes/SMB-Drive/MyParseApp1/files:/parse-server/files). All parse-server instances need to be able to read and write to the same storage in order for parse-server-fs-adapter to work properly with parse-server. If the file storage isn't centralized, parse-server will have trouble locating files and you will get random behavior on client-side.

parse-server-fs-adapter's People

Contributors

cbaker6 avatar davimacedo avatar dependabot[bot] avatar dtsolis avatar flovilmart avatar mtrezza avatar semantic-release-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

parse-server-fs-adapter's Issues

Unable to Upload Files

Both restFUL and an macOS app are able to download files no problem via parse api calls to a parse server and mono server.
Both servers are run in house on the same machine.
Updating data to parse is no problem.

However, I am unable to upload files in my macOS software.

But I tried using restFUL as a troubleshooting mechanism and found this :

curl -X POST
-H "X-Parse-Application-Id: myappid"
-H "X-Parse-REST-API-Key: myrestkey"
-H "Content-Type: image/jpg"
--data-binary '@1.jpg'
https://myserver.com/parse/files/1.jpg

Results in an error.

but

curl -X POST
-H "X-Parse-Application-Id: myappid"
-H "X-Parse-REST-API-Key: myrestkey"
-H "Content-Type: image/jpg"
--data-binary '@1.jpg'
http://myserver.com:1337/parse/files/1.jpg

does not result in an error.
Huh.
Unfortunately - connecting to the parse server via http isn't an option - OSX balks at it.

I can however retrieve a file via both HTTP and HTTPS CURL

Port not included from ParseServer's serverURL

Our serverURL is http://localhost:8080/parse but Parse.File data is sent to the client with url: "http://localhost/parse/files/...". The URLs are otherwise correct, they're just missing the port.

I'm new to Parse so apologies if I'm missing something.

Project isn't linted

New Feature / Enhancement Checklist

Current Limitation

The project isn't linted

Feature / Enhancement Description

Lint the project to make code conform to a standard

Example Use Case

Similar to parse-server project

Alternatives / Workarounds

Use as is.

3rd Party References

N/A

PERMISSION ERROR

: In order to use the FileSystemAdapter, write access to the server's file system is required.

at FileSystemAdapter._mkdir (/opt/bitnami/apps/parse/htdocs/node_modules/@parse/fs-files-adapter/index.js:106:17)

what folder need to permission change ?
any idea?

semantic-release 21.0.1 not compatible with current packages

New Issue Checklist

Issue Description

semantic-release 21.0.1 isn't compatible with current dev dependencies and #41 should be reverted. It appears to me that dependabot allowed an incorrect update in #41 and somehow forced an update to the package-lock. The warnings and errors in logs below look explicit and state a number of problems like peer semantic-release@">=16.0.0 <18.0.0" from @semantic-release/[email protected].

#41 updated from [email protected] to [email protected] which are outside the peer dependency windows in the logs. It seems if that update was to be done properly, other semantic-release updates should have been done at the same time.

In addition, the parse-server receives way more updates than this repo and it's versions aren't this high: https://github.com/parse-community/parse-server/blob/9674d4a2c0a9d0cda112056a6a2b1629931f37a3/package.json#L75-L107

Steps to reproduce

Use npm i to install a new package

Actual Outcome

Expected Outcome

  • Using npm i to install new packages should not have errors due to semantic-release
  • CI should pass successfully

Environment

Client

  • Parse Server Push Adapter version: 2.0.0

Server

  • Parse Server version: 6.2.0
  • Operating system: Linux
  • Local or remote host (AWS, Azure, Google Cloud, Heroku, Digital Ocean, etc): Local (docker)

Database

  • System (MongoDB or Postgres): Postgres
  • Database version: 15
  • Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): Local (docker)

Logs

npm WARN old lockfile 
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile 
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile 
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: @parse/[email protected]
npm WARN Found: [email protected]
npm WARN node_modules/semantic-release
npm WARN   peer semantic-release@">=16.0.0 <18.0.0" from @semantic-release/[email protected]
npm WARN   6 more (@semantic-release/changelog, @semantic-release/npm, ...)
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer semantic-release@">=16.0.0 <18.0.0" from @semantic-release/[email protected]
npm WARN node_modules/@semantic-release/commit-analyzer
npm WARN   @semantic-release/commit-analyzer@"^8.0.0" from [email protected]
npm WARN   node_modules/semantic-release
npm WARN   1 more (the root project)
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: @parse/[email protected]
npm WARN Found: [email protected]
npm WARN node_modules/semantic-release
npm WARN   peer semantic-release@">=16.0.0 <18.0.0" from @semantic-release/[email protected]
npm WARN   6 more (@semantic-release/changelog, @semantic-release/npm, ...)
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer semantic-release@">=15.8.0 <18.0.0" from @semantic-release/[email protected]
npm WARN node_modules/@semantic-release/changelog
npm WARN   dev @semantic-release/changelog@"5.0.1" from the root project
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: @parse/[email protected]
npm WARN Found: [email protected]
npm WARN node_modules/semantic-release
npm WARN   peer semantic-release@">=16.0.0 <18.0.0" from @semantic-release/[email protected]
npm WARN   6 more (@semantic-release/changelog, @semantic-release/npm, ...)
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer semantic-release@">=16.0.0 <18.0.0" from @semantic-release/[email protected]
npm WARN node_modules/@semantic-release/npm
npm WARN   @semantic-release/npm@"^7.0.0" from [email protected]
npm WARN   1 more (the root project)
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: @parse/[email protected]
npm WARN Found: [email protected]
npm WARN node_modules/semantic-release
npm WARN   peer semantic-release@">=16.0.0 <18.0.0" from @semantic-release/[email protected]
npm WARN   6 more (@semantic-release/changelog, @semantic-release/npm, ...)
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer semantic-release@">=16.0.0 <18.0.0" from @semantic-release/[email protected]
npm WARN node_modules/@semantic-release/git
npm WARN   dev @semantic-release/git@"9.0.0" from the root project
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: @parse/[email protected]
npm WARN Found: [email protected]
npm WARN node_modules/semantic-release
npm WARN   peer semantic-release@">=16.0.0 <18.0.0" from @semantic-release/[email protected]
npm WARN   6 more (@semantic-release/changelog, @semantic-release/npm, ...)
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer semantic-release@">=16.0.0 <18.0.0" from @semantic-release/[email protected]
npm WARN node_modules/@semantic-release/github
npm WARN   @semantic-release/github@"^7.0.0" from [email protected]
npm WARN   1 more (the root project)
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: @parse/[email protected]
npm WARN Found: [email protected]
npm WARN node_modules/semantic-release
npm WARN   peer semantic-release@">=16.0.0 <18.0.0" from @semantic-release/[email protected]
npm WARN   6 more (@semantic-release/changelog, @semantic-release/npm, ...)
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer semantic-release@">=15.8.0 <18.0.0" from @semantic-release/[email protected]
npm WARN node_modules/@semantic-release/release-notes-generator
npm WARN   @semantic-release/release-notes-generator@"^9.0.0" from [email protected]
npm WARN   1 more (the root project)
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR! 
npm ERR! While resolving: @parse/[email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/semantic-release
npm ERR!   dev semantic-release@"21.0.1" from the root project
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peer semantic-release@">=16.0.0 <18.0.0" from @semantic-release/[email protected]
npm ERR! node_modules/@semantic-release/commit-analyzer
npm ERR!   dev @semantic-release/commit-analyzer@"8.0.1" from the root project
npm ERR! 
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.

Unable to customise filesSubDirectory on FSFilesAdapter

Hello how are you ?

I use this on parse server,

var fsAdapter = new FSFilesAdapter({
"filesSubDirectory": "my/files/folder",
"encryptionKey": "someKey"
});

var api = new ParseServer({
appId: 'my_app',
masterKey: 'master_key',
filesAdapter: fsAdapter
})

When server start, FSFilesAdapter files is on files/my/files/folder instead of my/files/folder.

This may be due to this section of your code, line 180:

FileSystemAdapter.prototype._getApplicationDir = function() {
if (this._filesDir) {
return path.join('files', this._filesDir);
} else {
return 'files';
}
}

In getFileLocation function, config.mount returns undefined.

In my server, I am getting file urls as
http://xx.xx.xx.xx/undefined/files/xxxxxxxx/xxxxxxxx_file_933781.mp4

It turns out getFileLocation function is appending config.mount ( and it is undefined ).

But when I log the config object, there is property named config.mountPath, by using config.mountPath instead of config.mount, server returns proper file url.

This could be due to parse server versions, so someone experienced can submit the PR.

Cannot read property 'getFileData' of undefined

I'm testing the fs-adapter and it works on creating a file on disk. I got an error when I try to retrieve the file though; this is the stack trace

error: Uncaught internal server error. [TypeError: Cannot read property 'getFileData' of undefined] TypeError: Cannot read property 'getFileData' of undefined at getHandler (/root/parse/node_modules/parse-server/lib/Routers/FilesRouter.js:69:22) at Layer.handle [as handle_request] (/root/parse/node_modules/parse-server/node_modules/express/lib/router/layer.js:95:5) at next (/root/parse/node_modules/parse-server/node_modules/express/lib/router/route.js:131:13) at Route.dispatch (/root/parse/node_modules/parse-server/node_modules/express/lib/router/route.js:112:3) at Layer.handle [as handle_request] (/root/parse/node_modules/parse-server/node_modules/express/lib/router/layer.js:95:5) at /root/parse/node_modules/parse-server/node_modules/express/lib/router/index.js:277:22 at param (/root/parse/node_modules/parse-server/node_modules/express/lib/router/index.js:349:14) at param (/root/parse/node_modules/parse-server/node_modules/express/lib/router/index.js:365:14) at param (/root/parse/node_modules/parse-server/node_modules/express/lib/router/index.js:365:14) at Function.process_params (/root/parse/node_modules/parse-server/node_modules/express/lib/router/index.js:410:3)

I got that when I specify the 'filesSubDirectory' option while creating the adapter. Instead when I leave options blank I receive an {"error": "unauthorized"}

Am I missing something?

Allow files to be stored on external media

New Issue Checklist

Issue Description

Suggestion: Allow files to be stored on external media

Context :

I want to suggest an improvement that might make the adapter more flexible and adaptable to different file storage needs.

Steps to reproduce

Actual Outcome

Currently, the @parse/fs-files-adapter restricts file storage to the Parse application's parent directory. This can be convenient in many cases, but it can also be cumbersome for users who want to store files on external media, such as a mounted file system.

Expected Outcome

I suggest that the adapter provide the ability to specify a custom storage location for uploaded files, for example, an absolute directory path. This would allow adapter users to choose where they want to store their files based on their specific needs.>

Environment

Client

  • Parse Server Push Adapter version: n/a

Server

  • Parse Server version: 6.2.1
  • Operating system: Ubuntu server
  • Local or remote host (AWS, Azure, Google Cloud, Heroku, Digital Ocean, etc): Local

Database

  • System (MongoDB or Postgres): MongoDB
  • Database version: 4.2.19
  • Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): Local

Logs

getFileLocation return invalid URL

the getFileLocation function is missing the applicationId;

function getFileLocation(config, filename) {
return config.mount + '/files/' + config.applicationId + '/' + encodeURIComponent(filename);
}

please fix it

npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue

npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
[email protected] /home/pedro/bizzkuit_parse
└─┬ [email protected]
├─┬ [email protected]
│ ├── [email protected]
│ ├─┬ [email protected]
│ │ └─┬ [email protected]
│ │ ├── [email protected]
│ │ └── [email protected]
│ └── [email protected]
└── [email protected]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.