Hi,
I'm trying to create a file, which I want to stream a 115MB file from S3 into but failing horribly. Could you please help me in the right direction?

My code looks like this
The directory does exist and is writable
ufs looks like

ufs
    .use(vol)
    .use(fs)

console.log('test createwritestream ufs')
ufs.createWriteStream('/tmp/ufs.tgz')
if(ufs.existsSync('/tmp/ufs.tgz')) {
  console.log('/tmp/ufs.tgz create success')
}
else {
  console.log('/tmp/ufs.tgz create failed')
}

console.log('test createwritestream fs')
fs.createWriteStream('/tmp/fs.tgz')
if(fs.existsSync('/tmp/fs.tgz')) {
  console.log('/tmp/fs.tgz create success')
}
else {
  console.log('/tmp/ufs.tgz create failed')
}

The result is

2019-12-20T06:49:43.334Z	b1d017eb-a1ea-49d4-8339-c30d07f779a1	INFO	{ Error: ENOENT: no such file or directory, stat '/tmp/ufs.tgz'
    at Object.statSync (fs.js:855:3)
    at Union.createWriteStream (/var/task/node_modules/unionfs/lib/union.js:340:28)
    at Promise (/var/task/src/global/run.js:200:13)
    at new Promise (<anonymous>)
    at downloadFileFromS3 (/var/task/src/global/run.js:181:12)
    at Runtime.module.exports.execute [as handler] (/var/task/src/global/run.js:278:9)
  errno: -2,
  syscall: 'stat',
  code: 'ENOENT',
  path: '/tmp/ufs.tgz' }

The version I am using is:

"unionfs": {
      "version": "4.2.1",
      "resolved": "https://registry.npmjs.org/unionfs/-/unionfs-4.2.1.tgz",
      "integrity": "sha512-QvdUyG3+m3Nh8YQfCV1mzOvBp7izXoEpGMZzqupKrzFY7cWuayKr8JRdN7CwNcNVjXYR2AZBeY+hStPKFUAT+w==",
      "requires": {
        "fs-monkey": "^0.3.3"
      }
    },

Thanks!

Whilst there is a licence defined in the README, this isn't reflected in package.json, which causes some issues with automatic licence check tools. Please could you add a suitable entry?

I don't always use fs, but when I do, I prefer fs/promises. 😄

import fs from "fs";
import { Union } from "unionfs";
import { IFS } from "unionfs/lib/fs";

type FS_Promises = (typeof fs)["promises"];

type UseFSPromises = FS_Promises & {
  use(filesystem: FS_Promises): UseFSPromises
};

export function UnionFSPromises() : UseFSPromises
{
  const unionFS = new Union;
  const promises = unionFS.promises as UseFSPromises;
  promises.use = function(filesystem: FS_Promises) : UseFSPromises {
    unionFS.use({ promises: filesystem } as IFS);
    return this;
  }

  return promises;
}

On registry https://registry.npmjs.org/, the "latest" version (v6.24.1) of dependency babel-preset-es2015 has the following deprecation notice:

🙌 Thanks for using Babel: we recommend using babel-preset-env now: please read babeljs.io/env to update!

Marking the latest version of an npm package as deprecated results in the entire package being considered deprecated, so contact the package author you think this is a mistake.

Please take the actions necessary to rename or substitute this deprecated package and commit to your base branch. If you wish to ignore this deprecation warning and continue using babel-preset-es2015 as-is, please add it to your ignoreDeps array in Renovate config before closing this issue, otherwise another issue will be recreated the next time Renovate runs.

Affected package file(s): package.json

I was trying to use this library as a filesystem for webpack, but the lack of support for functions like mkdirp makes me unable to do so. There's no point to me to use unionfs with memfs+fs if it doesn't allow me to at least use all of memfs functions: https://github.com/streamich/memfs/blob/master/docs/api-status.md

When I try to use a memfs Volume I get the following errors:

With strict mode:

Argument of type 'Volume' is not assignable to parameter of type 'IFS'.
  Types of property 'ftruncateSync' are incompatible.
    Type '(fd: number, len?: number | undefined) => void' is not assignable to type '(fd: number, len?: number | null | undefined) => void'.
      Types of parameters 'len' and 'len' are incompatible.
        Type 'number | null | undefined' is not assignable to type 'number | undefined'.
          Type 'null' is not assignable to type 'number | undefined'.ts(2345)

Without strict mode:

Argument of type 'Volume' is not assignable to parameter of type 'IFS'.
  The types returned by 'statSync(...)' are incompatible between these types.
    Type 'Stats<number>' is missing the following properties from type 'BigIntStats': atimeNs, mtimeNs, ctimeNs, birthtimeNsts(2345)

This regression was introduced in 4.2.0
4.1.0 and prior work as expected.
Node version is v10.12.0

Sample code:

const realFs = require('fs');
const virtualFs = new (require('memfs').Volume)();
const unionFs = new (require('unionfs').Union)();

virtualFs.fromJSON({
	'does-not-exist-in-real-fs/file.txt': ''
});

unionFs.use(virtualFs);
unionFs.use(realFs);

try { console.log('realFs',    realFs.readdirSync('does-not-exist-in-real-fs'));    } catch(e) {console.error(e.message)}
try { console.log('virtualFs', virtualFs.readdirSync('does-not-exist-in-real-fs')); } catch(e) {console.error(e.message)}
try { console.log('unionFs',   unionFs.readdirSync('does-not-exist-in-real-fs'));   } catch(e) {console.error(e.message)}

Expected result:

Directory can be read in virtual fs, union fs but not real fs.

ENOENT: no such file or directory, scandir 'does-not-exist-in-real-fs'
virtualFs [ 'file.txt' ]
unionFs [ 'file.txt' ]

Actual result:

Directory can be read in virtual fs, but not union fs and real fs.

ENOENT: no such file or directory, scandir 'does-not-exist-in-real-fs'
virtualFs [ 'file.txt' ]
ENOENT: no such file or directory, scandir 'does-not-exist-in-real-fs'

Unionfs and memfs are great. Admirably, they stick to the FS api pretty closely.

I am looking to support some more advanced use cases involving FSs that are more varied and deserve their own scope. They will lean quite heavily on both libraries and I'm hoping this ticket can lead to some suggestions about how this could be best achieved. Prior art

I hope to support the following use cases:

• a proxy wrapper for a volume to ensure directories exist before files are created
• compare 2 volumes and create a diff
• copy contents of one volume onto another
• support other FS-like volumes such as ZipFS etc

It's in very early stages but these are the exports i'm hoping to see be made available to progress. If I'm missing something please do comment.

PRs:

• method metadatas
• IFS

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you can benefit from your bug fixes and new features again.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can fix this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here are some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

Invalid npm token.

The npm token configured in the NPM_TOKEN environment variable must be a valid token allowing to publish to the registry https://registry.npmjs.org/.

If you are using Two Factor Authentication for your account, set its level to "Authorization only" in your account settings. semantic-release cannot publish with the default "
Authorization and writes" level.

Please make sure to set the NPM_TOKEN environment variable in your CI with the exact value of the npm token.

Good luck with your project ✨

Your semantic-release bot 📦🚀

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

Invalid npm token.

The npm token configured in the NPM_TOKEN environment variable must be a valid token allowing to publish to the registry https://registry.npmjs.org/.

If you are using Two-Factor Authentication, make configure the auth-only level is supported. semantic-release cannot publish with the default auth-and-writes level.

Please make sure to set the NPM_TOKEN environment variable in your CI with the exact value of the npm token.

Good luck with your project ✨

Your semantic-release bot 📦🚀

On registry https://registry.yarnpkg.com/, the "latest" version (v6.24.1) of dependency babel-preset-es2015 has the following deprecation notice:

🙌 Thanks for using Babel: we recommend using babel-preset-env now: please read babeljs.io/env to update!

Marking the latest version of an npm package as deprecated results in the entire package being considered deprecated, so contact the package author you think this is a mistake.

Affected package file(s): package.json

If you don't care about this, you can close this issue and not be warned about babel-preset-es2015's deprecation again. If you would like to completely disable all future deprecation warnings then add the following to your config:

"suppressNotifications": ["deprecationWarningIssues"]

Possibly related to streamich/memfs#116

Not sure if this is a bug or a feature request. Currently, (at least) fs.watch() calls do not fall through to the other file systems that are joint in the unionfs. Was going to say it would be good if on detection of a ENOENT error on the first file system, if unionfs tried the other filesystems in the union for the file, but that will cause issues when you then create the folder/file on the top unionfs file system.

const patchFs = require('fs-monkey').patchFs;
const Volume = require('memfs').Volume;
const ufs = require('unionfs').ufs;
const fs = require('fs');
const path = require('path');
 
const vol = new Volume();
const fs2 = Object.assign({}, fs);
 
ufs.use(fs2).use(vol);
 
patchFs(ufs);
 
var dir = ufs.readdirSync('./');
console.log(dir);

fs.watch('./', { recurse: true }, (eventType, filename) => console.log('watch event', eventType, filename));

The readdir method fails on an empty directory, if it is not present on the first filesystem.

Failing test case:

import { Volume } from 'memfs';

it('does not throw when directory is empty and first fs fails', () => {
  const vol = Volume.fromJSON({});
  const vol2 = Volume.fromJSON({
    '/foo': null,
  });

  const ufs = new Union();
  ufs.use(vol as any);
  ufs.use(vol2 as any);
  expect(() => ufs.readdirSync('/foo')).not.toThrow();
});

However, the test passes if the fs order is swapped:

it('does not throw when directory is empty and second fs fails', () => {
  const vol = Volume.fromJSON({
    '/foo': null,
  });
  const vol2 = Volume.fromJSON({});

  const ufs = new Union();
  ufs.use(vol as any);
  ufs.use(vol2 as any);
  expect(() => ufs.readdirSync('/foo')).not.toThrow();
});

It also passes if the directory is not empty:

it('does not throw when directory is not empty and first fs fails', () => {
  const vol = Volume.fromJSON({});
  const vol2 = Volume.fromJSON({
    '/foo/child': null,
  });

  const ufs = new Union();
  ufs.use(vol as any);
  ufs.use(vol2 as any);
  expect(() => ufs.readdirSync('/foo')).not.toThrow();
});

If you try and do (admittedly weird) things with graceful-fs, it expects fs (which can be replaced by unionfs) to have ReadStream and WriteStream.

Can you expose those on unionfs?

https://github.com/isaacs/node-graceful-fs/blob/master/graceful-fs.js#L156

My code is as follow

const  fs = require('fs');
const unionfs = require('unionfs').default;
const _fs = unionfs.use(fs);
console.log(_fs.constants);
// undefined;

I can not figure out why the fs.constants losted, I have to patch on it like this

jest.mock('fs', () => {
  const fs = jest.requireActual('fs');
  const unionfs = require('unionfs').default;
  const _fs = unionfs.use(fs);
  _fs.constants = fs.constants;
  return _fs;
});

The latest release 4.5.0 is missing all lib files.

The only files present in this version of the package are:

• LICENSE
• package.json
• README.md

nice. i am considering using this. is it possible to dump and persist this to a real disk from in-memory file system?

We used Travis for running CI which included doing automatic releases using semantic-release; that has stopped working at some point, so I've migrated us to GitHub Actions but need the NPM_TOKEN setup so that we can resume releases.

This has to be done by @streamich - could we also rename the default branch to main at the same time?

Union methods frequently use this in their implementations. Calling them with a different context breaks them:

const fs = require('fs');
const ufs = new (require('unionfs').Union);

[].find(fs.existsSync); // undefined
[].find(ufs.existsSync); // TypeError: Cannot read property 'fss' of undefined

fs.existsSync.call(null, 'foo') // false
ufs.existsSync.call(null, 'foo') // TypeError: Cannot read property 'fss' of null

While this might be a questionable approach, this behaviour breaks unionfs compatibility with native fs. In my case this error originated from a different package in test environment where unionfs was mocking real fs.

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: Cannot find preset's package (github>whitesource/merge-confidence:beta)

Please upgrade fs-monkey dependency to the latest one (v1.0.3).

I encountered an issue when rmSync function is not available when using memfs with uninonfs. After some investigation I found out that the version of fs-monkey is different and doesn't include new implemented methods.

I wanted to create PR myself but due to permissions this issue was created instead.

I was wondering if unionfs supports the fs promise api that memfs supports?

node version: v10.15.3

Reproduce:

var { Volume } = require('memfs')
var { Union } = require('unionfs')
var mkdirp = require('mkdirp')
var fs = require('fs')

var mfs = Volume.fromJSON({ '/some/path': 'content' })
// throw "file already exists" Error, as expected
mkdirp.sync('/some/path', { fs: mfs })

var ufs1 = new Union().use(mfs).use(fs)
// throw "file already exists" Error, as expected
mkdirp.sync('/some/path', { fs: ufs1 })

var ufs2 = new Union().use(fs).use(mfs)
// throw "Maximum call stack size exceeded" Error, not expected
mkdirp.sync('/some/path', { fs: ufs2 })

/**
 * Error:
{ RangeError: Maximum call stack size exceeded
    at Object.uvException (internal/errors.js:250:21)
    at handleErrorFromBinding (fs.js:112:24)
    at Object.mkdirSync (fs.js:752:3)
    at Union.syncMethod (/Users/c4605/workSpace/pryo/fs-tree/node_modules/unionfs/lib/union.js:308:35)
    at Union.this_1.(anonymous function) [as mkdirSync] (/Users/c4605/workSpace/pryo/fs-tree/node_modules/unionfs/lib/union.js:60:34)
    at sync (/Users/c4605/workSpace/pryo/fs-tree/node_modules/mkdirp/index.js:71:13)
    at sync (/Users/c4605/workSpace/pryo/fs-tree/node_modules/mkdirp/index.js:77:24)
    at sync (/Users/c4605/workSpace/pryo/fs-tree/node_modules/mkdirp/index.js:78:17)
    at sync (/Users/c4605/workSpace/pryo/fs-tree/node_modules/mkdirp/index.js:78:17)
    at sync (/Users/c4605/workSpace/pryo/fs-tree/node_modules/mkdirp/index.js:78:17)
  prev:
   { RangeError: Maximum call stack size exceeded
       at Arguments.slice (<anonymous>)
       at extend (/Users/c4605/workSpace/pryo/fs-tree/node_modules/fast-extend/index.js:11:47)
       at getMkdirOptions (/Users/c4605/workSpace/pryo/fs-tree/node_modules/memfs/lib/volume.js:248:16)
       at Volume.mkdirSync (/Users/c4605/workSpace/pryo/fs-tree/node_modules/memfs/lib/volume.js:1525:20)
       at Union.syncMethod (/Users/c4605/workSpace/pryo/fs-tree/node_modules/unionfs/lib/union.js:308:35)
       at Union.this_1.(anonymous function) [as mkdirSync] (/Users/c4605/workSpace/pryo/fs-tree/node_modules/unionfs/lib/union.js:60:34)
       at sync (/Users/c4605/workSpace/pryo/fs-tree/node_modules/mkdirp/index.js:71:13)
       at sync (/Users/c4605/workSpace/pryo/fs-tree/node_modules/mkdirp/index.js:77:24)
       at sync (/Users/c4605/workSpace/pryo/fs-tree/node_modules/mkdirp/index.js:78:17)
       at sync (/Users/c4605/workSpace/pryo/fs-tree/node_modules/mkdirp/index.js:78:17) prev: null } }
 */

The readdir method is not working properly since it fails if one of the file systems registered on the union file system fails.

Environment:

• Node: v10.16.3
• unionfs: 0b5f0368af01679b31c0ed1017716bf396c63440

Sample:

This is a sample of what I'm trying to do:

const realFs = require('fs');
const virtualFs = new (require('memfs').Volume)();
const unionFs = new (require('unionfs').Union)();

virtualFs.fromJSON({
	'does-not-exist-in-real-fs/file.txt': ''
});

unionFs.use(virtualFs);
unionFs.use(realFs);

Current behavior:

If readdir has an error in one of the file systems registered on unionfs, it will have an error also in unionfs.

virtualFs.readdir('does-not-exist-in-real-fs', callback)); // called without error
unionFs.readdir('does-not-exist-in-real-fs', callback)); // called with error
realFs.readdir('does-not-exist-in-real-fs', callback)); // called with error

Expected result:

The directory can be read in the virtual file system, union file system but not the real file system.

virtualFs.readdir('does-not-exist-in-real-fs', callback)); // called without error
unionFs.readdir('does-not-exist-in-real-fs', callback)); // called without error
realFs.readdir('does-not-exist-in-real-fs', callback)); // called with error

Description

Copying a file from an existing location in one fs module into an existing location in another fs module fails

Steps to reproduce

With file

const fs = require('fs')
const path = require('path')

const { vol } = require('memfs')
const { ufs } = require('unionfs')

ufs.use(vol).use(fs)

const file1 = 'testfile.txt'
const file2 = 'somefolder/testfile.txt'

try {
    vol.mkdirSync(path.dirname(file2), { recursive: true })
    fs.writeFileSync(file1, '')
    ufs.copyFileSync(file1, file2)
} finally {
    if (fs.existsSync(file1)) fs.unlinkSync(file1)
}

Results

./node_modules/unionfs/lib/union.js:636
                    throw err;
                    ^

Error: ENOENT: no such file or directory, open 'testfile.txt'
    at createError (./node_modules/memfs/lib/volume.js:130:17)
    at Volume.openFile (./node_modules/memfs/lib/volume.js:818:15)
    at Volume.openBase (./node_modules/memfs/lib/volume.js:822:25)
    at Volume.openSync (./node_modules/memfs/lib/volume.js:833:21)
    at Volume.readFileBase (./node_modules/memfs/lib/volume.js:919:23)
    at Volume.readFileSync (./node_modules/memfs/lib/volume.js:934:21)
    at Volume.copyFileBase (./node_modules/memfs/lib/volume.js:1113:24)
    at Volume.copyFileSync (./node_modules/memfs/lib/volume.js:1127:21)
    at Union.syncMethod (./node_modules/unionfs/lib/union.js:630:35)
    at Union.this_1.<computed> [as copyFileSync] (./node_modules/unionfs/lib/union.js:525:34) {
  code: 'ENOENT',
  prev: Error: ENOENT: no such file or directory, copyfile 'testfile.txt' -> 'somefolder/testfile.txt'
      at Object.copyFileSync (fs.js:1801:3)
      at Union.syncMethod (./node_modules/unionfs/lib/union.js:630:35)
      at Union.this_1.<computed> [as copyFileSync] (./node_modules/unionfs/lib/union.js:525:34)
      at Object.<anonymous> (./example.js:16:9)
      at Module._compile (internal/modules/cjs/loader.js:936:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:947:10)
      at Module.load (internal/modules/cjs/loader.js:790:32)
      at Function.Module._load (internal/modules/cjs/loader.js:703:12)
      at Function.Module.runMain (internal/modules/cjs/loader.js:999:10)
      at internal/main/run_main_module.js:17:11 {
    errno: -2,
    syscall: 'copyfile',
    code: 'ENOENT',
    path: 'testfile.txt',
    dest: 'somefolder/testfile.txt',
    prev: null
  }
}

It seems that this library is coding by typescript, so why not exist a type file?

Hi @streamich
Great collection of fs libraries! Super useful and well made!

Quick Q: I want to use unionfs to combine an in-memory fs and the official (disk) fs, however, I only want to allow a particular folder to be accessible via the official fs. How would you recommend I do something like that with your tools?

hey @streamich

this wont work as expected, what wrong here. it searched the PRs and Issues, and Streams should be resolved somehow (it probably uses all your fs related packages):

//patching everything
const { patchRequire } = require("fs-monkey");
const { ufs } = require("unionfs");
const { vol } = require("memfs");
const mock = require("mock-require");
const realFs = require("fs");

vol.fromJSON(
  {
    "./dir": null,
    "./file.json": "{\"foo\":\"bar\"}"
  },
  __dirname
);

ufs.use(vol).use(realFs);

patchRequire(ufs);
mock("fs", ufs);


//the code inside the app
const fs = require("fs");

// a file from the real FS with or without UFS
console.log("realfs buffer", fs.readFileSync(__dirname + "/package.json")); //buffer
console.log("realfs ufs stream", fs.createReadStream(__dirname + "/package.json")); //undefined
console.log("realfs stream", realFs.createReadStream(__dirname + "/package.json")); //stream
console.log(require(__dirname + "/package.json")); //works

// a file from the memory fs with or without UFS
console.log("memfs buffer", fs.readFileSync(__dirname + "/file.json")); //buffer
console.log("memfs stream", vol.createReadStream(__dirname + "/file.json")); //stream
console.log(require(__dirname + "/file.json")); //works
console.log("memfs stream through ufs", fs.createReadStream(__dirname + "/file.json")); //undefined + ENOENT error

it would be awesome if you can help me?

If im understanding the code correctly it should work (its testing all underlying fs's for createReadStream (continues if one errors, that might be the problem?)

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• chore(deps): lock file maintenance

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update dependency ts-jest to v29.1.2
• chore(deps): update dependency @types/node to v20
• chore(deps): update semantic-release monorepo (major) (@semantic-release/npm, semantic-release)
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

real file

../test/456 => 999999999999999\n

=========

import { fs as memfs, vol, Volume, createFsFromVolume, IFs } from 'memfs';
import { ufs } from 'unionfs';
import * as fsNode from 'fs';

let vol2 = Volume.fromJSON({
	'../test': null,
}, );

let memfs2 = createFsFromVolume(vol2);

export const fs: IFs = ufs
	.use(fsNode)
	.use(memfs2)
;

console.log(1, fs.readFileSync('../test/456').toString());

fs.appendFileSync('../test/456', '123');

console.log(2, fs.readFileSync('../test/456').toString());

expect is 999999999999999\n123 but got 123 only at step 2

Testing a module that uses readline with a unionfs createReadStream does not work.

Error encountered:

(node:17269) UnhandledPromiseRejectionWarning: TypeError: input.on is not a function
    at new Interface (readline.js:184:11)
    at Object.createInterface (readline.js:63:10)

Test file:

const fs = require('fs');     
const readline = require('readline');
const mfs = require('memfs').fs;
const ufs = require('unionfs').ufs;
const util = require('util'); 
                              
const contents = `this        
is a                          
test                          
file                          
`;                            
                              
mfs.mkdirSync('/tmp');        
                              
ufs.use(fs).use(mfs);         
                              
const onLine = (type) => (line) => console.log(type + ': ' + line);
const onEnd = (type) => () => console.log(type + ':end');
                              
const systems = { fs, mfs, ufs };
                              
Object.keys(systems).forEach(async (key) => {
  const tfs = systems[key];   
  tfs.writeFileSync('/tmp/testfile', contents);
  const writeFile = util.promisify(tfs.writeFile);
  await writeFile('/tmp/testfile1', contents);
                              
  const rl = readline.createInterface({
    input: tfs.createReadStream('/tmp/testfile')
  });                         
                              
  rl.on('line', onLine(key)); 
  rl.on('end', onEnd(key));   
});

Actual file under test:
https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/lib/file.js (https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/src/lib/file.ts)

Using files:
https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/lib/file.test.js (https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/src/lib/file.test.ts)
https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/test/lib/unionfs.js (https://gitlab.com/bytesnz/keeps-on-ticking/blob/6-add-function-testing/src/test/lib/unionfs.ts)

Hi! Thanks for this very useful collection of modules :)

I'm using unionfs to run some fs intensive tests. I have basically a layer shared among all tests (real fs), and a layer for test-specific input files, and I'd like to have a third layer to collect all the test write operations output. Or at least, make sure that any write operations only affects the test-specific layer, not the shared one.

Unfortunately, it seems that write operations will target the first fs where the parent directory exists:

import { Volume } from 'memfs'
import { Union } from 'unionfs'

const shared = Volume.fromJSON({
  '/app/foo': 'foo',
})

const specific = new Volume()

const fs = new Union()
  .use(shared)
  .use(specific)

fs.writeFileSync('/app/bar', 'bar0', 'utf8')

console.log(
  shared.existsSync('/app/bar'), // expected: false, actual: true
  specific.existsSync('/app/bar'), // espected: true, actual: false
)

specific.mkdirSync('/app')

fs.writeFileSync('/app/bar', 'bar1', 'utf8')

console.log(
  shared.existsSync('/app/bar'), // expected: false, actual: true
  specific.existsSync('/app/bar'), // expected: true, actual: true
)

One way to achieve this would be to have a way to specify the fs to use for write operations, and automatically creates directories in the write fs when they exists in previous fs volumes. Maybe something like this:

const fs = new Union()
  .use(shared)
  .use(specific)
  .writeTo(specific)

// considering above fixtures
fs.writeFileSync('/app/baz', ...) // OK
fs.writeFileSync('/other/foo', ...) // ENOENT

I think this also relates to #181, where a behaviour similar to what I'm describing for directories would be expected when appending to files.

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

Invalid npm token.

The npm token configured in the NPM_TOKEN environment variable must be a valid token allowing to publish to the registry https://registry.npmjs.org/.

If you are using Two-Factor Authentication, make configure the auth-only level is supported. semantic-release cannot publish with the default auth-and-writes level.

Please make sure to set the NPM_TOKEN environment variable in your CI with the exact value of the npm token.

Good luck with your project ✨

Your semantic-release bot 📦🚀

Look at this:

#8 (comment)

I thought the unionfs is trying to operate on underlay file systems in the same order as the use, but after tested it I found it's in reversed order.

It's better to be mentioned in the document.

const {Volume} = require('memfs');
const {ufs} = require('unionfs');

const vol1 = Volume.fromJSON({'/test': '1'});
const vol2 = Volume.fromJSON({'/test': '2'});

ufs
    .use(vol1)
    .use(vol2);

console.log(ufs.readFileSync('/test', 'utf8'));  // 2, NOT 1

What if more than one filesystem have the same path? Is it first-come first-serve?

Thanks!

Description

Creating a folder and then writing directly to a file in the folder is a normal usecase that unionfs does not support.

Steps to reproduce

With file

const fs = require('fs')

const { vol } = require('memfs')
const { ufs } = require('unionfs')

ufs.use(vol).use(fs)

const runExampleWith = fsInstance => {
    try {
        fsInstance.mkdirSync('example', { recursive: true })
        const stream = fsInstance.createWriteStream('example/file.txt')
        stream.write('test', () => stream.end())
    } finally {
        fsInstance.rmdirSync('example', { recursive: true })
    }
}

Using the default node fs works

runExampleWith(fs)

Using memfs directly works

runExampleWith(vol)

Using the combined ufs fails with

runExampleWith(ufs)
// Error: ENOENT: no such file or directory, open 'example/file.txt'

Cause

Union fs checks file exists before attempting to use the file systems

Suggestion

import {dirname} from 'path' 
// ...
fs.statSync(dirname(path)); //we simply stat first to exit early for mocked fs'es

Related

• #384