Using Angular 9 - I get this error:

    ERROR in node_modules/etebase/dist/lib/Crypto.d.ts:3:31 - error TS2307: Cannot find module 'react-native-sodium'.
    
    3 import type rnsodiumType from "react-native-sodium";
                                    ~~~~~~~~~~~~~~~~~~~~~

So do I need to install react-native-sodium even if I dont use any of it?

[Delete, pls >.>]

I'm getting a 413 Payload Too Large error when trying to create an Item with binary content that's 6MB. I haven't tested where the cutoff is yet, but I was hoping there was some general advice on how to handle large files.

I need to be able to handle single files that are up to 1GB or so of binary data.

Should I zip the file up and break it into separate Items? Or is there some way to handle this with Etebase that I'm not seeing/understanding.

Thanks!

Trying to use the etebase library in an Angular project initialized with strict option fails. Steps to reproduce:

1. Create minimal project:

$ cd /tmp
$ ng new --defaults --strict test
CREATE test/README.md (1013 bytes)
...
CREATE test/e2e/src/app.po.ts (274 bytes)
✔ Packages installed successfully.
    Successfully initialized git.
$ cd test
$

2. Install and use etebase.

$ npm install --save etebase
...
+ [email protected]
added 11 packages from 13 contributors and audited 1626 packages in 14.594s
...
$ $EDITOR src/app/app.component.ts
...
$ git diff src/app/app.component.ts
diff --git a/src/app/app.component.ts b/src/app/app.component.ts
index 29d65ec..bdd1e6a 100644
--- a/src/app/app.component.ts
+++ b/src/app/app.component.ts
@@ -1,4 +1,5 @@
 import { Component } from '@angular/core';
+import { Account } from 'etebase';

 @Component({
   selector: 'app-root',
@@ -7,4 +8,7 @@ import { Component } from '@angular/core';
 })
 export class AppComponent {
   title = 'test';
+  public test(): void {
+    Account.login("username", "password", "url");
+  }
 }
$

3. Running test now fails:

$ npm run test -- --watch=false

> [email protected] test /tmp/test
> ng test "--watch=false"
...
✔ Browser application bundle generation complete.

Error: node_modules/etebase/dist/lib/Crypto.d.ts:1:21 - error TS7016: Could not find a declaration file for module 'libsodium-wrappers'. '/tmp/test/node_modules/libsodium-wrappers/dist/modules/libsodium-wrappers.js' implicitly has an 'any' type.
  Try `npm i --save-dev @types/libsodium-wrappers` if it exists or add a new declaration (.d.ts) file containing `declare module 'libsodium-wrappers';`

1 import _sodium from "libsodium-wrappers";
                      ~~~~~~~~~~~~~~~~~~~~


Error: node_modules/etebase/dist/lib/Helpers.d.ts:1:21 - error TS7016: Could not find a declaration file for module 'libsodium-wrappers'. '/tmp/test/node_modules/libsodium-wrappers/dist/modules/libsodium-wrappers.js' implicitly has an 'any' type.
  Try `npm i --save-dev @types/libsodium-wrappers` if it exists or add a new declaration (.d.ts) file containing `declare module 'libsodium-wrappers';`

1 import _sodium from "libsodium-wrappers";
                      ~~~~~~~~~~~~~~~~~~~~


Error: node_modules/etebase/dist/lib/OnlineManagers.d.ts:1:17 - error TS7016: Could not find a declaration file for module 'urijs'. '/tmp/test/node_modules/urijs/src/URI.js' implicitly has an 'any' type.
  Try `npm i --save-dev @types/urijs` if it exists or add a new declaration (.d.ts) file containing `declare module 'urijs';`

1 import URI from "urijs";
                  ~~~~~~~



npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] test: `ng test "--watch=false"`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] test script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     $HOME/.npm/_logs/2021-03-03T20_38_58_942Z-debug.log
$

4. Installing the missing type packages makes the errors a bit less obscure, but does not fix the underlying problem:

$ npm i --save-dev @types/libsodium-wrappers
...
+ @types/[email protected]
added 1 package from 1 contributor, removed 1 package and audited 1627 packages in 15.381s
...
$ npm i --save-dev @types/urijs
...
+ @types/[email protected]
added 1 package from 10 contributors and audited 1628 packages in 14.163s
...
$ npm run test -- --watch=false
...
✔ Browser application bundle generation complete.

Error: node_modules/etebase/dist/lib/Crypto.d.ts:1:8 - error TS1192: Module '"/tmp/test/node_modules/@types/libsodium-wrappers/index"' has no default export.

1 import _sodium from "libsodium-wrappers";
         ~~~~~~~


Error: node_modules/etebase/dist/lib/Helpers.d.ts:1:8 - error TS1192: Module '"/tmp/test/node_modules/@types/libsodium-wrappers/index"' has no default export.

1 import _sodium from "libsodium-wrappers";
         ~~~~~~~


Error: node_modules/etebase/dist/lib/OnlineManagers.d.ts:1:8 - error TS1259: Module '"/tmp/test/node_modules/@types/urijs/index"' can only be default-imported using the 'allowSyntheticDefaultImports' flag

1 import URI from "urijs";
         ~~~

  node_modules/@types/urijs/index.d.ts:26:1
    26 export = URI;
       ~~~~~~~~~~~~~
    This module is declared with using 'export =', and can only be used with a default import when using the 'allowSyntheticDefaultImports' flag.



npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] test: `ng test "--watch=false"`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] test script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     $HOME/.npm/_logs/2021-03-03T20_40_06_933Z-debug.log
$

If the project is created with just ng new --defaults test then no errors are generated. The main difference between strict and unstrict is the following difference in tsconfig.json:

--- test/tsconfig.json  2021-03-03 21:57:54.304950915 +0100
+++ test.strict/tsconfig.json   2021-03-03 21:59:09.016755082 +0100
@@ -4,6 +4,10 @@
   "compilerOptions": {
     "baseUrl": "./",
     "outDir": "./dist/out-tsc",
+    "forceConsistentCasingInFileNames": true,
+    "strict": true,
+    "noImplicitReturns": true,
+    "noFallthroughCasesInSwitch": true,
     "sourceMap": true,
     "declaration": false,
     "downlevelIteration": true,
@@ -18,6 +22,9 @@
     ]
   },
   "angularCompilerOptions": {
-    "enableI18nLegacyMessageIdFormat": false
+    "enableI18nLegacyMessageIdFormat": false,
+    "strictInjectionParameters": true,
+    "strictInputAccessModifiers": true,
+    "strictTemplates": true
   }
 }

Testing with a project created with without --strict but just adding "strict": true, to compilerOptions in tsconfig.json also produces the errors, thus compilerOptions.strict: true is the triggering factor.

Hello.
etebase-js at this point exposes an ES module entrypoint, but that isn't enough for use with import maps. If I understood correctly, the import maps get hung up on the crypto import of lib-sodium. I'm not sure how to fix this, but I believe it has something to do with export maps which explicitly state entyrpoints for submodules.

Thanks

https://github.com/WICG/import-maps

https://docs.skypack.dev/package-authors/package-checks#export-map

Hi,

I am evaluating etebase as a possible backend for something I am working on. Congrats for creating all this by the way as it looks super cool :-)

I managed to deploy my own server instance locally which seems to work fine. I can connect to admin, create new users, then log them in and create a calendar with etebase-web.

When I test etebase-js with that instance I get the following results:

Test Suites: 3 failed, 2 passed, 5 total
Tests: 25 failed, 1 skipped, 10 passed, 36 total
Snapshots: 0 total
Time: 5.38s
Ran all test suites.

The server logs are:

INFO: 127.0.0.1:50912 - "POST /api/v1/authentication/signup/ HTTP/1.1" 409 Conflict
INFO: 127.0.0.1:50913 - "POST /api/v1/authentication/signup/ HTTP/1.1" 409 Conflict
INFO: 127.0.0.1:50914 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50915 - "GET /api/v1/authentication/is_etebase/ HTTP/1.1" 200 OK
INFO: 127.0.0.1:50916 - "GET /api/api/v1/authentication/is_etebase/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50917 - "POST /api/api/v1/authentication/login_challenge/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50918 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50919 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50920 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50921 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50922 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50923 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50924 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50925 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50926 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50927 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50928 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50929 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50930 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50931 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50932 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50933 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50934 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50935 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50936 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50937 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50938 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50939 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50940 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50941 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50942 - "POST /api/v1/test/authentication/reset/ HTTP/1.1" 404 Not Found

Can anyone shed some light upon this issue?

You need to run the server in debug mode for the tests to work.

Closing as the question was answered.

Originally posted by @tasn in #39 (comment)

I did not find the usual button to reopen the issue on #39 so I created a new one. When running the server in debug mode I still get failures:

Test Suites: 3 failed, 2 passed, 5 total
Tests: 17 failed, 1 skipped, 18 passed, 36 total
Snapshots: 0 total
Time: 6.365s
Ran all test suites.
error Command failed with exit code 1.

Most errors are about trying to mkdir the home directory ( which already exists ):

500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
Traceback (most recent call last):
File "FILTERED/etebase/etebase/.venv/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 396, in run_asgi
result = await app(self.scope, self.receive, self.send)
[...]
File "/usr/local/Cellar/[email protected]/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/usr/local/Cellar/[email protected]/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/usr/local/Cellar/[email protected]/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
[Previous line repeated 5 more times]
File "/usr/local/Cellar/[email protected]/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 225, in makedirs
mkdir(name, mode)
OSError: [Errno 45] Operation not supported: <MY_HOME_DIRECTORY>

It can significantly reduce the bundle size. Here's how:
jedisct1/libsodium.js#156 (comment)

This one is tricky and I don't have time to make a project that reproduces it, but wanted to note it here just in case anyone has some bright ideas.

When calling login or signup from the top App component of my app, everything works fine, but when I call them from components further down the tree, the ready function in Crypto.ts returns false or never returns. If I comment out all the ready checks, everything works fine.

I'm not sure of an elegant way to handle this and if there would be downstream effects to removing this call for React Native. Or possibly this is a problem with my own setup. Although I did remove all but three components in my component tree in order to find this function as the culprit. So it was as barebones as my app could get but of course there are other libraries being loaded that may be conflicting somehow.

Here's the error I received immediately after installing react-native-etebase:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':react-native-sodium:mergeDebugNativeLibs'.
> A failure occurred while executing com.android.build.gradle.internal.tasks.MergeJavaResWorkAction
   > 2 files found with path 'lib/armeabi-v7a/libsodium.so'.
     If you are using jniLibs and CMake IMPORTED targets, see
     https://developer.android.com/r/tools/jniLibs-vs-imported-targets

The problem is explained in the link from the error, but it ultimately appears to be the react-native-sodium dependency that's at fault.

I was able to get it working by editing the react-native-sodium grade.build file like this:

    sourceSets {
        main {
//            jniLibs.srcDirs = ['./lib']
        }
    }

But of course this will be overwritten whenever the package is updated or reinstalled. I don't know enough about Gradle to overwrite this in my project's gradle.build, so if anyone knows how to do that, I think it would be helpful.

This could potentially happen with other packages as well, not just react-native-sodium. Here's how other people have solved the issue, but the solutions didn't work for me:

https://stackoverflow.com/questions/62088079/error-when-building-project-with-ndk-support-after-updating-to-android-studio-4

If you setContent with repetitive data on a collection, an error is thrown when trying to get that content again.

 TypeError: Cannot read properties of undefined (reading 'length')
    at reduce (/etebase-js/src/Crypto.ts:36:49)
    at Array.reduce (<anonymous>)
    at concatArrayBuffersArrays (/etebase-js/src/Crypto.ts:36:26)
    ...

It looks like the index list and/or chunks created here are the issue (the indices are pointers at the original chunks array, but the chunks array is mutated here, so the pointers are incorrect?)

The following tests illustrate the issue (large files are fine, small files are fine, but repetitive data (that would result in chunks that would be deduplicated) throws the error

describe.only("chunking files", () => {
  it("Duplicate Chunks", async () => {
    const collectionManager = etebase.getCollectionManager();
    const col = await collectionManager.create(colType, {}, "");

    const buf = randomBytesDeterministic(10 * 1024, new Uint8Array(32)); // 10kb of psuedorandom data
    const content = JSON.stringify([buf, buf, buf, buf]);

    await col.setContent(content);

    await collectionManager.transaction(col);
    const decryptedContent = await col.getContent(); // <---
    /*
  TypeError: Cannot read properties of undefined (reading 'length')
    at reduce (/etebase-js/src/Crypto.ts:36:49)
    at Array.reduce (<anonymous>)
    at concatArrayBuffersArrays (/etebase-js/src/Crypto.ts:36:26)
    ...
  */

    const out = to_string(decryptedContent);
    expect(out).toEqual(content);
  });

  it("Regular Chunks", async () => {
    const collectionManager = etebase.getCollectionManager();
    const col = await collectionManager.create(colType, {}, "");

    const buf = randomBytesDeterministic(100 * 1024, new Uint8Array(32));
    const content = JSON.stringify(buf);

    await col.setContent(content);

    await collectionManager.transaction(col);
    const decryptedContent = await col.getContent();

    const out = to_string(decryptedContent);
    expect(out).toEqual(content);
  });
  
  it("Small file, no chunks", async () => {
    const collectionManager = etebase.getCollectionManager();
    const col = await collectionManager.create(colType, {}, "");

    const buf = randomBytesDeterministic(10, new Uint8Array(32));
    const content = JSON.stringify(buf);

    await col.setContent(content);

    await collectionManager.transaction(col);
    const decryptedContent = await col.getContent();

    const out = to_string(decryptedContent);
    expect(out).toEqual(content);
  });
});

I'll do a PR with a fix shortly

I'm trying to create an Item with 1MB of binary content in React Native, and there are two methods running over 13 seconds each:

CryptoManager.encrypt

and

CryptoManager.calculateMac

I believe it's due to the sodium methods being called:

sodium.crypto_aead_xchacha20poly1305_ietf_encrypt

sodium.crypto_generichash

I'm attempting fix this with a PR, by detecting the React Native environment similar to how the library already detects it here:

But I'm unsure which methods from react-native-sodium to use and how to implement them. I tried swapping in rnsodium.crypto_secretbox_easy for crypto_aead_xchacha20poly1305_ietf_encrypt, but getting an unhelpful FAILURE error, even after converting all Uint8Arrays to base64.

Any suggestions here? Is this the right approach? Can you think of what the issue might be with using crypto_secretbox_easy?

It looks like the error comes from this line in react-native-sodium:

https://github.com/lyubo/react-native-sodium/blob/41c8a0e785ec136f487ab85a27202919a0b2222c/ios/RCTSodium/RCTSodium.m#L138

Which points towards me not converting the data properly.

Any help is appreciated, thanks!

Hi! Where is the Local Cache stored?

Automatically translated.