Git Product home page Git Product logo

hoodik's Introduction

Hoodik

Hoodik

Hoodik is a lightweight, secure, and self-hosted cloud storage solution. It's designed and built with Rust and Vue, focusing on end-to-end encryption that shields your data from prying eyes and hackers. Hoodik supports file uploading and downloading, making it easy for you to share files with other users. The simple and intuitive web interface makes file management a breeze. Thanks to Rust's focus on speed and performance, your data transfers will be lightning fast.

Hoodik

Features

Hoodik is designed with a central goal: to store your files securely. Files are encrypted and decrypted on your device during download and upload.

To ensure end-to-end encryption remains fast and efficient while enabling file sharing among application users, a hybrid encryption approach is used:

  • Upon registration, each user receives a generated RSA key pair.
  • We store your keys, encrypted with your passphrase, with your information on the server (choose a robust passphrase).
  • Files are encrypted with a randomly generated AES key during upload.
  • The file's AES key is encrypted with the user's public key and stored in the database, linking the user and the file.

We've created a mechanism to enable search through your files without leaving plaintext metadata in the database:

  • Any searchable data about the file (like name, metadata, etc.) is tokenized.
  • The resulting tokens are hashed and stored in the database as file tokens.
  • When you perform a search, we perform the same operation on your search query and transmit it to the server.
  • The server matches tokens to the query and fetches the corresponding files from the database.

The process for publicly sharing links to files protects the actual file's AES key:

  • A random AES key is generated for the link.
  • The file metadata is encrypted with the link key.
  • The original file's AES key is encrypted with the link key.
  • The link key is encrypted with the owner's RSA key (enabling the owner to retrieve the key anytime).
  • When someone clicks the link, the link key will either be included in the link https://.../links/{id}#link-key, or they need to input it in the client app before starting the download.
  • On the download request, the link key is sent to the server where the actual file key is decrypted in memory.
  • The file content is streamed for download and is decrypted in memory.

For RSA, we employ 2048-bit PKCS#1 keys, and for AES, we use AEAD Ascon-128a. You can find detailed usage of the crypto in the cryptfns workspace member. We chose this encryption setup because it offers impressive performance results.

Files are stored in chunks and each chunk is encrypted individually. This enables concurrent uploading and downloading of chunks to offset encryption overhead.

*Just to note, in the case of downloading publicly linked files, the shared key only unlocks the link. The actual file key is encrypted within the link and decrypts the file as it downloads. This design ensures the person receiving the shared link never gets the file key.

*We provide the option of server-based encryption and decryption as a fallback solution if the client runs on a device with limited computing power. However, this feature is expected to be used rarely.

Installing via Docker

While the application itself can handle incoming traffic, we recommend using a reverse proxy, such as Nginx Proxy Manager, for optimal results.

docker run --name hoodik -it -d \
  -e DATA_DIR='/data' \
  -e APP_URL='https://my-app.local' \
  -e SSL_CERT_FILE='/data/my-cert-file.crt.pem' \
  -e SSL_KEY_FILE='/data/my-key-file.key.pem' \
  -e MAILER_TYPE='smtp' \
  -e SMTP_ADDRESS='smtp.gmail.com' \
  -e SMTP_USERNAME='[email protected]' \
  -e SMTP_PASSWORD='google-account-app-password' \
  -e SMTP_PORT='465' \
  -e SMTP_DEFAULT_FROM='Hoodik Drive <[email protected]>' \
  --volume "$(pwd)/data:/data" \
  -p 4554:5443 \
  hudik/hoodik:latest

Database

Hoodik supports either Sqlite or Postgres databases. Sqlite is enabled by default and it creates a database file in your DATA_DIR right out of the box. If you prefer an external Postgres database, simply provide the DATABASE_URL for your Postgres connection.

Please take note: The databases used with Hoodik are not interchangeable. Should you decide to switch from one database type to another after you've begun using the application, this could result in the loss of all your data.

Configuration

For a more detailed application configuration, please review our environment example.

Contributors

hoodik's People

Contributors

htunlogic avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

hoodik's Issues

Enhance session handling

Enhance the session handling while user is logged in.
Also, add triggers to move the user from main page when session expires or some authentication error happens.

IP is not logged correctly into the activity logs

Hoodik doesn't properly log the IPs of the users on the User Sessions

Logs when Docker Container isn't deattatched:
image

Web app:
image

Some details to note:
I've installed Hoodik via Docker, and used Nginx Proxy Manager to proxy everything.

Also, one question, is it possible to disable HTTPS on Hoodik, so connections can be accepted by Nginx Proxy manager and therefore use a Lets Encrypt cert issued by Nginx proxy manager to encrypt outside connections?

Error setting up hoodik

Starting Hoodik v1.0.0 on 0.0.0.0:5443
-- Using data_dir: /data
-- Using ssl cert: /data/hoodik.crt.pem
-- Using ssl key: /data/hoodik.key.pem
-- RUST_LOG=Some("hoodik=debug,auth=debug,error=debug,entity=debug,storage=debug,context=debug,util=debug,cryptfns=debug,actix_web=debug")
------------------------------------------
Error: SmtpError(lettre::transport::smtp::Error { kind: Connection, source: Failure(Ssl(Error { code: ErrorCode(1), cause: Some(Ssl(ErrorStack([Error { code: 167772427, library: "SSL routines", function: "ssl3_get_record", reason: "wrong version number", file: "../ssl/record/ssl3_record.c", line: 354 }]))) }, X509VerifyResult { code: 0, error: "ok" })) })

Rename files

Add option to rename files, this rename shouldn't affect the already created link

Multi-bytes characters in dir/file names are not encrypted/decrypted correctly

When I uploaded/created directories/files with Japanese characters, I found that names are broken and displayed in wrong ASCII characters.

In the screenshot below, I've created a directory named "かきくけこ" under the directory "あいうえお" (in Japanese characters "Hiragana").
However, Hoodik shows that directory as "BDFHJ / KMOQS" instead of "あいうえお / かきくけこ".

image

Environment

  • macOS Sonoma (M2 chip)
  • Docker (latest v1.1.1)
  • Database: reproduced in both PostgreSQL and SQLite
  • Browser: macOS Vivaldi / Android Chrome

Push container images to GHCR

Dockerhub has pull rate limits and have also shown in recent history to be pretty hostile to open source developers.

Please consider publishing the container image to Github's container registry (GHCR) as well. Publishing the container image to GHCR also is integrated with Github and is shown under the packages section on your repo's landing page.

Write proper readme

Write proper readme for the application as a whole.

Readme should describe how to install the application and use it.

It should point to the environment variables and possible configurations.

It should also give out some basic information about the code structure and what each crate or package does.

httputil: ReverseProxy read error during body copy: net/http: server replied with more than declared Content-Length; truncated

I am currently using a tailscale reverse proxy to serve up hoodik (https://tailscale.com/kb/1312/serve); and I am seeing this error when I am trying to download content (including image previews).

httputil: ReverseProxy read error during body copy: net/http: server replied with more than declared Content-Length; truncated

Debugging on hoodik itself didnt show any errors or warnings; and that is the only error I get in the proxy logs. The result however is messages like:

image

and

image

Removing the reverse proxy fixes it - however I then run into issues with certificate validity and I cant seem to get hoodik to accept certs from letsencrypt.. but I would really prefer to get the reverse proxy working anyway.

Anyone seen this before?

Add public links

Add ability for user to share his files with a public link.

User would select one or more files and create a shared link, the link would be displayed to the user together with a password, the password would unlock each of the files shared. The password would be AES encrypted key.

Links should be viewed as directory that has N files in it.

Structure to consider:

  • Links
    • id
    • name
    • createdAt
  • LinkFiles
    • id
    • downloads
    • encrypted_metadata
    • file_id

Links can be revoked and once they are the LinkFiles entities are deleted in order for the encrypted_metadata locked with link key is destroyed.

Clicking the link would open up the frontend page where user would have an option to enter a password (or the password could be integrated in the link itself), he would get a view with all the files shared with him (as if he opened a directory).

Download button should be next to each file and user should be able to download each file.

Alternatively, the link could be generated in form of a quick link and it would lead to backend directly, where the password would be integrated in the link, backend would grab all the files and start decrypting them and pushing into a single .zip file and then it should stream it directly to the user.

Rework the config crate

Config crate is currently a hot pile and should be redone to be more readable and modular.

2fa error

Hi, any help 2fa not working using yubico

[2024-03-23T13:10:18Z DEBUG actix_web::middleware::logger] Error in response: Validation(ValidationErrors { errors: {"secret": ValidationError { field: "secret", errors: ["invalid_otp_token"] }} })

Add support for LDAP/OpenID

Hoodik looks to be exactly what I was looking for, a simple and fast way for me to upload and share files with a small group of people (and vice versa), other solutions feel overkill.

However, I currently have a big issue with using it since I host it behind Authelia with LLDAP, but it can't integrate with Hoodik as it is, so for my users, it means they need to authenticate with Authelia and then authenticate extra with their Hoodik account. I would love to see integration with this so I could have SSO working properly. Thanks!

Add OIDC support

It would be great if I could use Authelia for my family to sign in with instead of managing accounts here as well.

Thanks!

Details view for files

Enable details view for files that will display some general information about the file that we have for it.

Fix authentication when downloading and uploading

Downloading and uploading are long running processes which may need special tokens that enable them to do that, currently, after the session is refreshed download or upload will fail with 401 because the token that its being used has expired.

Ideally, we would create a token for a specific file which would be valid only to interact with that file (upload and download).

The token would be generated before the upload/download starts and would be sent in the worker together with the file. It would be locked only to a single path and method and would have some longer lived life (24-48h)

Create Android App for file syncing

I think this one would make a really big difference in adoption is the ability to sync Android folders.

There are so little offer for cloud storage written in Rust, hope this one takes off.

Congrats on the work so far.

Uploading any ZIP file will cause in a Chunk size mismatch

After installing Hoodik locally using Docker, I can't seem to upload any ZIP files. PNG files do work fine, though.

Chrome version: 118.0.5993.118
OS version: Windows 10 Pro 22H2 19045.3570

docker-compose file:

version: '3'
services:
  hoodik:
    container_name: hoodik
    image: hudik/hoodik:latest
    environment:
      DATA_DIR: /data
      APP_URL: https://localhost:4554/
      SSL_CERT_FILE: /data/my-cert-file.crt.pem
      SSL_KEY_FILE: /data/my-key-file.key.pem
      MAILER_TYPE: smtp
      SMTP_ADDRESS: <redacted>
      SMTP_USERNAME: <redacted>
      SMTP_PASSWORD: <redacted>
      SMTP_PORT: 465
      SMTP_DEFAULT_FROM: 'Hoodik Drive <<redacted>>'
    volumes:
      - ./data:/data
    ports:
      - "4554:5443"

Console log:

[+] Running 1/0
  Container hoodik  Running                                                                                                                                                                                             0.0s 
Attaching to hoodik
hoodik  | [2023-11-03T10:02:46Z INFO  actix_web::middleware::logger] 172.24.0.1 "POST /api/auth/login HTTP/2.0" 200 4532 "https://localhost:4554/auth/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36" 0.225599
hoodik  | [2023-11-03T10:02:46Z INFO  actix_web::middleware::logger] 172.24.0.1 "GET /api/storage HTTP/2.0" 200 28 "https://localhost:4554/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36" 0.002213
hoodik  | [2023-11-03T10:02:46Z INFO  actix_web::middleware::logger] 172.24.0.1 "GET /api/storage?dirs_only=true&is_owner=true HTTP/2.0" 200 28 "https://localhost:4554/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36" 0.002049
hoodik  | [2023-11-03T10:02:46Z INFO  actix_web::middleware::logger] 172.24.0.1 "POST /api/storage/stats HTTP/2.0" 200 40 "https://localhost:4554/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36" 0.003152
hoodik  | [2023-11-03T10:02:49Z DEBUG actix_web::middleware::logger] Error in response: NotFound("file_not_found")
hoodik  | [2023-11-03T10:02:49Z INFO  actix_web::middleware::logger] 172.24.0.1 "GET /api/storage/1d62e60a9d7b85ac1dbbe521841f037ecdbf70cd61bc750785d3ab86cd0a837e/name-hash HTTP/2.0" 404 43 "https://localhost:4554/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36" 0.001861
hoodik  | [2023-11-03T10:02:49Z INFO  actix_web::middleware::logger] 172.24.0.1 "POST /api/storage HTTP/2.0" 200 884 "https://localhost:4554/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36" 0.058607
hoodik  | [2023-11-03T10:02:50Z INFO  actix_web::middleware::logger] 172.24.0.1 "POST /api/storage/35062160-ec66-404a-abf0-c67ca7a64b61?chunk=4&checksum=3149&checksum_function=crc16 HTTP/2.0" 200 884 "https://localhost:4554/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36" 0.611922
hoodik  | [2023-11-03T10:02:56Z DEBUG actix_web::middleware::logger] Error in response: Validation(ValidationErrors { errors: {"chunk": ValidationError { field: "chunk", errors: ["chunk_size_mismatch: expected max 3614306.5, but received 4194320"] }} })
hoodik  | [2023-11-03T10:02:56Z INFO  actix_web::middleware::logger] 172.24.0.1 "POST /api/storage/35062160-ec66-404a-abf0-c67ca7a64b61?chunk=3&checksum=7c63&checksum_function=crc16 HTTP/2.0" 422 158 "https://localhost:4554/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36" 5.788935
hoodik  | [2023-11-03T10:02:56Z DEBUG actix_web::middleware::logger] Error in response: Validation(ValidationErrors { errors: {"chunk": ValidationError { field: "chunk", errors: ["chunk_size_mismatch: expected max 3614306.5, but received 4194320"] }} })
hoodik  | [2023-11-03T10:02:56Z INFO  actix_web::middleware::logger] 172.24.0.1 "POST /api/storage/35062160-ec66-404a-abf0-c67ca7a64b61?chunk=2&checksum=e96e&checksum_function=crc16 HTTP/2.0" 422 158 "https://localhost:4554/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36" 5.848923
hoodik  | [2023-11-03T10:02:56Z DEBUG actix_web::middleware::logger] Error in response: Validation(ValidationErrors { errors: {"chunk": ValidationError { field: "chunk", errors: ["chunk_size_mismatch: expected max 3614306.5, but received 4194320"] }} })
hoodik  | [2023-11-03T10:02:56Z INFO  actix_web::middleware::logger] 172.24.0.1 "POST /api/storage/35062160-ec66-404a-abf0-c67ca7a64b61?chunk=0&checksum=e92c&checksum_function=crc16 HTTP/2.0" 422 158 "https://localhost:4554/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36" 6.009717
hoodik  | [2023-11-03T10:02:56Z DEBUG actix_web::middleware::logger] Error in response: Validation(ValidationErrors { errors: {"chunk": ValidationError { field: "chunk", errors: ["chunk_size_mismatch: expected max 3614306.5, but received 4194320"] }} })
hoodik  | [2023-11-03T10:02:56Z INFO  actix_web::middleware::logger] 172.24.0.1 "POST /api/storage/35062160-ec66-404a-abf0-c67ca7a64b61?chunk=1&checksum=a63e&checksum_function=crc16 HTTP/2.0" 422 158 "https://localhost:4554/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36" 5.969061```

Add admins and admins dashboard

Admin should have a dashboard where he can see all the users in the system, the number of files they have and the size those files are taking.

He should also have insight into how much space total is used by all the files stored on the server

Admin: Invitation page

where other users can be invited to register. This could be used in case the admin has closed the registrations and didn't define any specific emails or patterns.

Enhance preview view

Preview view should be made better, to enable proper image scaling and zoom for images.
Also, it should support preview of other common files, like PDF-s.

Enable user to change his password

User should be able to change his password.

In order to do that, user must be logged in with unlocked private key. When the password is changed, the key locked again with the new password must be sent with the new password and the user object will be updated with new credentials.

In case user forgets his password and wants to reset it, he can be offered to login with private key directly, and then he can be sent to the same screen where he will be able to change his password.

Private key signature needs to be verified before its being locked and swapped.

All the files and folders will stay as they were, and nothing else should change.

In case user loses his private key and password, there is nothing we can do, all the files will remain locked forever.

TLDR: User must have either valid password or private key to access his account

Re-send verification email

Add an ability to newly registered user to re-send the verification email in case he missed the already sent one.

That could be re-used as a call to the backend from either not logged-in page, or a quick button for logged-in user that sets the email automatically.

This could be also expanded into settings where admin can decide if the user can log in without activated email or not.

Admin should be verified right away when registered.

View authenticated sessions

Enable user to view his authenticated sessions and history of accessing the application.
User should be able to terminate any active session from himself.

Add sharing capability for files

FIles can be shared between users on the platform.

User can select file that he wants to share, and then click share button and insert users email, if user exists on the platform UserFiles entity should be created for that other user and he should get a notification that the file was shared with him.

Shares will be displayed on a separate page, there should be "Shared with me" and "My shares" where user would be able to see files shared with him and also files he shared.

Shares don't support sharing folders, only files can be selected to be shared.

Note: If user doesn't exist on the platform, user should be prompted that it cannot be directly shared and should be taken to sending a public link to the user that is not on the platform.

Removing the unraid-template completly from the repo

Hey, i have seen, that the unraid template is still visible in the git history of this project and it has some (i guess) sensitive informtion in it.
I´d suggest to run
git filter-branch --tree-filter 'rm -f unraid-template.xml' HEAD
followed by
git push origin --force --all
to completly remove that file from the git history and making it (afaik) unrecoverable.
Those commands should go through all git commits and remove that file completely.

The process is described here https://www.linkedin.com/pulse/how-remove-file-from-github-repository-has-already-been-daniel-awde

Upload Speed seems quite slow

Hey, i am not sure if this is 100% a hoodik issue, but it seems like it. If i try to upload a file, i get only 1/4 of the max speed of my connection. For my internet connection its 40Mbits, but i only get max 10Mbits in hoodik. If i try to connect to hoodik via local connection, without going through the internet, i get 25Mbits instead of 1Gbit which is my max local networking.

So in both cases its exactly 1/4 which cant be just a random coincedence imo. I also tested with nextcloud for example or filerun and got the max speed both times

Fix light mode

Fix light mode in theme everywhere to look presentable as the dark mode is.

Auth Question/Enhancement

How is authentication and user creation handled within Hoodik? Is it possible to disable account creation and instead just use OAuth or SAML instead? I tried to look through the docs and I didn't see anything.

Add admin settings

Settings should be stored directly on the disk in form of a JSON file so it could be edited without the application running.

Currently, settings should include:

  • Registration

    • registration allowed (should always allow the first user)
    • allowed emails
    • allowed email domains
  • ???

Add email service

Add email service that will handle any kind of email being sent through it.

For start it must support email confirmation after the registration.

The service needs to be able to function with multiple email providers and methods, initial method should be a simple SMTP.

The service should be attached to the context and available through the application to be used.

Add validation for file size

Currently, malicious API user could have a file of lets say 100MB, say to the create_file that the file is 10kb and has 10 chunks. Then proceed and upload 10 x 10mb chunks.

The validation needs to calculate the maximum size of the chunk (size / chunks) and then monitor that uploading chunk is not greater then the size defined.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.