Git Product home page Git Product logo

linkding-on-fly's Introduction

linkding on fly

๐Ÿ”– Run the self-hosted bookmark service linkding on fly.io. Automatically backup the bookmark database to Backblaze B2 with litestream.

Pricing

Assuming one 256MB VM and a 3GB volume, this setup fits within Fly's free tier. 1 Backups with Backblaze B2 are free as well. 2

Prerequisites

Instructions below assume that you have cloned this repository to your local computer:

git clone https://github.com/fspoettel/linkding-on-fly.git && cd linkding-on-fly

Litestream - Create Backblaze B2 Bucket and Application Key

Log into Backblaze B2 and create a bucket. Once created, you will see the bucket's name and endpoint. You will use these later to populate LITESTREAM_REPLICA_BUCKET and LITESTREAM_REPLICA_ENDPOINT in the fly.toml configuration.

Next, create an application key for the bucket. Once created, you will see the keyID and applicationKey. You will add these later to Fly's secret store, save them for step 3 below.

Note
If you want to use another storage provider, check litestream's "Replica Guides" section and adjust the config as needed.

Usage

  1. Login to flyctl:

    flyctl auth login
  2. Generate fly app and create the fly.toml:

    Alternative: Generating from template

    You can generate the fly.toml from the template provided in this repository.

    1. Install envsubst if you don't have it already:

      # macOS
      brew install gettext
    2. Copy the .env.sample file to .env, fill in the values and source them:

      cp .env.sample .env
      # vim .env
      source .env
    3. Generate the fly.toml from the template:

      envsubst < templates/fly.toml > fly.toml
    4. Proceed to step 3.

    # Generate the initial fly.toml
    # When asked, don't setup Postgres or Redis.
    flyctl launch

    Next, open the fly.toml and add the following env and mounts sections (populating LITESTREAM_REPLICA_ENDPOINT and LITESTREAM_REPLICA_BUCKET):

    [env]
      # linkding's internal port, should be 8080 on fly.
      LD_SERVER_PORT="8080"
      # Path to linkding's sqlite database.
      DB_PATH="/etc/linkding/data/db.sqlite3"
      # B2 replica path.
      LITESTREAM_REPLICA_PATH="linkding_replica.sqlite3"
      # B2 endpoint.
      LITESTREAM_REPLICA_ENDPOINT="<Backblaze B2 endpoint>"
      # B2 bucket name.
      LITESTREAM_REPLICA_BUCKET="<Backblaze B2 bucket name>"
    
    [mounts]
      source="linkding_data"
      destination="/etc/linkding/data"
  3. Add the Backblaze application key to fly's secret store

    flyctl secrets set LITESTREAM_ACCESS_KEY_ID="<keyId>" LITESTREAM_SECRET_ACCESS_KEY="<applicationKey>"
  4. Create a persistent volume to store the linkding application data:

    # List available regions via: flyctl platform regions
    flyctl volumes create linkding_data --region <region code> --size 1

    Note
    Fly's free tier includes 3GB of storage across your VMs. Since linkding is very light on storage, a 1GB volume will be more than enough for most use cases. It's possible to change volume size later. A how-to can be found in the "Verify Backups / Scale Persistent Volume" section below.

  5. Add the linkding superuser credentials to fly's secret store:

    flyctl secrets set LD_SUPERUSER_NAME="<username>" LD_SUPERUSER_PASSWORD="<password>"
  6. Deploy linkding to fly:

    flyctl deploy

    Note
    The Dockerfile contains overridable build arguments: ALPINE_IMAGE_TAG, LINKDING_IMAGE_TAG and LITESTREAM_VERSION which can overridden by passing them to flyctl deploy like --build-arg LITESTREAM_VERSION=v0.3.11 etc.

That's it! ๐Ÿš€ - If all goes well, you can now access linkding by running flyctl open. You should see the linkding login page and be able to log in with the superuser credentials you set in step 5.

If you wish, you can configure a custom domain for your install.

Verify the Installation

  • You should be able to log into your linkding instance.
  • There should be an initial replica of your database in your B2 bucket.
  • Your user data should survive a restart of the VM.

Verify Backups / Scale Persistent Volume

Litestream continuously backs up your database by persisting its WAL to the Backblaze B2 bucket, once per second.

There are two ways to verify these backups:

  1. Run the docker image locally or on a second VM. Verify the DB restores correctly.
  2. Swap the fly volume for a new one and verify the DB restores correctly.

We will focus on 2 as it simulates an actual data loss scenario. This procedure can also be used to scale your volume to a different size.

Start by making a manual backup of your data:

  1. SSH into the VM and copy the DB to a remote. If only you are using your instance, you can also export bookmarks as HTML.
  2. Make a snapshot of the B2 bucket in the B2 admin panel.

Now list all fly volumes and note the id of the linkding_data volume. Then, delete the volume:

flyctl volumes list
flyctl volumes delete <id>

This will result in a dead VM after a few seconds. Create a new linkding_data volume. Your application should automatically attempt to restart. If not, restart it manually.

When the application starts, you should see the successful restore in the logs:

[info] No database found, attempt to restore from a replica.
[info] Finished restoring the database.
[info] Starting litestream & linkding service.

Troubleshooting

Litestream is logging 403 errors

Check that your B2 secrets and environment variables are correct.

Fly ssh does not connect

Check the output of flyctl doctor, every line should be marked as PASSED. If Pinging WireGuard fails, try flyctl wireguard reset and flyctl agent restart.

Fly does not pull in the latest version of linkding

  • Override the Dockerfile build argument LINKDING_IMAGE_TAG: flyctl deploy --build-arg LINKDING_IMAGE_TAG=<tag>
  • Run flyctl deploy with the --no-cache option.

Create a linkding superuser manually

If you have never used fly's SSH console before, begin by setting up fly's ssh-agent:

flyctl ssh establish

# use agent if possible, otherwise follow on-screen instructions.
flyctl ssh issue --agent

Then, run flyctl ssh console to get an interactive shell in your running container. You can now create a superuser by running the createsuperuser command and entering a password.

cd /etc/linkding
python manage.py createsuperuser --username=<your_username> --email=<your_email>
exit

Footnotes

  1. Otherwise the VM is ~$2 per month. $0.15/GB per month for the persistent volume. โ†ฉ

  2. The first 10GB are free, then $0.005 per GB. โ†ฉ

  3. https://fly.io/docs/getting-started/installing-flyctl/ โ†ฉ

linkding-on-fly's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

linkding-on-fly's Issues

[Request] Can you provide steps for fl0.com

I tried to use fly, but it is asking me to add card details even for 256MB machine,

so I am trying to imitate the same steps on fl0 - https://www.fl0.com
by setting the same environment variables in the fl0 machine on website, and using Dockerfile
but for some reason, it finishes the build but fails to deploy with Status Reason: CrashLoopBackoff

Could you please help? I have not used Dockerfile before so I am not sure why it is crashing
Thank you

Dockerfile
ARG APP_NAME=linkding
ARG ALPINE_IMAGE_TAG=3.14
ARG LINKDING_IMAGE_TAG=latest

FROM docker.io/alpine:$ALPINE_IMAGE_TAG as builder
ARG APP_NAME
ENV APP_NAME=$APP_NAME

ARG LITESTREAM_VERSION=v0.3.11
# Download the static build of Litestream directly into the path & make it executable.
# This is done in the builder and copied as the chmod doubles the size.
ADD https://github.com/benbjohnson/litestream/releases/download/$LITESTREAM_VERSION/litestream-$LITESTREAM_VERSION-linux-amd64.tar.gz /tmp/litestream.tar.gz
RUN tar -C /usr/local/bin -xzf /tmp/litestream.tar.gz

# Pull linkding docker image.
FROM docker.io/sissbruecker/linkding:$LINKDING_IMAGE_TAG
ARG APP_NAME
ENV APP_NAME=$APP_NAME
VOLUME /etc/linkding
VOLUME /etc/linkding/data
ENV DB_PATH="/etc/linkding/data/db.sqlite3"
ENV LD_SERVER_PORT="9090"
ENV LITESTREAM_REPLICA_PATH="linkding_replica.sqlite3"
ENV LITESTREAM_REPLICA_BUCKET="somename"
ENV LITESTREAM_REPLICA_ENDPOINT="someurl.backblazeb2.com"
# Copy Litestream from builder.
COPY --from=builder /usr/local/bin/litestream /usr/local/bin/litestream

# Copy Litestream configuration file.
COPY etc/litestream.yml /etc/litestream.yml

# Copy custom uwsgi. This allows to run with 256MB RAM.
COPY uwsgi.ini /etc/linkding/uwsgi.ini

# Copy startup script and make it executable.
COPY scripts/run.sh /scripts/run.sh
RUN chmod +x /scripts/run.sh

# Litestream spawns linkding's webserver as subprocess.
CMD ["/scripts/run.sh"]

Failure on fly.io

Hi, followed the guide exactly. After I deploy to fly.io and go to live logs i see that;

2024-03-27T14:54:52.830 app[3287965c063418] ... [info] ERROR Error: failed to spawn command: /scripts/run.sh: No such file or directory (os error 2)

Experience running on fly.io with 256MB

With 256MB RAM, there are frequent Out Of Memory (OOM) crashes. The idle RAM is about 220 MB.

Without further configuration, import via html file is okay as long as 300 bookmarks per file. More than that and it crashed with OOM.

For the mass import, I run the linkding in local docker and do the import. Then backup sqlite file (according to the documentation) and replace the sqlite file on fly.io. I have to restart the fly.io. The existing user and user from local import both works.

I followed the fly.io documentation to enable swap and it reduced idle RAM to 180MB.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.