Git Product home page Git Product logo

precis's Introduction

The Precis logo... for now

Precis (properly Précis, pronounced "pray-see") is a extensibility-oriented RSS reader that can use LLMs to summarize and synthesize information from numerous different sources, with an emphasis on timely delivery of information via notifications.

The following components of the app are extensible:

  1. Summarization - LLMs including Ollama and OpenAI
  2. Content Retrieval - requests or playwright
  3. Notification - currently - matrix or slack, but support for other protocols such as discord or mattermost, or push-based services such as gotify or ntfy, or even a message bus such as kafka would be possible
  4. Storage - At this time, we support two reasonable embedded DBs - tinydb or lmdb - defaults to tinydb. You can add support for your database of choice if you can implement about 20 shared transactions.

The Summarization and Notification handlers also support a null handler that does nothing. Good for testing or if you don't care about notifications and summaries. The null handler is the default.

Precis also supports themes.

For an extended overview of why this project exists, see my blog.

Architecture

Precis is a FastAPI monolith that serves fully static pages styled by Tailwind CSS using DaisyUI components. It uses some query-parameter and redirect chicanery to fake interactivity. We'll probably add actual interactivity at some point.

Deployment and Development

Environment

Precis uses the following environment variables to define its instance. It is, therefore, possible to have multiple instances of Precis running at the same time, as long as you maintain separation between their storage and config directories.

These are expected to be set as environment variables.

CONFIG_DIR - the directory where config files live, if you use them
DATA_DIR - the directory where the database should live
RSS_BASE_URL - the base url (without trailing slash of the Precis application - used for linking)
PRECIS_STORAGE_HANDLER - the name of the storage handler to use.

Different features may require different environment variables. Developers should strive to set reasonable defaults and document the full range of them in the docker-compose.yml

Deployment via Docker-Compose:

The easiest way to run Precis is via docker-compose:

docker compose up

Feel free to edit the docker-compose.yml as needed to fit your environment.

Pre-Requisites

  • Python 3.11 (for dev, we recommend you use pyenv)
  • Node 20 or higher (use nvm)

Development Instructions

To install, create a fresh venv and then:

make dev

Then to develop, in one terminal start tailwind by doing make tw. Then, in other start the main app by doing make run.

Features

OPML Import/Export

Precis supports exporting your current set of feeds as OPML, as well as importing feeds from other OPML files. You can find options on the /feeds/ page, or use the CLI as described below.

A couple caveats:

  1. When importing an OPML file, if the feed already exists then it will be upserted. Any properties that differ from the OPML file will change to match.
  2. When importing an OPML file, Precis will take the first category as the category of the feed. You may always change the category at a later time.
  3. When importing and exporting, we the map the feed name, feed url, and category to the outline item text, xml_url, and categories[0] attributes, respectively. Please ensure that your input file uses the same conventions.

Backups

Precis supports exporting and importing point-in-time backups of the entire application state. You can find these options on the /about/ page, or use the CLI as described below. One of the design goals of this functionality is supporting the ability to combine two different Precis instances, so the default behavior of the import functionality is to upsert.

Users should be careful about potentially importing the same backup twice, and importing backups that have overlapping content may cause unexpected behavior.

Finally - the backups will contain any API keys or other secrets that you've configured Precis to use, so they should themselves be treated as secrets.

Composable Configurations

Precis supports reading configurations from YML files that are stored in the CONFIG_DIR as an alternative to manually configuring them in the UI. This lets you bootstrap your configuration and/or manage configurations in source control. Of course, even if you use composable configurations, you can still add additional feeds or change settings in the UI.

Composable configurations can be used to configure feeds, settings, and handlers, and each of these can be ingested from the source file using the cli. Below are links to the example configs off of which you can base your own, as well as the command to import those configurations in the CLI.

  • Settings are configured in ${DATA_DIR}/settings.yml and imported using precis load-settings
  • Handlers are configured in ${DATA_DIR}/handlers.yml and imported using precis load-handlers
  • Feeds are configured in ${DATA_DIR}/feeds.yml and imported using precis load-feeds

CLI

Precis includes a CLI tool that can be used to manage the application. Currently, it supports exporting and importing JSON backups and OPML files and loading Composable Configurations (as described above). It should be available if you start a shell in the docker container, or if you activate the virtualenv where Precis is installed. It respects the DATA_DIR and CONFIG_DIR that you configured, if any.

❯ precis --help
Usage: precis [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  backup         Write a json-format backup of the current Precis state...
  export-opml    Write a opml-format list of the feeds configured in...
  import-opml    Import an opml-formatted feed list into Precis
  load-feeds     Load feeds from a YML-formatted feeds.yml file in the...
  load-handlers  Load global handlers from a YML-formatted handlers.yml...
  load-settings  Load global settings from a YML-formatted settings.yml...
  restore        Restore a json-format backup of the Precis state

UI Tour

After initial onboarding, you'll be brought to the feeds page. The feeds page

You can then view the feed entries for each feed. The feed entries page

When you read a feed entry you'll get the full text of the article, as well a summary if you have a summarization handler configured. The read page

Global settings can be configured in the UI The settings page

Configuring individual handlers is as simple as following an JSON Schema spec. (At some point we'll make a real UI) The handler config page

precis's People

Contributors

leozqin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

b08240

precis's Issues

Validate rss feed on feed creation

INFO:     Polling feed 786cde34ad8bced802812fdb4610db00: Knock LA
rss-1  | INFO:     Retrieved poll state: None
rss-1  | ERROR:    Traceback (most recent call last):
rss-1  |   File "/usr/local/lib/python3.12/site-packages/fastapi_utils/tasks.py", line 64, in loop
rss-1  |     await func()  # type: ignore
rss-1  |     ^^^^^^^^^^^^
rss-1  |   File "/app/app/app.py", line 33, in poll_feeds
rss-1  |     await rss.check_feeds()
rss-1  |   File "/app/app/rss.py", line 90, in check_feeds
rss-1  |     await self._check_feed(feed=feed)
rss-1  |   File "/app/app/rss.py", line 67, in _check_feed
rss-1  |     earliest_init_entry = min([timegm(i.published_parsed) for i in entries])
rss-1  |                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
rss-1  | ValueError: min() iterable argument is empty

We also need to pop a UI element in the feed page when this happens

langchain summarization provider

This would be cool because it unlocks a whole world of llm providers, but I could never get it to work well with a modest self-hosted model

GHCR integration

When I add a tag, build and push a new image to GHCR. Set version and latest tag.

about page

  • version
  • storage handler
  • environment variables?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.