th3r00t / pyshelf Goto Github PK

View Code? Open in Web Editor NEW

519.0 24.0 32.0 109.38 MB

A simple terminal based ebook server

License: GNU General Public License v3.0

Python 7.20% JavaScript 87.13% HTML 5.66%

ebook-server python3 terminal-based hacktoberfest

pyshelf's Introduction

pyShelf 0.7.0

Terminal based ebook server. Open source & Lightweight.

Having used Calibre for hosting my eBook collection in the past, I found myself frustrated having to install X on my server, or manage my library externally, Thus I have decided to spin up my own.

You dont need an X server to host a website, or your Movie & Tv collection, so why should you need one to host ebooks?

Other solutions require you to have access to an X server to at the very least generate your book database, pyShelf doesnt. We aim to provide a fully featured ebook server with minimal requirements, and no reliance on X whatsoever.

Follow or influence development @

Discord

General Notes

You may need to click import books a second time if your import fails on the first attempt

0.7.0 Patch Notes.

New Features

Administration System
PDF Support

Current Features

Recursive Scanning
Django based frontend
Seach via author, title, & file name fields.
Download System
Automated Collections
- A work in progress, the collections are based on your folder structure.
User System
Per User Favorites
Expanded book information view
Websocket server
- currently only responds to ping, and importBooks, more responders are planned.
Full Docker integration.
On Demand Importing

Branch	Support	Feature set
Master	Bugs get priority	Most stable branch, may be behind in the core feature set
Development	Please report all bugs	Most active branch, this branch is a rolling release, containing the latest features. There will be bugs here hopefully nothing service killing
Others	Here there be dragons	These branches are used for day to day development, nothing here should be considered stable.

Currently Supported Formats

epub
mobi
pdf

Installation & Support Information

Installation

Docker

The official Docker image for pyShelf is pyshelf/pyshelf. The easiest way to get pyShelf running is through docker-compose. An example docker-compose.yml is included in the repo.

You'll need a .env file wich sets the LOCAL_BOOK_DIR variable, for example:

LOCAL_BOOK_DIR=/home/someone/books

The Docker image is still new, so there could still be some issues and missing features. Feel free to create a bug-issue when you encounter a bug. Development of the Docker image is discussed in #53 . Currently the database needs to be PostgreSQL with the account details shown in the example docker-compose.yml.

Default User/Pass for web interface: pyshelf/pyshelf

Self Hosted

This is targeted towards Network Administrators, and home enthusiasts whom I assume will know how to setup a Django app, and a PostgreSQL server. For those unfamiliar with the required setup please use the included docker-compose.yml

Pre-req Dependencies

gcc
python3
pip
postgresql

Once your database is ready very little is required to get the system up and running:

From the main directory

pip install -r requirements.txt

./configure

cd src/ && daphne frontend.asgi:application add -b 0.0.0.0 -p 8000 as required to specify which interface's and port to bind too

As of 0.6.0 Django is being served up via Daphne, and the static files are served up via whitenoise.

Import Books

The first step is to login, after logging in the button whill show your username, click on it once again, and a new menu will pop up with the option to logout, or import books.

In Progress

Organizational tools.

Improved cover image storage, and acquisition.

OPDS Support

Support for other formats

.mobi
.pdf
.cbz
.zip (Zipped book folders, is this a new idea? (Consider storing your library folders zipped and retrieving a book on demand))

Future Goals

Terminal Backend for catalogue maintenance

Development

pre-commit Before developing, run pre-commit install See the documentation for more information.
'Doxygen' Any changes to source should be documented and have run doxygen doxygen.conf prior to commiting.
'sem-ver' Before advancing version numbers be sure to set PROJECT_NUMBER in doxygen.conf accordingly.

pyshelf's People

Contributors

Stargazers

Watchers

pyshelf's Issues

Add Support for Other Architectures

ARM Support has been asked for on reddit. Will start working on this once i buy a Raspberry pi

Calibre support

it would be nice to be able to have a calibre plugin:

pull ebook from pyshelf
push ebook to pyshelf

calibre server may not be greate, but the user part is good for local desktop

[Feature] ebook friendly ui

It would be nice to have a separate wifi enabled ebook friendly page. It should be simple html supported by outdated browsers and contrast e-ink friendly design.

Website is down

https://pyshelf.com/

Is it permanent?

Ref. awesome-selfhosted/awesome-selfhosted#2266

Thanks

Features - github support

i would like to have the ability to add "github"library so user could sync there library to a public/private repository on github. so it could ease the sync library setup for backup and stuff

Django settings contain sensitive data and debug is default on

In src/frontend/settings.py I've found the following:

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = "@(9b9jslgg41u1u=mr)-2*-n2x0vef0zsy39*z@sz18&tvow18"

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = TEMPLATE_DEBUG = True
if DEBUG is True:
    from pudb.remote import set_trace
ALLOWED_HOSTS = CONFIG.allowed_hosts

I don't think the secret key shoud be visible here? DEBUG = TEMPLATE_DEBUG = True should better default to False

Book Tracking System

Design a system (flat file?) to check if files in book directory have already been processed

Allow user defined config variables as docker environment variables

It would be great to be able to set the config.json variables as docker environment variables when spinning up the container. Right now the only way to customize them is to map a customized config.json file into the container.

Value too long for type character results in failure during import books

During initial setup I select import and then, after several books have been processed, I get the following error.

...
...
...
2020-10-16 18:31:46.409 | INFO     | backend.lib.storage:insert_book:80 - American Exceptionalism and American Innocence
2020-10-16 18:31:46,480 ERROR    Error in connection handler
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/websockets/server.py", line 191, in handler
    await self.ws_handler(self, path)
  File "./backend/lib/pyShelf.py", line 84, in socketio
    await self.runImport()
  File "./backend/lib/pyShelf.py", line 69, in runImport
    Catalogue(self.config).import_books()
  File "./backend/lib/library.py", line 281, in import_books
    db.insert_book(book)
  File "./backend/lib/storage.py", line 86, in insert_book
    raise e
  File "./backend/lib/storage.py", line 65, in insert_book
    self.cursor.execute(
psycopg2.errors.StringDataRightTruncation: value too long for type character varying(255)
2020-10-16 18:31:46,491 WARNING  Executing <Task finished name='Task-163' coro=<WebSocketServerProtocol.handler() done, defined at /usr/local/lib/python3.8/dist-packages/websockets/server.py:118> result=None created at /usr/local/lib/python3.8/dist-packages/websockets/server.py:116> took 10.273 seconds

Binary image display in template

Having some trouble rendering the bin data stored in the books table, cover column. I have created a custom filter, bin_2_img, and it seems to return the binary I'm looking for. The template is including the the returned data, in the requisite img src tag. however the image is still not rendering.

Merits of db stored images

One should endeavor not to store images in a database, instead storing a path

Of this I am very aware, however for the time being the images are ideally originally stored in side each epubs file. Epub files are semi-analogous to zip files. In order to minimize repeatedly opening and querying the file each time the image is needed I chose to store it instead.

Future releases will most likely be changed to extract the image prior to storing the book and renaming it to match the book title, or file name then storing it in the directory of the book, and a link to the new file in the db instead.

Installation Options

Release 0.4.0 Requires an installer.

Page Count

Figure out how to get page count from epub. I am unsure as to if this is a standard count, or if it would be different depending on device.

More research is needed

Import books times out when executed from web ui

Seems the async operation is timing out before starting, this is a new bug that wasnt present the last time I installed pyshelf. As I have been unhappy with the current websocket implementation I may take this opportunity to revisit that as well. For now books imports only working via calling import books from the install directory.

Add ability to import books from frontend

Adding this functionality will allow the docker release to be updated in-situ. Currently it is not possible to update the library once deployed

Add ability to directly E-mail Mobi files to Kindle E-mail addresses

Calibre has the capability to send MOBI files directly to the Kindle by E-mailing to a kindle specific address. See here. It says that Gmail does not work, but I've been using it just fine.

Would be great if pyshelf supported this.

Add user

i would like to be able to have a main admin user and then from it add user with different privileges (and maybe api key would be nice)

Features - link pyshelf to pyshelf

would be nice to add a way to link multiple pyshelf,

so you will have access to the whole DB of every linked pyshelf, but locally download the book on request only

Features - attached ressources

Since i have a big technical ebook library(developper ebook, with code example) i would like to be able to attache a zip file to an ebook so when i read it i could host it and download the attachement from the pyshelf ui

This could bring pyshelf as a technical book library reference!!!

sqlite support ?

Any chance of sqlite support? It'd simplify a docker deploy as it'd be one less dependency.

Dynamically display books in html

Having now began the front-end interface I am looking for a way to display the books dynamically in HTML based on available screen size, and set the rest up in a pager system.

Normally I would do this easily in Django, or Flask however my intention is to avoid those as requirements, and use the built in http.server module.

Incorrect Website

Hey,

It seems like the pyShelf website redirects to the below URI and renders an unrelated website for "Soulwich: Asian sandwiches & more".

URI: http://162.255.85.132/?SID=rt6t135fm9bknvaptrrh6bf0dm

Image retrieval

Cover images are being acquired from a regex search of the epubs file contents. Cover could be attained by scraping the opf for the requisite meta data. If the current search does not render results, I am hooking in the DuckDuckGo search api and returning the first image result.

Linux Install importBooks Issue

The hashbang at the top of the importBooks file is currently just #!python

This doesn't play well with most linux systems as it expects a full path. Should be changed to something along the lines of #!/usr/bin/python so it looks like this and executes correctly.

See: This stackoverflow question