Git Product home page Git Product logo

fimfic2epub's Introduction

NPM

fimfic2epub logo

fimfic2epub

This is a tool to generate better EPUB ebooks from Fimfiction stories. It's also a Chrome/Firefox extension, replacing the default EPUB download option with this tool. If you're on an Arch Linux-based system you can install it from the Arch User Repository.

Screenshot of the web extension

Features

  • The generated ebook is in modern EPUB3 format with fallbacks for older EPUB2 reading systems
  • Improved styling and formatting of content compared to Fimfiction's export options
  • Cover image can be changed from an image file or url
  • Downloads and embeds artwork from the story inside the EPUB file, including YouTube thumbnails, for optimal offline reading and archiving (optional)
  • Rating, tags, status, story description and more info are available on the title page
  • The table of contents page includes chapter modification dates and word counts
  • Option to put all author notes in an index at the end of the ebook
  • Option to not add a title heading for chapters (in case the story has its own)
  • Tweak paragraph style from double-spaced to indented (similar to book typesetting, may not look good on every story)
  • Emoji, icon and webp support (webp images gets converted to png)
  • Calculate the Flesch reading ease value of the story
  • Customize metadata of the generated ebook, such as title, author, subjects and description
  • Command line tool with same features as the web extension

Demo

You can have a look at what a generated EPUB looks like here. It was generated from the story Summer Island.

Usage (web extension)

You can download the Chome extension from Chrome Web Store and Firefox Add-ons

Installation & usage (command line)

You can install the tool by running npm install -g fimfic2epub. On Arch Linux-based systems you can install it from the AUR like this: yay -S fimfic2epub (replace yay with your favorite AUR-helper)

You can then run the tool it like this:

$ fimfic2epub <story id/url> [<optional filename>]

By default the EPUB will be saved in the current working directory with the filename Author - Title.epub. Run fimfic2epub -h to see a list of all flags.

  Usage: fimfic2epub [options] <story> [filename]

  Tool to generate improved EPUB ebooks from Fimfiction stories

  Options:

    -V, --version                output the version number
    -d, --dir <path>             Directory to store ebook in. Is prepended to filename
    -t, --title <value>          Set the title of the story
    -a, --author <value>         Set the author of the story
    -T, --typogrify              Enable typographic fixes (smart quotes, dashes, ellipsis, ordinal)
    -c, --no-comments-link       Don't add link to online comments
    -H, --no-headings            Don't add headings to chapters (includes chapter title, duration and word count)
    -W, --no-chapter-word-count  Don't add word count to chapter headings
    -D, --no-chapter-duration    Don't add time to read to chapter headings
    -b, --no-bars                Don't add chapter bars to show reading progress
    -r, --no-reading-ease        Don't calculate Flesch reading ease
    -e, --no-external            Don't embed external resources, such as images (breaks EPUB spec)
    -n, --no-notes               Don't include author notes
    -i, --notes-index            Create an index with all author notes at the end of the ebook
    -p, --paragraphs <style>     Select a paragraph style <spaced|indented|indentedall|both> (default: spaced)
    -k, --kepubify               Add extra <span> elements for Kobo EPUB (KEPUB) format
    -j, --join-subjects          Join dc:subjects to a single value
    -w, --wpm <number>           Words per minute. Set to 0 to disable reading time estimations (default: 200)
    -C, --cover <url>            Set cover image url
    -h, --help                   output usage information

Examples

Download with automatic filename:
$ fimfic2epub 289663
$ fimfic2epub https://www.fimfiction.net/story/289663/summer-island

Download and save to a specified dir/filename:
$ fimfic2epub 289663 path/to/file.epub
$ fimfic2epub --dir path/to/my/dir 289663 ebook_%id%.epub # %id% gets replaced by the story id

Building

Make sure Node.js is installed. After you've cloned this repository, run npm install and npm run build to build it. This project uses gulp.

Development

When developing the code, you can run npm run dev watch. This will rebuild the code quickly and lint it when you save. To just lint, run npm run dev lint and to clean, run npm run dev clean. To build for production, just run npm run build.

To test the Chrome extension, click Load Unpacked and pick the extension/ directory. To test the Firefox extension, go to Add-ons and Debug Add-ons. Then Load Temporary Add-on and pick the extension.xpi file. To reload them after a rebuild, click the Reload button in Chrome and/or Firefox.

License

MIT

fimfic2epub's People

Contributors

benny- avatar daniel-j avatar dependabot[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

fimfic2epub's Issues

Feature Request: Link to Chapter Comments

One of the cool parts about Nyerguds's converter is at the end of each chapter, there is a hyperlink that leads directly to that chapter's specific comments. This allows a reader to leave comments with a press of a button and greatly increases the likelihood of comments. I think it's especially important to have one at the end of the book. While I know there's a link at the beginning of the book, having one right at hand makes it easier (and folks tend to be lazy, let's just be honest).

Empty chapters and missing description

Every chapter is empty except for the title and the view comments button.
And every story description shows the following:

Groups

Groups
Blog Posts
Users
Settings
News
FAQ
Rules
Writing Guide
Chat

`Internal Server Error` for fics with no cover art

A few (not very good) examples I tested at your website:

https://www.fimfiction.net/story/315940/the-duke-abides
https://www.fimfiction.net/story/453956/the-broken-treaty
https://www.fimfiction.net/story/424744/a-lightning-in-the-dark

I noticed you made (and closed) a similar pull request, but this doesn't look to be intended behavior.

Other fics work just fine.

Suggestion: More Visible Integration

The icon that FimFiction uses for EPUB downloads is itty-bitty-tiny-whiny. If it's possible, it would be great if there was some sort of extra button allowing direct downloads while on a synopsis page (and only the synopsis page). I don't know how easily that is to add, but it would be a nice feature.

In addition (or as an alternate idea) the button in Chrome toolbar should give you the option to create an EPUB from the page as well. As of now, it serves no purpose.

While I realize the Bookmarklet would do a similar thing, it would be nice to have a dynamic option.

Suggestion: Visible Progress Bar Upon Download

When you first click the .EPUB button on FimFiction, it appears that nothing has happened. While I know you need to wait, let's be honest, most folks won't. If there was a way to superimpose some sort of progress bar or "stand by" floating image in Chrome (even a small one), it would be a good UI addition.

Bug: Links in Author's Notes do not appear as hyperlinks in Dark Settings in iBooks

Yeah, apparently Apple's coding for the dark view is simply terrible.

When looking at the Author's Notes field in either of the two dark modes in iBooks (Mac or iOS), hyperlinks appear as normal text instead of blue text (or blue text with an underline). This issue does not occur for links in any other area (such as the synopsis).

screen shot 2016-08-24 at 9 06 56 am
screen shot 2016-08-24 at 9 06 52 am
screen shot 2016-08-24 at 9 06 49 am
screen shot 2016-08-24 at 9 06 44 am

Fimfic 2 epub has stopped working

I'm not really sure what else to say. When I select download, either by the dropdown arrow and "download as epub", or by clicking on the extension icon, nothing happens.

Browser console log:

e is undefined  fimfic2epub.js:86
	A/< jar:file:///Users/michael/Library/Application%20Support/Firefox/Profiles/0lyhtfoi.No%20addons/extensions/[email protected]!/build/fimfic2epub.js:86:24352
	runSafeSyncWithoutClone resource://gre/modules/ExtensionUtils.jsm:71:14
	runSafeWithoutClone resource://gre/modules/ExtensionCommon.jsm:133:38
	wrapPromise/</< resource://gre/modules/ExtensionCommon.jsm:322:15
	withLastError resource://gre/modules/ExtensionCommon.jsm:262:14
	wrapPromise/< resource://gre/modules/ExtensionCommon.jsm:316:11
Unchecked lastError value: Error: Please set webextensions.storage.sync.enabled to true in about:config  ExtensionCommon.jsm:265
	withLastError resource://gre/modules/ExtensionCommon.jsm:265:9
	wrapPromise/< resource://gre/modules/ExtensionCommon.jsm:316:11
uncaught exception: undefined

Feature Request: Word Count in Synopsis

A recent feature I suggested in Nyerguds's converter (that he added in 1.14.5) was to add a word count in the synopsis. In iBooks, there's no easy way to tell how long the book is as a whole. This is more a quality-of-life thing than anything else.

Bug Report: Meta-Data "Tags" are not showing properly in iBooks.

After generating the .EPUB, only the first tag in the book's listing on FiMFiction appears in the meta-data when viewing the book by category in iBooks.

Upon further inspection in Sigil, this application creates separate "dc:subject" fields for each tag, while Nyergunds's Converter uses a single "dc:subject" field with each tag separated by a comma and a space.

title.xhtml contains invalid XML/XHTML

Reproducible steps:

npm install [email protected]
./node_modules/fimfic2epub/bin/fimfic2epub 369464

Now open /OEBPS/Text/title.xhtml in the downloaded .epub file in for example firefox. You will get a XML parse exception.

XML Parsing Error: not well-formed
Line Number 26, Column 1203:

File doesn't download

Since this morning, trying to download a file silently fails. The progress bar says "completed," but no actual file download begins afterwards.

Doesnā€™t include authorā€™s notes

The feature to include the authorā€™s notes in the EPUB appears to be broken. Itā€™s not working on any of the stories I use this tool on (all of which have clear authorā€™s notes). I tried toggling ā€œPut all notes at the end of the ebookā€ to see if that changed anything, but it didnā€™t. Meanwhile, authorā€™s notes are saved when using FimFicā€™s default EPUB download.

[Suggestion] Download multiples of the same image separately

At the moment, if the extension detects that a fic uses the same image multiple times, it'll just download the image once and use place it wherever the image appears in the fic.

However, as there are numerous fics that have errors like the ones below, and the extension treats the error messages as if they're all the same image.
image

I've been trying to manually replace some of the images using Calibre, but because every time an error image is shown it directs back to the same file, replacing one image will replace all of them.

Would it be possible to add an option to grab all images separately, even if they are the same file? Hopefully that all makes sense.

Feature Request: Customize Metadata (Example: Covers)

Custom covers would be top of the list for me for options. I always create custom covers for FiMFiction stories and I'd like to be able to directly embed them into the .EPUB. Having to add them in Sigil after the fact creates havoc in iBooks unless very specific steps are taken.

Implementation Suggestion: A feature in the Google Chrome Extension (such as a "Customize EPUB" checkbox) that gives a pop-up with the basic metadata if checked.

Here are the features I normally adjust:

  • Tags - I always remove the My Little Pony & My Little Pony: Friendship is Magic tags, as they aren't required for me and they're so long that it makes any other tag impossible to read on an iOS device.
  • Cover (Both Large Cover and Small Cover) - Nyerguds's program only allows you to generate covers from web URLs. I don't know if you have this limitation or not, but being able to select a local file would be a nice touch, but the sheer amount of image hosting options out there, this is not required at all.

Chapter Headers missing and external Pictures not being downloaded in CLI Version

I noticed some weirdness lately with the CLI Version where it stopped Downloading External Pictures and only embeds the Links.
In Addition there is no Header created anymore despite the -H not being set.

commandline in my script used: fimfic2epub -T -c -W -D -b -r -w 0 -j $link
Arch Linux
Kernel: 5.12.14-zen1-1-zen
fimfic2epub 1.7.55-1
npm: 7.17.0-1
semver: 7.3.5-2
nodejs: nodejs 16.5.0-1
pango 1:1.48.6-1
libjpeg-turbo 2.1.0-1

Bug: Long Road to Friendship gets stuck at Chapter 42/43

screen shot 2016-09-06 at 8 48 11 am

Long Road to Friendship

I've attempted to download this story multiple times on both Mac and PC in Chrome. I've even tried the manual website (which eventually responds with a 504 Gateway Time-out), however it seems like it keeps getting stuck. I've tried other stories with a large number of chapters & word counts (Fallout: Equestria) as well as other stories by Albinocorn, but everything else seems fine. I'm not sure what's causing this particular story to cause the app to hang.

Bug: Table of Contents Errors (No Page Numbers) causing failure to sync with Cloud-based iOS systems

I know I already brought this up, but I ran into a more serious problem (at least for me).

Story Used for the test: http://www.fimfiction.net/story/8657/the-dresden-fillies-strange-friends

Symptom:
Page numbers are not being generated in iBooks for Mac OS X or iOS.
screen shot 2016-09-12 at 8 58 08 am

Result:
Notes, reading location, bookmarks and highlights are not syncing through iCloud to multiple devices (iOS or Mac OS).
Creating notes, reading location, bookmarks or highlights on an iPad will not sync to iPhone or Mac (or any combination of the three). I've tested this with multiple books, however, when I generated .EPUBs based on Nyerguds's EPUB Tool, they worked perfectly (as they had before).

Here's what I'm trying to figure out with the following data:
Why does the Nyerguds Application work correctly with page numbers, table of contents as well as syncing notes, reading location, bookmarks and highlights while the Chrome Extension doesn't?

I hope to figure this out, because I constantly flip back and forth between my iPhone and my iPad. In addition, I often save notes and bookmarks for the various follow-ups I do. While there are workarounds for both issues, when the Nyerguds application supports this by default, it ends up being less time to just hop into Parallels and run that application to generate the EPUB than to use the Chrome Extension.

I compared the generated files from both tools and here's what I've discovered:

Djazz Application:
Files are in XHTML format.
There is no toc.xhtml file by default. There is only a nav.xhtml - I tried simply changing this to toc.xhtml, but this had no effect.

Original nav.xhtml Table of Contents generated by Djazz Extension:

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html>

<html xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops">
<head>
    <meta charset="utf-8"/>
    <link rel="stylesheet" type="text/css" href="../Styles/style.css"/>
    <title>Contents</title>
  </head>
  <body id="navpage">
    <nav id="toc" epub:type="toc">
      <h3>Contents</h3>
      <ol>
        <li hidden="">
          <a href="../Text/cover.xhtml">Cover</a>
        </li>
        <li>
          <a href="../Text/chapter_001.xhtml" class="leftalign">Prologue</a>
        </li>
        <li>
          <a href="../Text/chapter_002.xhtml" class="leftalign">Chapter One</a>
        </li>
        <li>
          <a href="../Text/chapter_003.xhtml" class="leftalign">Chapter Two</a>
        </li>
        <li>
          <a href="../Text/chapter_004.xhtml" class="leftalign">Chapter Three</a>
        </li>
        <li>
          <a href="../Text/chapter_005.xhtml" class="leftalign">Chapter Four</a>
        </li>
        <li>
          <a href="../Text/chapter_006.xhtml" class="leftalign">Chapter Five</a>
        </li>
        <li>
          <a href="../Text/chapter_007.xhtml" class="leftalign">Chapter Six</a>
        </li>
        <li>
          <a href="../Text/chapter_008.xhtml" class="leftalign">Chapter Seven</a>
        </li>
        <li>
          <a href="../Text/chapter_009.xhtml" class="leftalign">Chapter Eight</a>
        </li>
        <li>
          <a href="../Text/chapter_010.xhtml" class="leftalign">Chapter Nine</a>
        </li>
        <li>
          <a href="../Text/chapter_011.xhtml" class="leftalign">Chapter Ten</a>
        </li>
        <li>
          <a href="../Text/chapter_012.xhtml" class="leftalign">Chapter Eleven</a>
        </li>
        <li>
          <a href="../Text/chapter_013.xhtml" class="leftalign">Chapter Twelve</a>
        </li>
      </ol>
    </nav>
</body>
</html>

When I open the file in Sigil and go to Tools -> Table of Contents -> "Create HTML Table of Contents," it replaces either nav.xhtml or toc.xhtml with the following data:

HTML Table of Contents generated by Sigil from Djazz Extension:

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops" lang="en" xml:lang="en">
<head>
  <meta charset="utf-8" />
  <style type="text/css">
    nav#landmarks, nav#page-list { display:none; }
    ol { list-style-type: none; }
  </style>
</head>
<body epub:type="frontmatter">
  <nav epub:type="toc" id="toc">
  </nav>
  <nav epub:type="landmarks" id="landmarks" hidden="">
  </nav>
</body>
</html>

However, I should note that the "Generate Table of Contents" and "Edit Table of Contents" appear to work before using the "Generate HTML Table of Contents," but they will not work after.

Nyerguds Application:
Files are in HTML format.
When I open the file in Sigil and go to Tools -> Table of Contents -> "Create HTML Table of Contents," it creates a normal, functional table of contents.

Original toc.html Table of Contents generated by Sigil from Nyerguds Application:

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
  "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
        <title>The Dresden Fillies: Strange Friends</title>
        <link rel="stylesheet" type="text/css" href="../Styles/style.css"/>
    </head>
    <body>
<div style="text-align:center"><h3>Table of Contents</h3>
</div>
<p class="double"><a href="../Text/ch0001_26220.html">Prologue</a>
</p>
<p><a href="../Text/ch0002_26223.html">Chapter One</a>
</p>
<p><a href="../Text/ch0003_26620.html">Chapter Two</a>
</p>
<p><a href="../Text/ch0004_26674.html">Chapter Three</a>
</p>
<p><a href="../Text/ch0005_26910.html">Chapter Four</a>
</p>
<p><a href="../Text/ch0006_27037.html">Chapter Five</a>
</p>
<p><a href="../Text/ch0007_27245.html">Chapter Six</a>
</p>
<p><a href="../Text/ch0008_27326.html">Chapter Seven</a>
</p>
<p><a href="../Text/ch0009_27670.html">Chapter Eight</a>
</p>
<p><a href="../Text/ch0010_27831.html">Chapter Nine</a>
</p>
<p><a href="../Text/ch0011_28028.html">Chapter Ten</a>
</p>
<p><a href="../Text/ch0012_28184.html">Chapter Eleven</a>
</p>
<p><a href="../Text/ch0013_28466.html">Chapter Twelve</a>
</p>
</body>
</html>

HTML Table of Contents generated by Sigil from Nyerguds Application (Results in XHTML file: toc.xhtml):

<?xml version="1.0"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
   "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Contents</title>
<link href="../Styles/sgc-toc.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div class="sgc-toc-title">Table of Contents</div>
<div class="sgc-toc-level-1">
  <a href="../Text/title.html">Cover</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/synopsis.html">Synopsis</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/toc.html">Table of Contents</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/ch0001_26220.html">Prologue</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/ch0002_26223.html">Chapter One</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/ch0003_26620.html">Chapter Two</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/ch0004_26674.html">Chapter Three</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/ch0005_26910.html">Chapter Four</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/ch0006_27037.html">Chapter Five</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/ch0007_27245.html">Chapter Six</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/ch0008_27326.html">Chapter Seven</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/ch0009_27670.html">Chapter Eight</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/ch0010_27831.html">Chapter Nine</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/ch0011_28028.html">Chapter Ten</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/ch0012_28184.html">Chapter Eleven</a>
</div>
<div class="sgc-toc-level-1">
  <a href="../Text/ch0013_28466.html">Chapter Twelve</a>
</div>
</body></html>

Broken as of 06/30/18 (stuck at ā€œFetching metadataā€¦ā€)

As of today, the extension doesnā€™t work on any fic Iā€™ve tried it on. When I click the icon, the downloader window appears, but the progress immediately stays stuck at the first step, ā€œFetching metadataā€¦ā€. I think they changed something under the hood at the site that caused this.

Using fimfic2epub v1.7.33
Firefox 61.0 (x64)
Windows 8.1 Pro (x64)
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    šŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. šŸ“ŠšŸ“ˆšŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ā¤ļø Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.