gutenbergtools / ebookconverter Goto Github PK
View Code? Open in Web Editor NEWcode that orchestrates ebook conversion for project gutenberg
License: GNU General Public License v3.0
code that orchestrates ebook conversion for project gutenberg
License: GNU General Public License v3.0
I'm not sure if this is for ebookconverter or elsewhere in the processing chain.
For HTML and plain text, ebookmaker adds the header+metadata and footer to posted books.
For HTML, I would like to stop pushing *-h.zip and instead have that created after the header+metadata and footer is added. The *-h.zip can then go in cache/epub/xxx rather than in 1/2/3/...
The *-h.zip is a very useful format since it allows download of the HTML file plus all assets. But I'd like it to have the correct up-to-date metadata from the catalog whenever the other generated formats are built.
Per an email exchange October 26-28, it seems that the older code that put dcterms:type has been replaced, and some functionality was lost (see below).
The desire is for the XML/RDF file to have values that indicate all of the main categories like text, plus special categories listed under Browsing Options at https://www.gutenberg.org/ebooks: Special Categories: Audio Book, computer-generated Audio Book, human-read Compilations Data Music, recorded Music, Sheet Other recordings Pictures, moving Pictures, still.
Email explaining the issue:
Until a month ago, the rdf for these (non-book) files would have had the info
you're looking for in an attribute called dcterms:type.Before last month, non-book RDF files were those generated by a script that last
ran more than 10 years ago.A month ago we started re-generating RDF for non-book items in PG with the script
that has been making rdf for book items for the past 10 years, which for unknown
reasons does not emit a dcterms:type attribute.It should not be a lot of work for us to add this attribute, but it is not likely
to be done until the later part of November, or later depending on resource
availability.In the meantime, the information you really want is in this file (along with some
info that will never be in rdf files):https://github.com/gitenberg-dev/gitberg/blob/master/gitenberg/data/missing.tsv
I advise using an RDF parser to read the rdf files; when we update third party
modules in the generating script, I would expect them to reproduce the RDF graph but
not necessarily its xml represeantation.
Per some emails on December 9-10, there is an issue where Facebook postings to @gutenberg_new do not have a preview image.
I wonder whether this might be a timing issue, where the posting is sent before the landing page is fully ready.
It will be good to investigate this possibility and, if needed, add a delay before the social media postings.
I've encountered 4 missing ebooks from the gutenberg website: 38200
, 57983
, 64156
, 65643
. These pages contain no data, and only have a link to the RDF file. I discovered these because the RDF files also have blank data.
In the July 2021 newsletter, 65643 is listed as, "The lives of celebrated travellers, Vol. 2, by James Augustus St. John"
Related, but I guess not an eText release is ID 90907
, which exists in the rdf-files.tar.zip
offline archive, and also contains blank data.
P.S. I wasn't sure where to raise this issue so please point me to the correct place if needed.
For the backend notification code (Notifier.py), please consider this request: "Is it possible to have the name of the book and a link to the landing page. For me, that would make these success messages useful and meaningful."
Sample: https://www.gutenberg.org/ebooks/1399
1/3/9/1399/1399-rst/images/cover.jpg is correctly inserted into the EPUB and MOBI "with images" files.
However, the cover formats are not created in cache/epub/1399, i.e. pg1399.cover.small.jpg and pg1399.cover.medium.jpg. And therefore, there is no cover image to display for the landing page. These supplemental files should be generated.
for this reason, the rdf file does not keep up with metadata updated in the cataloging interface
Now that subtitles for new books are uniformly being handled using marc subfields in the database, we need to determine the best way to handle them in RDF. Also check OPDS?
Edit | Delete | 508 - Creation / Production Credits Note | 0 | Eric Schmidt (This file was produced from images generously made available by The Internet Archive) | |
---|---|---|---|---|---|
Edit | Delete | 508 - Creation / Production Credits Note | 0 | Eric Schmidt, Wouter Franssen and the Online Distributed Proofreading Team at https://www.pgdp.net (This file was produced from images generously made available by The Internet Archive) |
On Mar 14, 2023, at 6:02 PM, Jacqueline Jeremy wrote:Hi Eric
The files for Ebook #68957 were replaced--the original files had many
errors, and did not include many illustrations. The autogenerated
files appear to have the right content, but the credit line is not
correct. Can you change it be the same as in the submitted files, i.e.
Produced by: Eric Schmidt, Wouter Franssen and the Online Distributed
Proofreading Team at https://www.pgdp.net (This file was produced from
images generously made available by The Internet Archive)
Edit Delete 508 - Creation / Production Credits Note 0 Eric Schmidt (This file was produced from images generously made available by The Internet Archive)
Edit Delete 508 - Creation / Production Credits Note 0 Eric Schmidt, Wouter Franssen and the Online Distributed Proofreading Team at https://www.pgdp.net/ (This file was produced from images generously made available by The Internet Archive)
So now you have to tell me what you did, how it was submitted, etc., so I can fix the bug, if that's what it is.
Eric
On Mar 14, 2023, at 6:02 PM, Jacqueline Jeremy wrote:
Hi Eric
The files for Ebook #68957 were replaced--the original files had many
errors, and did not include many illustrations. The autogenerated
files appear to have the right content, but the credit line is not
correct. Can you change it be the same as in the submitted files, i.e.
Produced by: Eric Schmidt, Wouter Franssen and the Online Distributed
Proofreading Team at https://www.pgdp.net/ (This file was produced from
images generously made available by The Internet Archive)
Here's an outline of one possible semiautomated workflow for migrating structured credit and update data into the existing database:
Need to address various forms of credit/update sources.
*** START OF...
markerFrom @gbnewby
I thought this was raised earlier but didn't see an open issue for it.
The production team (whitewashers) consensus is they do not desire to get "Good news, PG whitewasher(s)" emails when a new title is successfully processed on ibiblio.
They only want to get an email if there is an error (not just a warning) during processing.
Thanks.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.