I don't know if it's supposed to be like this, but I can't use URLs formatted in https://www.royalroadl.com/fiction/XXXX/title
So I created a batch that should work with all URLs.
(P.S There's another batch under this one, but it's a pathetic version of the first batch)
These are both majorly flawed, but they should work for most books, and the first one should work for all books, including pasting in chapter URLs, and chapter URLs with the "d%C3%A9j%C3%A0-vu".

@SETLOCAL enableextensions enabledelayedexpansion
@echo off
rem "Edit Information Here"

set programVersion="1.2.5"
set allocateMemorySize="1024"

rem "For Memory Size, 1024 is 1G, 2048 is 2G, etc."
rem "Above 1G will not work on all computers"

rem "Don't Edit Beyond This"
set /p bookURL="Enter RoyalRoadL Book URL: "
call:extractLeadingNumbers %bookURL% bookURLnumber
set bookURLparsed="https://www.royalroadl.com/fiction/%bookURLnumber%"

java -Xmx%allocateMemorySize%m -Xms%allocateMemorySize%m -jar royalroadl-downloader-assembly-%programVersion%.jar %bookURLparsed%

pause

:extractLeadingNumbers inputString returnVar
setlocal enableextensions disabledelayedexpansion
set "string=%~1"
for /f "tokens=1-2 delims=0123456789 " %%a in ("%string:^"=%") do set "delimiters=%%a%%b"
for /f "delims=%delimiters% " %%a in ("%string:^"=%") do set "numbers=%%a"
endlocal & set "%~2=%numbers%"
goto :eof

endlocal

aaaaaaand one that works with books with four digit book titles

@SETLOCAL enableextensions enabledelayedexpansion
@echo off

rem "Edit Information Here"

set programVersion="1.2.5"
set allocateMemorySize="1024"

rem "For Memory Size, 1024 is 1G, 2048 is 2G, etc."
rem "Above 1G will not work on all computers"

rem "Don't Edit Beyond This"
set /p bookURL="Enter RoyalRoadL Book URL: "

if not x%bookURL:https:=%==x%bookURL% if not x%bookURL:www.=%==x%bookURL% call:httpsWWW
if not x%bookURL:https:=%==x%bookURL% if x%bookURL:www.=%==x%bookURL% call:httpsnoWWW

if not x%bookURL:http:=%==x%bookURL% if not x%bookURL:www.=%==x%bookURL% call:httpWWW
if not x%bookURL:http:=%==x%bookURL% if x%bookURL:www.=%==x%bookURL% call:httpnoWWW

java -Xmx%allocateMemorySize%m -Xms%allocateMemorySize%m -jar royalroadl-downloader-assembly-%programVersion%.jar %bookURLparsed%

pause

:httpsWWW
set bookURLparsed=%bookURL:~0,39%
goto:eof

:httpsnoWWW
set bookURLparsed=%bookURL:~0,36%
goto:eof

:httpWWW
set bookURLparsed=%bookURL:~0,38%
goto:eof

:httpnoWWW
set bookURLparsed=%bookURL:~0,35%
goto:eof

endlocal

running the code on kinda old laptop and suddenly the program spitting out of memory error. the novel i'm trying to download is mythran master of all continued (https://royalroadl.com/fiction/8220/mythrans-master-of-all-continued) which is around 300++ chapter

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.io.ByteArrayOutputStream.grow(Unknown Source)
at java.io.ByteArrayOutputStream.ensureCapacity(Unknown Source)
at java.io.ByteArrayOutputStream.write(Unknown Source)
at org.jsoup.helper.DataUtil.readToByteBuffer(DataUtil.java:167)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:
707)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:
628)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:260)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser.executeRequest(Jso
upBrowser.scala:65)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser$$anonfun$3.apply(J
soupBrowser.scala:76)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser$$anonfun$3.apply(J
soupBrowser.scala:76)
at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52)
at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser.get(JsoupBrowser.s
cala:33)
at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:62)
at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:59)
at scala.collection.parallel.AugmentedIterableIterator$class.map2combine
r(RemainsIterator.scala:115)
at scala.collection.parallel.immutable.ParVector$ParVectorIterator.map2c
ombiner(ParVector.scala:62)
at scala.collection.parallel.ParIterableLike$Map.leaf(ParIterableLike.sc
ala:1054)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.
scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:4
8)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:4
8)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$Map.tryLeaf(ParIterableLike
.scala:1051)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class
.internal(Tasks.scala:169)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTa
sk.internal(Tasks.scala:443)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class
.compute(Tasks.scala:149)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTa
sk.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:1
60)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool
.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:19
79)

The behavior of downloading images and embedding them into the document is useful in some situations, but not all of them. It would in some situations be more useful to leave the links as is or download the images into a folder and link to them.

Hi, I've just tried to use this downloader on the site and I'm getting this error.

c:\rrld>java -jar royalroadl-downloader-assembly-1.2.2.jar http://royalroadl.com/fiction/5029/
[scallop] Error: Validation failure for 'fiction-link' option parameters: http://royalroadl.com/fiction/5029/

Is it something I'm doing wrong or has rrl changed there site. I've used this tool before without a problem.

Hi,

I love the program but when i tryed using it today i got the following error.

Exception in thread "main" org.jsoup.HttpStatusException: HTTP error fetching URL. Status=429, URL=https://www.royalroad.com/fiction/34592/core-chronicles/chapter/532426/101
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:679)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:628)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:260)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser.executeRequest(JsoupBrowser.scala:65)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser$$anonfun$3.apply(JsoupBrowser.scala:76)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser$$anonfun$3.apply(JsoupBrowser.scala:76)
at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52)
at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser.get(JsoupBrowser.scala:33)
at com.aivean.royalroad.Main$$anonfun$4$$anonfun$apply$1.apply(Main.scala:69)
at com.aivean.royalroad.Main$$anonfun$4$$anonfun$apply$1.apply(Main.scala:69)
at scala.util.Try$.apply(Try.scala:192)
at com.aivean.royalroad.Main$.retry(Main.scala:60)
at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:69)
at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:66)
at scala.collection.parallel.AugmentedIterableIterator$class.map2combiner(RemainsIterator.scala:115)
at scala.collection.parallel.immutable.ParVector$ParVectorIterator.map2combiner(ParVector.scala:62)
at scala.collection.parallel.ParIterableLike$Map.leaf(ParIterableLike.scala:1054)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$Map.tryLeaf(ParIterableLike.scala:1051)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.internal(Tasks.scala:159)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:443)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:149)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinTask.doJoin(ForkJoinTask.java:341)
at scala.concurrent.forkjoin.ForkJoinTask.join(ForkJoinTask.java:673)
at scala.collection.parallel.ForkJoinTasks$WrappedTask$class.sync(Tasks.scala:378)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.sync(Tasks.scala:443)
at scala.collection.parallel.ForkJoinTasks$class.executeAndWaitResult(Tasks.scala:426)
at scala.collection.parallel.ForkJoinTaskSupport.executeAndWaitResult(TaskSupport.scala:56)
at scala.collection.parallel.ExecutionContextTasks$class.executeAndWaitResult(Tasks.scala:558)
at scala.collection.parallel.ExecutionContextTaskSupport.executeAndWaitResult(TaskSupport.scala:80)
at scala.collection.parallel.ParIterableLike$ResultMapping.leaf(ParIterableLike.scala:958)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$ResultMapping.tryLeaf(ParIterableLike.scala:953)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:152)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Exception in thread "main" java.util.NoSuchElementException
        at net.ruippeixotog.scalascraper.browser.JsoupBrowser$JsoupElement.attr(JsoupBrowser.scala:98)
        at com.aivean.royalroad.Main$$anonfun$8$$anonfun$apply$1.applyOrElse(Main.scala:117)
        at com.aivean.royalroad.Main$$anonfun$8$$anonfun$apply$1.applyOrElse(Main.scala:115)
        at scala.PartialFunction$$anonfun$runWith$1.apply(PartialFunction.scala:141)
        at scala.PartialFunction$$anonfun$runWith$1.apply(PartialFunction.scala:140)
        at scala.collection.Iterator$class.foreach(Iterator.scala:891)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at net.ruippeixotog.scalascraper.model.LazyElementQuery.foreach(ElementQuery.scala:40)
        at scala.collection.TraversableLike$class.collect(TraversableLike.scala:271)
        at net.ruippeixotog.scalascraper.model.LazyElementQuery.collect(ElementQuery.scala:40)
        at com.aivean.royalroad.Main$$anonfun$8.apply(Main.scala:115)
        at com.aivean.royalroad.Main$$anonfun$8.apply(Main.scala:96)
        at scala.collection.immutable.Stream.foreach(Stream.scala:594)
        at com.aivean.royalroad.Main$.delayedEndpoint$com$aivean$royalroad$Main$1(Main.scala:96)
        at com.aivean.royalroad.Main$delayedInit$body.apply(Main.scala:17)
        at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
        at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
        at scala.App$$anonfun$main$1.apply(App.scala:76)
        at scala.App$$anonfun$main$1.apply(App.scala:76)
        at scala.collection.immutable.List.foreach(List.scala:392)
        at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
        at scala.App$class.main(App.scala:76)
        at com.aivean.royalroad.Main$.main(Main.scala:17)
        at com.aivean.royalroad.Main.main(Main.scala)

ENV:
JDK: openjdk version "21.0.1" 2023-10-17 LTS
downloader: Royalroad downloader v2.7.0
log: failed-download.log

This is the command I get when I try to use the .jar
[scallop] Error: Validation failure for 'fiction-link' option parameters: https://royalroadl.com/fiction/XXX

royalroadl seems to have entirely changed the site format so the program no longer works

It would be useful to generate a title page with these details:

• Link to the work on Royal Road
• Title
• Author name
• Datetime the work was scraped

I mostly just wanted a link to get back to the fiction page.

Threw an ObjectNotFound; CommandNotFoundException when i included the chapter part after the link, worked without.

Can you add a unique class to identify all chapter title HTML tags? Should be trivial, considering what I understand of the code, but I don't practice scala and don't have any env set up to make a pr. This change would ease converting (primarily structure deduction) to other formats. Many fictions don't use traditional keywords signifying a chapter so detecting those correctly may be a bother. Given that so far the only h1 tags in the output I have seen are the chapter titles and assuming all fictions fave no h1 tags in the text then structure deduction is still trivial but better be safe than sorry.

when trying to download forgotten conqueror, kestix novels or andur novels i get the same no PSK available issues then security and scala issues.

Exception in thread "main" javax.net.ssl.SSLException: No PSK available. Unable to resume.
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:129)
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:308)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:255)
at java.base/sun.security.ssl.ServerHello$T13ServerHelloConsumer.consume(ServerHello.java:1224)
at java.base/sun.security.ssl.ServerHello$ServerHelloConsumer.onServerHello(ServerHello.java:984)
at java.base/sun.security.ssl.ServerHello$ServerHelloConsumer.consume(ServerHello.java:872)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:178)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1152)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1063)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:567)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:163)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:651)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:628)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:260)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser.executeRequest(JsoupBrowser.scala:65)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser$$anonfun$3.apply(JsoupBrowser.scala:76)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser$$anonfun$3.apply(JsoupBrowser.scala:76)
at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52)
at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser.get(JsoupBrowser.scala:33)
at com.aivean.royalroad.Main$$anonfun$4$$anonfun$apply$1.apply(Main.scala:69)
at com.aivean.royalroad.Main$$anonfun$4$$anonfun$apply$1.apply(Main.scala:69)
at scala.util.Try$.apply(Try.scala:192)
at com.aivean.royalroad.Main$.retry(Main.scala:60)
at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:69)
at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:66)
at scala.collection.parallel.AugmentedIterableIterator$class.map2combiner(RemainsIterator.scala:115)
at scala.collection.parallel.immutable.ParVector$ParVectorIterator.map2combiner(ParVector.scala:62)
at scala.collection.parallel.ParIterableLike$Map.leaf(ParIterableLike.scala:1054)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$Map.tryLeaf(ParIterableLike.scala:1051)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.internal(Tasks.scala:159)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:443)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:149)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

It'd be nice if there was a flag to have author's notes for chapters added in. Most of the time it's better left off, but for stories where they don't just plug their links or whatever it would be handy.

As it says, it runs out of memory if the story is too large

I was checking why the embedded images do not render for me.
During that I noticed that the embedded images were of type "data:text/html" and not a variation of "data:image/"

Attached is an example of the originally downloaded file with RR-downloader and the decoded base64 data.
rrtest.zip
It seems that instead of embedding the image an Imgur embed HTML page was generated with Google & FB tracking included?

TLDR: Image embedding embeds a reference html page for Imgur instead of the image data

Hey, figured it would be good to note that if people have the 32-bit version of java installed this command fails due to the colossal size of the story.

java -jar .\royalroad-downloader-assembly-2.1.0.jar https://www.royalroad.com/fiction/11209/the-legend-of-randidly-ghosthound

The previous solution #8, didn't work. I resolved it by switching to 64 and then running with -Xmx6G.

since the current version of this build has you type the 5 digit number instead of whole url, can you add a step between the first and immediate download to have it confirm the title of the book?

thanks! i love this tool

encountered with http://royalroadl.com/fiction/4293, which has over 100 chapters.
From what i can tell, the app cannot handle this story because it has so many chapters, the read times out

Exception in thread "main" java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at java.io.BufferedInputStream.fill(Unknown Source) at java.io.BufferedInputStream.read1(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source) at sun.net.www.http.HttpClient.parseHTTP(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source) at java.net.HttpURLConnection.getResponseCode(Unknown Source) at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:516) at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:493) at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:205) at net.ruippeixotog.scalascraper.browser.Browser.executeRequest(Browser.scala:42) at net.ruippeixotog.scalascraper.browser.Browser$$anonfun$4.apply(Browser.scala:65) at net.ruippeixotog.scalascraper.browser.Browser$$anonfun$4.apply(Browser.scala:65) at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52) at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52) at net.ruippeixotog.scalascraper.browser.Browser.get(Browser.scala:20) at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:51) at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:51) at scala.collection.parallel.mutable.ParArray$Map.leaf(ParArray.scala:657) at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49) at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48) at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48) at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51) at scala.collection.parallel.mutable.ParArray$Map.tryLeaf(ParArray.scala:648) at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.internal(Tasks.scala:169) at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:443) at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:149) at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443) at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Exception in thread "main" javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:128)
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:308)
at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:279)
at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:181)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1152)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1063)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:567)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:163)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:651)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:628)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:260)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser.executeRequest(JsoupBrowser.scala:65)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser$$anonfun$3.apply(JsoupBrowser.scala:76)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser$$anonfun$3.apply(JsoupBrowser.scala:76)
at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52)
at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser.get(JsoupBrowser.scala:33)
at com.aivean.royalroad.Main$$anonfun$4$$anonfun$apply$1$$anonfun$apply$2.apply(Main.scala:92)
at com.aivean.royalroad.Main$$anonfun$4$$anonfun$apply$1$$anonfun$apply$2.apply(Main.scala:92)
at scala.util.Try$.apply(Try.scala:192)
at com.aivean.royalroad.Main$.backpressure(Main.scala:68)
at com.aivean.royalroad.Main$$anonfun$4$$anonfun$apply$1.apply(Main.scala:92)
at com.aivean.royalroad.Main$$anonfun$4$$anonfun$apply$1.apply(Main.scala:92)
at scala.util.Try$.apply(Try.scala:192)
at com.aivean.royalroad.Main$.retry(Main.scala:83)
at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:92)
at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:89)
at scala.collection.parallel.AugmentedIterableIterator$class.map2combiner(RemainsIterator.scala:115)
at scala.collection.parallel.immutable.ParVector$ParVectorIterator.map2combiner(ParVector.scala:62)
at scala.collection.parallel.ParIterableLike$Map.leaf(ParIterableLike.scala:1054)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$Map.tryLeaf(ParIterableLike.scala:1051)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.internal(Tasks.scala:159)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:443)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:149)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinTask.doJoin(ForkJoinTask.java:341)
at scala.concurrent.forkjoin.ForkJoinTask.join(ForkJoinTask.java:673)
at scala.collection.parallel.ForkJoinTasks$WrappedTask$class.sync(Tasks.scala:378)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.sync(Tasks.scala:443)
at scala.collection.parallel.ForkJoinTasks$class.executeAndWaitResult(Tasks.scala:426)
at scala.collection.parallel.ForkJoinTaskSupport.executeAndWaitResult(TaskSupport.scala:56)
at scala.collection.parallel.ExecutionContextTasks$class.executeAndWaitResult(Tasks.scala:558)
at scala.collection.parallel.ExecutionContextTaskSupport.executeAndWaitResult(TaskSupport.scala:80)
at scala.collection.parallel.ParIterableLike$ResultMapping.leaf(ParIterableLike.scala:958)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$ResultMapping.tryLeaf(ParIterableLike.scala:953)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:152)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Hi, I m not that technical but managed to get the downloader to work and already got a few books just fine.

However, when downloading this book specifically, the HTML comes back empty. The downloader identifies the book just fine, with name and all just no content there. Any help would be greatly appreciated :)

The book: https://www.royalroad.com/fiction/26727/arkendrithyst

The execute I'm using (works for other books): java -jar royalroad-downloader-assembly-2.4.0.jar http://royalroadl.com/fiction/26727

With Royal Road allowing to organize chapters into Volumes it would be useful to download a specific Volume.

On a sidenote: in pair with --from-chapter a --to-chapter equivalent would be useful. Also using RR chapter id (https://www.royalroad.com/fiction/49717/the-rise-of-the-winter-wolf/chapter/812891/b1-prologue -> 812891) to specify chapter range would make more sense than counting chapters. Yes, that would require looping over the entire chapter URL list once before the download but it is simpler for user-side. An universal option would be something like --chapter-range that allows standard range expressions e.g. "812891-812991,812993,812995" with addition of something like a wildcard symbol to denote an open range.

I digress, the feature request is only about the Volume download.

Try to download the book.
https://royalroadl.com/fiction/1866/demon-hero-reaper-saviour/chapter/16869/chapter-12-d%C3%A9j%C3%A0-vu
I assume it's an encoding oddity, shouldn't be hard to fix.

Love your tool. New error. 🐑

`downloading: http://royalroadl.com/fiction/15935/there-is-no-epic-loot-here-only
-puns/chapter/201782/52-griminology
Exception in thread "main" org.jsoup.HttpStatusException: HTTP error fetching UR
L. Status=520, URL=http://royalroadl.com/fiction/15935/there-is-no-epic-loot-her
e-only-puns/chapter/185868/6-the-capped-limit
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:
679)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:
628)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:260)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser.executeRequest(Jso
upBrowser.scala:65)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser$$anonfun$3.apply(J
soupBrowser.scala:76)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser$$anonfun$3.apply(J
soupBrowser.scala:76)
at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52)
at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:52)
at net.ruippeixotog.scalascraper.browser.JsoupBrowser.get(JsoupBrowser.s
cala:33)
at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:62)
at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:59)
at scala.collection.parallel.AugmentedIterableIterator$class.map2combine
r(RemainsIterator.scala:115)
at scala.collection.parallel.immutable.ParVector$ParVectorIterator.map2c
ombiner(ParVector.scala:62)
at scala.collection.parallel.ParIterableLike$Map.leaf(ParIterableLike.sc
ala:1054)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.
scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:4
8)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:4
8)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$Map.tryLeaf(ParIterableLike
.scala:1051)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class
.internal(Tasks.scala:169)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTa
sk.internal(Tasks.scala:443)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class
.compute(Tasks.scala:149)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTa
sk.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:1
60)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinTask.doJoin(ForkJoinTask.java:341)
at scala.concurrent.forkjoin.ForkJoinTask.join(ForkJoinTask.java:673)
at scala.collection.parallel.ForkJoinTasks$WrappedTask$class.sync(Tasks.
scala:378)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTa
sk.sync(Tasks.scala:443)
at scala.collection.parallel.ForkJoinTasks$class.executeAndWaitResult(Ta
sks.scala:426)
at scala.collection.parallel.ForkJoinTaskSupport.executeAndWaitResult(Ta
skSupport.scala:56)
at scala.collection.parallel.ParIterableLike$ResultMapping.leaf(ParItera
bleLike.scala:958)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.
scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:4
8)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:4
8)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$ResultMapping.tryLeaf(ParIt
erableLike.scala:953)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class
.compute(Tasks.scala:152)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTa
sk.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:1
60)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool
.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:19
79)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThre
ad.java:107)

C:\Users\User\Downloads>echo "java -jar royalroadl-downloader-assembly-1.2.3.jar
http://royalroadl.com/fiction/15935" error.txt
"java -jar royalroadl-downloader-assembly-1.2.3.jar http://royalroadl.com/fictio
n/15935" error.txt`

Hi,

Trying to download, but I get a validation error every time:

mstevens@osaka:~/royal-road-download % java -jar royalroadl-downloader-assembly-1.3.0.jar https://www.royalroadl.com/fiction/18619
[scallop] Error: Validation failure for 'from-chapter' option parameters: 

My java is:

mstevens@osaka:~/royal-road-download % java -version
java version "1.8.0_77"
Java(TM) SE Runtime Environment (build 1.8.0_77-b03)
Java HotSpot(TM) 64-Bit Server VM (build 25.77-b03, mixed mode)

Returns following when command is entered.

Exception

in thread "main" java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(Unknown Source)
at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:43)
at scala.collection.IterableLike$class.head(IterableLike.scala:107)
at scala.collection.AbstractIterable.head(Iterable.scala:54)
at net.ruippeixotog.scalascraper.scraper.ContentExtractors$$anonfun$4.apply(HtmlExtractor.scala:89)
at net.ruippeixotog.scalascraper.scraper.ContentExtractors$$anonfun$4.apply(HtmlExtractor.scala:89)
at net.ruippeixotog.scalascraper.scraper.SimpleExtractor.extract(HtmlExtractor.scala:63)
at net.ruippeixotog.scalascraper.dsl.ScrapingOps$ElementsScrapingOps$$anonfun$extract$1.apply(ScrapingOps.scala:16)
at scalaz.Monad$$anonfun$map$1$$anonfun$apply$2.apply(Monad.scala:14)
at scalaz.IdInstances$$anon$1.point(Id.scala:18)
at scalaz.Monad$$anonfun$map$1.apply(Monad.scala:14)
at scalaz.IdInstances$$anon$1.bind(Id.scala:20)
at scalaz.Monad$class.map(Monad.scala:14)
at scalaz.IdInstances$$anon$1.map(Id.scala:17)
at scalaz.syntax.FunctorOps.map(FunctorSyntax.scala:9)
at net.ruippeixotog.scalascraper.dsl.ScrapingOps$ElementsScrapingOps.extract(ScrapingOps.scala:16)
at net.ruippeixotog.scalascraper.dsl.ScrapingOps$ElementsScrapingOps.$greater$greater(ScrapingOps.scala:20)
at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:37)
at com.aivean.royalroad.Main$$anonfun$4.apply(Main.scala:34)
at scala.collection.parallel.mutable.ParArray$Map.leaf(ParArray.scala:657)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.mutable.ParArray$Map.tryLeaf(ParArray.scala:648)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.internal(Tasks.scala:159)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:443)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:149)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

I have used your downloader before and i like it. But when i tried using it today i got this message:
"Title: This Game We Call Life | RoyalRoadL
Saving as: This_Game_We_Call_Life_RoyalRoadL.html
done"
The story have 88 chapters and the html file created is 1kb large. I tried another story (10011) and i got the same result. then i switched to my laptop and got the same result on that to.
This_Game_We_Call_Life_RoyalRoadL.txt
I saved the html result as a txt file

I apologise in advance. This happens and I don't know how to fix this,

Just letting you know that the domain name for RoyalroadL has changed to https://www.royalroad.com/. You might want to make the required updates to the downloader.