Command line hangs at "Downloading audio:" about echo360 HOT 15 CLOSED

@soraxas Thanks for the update, it fixed the issue! All the videos got downloaded now. 😄

from echo360.

It's not possible to know what's happening with this output. You can try to run it with the debug flag --debug and check the output file echo360Downloader.log to see what operations it hangs at.

from echo360.

22mb log that is greek to me. any ideas? link

from echo360.

From the log it seems like it was able to retrieve addresses to download (things seems normal). During the download process, it is handled by external libraries so I cannot diagnose it from this log as they are not included.

If it works previously but suddenly it stops working, could it be a network problem? Or site issues? Or rather, when it hangs while downloading, are there any actual network activities in the background? Your OP description is very brief to pin down what's happening when you said it hangs

from echo360.

Not sure what other details I can give. Previously, when it worked, it had me login via chrome, then it gives some invalid first paint errors, and once it enumerated through all of the available lectures it began downloading one after another. Now, it does the exact same thing, but when it gets to the downloading step it says

Downloading "name of lecture"

just as it did before, but then a line comes up that I don't think happened previously that says "> Downloading audio:"
and then it just stops with the flashing carot and nothing else happens. It doesn't terminate. Just hangs indefinitely.

from echo360.

Is it that it hangs at every video or only some particular one? (well, you wouldn't know if it already hangs at the first video)

The module first tries to download directly with mp4 if your course provides it. If not, it will fall-back to using streaming method with m3u8. Strangely, some of your video are available as mp4 and some it needs to use m3u8 method.
The > Downloading audio you mentioned occurs in the m3u8 case as it needs to download the visual and audio separately.

The first available one seems to be mp4 so it shouldn't hangs on the first one did it? (I can't be sure of the ordering as it also sort based on aired date of lecture)

from echo360.

I have that same problem.
Last lines in log:

[DEBUG: 07-09 13:13] echo360.videos Trying from_json_mp4 method
[DEBUG: 07-09 13:13] echo360.videos Encountered exception: 'video'
[DEBUG: 07-09 13:13] echo360.videos Trying from_json_m3u8 method
[DEBUG: 07-09 13:13] echo360.videos Encountered exception: 'hasAvailableVideo'
[DEBUG: 07-09 13:13] echo360.videos Trying brute_force_all_mp4 method
[DEBUG: 07-09 13:13] selenium.webdriver.remote.remote_connection POST http://127.0.0.1:54595/session/ca17a9e21022ede8b00d40c039a5685c/url {"url": "https://echo360.org.uk/lesson/G_c698b4f7-62d7-421b-be22-7de8a3c7de32_90f1e50d-adaa-4c48-bbb9-b8853ac8c85f_2019-11-21T09:00:00.000_2019-11-21T09:55:00.000/classroom", "sessionId": "ca17a9e21022ede8b00d40c039a5685c"}
[DEBUG: 07-09 13:13] urllib3.connectionpool http://127.0.0.1:54595 "POST /session/ca17a9e21022ede8b00d40c039a5685c/url HTTP/1.1" 200 72
[DEBUG: 07-09 13:13] selenium.webdriver.remote.remote_connection Finished Request
[DEBUG: 07-09 13:13] selenium.webdriver.remote.remote_connection GET http://127.0.0.1:54595/session/ca17a9e21022ede8b00d40c039a5685c/source
{"sessionId": "ca17a9e21022ede8b00d40c039a5685c"}
[DEBUG: 07-09 13:13] urllib3.connectionpool http://127.0.0.1:54595 "GET /session/ca17a9e21022ede8b00d40c039a5685c/source HTTP/1.1" 200 284727
[DEBUG: 07-09 13:13] selenium.webdriver.remote.remote_connection Finished Request
[DEBUG: 07-09 13:13] echo360.videos Encountered exception: None were found.
[DEBUG: 07-09 13:13] echo360.videos Trying brute_force_all_m3u8 method
[DEBUG: 07-09 13:13] selenium.webdriver.remote.remote_connection POST http://127.0.0.1:54595/session/ca17a9e21022ede8b00d40c039a5685c/url {"url": "https://echo360.org.uk/lesson/G_c698b4f7-62d7-421b-be22-7de8a3c7de32_90f1e50d-adaa-4c48-bbb9-b8853ac8c85f_2019-11-21T09:00:00.000_2019-11-21T09:55:00.000/classroom", "sessionId": "ca17a9e21022ede8b00d40c039a5685c"}
[DEBUG: 07-09 13:13] urllib3.connectionpool http://127.0.0.1:54595 "POST /session/ca17a9e21022ede8b00d40c039a5685c/url HTTP/1.1" 200 72
[DEBUG: 07-09 13:13] selenium.webdriver.remote.remote_connection Finished Request
[DEBUG: 07-09 13:13] selenium.webdriver.remote.remote_connection GET http://127.0.0.1:54595/session/ca17a9e21022ede8b00d40c039a5685c/source
{"sessionId": "ca17a9e21022ede8b00d40c039a5685c"}
[DEBUG: 07-09 13:13] urllib3.connectionpool http://127.0.0.1:54595 "GET /session/ca17a9e21022ede8b00d40c039a5685c/source HTTP/1.1" 200 284727
[DEBUG: 07-09 13:13] selenium.webdriver.remote.remote_connection Finished Request
[DEBUG: 07-09 13:13] echo360.videos Found the following urls https://content.echo360.org.uk/0000.75f1696a-068b-4ce7-83c0-b03455c0fa36/7b2ca4a8-94d3-4802-aa3f-68f9160ab274/1/s1_av.m3u8
[DEBUG: 07-09 13:13] selenium.webdriver.remote.remote_connection GET http://127.0.0.1:54595/session/ca17a9e21022ede8b00d40c039a5685c/cookie
{"sessionId": "ca17a9e21022ede8b00d40c039a5685c"}
[DEBUG: 07-09 13:13] urllib3.connectionpool http://127.0.0.1:54595 "GET /session/ca17a9e21022ede8b00d40c039a5685c/cookie HTTP/1.1" 200 2687
[DEBUG: 07-09 13:13] selenium.webdriver.remote.remote_connection Finished Request
[DEBUG: 07-09 13:13] urllib3.connectionpool Starting new HTTPS connection (1): content.echo360.org.uk:443
[DEBUG: 07-09 13:13] urllib3.connectionpool https://content.echo360.org.uk:443 "GET /0000.75f1696a-068b-4ce7-83c0-b03455c0fa36/7b2ca4a8-94d3-4802-aa3f-68f9160ab274/1/s1_av.m3u8 HTTP/1.1" 200 491
[DEBUG: 07-09 13:13] selenium.webdriver.remote.remote_connection GET http://127.0.0.1:54595/session/ca17a9e21022ede8b00d40c039a5685c/cookie
{"sessionId": "ca17a9e21022ede8b00d40c039a5685c"}
[DEBUG: 07-09 13:13] urllib3.connectionpool http://127.0.0.1:54595 "GET /session/ca17a9e21022ede8b00d40c039a5685c/cookie HTTP/1.1" 200 2687
[DEBUG: 07-09 13:13] selenium.webdriver.remote.remote_connection Finished Request
[DEBUG: 07-09 13:13] urllib3.util.retry Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)
[DEBUG: 07-09 13:13] urllib3.connectionpool Starting new HTTPS connection (1): content.echo360.org.uk:443
[DEBUG: 07-09 13:13] urllib3.connectionpool https://content.echo360.org.uk:443 "GET /0000.75f1696a-068b-4ce7-83c0-b03455c0fa36/7b2ca4a8-94d3-4802-aa3f-68f9160ab274/1/s0q0.m3u8 HTTP/1.1" 200 None

from echo360.

It hangs before anything begins to download. It creates empty folders in the output folder, but nothing downloads. I also tried downloading specific videos that had downloaded previously using the --interactive flag, and that didn't work either.

from echo360.

Yes, I have the same problem. Hangs on "Downloading audio", and output folder is empty, nothing downloads. How is it meant to work normally? I also noticed that it says 0/2 videos at the start, but there are way more than two videos (there's a single video and a folder of videos on the home page).

Looking at the code, it reaches line 235 in downloader.py, where it calls videos.download(). It is then somehow freezing on line 262 in videos.py in download(), where it calls videos._download_url_to_dir() on line 147. So I guess we need to look closer at what's happening within videos._download_url_to_dir().

After putting down some print statements, it now seems that it is freezing on line 122 in hls_downloader.py. I put a print statement inside _worker(), but it doesn't print anything. So the point where it is freezing on is line 122 in hls_downloader.py, on this function call: self.pool.map(self._worker, ts_list). The value of ts_list is: [('https://content.echo360.org.au/{omitted}/{omitted}/1/s0q0.m4s', 0)]. I added {omitted}.

from echo360.

Hi @WeavingBird1917 thank you very much for your detailed analysis! Really appreciate it as it's much more useful for knowing what's happening than a not working statement.

Some explanation: The print statement in hls_downloader.py probably does not work because I utilised multithreading at that location to speed up with parallel downloads and probably overwritten by the progress updates. If wished, you can probably write to some temporary file to see what's happening.

Now, https://content.echo360.org.au/{omitted}/{omitted}/1/s0q0.m4s would be the audio file from some m3u8 file that the program extracted previously. Could you try to download the file from a browser to see if it works? (It will probably require you to authenticate yourself via echo360 first, so if it said something like permission issue try to use the same browser to login to your echo360 address first)

from echo360.

Hello @soraxas thanks for the explanation. I am able to download the file via the link, although I don't have any programs which can open that format at the moment.

I tried writing to a temp file within _worker(), but it doesn't seem to work (no files created).

from echo360.

I'm experiencing the same issue, it just hangs at "Downloading audio"

from echo360.

Hi @WeavingBird1917 based on your description I think I was able to pin point where it hangs (plus I think recently echo360 had changed their internal structure to restricts exposing some internal links).

Can you try to see if the latest commit fixes the hanging issue?

from echo360.

You are welcome :)

from echo360.

My issue is now fixed too! thanks!

from echo360.