snobu / destreamer Goto Github PK
View Code? Open in Web Editor NEWSave Microsoft Stream videos for offline enjoyment.
License: MIT License
Save Microsoft Stream videos for offline enjoyment.
License: MIT License
So, I've tried the new functionality for downloading a whole bunch of videos from a text file.
I've put the urls in the text file, one url per each line like it says on the instruction of readme.
I launched this:
node destreamer.js --username MYUSERNAME --outputdirectory "videos" \ --videoUrls list.txt
(Also tried using the correct escape character for powershell, same result), I get this:
Video URLs: [ 'list.txt' ]
Username: MYUSERNAME
Output Directory: videos
Video/Audio Quality: undefined
Using youtube-dl version 2020.03.08
Using ffmpeg version git-2020-03-15-c467328 Copyright (c) 2000-2020 the FFmpeg developers
Launching headless Chrome to perform the OpenID Connect dance...
Navigating to STS login page...
We are logged in.
Got cookie. Consuming cookie...
Calling Microsoft Stream API...
ApiGatewayUri: https://euwe-1.api.microsoftstream.com/api/
ApiGatewayVersion: 1.3-private
Fetching title and HLS URL...
Error when calling Microsoft Stream API: 400 undefined400
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid URL</h2>
<hr><p>HTTP Error 400. The request URL is invalid.</p>
</BODY></HTML>
And chromium crash, not even downloading the first video.
It seems an error calling microsoft stream API.. is it a bug?
First of all, congrats on releasing 2.0!
I have a minor but potentially important improvement for the logic for parsing video GUID from the URL. I don't know about you but when I copy the video URL it is in the form of: https://web.microsoftstream.com/video/<videoId>?channelId=<channelId>
Right now the parsing logic at
Line 108 in 0878508
?
.
I am making a quick pull request.
It worked fine for me even though I didn't change a similar logic logic at
Line 46 in 0878508
attempting this on gentoo with node v12.14.0, ffmpeg 4.2.2, latest youtube-dl
Output Directory: vids/ Video/Audio Quality: Using youtube-dl version 2020.03.24 Using ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers Launching headless Chrome to perform the OpenID Connect dance... Navigating to STS login page... We are logged in. Sorry, i mean "you". (node:134608) UnhandledPromiseRejectionWarning: TimeoutError: Navigation timeout of 30000 ms exceeded at /home/dexter/test/destreamer/node_modules/puppeteer/lib/LifecycleWatcher.js:142:21 -- ASYNC -- at Frame.<anonymous> (/home/dexter/test/destreamer/node_modules/puppeteer/lib/helper.js:111:15) at Page.goto (/home/dexter/test/destreamer/node_modules/puppeteer/lib/Page.js:672:49) at Page.<anonymous> (/home/dexter/test/destreamer/node_modules/puppeteer/lib/helper.js:112:23) at rentVideoForLater (/home/dexter/test/destreamer/destreamer.js:79:20) (node:134608) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
Any thoughts?
internal/modules/cjs/loader.js:983
throw err;
^
Error: Cannot find module 'C:\Users\Ahnaf Kabir\Downloads\destreamer-master\build\src\destreamer.js'
�[90m at Function.Module._resolveFilename (internal/modules/cjs/loader.js:980:15)�[39m
�[90m at Function.Module._load (internal/modules/cjs/loader.js:862:27)�[39m
�[90m at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12)�[39m
�[90m at internal/main/run_main_module.js:18:47�[39m {
code: �[32m'MODULE_NOT_FOUND'�[39m,
requireStack: []
}
Not sure if this is something GitHub Actions does wrong since those tests work very well on my Mac and Windows box..
> [email protected] test /home/runner/work/destreamer/destreamer
> mocha build/test
1) "before all" hook in "{root}"
2) "after all" hook in "{root}"
0 passing (2s)
2 failing
1) "before all" hook in "{root}":
Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()"
is called; if returning a Promise, ensure it resolves.
2) "after all" hook in "{root}":
TypeError: Cannot read property 'close' of undefined
at Context.after (build/test/test.js:28:19)
##[error]The operation was canceled.
That's from our CI GitHub Action (which now fails). Any ideas?
When I try to run the program, chromium crashes like this:
Here's what the command prompt looks like:
C:\destreamer-master>npm start https://web.microsoftstream.com/video/745f450f-7454-4399-83bf-460ba2b6956f
> [email protected] start C:\destreamer-master
> node ./destreamer.js "https://web.microsoftstream.com/video/745f450f-7454-4399-83bf-460ba2b6956f"
Using youtube-dl version 2019.08.13
Launching headless Chrome to perform the OpenID Connect dance...
(node:4524) UnhandledPromiseRejectionWarning: Error: Protocol error (Performance.enable): Target closed.
at C:\destreamer-master\node_modules\puppeteer\lib\Connection.js:183:56
at new Promise (<anonymous>)
at CDPSession.send (C:\destreamer-master\node_modules\puppeteer\lib\Connection.js:182:12)
at Page._initialize (C:\destreamer-master\node_modules\puppeteer\lib\Page.js:137:20)
at Function.create (C:\destreamer-master\node_modules\puppeteer\lib\Page.js:47:16)
at C:\destreamer-master\node_modules\puppeteer\lib\Target.js:74:32
at processTicksAndRejections (internal/process/task_queues.js:85:5)
at async Browser._createPageInContext (C:\destreamer-master\node_modules\puppeteer\lib\Browser.js:177:18)
at async rentVideoForLater (C:\destreamer-master\destreamer.js:45:18)
-- ASYNC --
at Target.<anonymous> (C:\destreamer-master\node_modules\puppeteer\lib\helper.js:111:15)
at Browser._createPageInContext (C:\destreamer-master\node_modules\puppeteer\lib\Browser.js:177:31)
at processTicksAndRejections (internal/process/task_queues.js:85:5)
at async rentVideoForLater (C:\destreamer-master\destreamer.js:45:18)
-- ASYNC --
at Browser.<anonymous> (C:\destreamer-master\node_modules\puppeteer\lib\helper.js:111:15)
at rentVideoForLater (C:\destreamer-master\destreamer.js:45:32)
at processTicksAndRejections (internal/process/task_queues.js:85:5)
(node:4524) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:4524) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Any help is much appreciated.
Thanks for making this, I'm getting a few people to record handover videos and this tool is a makes getting those recordings down into one place a breeze.
Let me know if I can help out with any contributions!
I got the below error.
ERROR: fixed output name but more than one file to download (node:35663) UnhandledPromiseRejectionWarning: Error: Command failed: youtube-dl --no-call-home --no-warnings --output "/Users/bwyyoung/Download_HDD//Meeting in "Lecture Room".mp4" --add-header Cookie:"Authorization=eyJUb2tl...........cEFzOGM9In0%3d; Signature=7lWoeLksDjJIzJP1LX7Vxoxh%252bmXr7PTXXFEKvuGtXKk%253d" "https://gkbzua4wayz4evghrorxwmd34a.streaming.mediaservices.windows.net/506e134b-38ff-4f69-8865-7556586f001c/1-5uzexnjuqyxqrzmo2qxxv7ngoc.ism/manifest(format=m3u8-aapl)" at checkExecSyncError (child_process.js:629:11) at Object.execSync (child_process.js:666:13) at rentVideoForLater (/Users/bwyyoung/Desktop/destreamer-dev/destreamer.ts:108:22) at process._tickCallback (internal/process/next_tick.js:68:7) (node:35663) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) (node:35663) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
I have created a list.txt file containing all my urls:
-one line per url
-every url is delimited with marks ("").
then i proced with the command line
node destreamer.js --username "loginemail" --outputDirectory "mydirectorypath" \ --videoUrls "mylist.txtpath"
the program times out with the following error
no such file or directory, open 'mylist.txtpath'
This works on the Mac (both inside VSCode and iTerm), needs testing on Windows.
We already have the thumbnail URL (it's in the same payload with the playbackUrl), we just need to fetch the image and render in the console.
term.drawImage('.thumbnail.png', { shrink: { width: 50, height: 50 } });
Crude implementation in the tokencache branch -
I have this error:
Output Directory: videos Using youtube-dl version 2020.03.08 Using ffmpeg version n4.2.2 Copyright (c) 2000-2019 the FFmpeg developers Launching headless Chrome to perform the OpenID Connect dance... Navigating to STS login page... We are logged in. Sorry, i mean "you". (node:17384) UnhandledPromiseRejectionWarning: TimeoutError: Navigation Timeout Exceeded: 30000ms exceeded at /home/livrea/Scrivania/destreamer-dev/node_modules/puppeteer/lib/LifecycleWatcher.js:142:21 -- ASYNC -- at Frame.<anonymous> (/home/livrea/Scrivania/destreamer-dev/node_modules/puppeteer/lib/helper.js:111:15) at Page.goto (/home/livrea/Scrivania/destreamer-dev/node_modules/puppeteer/lib/Page.js:674:49) at Page.<anonymous> (/home/livrea/Scrivania/destreamer-dev/node_modules/puppeteer/lib/helper.js:112:23) at rentVideoForLater (/home/livrea/Scrivania/destreamer-dev/destreamer.js:72:20) (node:17384) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag
--unhandled-rejections=strict(see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1) (node:17384) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
I could download some videos, this is great. But mainly I'd need the audio stream.
Sadly the downloaded MP4 file has no audio stream, only video. I asked one of my friends, he said in the real browser the video has sound, so the recording is correct.
Do I need to specify some parameters to get the audio too? Like the format?
hi, I use arch linux, I installed chromium (which I have always used for destreamer on other linux systems), but when I try to start the process I have this error.
Using youtube-dl version 2020.03.24
Using ffmpeg version n4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
Launching headless Chrome to perform the OpenID Connect dance...
(node:610) UnhandledPromiseRejectionWarning: Error: Failed to launch the browser process! spawn /home/oscar/destreamer/node_modules/puppeteer/.local-chromium/linux-722234/chrome-linux/chrome EACCES
TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/master/docs/troubleshooting.md
at onClose (/home/oscar/destreamer/node_modules/puppeteer/lib/Launcher.js:750:14)
at ChildProcess.<anonymous> (/home/oscar/destreamer/node_modules/puppeteer/lib/Launcher.js:741:65)
at ChildProcess.emit (events.js:315:20)
at Process.ChildProcess._handle.onexit (internal/child_process.js:273:12)
at onErrorNT (internal/child_process.js:467:16)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
(node:610) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:610) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Thank you
Hello,
Thanks for this awesome code ... works well ... could please add a video tutorial ?
sometimes i get some of this error but it continue downloading, the video lose some parts at the end:
[tcp @ 000002401db659c0] Connection to tcp://gd5xbgutd55iu6wqwypmuwj5ad.streaming.mediaservices.windows.net:443 failed: Error number -138 occurred
[crypto @ 000002401ddcd640] Unable to open resource: https://gd5xbgutd55iu6wqwypmuwj5ad.streaming.mediaservices.windows.net/a9f646e5-f1dc-49d4-b1ea-1b128677e6df/1-ipd7237ui44zx2bs2rjvad6y6g.ism/QualityLevels(250000)/Fragments(video=1260000000,format=m3u8-aapl)[hls @ 000002401d8aefc0] Failed to open segment 21 of playlist 0
also i get sometimes this error and the program closes:
"Navigating to STS login page...
We are logged in. Got cookie. Consuming cookie...
Accessing API...
Fetching title and HLS URL...
Error when calling Microsoft Stream API:403"
Note : the last error i started to get only after your latest changes in destreamer.ts
We need a token cache that:
~/.destreamer/token.cache
or something like that to be consistent with every other project out there that needs to save state).This will dramatically improve the user experience since within the span of one hour (Azure AD issues tokens with 1-hour lifetime) users can download videos without having to log back in.
Decode sample:
import * as jwt_decode from 'jwt-decode';
const token = 'eyJ0eXAi...';
const decoded = jwt_decode(token);
let now = Math.floor(Date.now() / 1000);
let exp = decoded['exp'];
let timeLeft = exp - now; // seconds
Stretch goal:
https://*.microsoftstream.com
just by doing an Open ID connect / OAuth 2.0 dance using the default system browser (drop puppeteer altogether).Using ffmpeg version git-2020-04-20-cacdac8 Copyright (c) 2000-2020 the FFmpeg developers
Access token still good for 59 minutes.
Fetching metadata...
RangeError: Invalid array length
at Object.parse (C:\Users\Mohamed Yahia\Desktop\jkhklj\destreamer\node_modules\cli-progress\lib\options.js:48:38)
at new SingleBar (C:\Users\Mohamed Yahia\Desktop\jkhklj\destreamer\node_modules\cli-progress\lib\single-bar.js:8:24)
at downloadVideo (C:\Users\Mohamed Yahia\Desktop\jkhklj\destreamer\build\src\destreamer.js:120:22)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
Unhandled error!
Timeout or fatal error, please check your downloads directory and try again
any help ?
Right now to make the execution a bit less annoying the login happens only at the beginning and if too much time the token expires, so if the queue is too long it may stop working mid download.
To fix this we can check before every download how much time remains before the token expires, I don't have the time right now.
We need to implement Mocha tests in the tokencache branch
@kylon do you want to take a look since last time you did a great job? (and I know absolutely nothing about them :D )
I think it would be helpful to add some kind of "already downloaded" check when you provide a file with urls and a download folder which already contains data.
Could it be useful?
I've pushed the master
branch as our 2.0 Release. Let's await some issues, if none show up let's make a fresh roadmap, that may include -
All other branches are now deprecated.
I'd rather delete them all (except for dev
since people are still pulling that to update their current destreamer). @kylon @lukaarma do you still need the other branches lingering around or can i obliterate them?
I've added a deprecation warning (both README and console.log in code) to the dev branch.
I try to download a Stream video, and it goes to 100% without errors, but then crashes with Unknown FFmpeg error
. Is there anything I can do to debug this?
ffmpeg
crashed for me even before the tokencache update, but at least it left the files so I could manually join audio and video. Thank you for any help.
Right now puppeteer must get a URL with a schema (https://). If URL schema is missing, this gets thrown -
Protocol error (Page.navigate): Cannot navigate to invalid URL
We should sanitize the URL before continuing execution and either prepend the schema if missing or throw back to user with a more meaningful error message.
Using youtube-dl version 2020.03.24
Using ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
Launching headless Chrome to perform the OpenID Connect dance...
Navigating to STS login page...
(node:936) UnhandledPromiseRejectionWarning: Error: Protocol error (Page.navigate): Cannot navigate to invalid URL
at /Users/rostyslavsalabay/destreamer-dev/node_modules/puppeteer/lib/Connection.js:183:56
at new Promise ()
at CDPSession.send (/Users/rostyslavsalabay/destreamer-dev/node_modules/puppeteer/lib/Connection.js:182:12)
at navigate (/Users/rostyslavsalabay/destreamer-dev/node_modules/puppeteer/lib/FrameManager.js:118:39)
at FrameManager.navigateFrame (/Users/rostyslavsalabay/destreamer-dev/node_modules/puppeteer/lib/FrameManager.js:95:7)
at Frame.goto (/Users/rostyslavsalabay/destreamer-dev/node_modules/puppeteer/lib/FrameManager.js:406:37)
at Frame. (/Users/rostyslavsalabay/destreamer-dev/node_modules/puppeteer/lib/helper.js:112:23)
at Page.goto (/Users/rostyslavsalabay/destreamer-dev/node_modules/puppeteer/lib/Page.js:672:49)
at Page. (/Users/rostyslavsalabay/destreamer-dev/node_modules/puppeteer/lib/helper.js:112:23)
at rentVideoForLater (/Users/rostyslavsalabay/destreamer-dev/destreamer.js:70:16)
-- ASYNC --
at Frame. (/Users/rostyslavsalabay/destreamer-dev/node_modules/puppeteer/lib/helper.js:111:15)
at Page.goto (/Users/rostyslavsalabay/destreamer-dev/node_modules/puppeteer/lib/Page.js:672:49)
at Page. (/Users/rostyslavsalabay/destreamer-dev/node_modules/puppeteer/lib/helper.js:112:23)
at rentVideoForLater (/Users/rostyslavsalabay/destreamer-dev/destreamer.js:70:16)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
(node:936) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict
(see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:936) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
I'm running Mac OS Catalina, when I run the script the following error appears at the terminal and Chromium opens with about:blank window. This script worked fine for me on the same computer an hour ago
thank you.
Hi, I don't know if it's a real problem, I don't care... I just signed up on GitHub because I wanted to thank you!
I'm using your code on a Mac (I don't know anything about coding!!!) and it works perfectly after I installed through brew everything I had to install!
I also passed it 4 URLs together and it got everything correctly!
I didn't modify anything in your file and I have to login every time I run the code even if I download one video after another but I don't care :)
Thank you again!
p.s. I just noticed it doesn't download by default the best version of the video (1080p) but sometimes it chooses a worse resolution. Is there a way to set the resolution in the command I run?
I have to check before if that resolution exists of course.
The current state of our youtube-dl progress output can be described as apocalyptic at best. For example -
I propose we either build our own logic to deal with the mess or we use something that already exists out there, like ytdl-core
. Example -
const readline = require('readline');
const fs = require('fs');
const ytdl = require('ytdl-core');
// TypeScript: import ytdl from 'ytdl-core'; with --esModuleInterop
// TypeScript: import * as ytdl from 'ytdl-core'; with --allowSyntheticDefaultImports
// TypeScript: import ytdl = require('ytdl-core'); with neither of the above
let video = ytdl('https://www.youtube.com/watch?v=UnSILVWDKL8');
let starttime;
video.pipe(fs.createWriteStream('video.mp4'));
video.once('response', () => {
starttime = Date.now();
});
video.on('progress', (chunkLength, downloaded, total) => {
const percent = downloaded / total;
const downloadedMinutes = (Date.now() - starttime) / 1000 / 60;
readline.cursorTo(process.stdout, 0);
process.stdout.write(`${(percent * 100).toFixed(2)}% downloaded `);
process.stdout.write(`(${(downloaded / 1024 / 1024).toFixed(2)}MB of ${(total / 1024 / 1024).toFixed(2)}MB)\n`);
process.stdout.write(`running for: ${downloadedMinutes.toFixed(2)}minutes`);
process.stdout.write(`, estimated time left: ${(downloadedMinutes / percent - downloadedMinutes).toFixed(2)}minutes `);
readline.moveCursor(process.stdout, 0, -1);
});
Result -
Moreover, terminal-kit
(which we already depend on) has a progress bar, would make for a really neat UI -
Passing --verbose
would change this neat output to raw youtube-dl output when debugging is necessary.
I've been looking through the source code but this seems like is not implemented (yet). If it is, could you please point me to the documentation?
chromium crashing on Windows x64... tried to change puppeteer and chromium versions but no luck.
what else can I do?
Hi I just want to first say that this project is awesome and helps me a lot!
I would just like to suggest the feature of adding download of captions/transcript. For the videos I have access to, the transcript is served as a JSON file. For example, for:
https://web.microsoftstream.com/video/VIDEO_ID?channelId=CHANEL_ID
The auto-generated transcript is served as JSON at:
https://cact-1.api.microsoftstream.com/api/videos/VIDEO_ID/events?$filter=Type%20eq%20%27transcript%27&api-version=1.3-private
I am not sure if cact
is region specific.
Maybe the easiest solution is to download the JSON file and convert it to a standard subtitles file format.
Just thought this would be a neat feature.
Transpiling TypeScript to JavaScript...
TSFILE: C:/StoneTemp/destreamer-dev/BrowserTests.js
TSFILE: C:/StoneTemp/destreamer-dev/destreamer.js
Using youtube-dl version 2019.11.22
Using ffmpeg version git-2019-11-26-59d264b Copyright (c) 2000-2019 the FFmpeg developers
Launching headless Chrome to perform the OpenID Connect dance...
Navigating to STS login page...
We are logged in. Sorry, i mean "you".
Got cookie. Consuming cookie...
Looking up AMS stream locator...
Video title is: Training
At this point Chrome's job is done, shutting it down...
Constructing HLS URL...
(node:5584) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'substring' of undefined
at rentVideoForLater (C:\StoneTemp\destreamer-dev\destreamer.js:83:27)
(node:5584) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:5584) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
tokencache
branch
Launching headless Chrome to perform the OpenID Connect dance...
Navigating to login page...
We are logged in.
Unhandled error!
Timeout or fatal error, please check your downloads and try again if necessary.
Error: Evaluation failed: ReferenceError: sessionInfo is not defined
at __puppeteer_evaluation_script__:3:26
at ExecutionContext._evaluateInternal (C:\destreamer\node_modules\puppeteer\lib\ExecutionContext.js:122:13)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async ExecutionContext.evaluate (C:\destreamer\node_modules\puppeteer\lib\ExecutionContext.js:48:12)
at async DoInteractiveLogin (C:\destreamer\destreamer.js:108:19)
at async main (C:\destreamer\destreamer.js:187:19)
-- ASYNC --
at ExecutionContext.<anonymous> (C:\destreamer\node_modules\puppeteer\lib\helper.js:111:15)
at DOMWorld.evaluate (C:\destreamer\node_modules\puppeteer\lib\DOMWorld.js:112:20)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async DoInteractiveLogin (C:\destreamer\destreamer.js:108:19)
at async main (C:\destreamer\destreamer.js:187:19)
-- ASYNC --
at Frame.<anonymous> (C:\destreamer\node_modules\puppeteer\lib\helper.js:111:15)
at Page.evaluate (C:\destreamer\node_modules\puppeteer\lib\Page.js:860:43)
at Page.<anonymous> (C:\destreamer\node_modules\puppeteer\lib\helper.js:112:23)
at DoInteractiveLogin (C:\destreamer\destreamer.js:108:30)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async main (C:\destreamer\destreamer.js:187:19)
C:\destreamer\destreamer.js:175
throw new Error('Killing process..\n');
^
Error: Killing process..
at process.<anonymous> (C:\destreamer\destreamer.js:175:11)
at process.emit (events.js:315:20)
at processPromiseRejections (internal/process/promises.js:209:33)
at processTicksAndRejections (internal/process/task_queues.js:98:32)
When trying to download a video from MS WebStream using destreamer.js, I am just seeing single line output as shown in the screenshot below, ie., Running in an elevated shell. I don't see any other command/browser windows getting opened. I tried to execute the command from a normal CMD as well as from a CMD with runas Administrator option. Still no success. Please help me.
We no longer need to pass the browser cookie down to youtube-dl, we can simply use the access token we get from sessionInfo (as Bearer).
Since we can simply call the MS Stream API for the HLS URL we only need the browser to get to the access token (also API endpoint and version but that's in the sessionInfo as well).
For example:
youtube-dl --no-call-home --no-warnings
--output "videos/blah blah.mp4"
--add-header Authorization:"Bearer ey......"
"https://xxxxx.azureedge.net/X-X-X-X/zzzz.ism/manifest(format=m3u8-aapl)"
Hi, sorry if this sounds like a very dumb question (I rarely use github) but I can't seem to build destreamer. Executing npm install
returns me with walls of warning such as:
npm WARN tar ENOENT: no such file or directory, open 'C:\Users\xx\Downloads\destreamer-master\node_modules\.staging\es-abstract-6e8ea733\2015\IsConcatSpreadable.js'
I am using node 13.13 with built-in npm (I have no idea how to figure out). I'll attach the log file if deemed necessary. Thanks!
Might add to the prereqs that you need ffmpeg setup and in path.
So, it was working for me till I updated the script with your latest version; i updated because sometimes it worked, other times it didn't (very strange) so I tried to update; I did npm install, npm run -s build, and added --no-sandbox to arguments, just like I did the first time I installed it.
Now, it gives me this:
Launching headless Chrome to perform the OpenID Connect dance... Navigating to STS login page... We are logged in. Sorry, i mean "you". (node:5580) UnhandledPromiseRejectionWarning: TimeoutError: Navigation Timeout Exceeded: 30000ms exceeded at C:\Users\xxx\Desktop\destreamer-dev\node_modules\puppeteer\lib\LifecycleWatcher.js:142:21 -- ASYNC -- at Frame.<anonymous> (C:\Users\xxx\Desktop\destreamer-dev\node_modules\puppeteer\lib\helper.js:111:15) at Page.goto (C:\Users\xxx\Desktop\destreamer-dev\node_modules\puppeteer\lib\Page.js:674:49) at Page.<anonymous> (C:\Users\xxx\Desktop\destreamer-dev\node_modules\puppeteer\lib\helper.js:112:23) at rentVideoForLater (C:\Users\xxx\Desktop\destreamer-dev\destreamer.js:72:20) (node:5580) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag
--unhandled-rejections=strict(see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1) (node:5580) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
I've also tried to reinstall everything, with the old version of the script (I had it zipped), but nothing ;(
Please help me, I'm using your script to download the videos of my class :(
PS if you need the logs you can tell me where to find them (I'm not pratical of nodejs)
I've noticed that on the new branch (2.0) you've discarded youtube-dl in favor of a full ffmpeg approach.
I've noticed also that video downloaded with the new version (2.0) weights a lot less than videos downloaded with the previous version of destreamer.
A 2 hours video with the previous version was about 400mb, now it's 170mb circa.
I'm wondering so if that means that video quality is bad, compared to the previous version, of it is applied another compression algorithm for the video.
So I'm wondering, now we can't decide for the output video quality, right? Having dropped youtube-dl and given that command line parameters related to output quality were passed to youtube-dl
Obviously that's not an issue, just want to know if videos downloaded with the 2.0 version have same "quality" of the previous version, and if not what I can do to download them with the same previous quality with this release.
Thanks :)
Hi all,
Tried to use destreamer 2.0.0 on Ubuntu 18.04.
I'm completely new to TypeScript so not quite sure why is this error showing up.
_____ub@____-ub-vm:~/destreamer$ ./destreamer.sh -i "https://web.microsoftstream.com/video/aa54_1a7-____-____-____-____________" -v
Using ffmpeg version 3.4.6-0ubuntu0.18.04.1 Copyright (c) 2000-2019 the FFmpeg developers
Video URLs:
[ 'https://web.microsoftstream.com/video/aa54_1a7-____-____-____-____________' ]
Access token still good for 46 minutes.
Unrecognized URL format in https://web.microsoftstream.com/video/aa54_1a7-____-____-____-____________: URL is not defined
It seems to be coming from:
Lines 102 to 117 in 0878508
Is there any possible mistake I made, or any try I can do?
Hello! This project looks awesome but unfortunately isn't working for me on windows or linux for a video i'm trying to download.
The microsoft streams video gives the error message:
Error code 0x20400003
and then the terminal just keeps giving off errors like this:
[mp4 @ 0000017ceca76780] Invalid DTS: 885060 PTS: 881820 in output stream 0:0, replacing by guess [mpegts @ 0000017cec6e7000] Invalid timestamps stream=0, pts=7367850, dts=7371090, size=6535 [mpegts @ 0000017cec6e7000] Invalid timestamps stream=0, pts=7371090, dts=7373790, size=4845 [hls @ 0000017cec59c880] Invalid timestamps stream=0, pts=7367850, dts=7371090, size=6535 [mp4 @ 0000017ceca76780] Invalid DTS: 891090 PTS: 887850 in output stream 0:0, replacing by guess [hls @ 0000017cec59c880] Invalid timestamps stream=0, pts=7371090, dts=7373790, size=4845 [mp4 @ 0000017ceca76780] Invalid DTS: 893790 PTS: 891090 in output stream 0:0, replacing by guess
Just hangs on hls after certain amount of time - running on Windows 10
We could take example from aria2c and allow options for each url in the txt file, like different output directories and so on
Sorry, I'm new to programming and typescript files.
Which is the username constant?
I have seen you wrote this in the file:
// Type in your username here (the one you use to
// login to Microsoft Stream).
const args: string[] = process.argv.slice(2); // TODO: Remove this
Is this it? How should I give my username to it (what format)? Sorry if this is a really obvious question but I couldn't figure it out.
RangeError: Invalid array length
at Object.parse (C:\destreamer\node_modules\cli-progress\lib\options.js:48:38)
at new SingleBar (C:\destreamer\node_modules\cli-progress\lib\single-bar.js:8:24)
at downloadVideo (C:\destreamer\build\src\destreamer.js:121:22)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
Unhandled error!
Timeout or fatal error, please check your downloads directory and try again
I keep this error when trying to download a video. I have tried to download it to a custom path but it's also not working.
When downloading a video, after the download is completed I get the following error:
Download finished: videos/<whatever>.mp4
TypeError: process.off is not a function
at downloadVideo (/home/mario/destreamer/build/src/destreamer.js:186:17)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:188:7)
The video downloads correctly so it doesn't seem like a big issue, but I thought you may want to know.
A lot of our video conference is just automatically uploaded to Streams, and no one cares their title, so all titles are simply "Meeting in General". This will cause error if I try to specify more than one --videoUrls
. Maybe you should append the video ID to the output filename, that is unique.
First of all, I want to say that this is wonderful and help me a lots.
There is a little bug that if two video have the exactly same title name, only the first video in the list will be downloaded because the output folder can not have two files with exactly the same name.
So, as in title, I get an error when I try to download multiple videos with the same command.
I've used:
node destreamer.js --username MYUSERNAME --outputdirectory "videos" \ --videoUrls "VIDEOURL1" \ "VIDEOURL2" \ "VIDEOURL3"
The videos have all different names, so that's not the problem.
It starts downloading the video related to VIDEOURL1, but when it correctly finish the first video, it gives me:
[ffmpeg] Downloaded 103911358 bytes [download] 100% of 99.10MiB in 07:59 [ffmpeg] Merging formats into "videos\NAMEOFTHEVIDEO.mp4" Deleting original file videos\NAMEOFTHEVIDEO.f629.mp4 (pass -k to keep) Deleting original file videos\NAMEOFTHEVIDEO.f137.mp4 (pass -k to keep) (node:1948) UnhandledPromiseRejectionWarning: Error: Protocol error (Page.navigate): Cannot navigate to invalid URL at C:\destreamer-dev\node_modules\puppeteer\lib\Connection.js:183:56 at new Promise (<anonymous>) at CDPSession.send (C:\destreamer-dev\node_modules\puppeteer\lib\Connection.js:182:12) at navigate (C:\destreamer-dev\node_modules\puppeteer\lib\FrameManager.js:118:39) at FrameManager.navigateFrame (C:\destreamer-dev\node_modules\puppeteer\lib\FrameManager.js:95:7) at Frame.goto (C:\destreamer-dev\node_modules\puppeteer\lib\FrameManager.js:406:37) at Frame.<anonymous> (C:\destreamer-dev\node_modules\puppeteer\lib\helper.js:112:23) at Page.goto (C:\destreamer-dev\node_modules\puppeteer\lib\Page.js:672:49) at Page.<anonymous> (C:\destreamer-dev\node_modules\puppeteer\lib\helper.js:112:23) at rentVideoForLater (C:\destreamer-dev\destreamer.js:110:20) -- ASYNC -- at Frame.<anonymous> (C:\destreamer-dev\node_modules\puppeteer\lib\helper.js:111:15) at Page.goto (C:\destreamer-dev\node_modules\puppeteer\lib\Page.js:672:49) at Page.<anonymous> (C:\destreamer-dev\node_modules\puppeteer\lib\helper.js:112:23) at rentVideoForLater (C:\destreamer-dev\destreamer.js:110:20) at runMicrotasks (<anonymous>) at processTicksAndRejections (internal/process/task_queues.js:97:5) (node:1948) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag
--unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1) (node:1948) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Obviously that's a minor issue, but hope it will get fixed
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.