Comments (6)
This works:
% shot-scraper install --help
Usage: shot-scraper install [OPTIONS]
Install Playwright browser needed by this tool.
Usage:
shot-scraper install
Options:
-h, --help Show this message and exit.
% shot-scraper install
Downloading Playwright build of chromium v965416 - 117.2 Mb [====================] 100% 0.0s
Playwright build of chromium v965416 downloaded to /Users/simon/Library/Caches/ms-playwright/chromium-965416
Downloading Playwright build of ffmpeg v1007 - 1.1 Mb [====================] 100% 0.0s
Playwright build of ffmpeg v1007 downloaded to /Users/simon/Library/Caches/ms-playwright/ffmpeg-1007
from shot-scraper.
To get Playwright working you need to run:
playwright install
To install the browser(s) it needs.
Problem is, if you pipx install shot-scraper
you'll have shot-scraper
on your PATH but you won't have playwright
- so that command won't work without extra fiddling.
I solved this for pip install
for plugins in Datasette with the datasette install ...
command. Same trick can work here.
from shot-scraper.
Not sure how to test this on my local machine since I've already run playwright install
once.
from shot-scraper.
https://playwright.bootcss.com/python/docs/installation says:
Each version of Playwright needs specific versions of browser binaries to operate. By default Playwright downloads Chromium, WebKit and Firefox browsers into the OS-specific cache folders:
%USERPROFILE%\AppData\Local\ms-playwright
on Windows~/Library/Caches/ms-playwright
on MacOS~/.cache/ms-playwright
on Linux
from shot-scraper.
Confirmed: I can break my Playwright by removing that directory on macOS:
(shot-scraper) shot-scraper % ls -lah ~/Library/Caches/ms-playwright
total 0
drwxr-xr-x 7 simon staff 224B Mar 8 18:35 .
drwx------+ 204 simon staff 6.4K Mar 8 21:25 ..
drwxr-xr-x 5 simon staff 160B Mar 8 18:35 .links
drwxr-xr-x 4 simon staff 128B Mar 8 13:56 chromium-965416
drwxr-xr-x 5 simon staff 160B Mar 8 13:56 ffmpeg-1007
drwxr-xr-x 4 simon staff 128B Mar 8 13:56 firefox-1316
drwxr-xr-x 16 simon staff 512B Mar 8 13:57 webkit-1609
(shot-scraper) shot-scraper % shot-scraper http://www.example.com/ -o /tmp/example.png
Screenshot of 'http://www.example.com/' written to '/tmp/example.png'
(shot-scraper) shot-scraper % rm -rf ~/Library/Caches/ms-playwright
(shot-scraper) shot-scraper % shot-scraper http://www.example.com/ -o /tmp/example.png
Traceback (most recent call last):
File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/bin/shot-scraper", line 33, in <module>
sys.exit(load_entry_point('shot-scraper', 'console_scripts', 'shot-scraper')())
File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/Users/simon/Dropbox/Development/shot-scraper/shot_scraper/cli.py", line 30, in shot
browser = p.chromium.launch()
File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/sync_api/_generated.py", line 11417, in launch
self._sync(
File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/_impl/_sync_base.py", line 111, in _sync
return task.result()
File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/_impl/_browser_type.py", line 90, in launch
Browser, from_channel(await self._channel.send("launch", params))
File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 39, in send
return await self.inner_send(method, params, False)
File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 63, in inner_send
result = next(iter(done)).result()
playwright._impl._api_types.Error: Executable doesn't exist at /Users/simon/Library/Caches/ms-playwright/chromium-965416/chrome-mac/Chromium.app/Contents/MacOS/Chromium
╔═════════════════════════════════════════════════════════════════════════╗
║ Looks like Playwright Test or Playwright was just installed or updated. ║
║ Please run the following command to download new browsers: ║
║ ║
║ playwright install ║
║ ║
║ <3 Playwright Team ║
╚═════════════════════════════════════════════════════════════════════════╝
from shot-scraper.
Here's how datasette install
works: https://github.com/simonw/datasette/blob/dd94157f8958bdfe9f45575add934ccf1aba6d63/datasette/cli.py#L309-L316
args = ["pip", "install"]
if upgrade:
args += ["--upgrade"]
args += list(packages)
sys.argv = args
run_module("pip", run_name="__main__")
from shot-scraper.
Related Issues (20)
- Experimental feature: heap snapshots HOT 1
- interactive mode doesn't respect viewport size for initial window HOT 2
- bash interpolation inside shots.yml HOT 1
- Interactive automated browsing? (GPT-4V autopilot, simple scripting, etc) HOT 2
- Auto scraper?
- Failing test: Selector with a wait HOT 1
- Proposal: Add an option to make device scale factor flexible, in addition to the existing --retina argument HOT 1
- Proposal: Add an option to pass custom browser arguments to Playwright when launching the browser HOT 4
- Support for HTTP Basic Authentication HOT 10
- Conflict: an artifact with this name already exists on the workflow run HOT 2
- shot-scraper video command HOT 3
- Videos not screenshotted: "Sorry, we are having trouble playing this video" HOT 2
- Add a high level Python API HOT 5
- Add a new `--save-har=my.har` option to save requests to HAR file
- --init-script support HOT 3
- Ability to run `shot-scraper javascript` against several URLs at once HOT 3
- Can the browser be shown while shot-scraper is working? HOT 5
- Take screenshot in case of an error
- How can I avoid 403 error?
- How to allow insecure HTTPS connection with broken certs? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from shot-scraper.