Git Product home page Git Product logo

Comments (6)

simonw avatar simonw commented on September 28, 2024 1

This works:

% shot-scraper install --help
Usage: shot-scraper install [OPTIONS]

  Install Playwright browser needed by this tool.

  Usage:

      shot-scraper install

Options:
  -h, --help  Show this message and exit.
% shot-scraper install                 
Downloading Playwright build of chromium v965416 - 117.2 Mb [====================] 100% 0.0s 
Playwright build of chromium v965416 downloaded to /Users/simon/Library/Caches/ms-playwright/chromium-965416
Downloading Playwright build of ffmpeg v1007 - 1.1 Mb [====================] 100% 0.0s 
Playwright build of ffmpeg v1007 downloaded to /Users/simon/Library/Caches/ms-playwright/ffmpeg-1007

from shot-scraper.

simonw avatar simonw commented on September 28, 2024

To get Playwright working you need to run:

playwright install

To install the browser(s) it needs.

Problem is, if you pipx install shot-scraper you'll have shot-scraper on your PATH but you won't have playwright - so that command won't work without extra fiddling.

I solved this for pip install for plugins in Datasette with the datasette install ... command. Same trick can work here.

from shot-scraper.

simonw avatar simonw commented on September 28, 2024

Not sure how to test this on my local machine since I've already run playwright install once.

from shot-scraper.

simonw avatar simonw commented on September 28, 2024

https://playwright.bootcss.com/python/docs/installation says:

Each version of Playwright needs specific versions of browser binaries to operate. By default Playwright downloads Chromium, WebKit and Firefox browsers into the OS-specific cache folders:

  • %USERPROFILE%\AppData\Local\ms-playwright on Windows
  • ~/Library/Caches/ms-playwright on MacOS
  • ~/.cache/ms-playwright on Linux

from shot-scraper.

simonw avatar simonw commented on September 28, 2024

Confirmed: I can break my Playwright by removing that directory on macOS:

(shot-scraper) shot-scraper % ls -lah ~/Library/Caches/ms-playwright
total 0
drwxr-xr-x    7 simon  staff   224B Mar  8 18:35 .
drwx------+ 204 simon  staff   6.4K Mar  8 21:25 ..
drwxr-xr-x    5 simon  staff   160B Mar  8 18:35 .links
drwxr-xr-x    4 simon  staff   128B Mar  8 13:56 chromium-965416
drwxr-xr-x    5 simon  staff   160B Mar  8 13:56 ffmpeg-1007
drwxr-xr-x    4 simon  staff   128B Mar  8 13:56 firefox-1316
drwxr-xr-x   16 simon  staff   512B Mar  8 13:57 webkit-1609
(shot-scraper) shot-scraper % shot-scraper http://www.example.com/ -o /tmp/example.png
Screenshot of 'http://www.example.com/' written to '/tmp/example.png'
(shot-scraper) shot-scraper % rm -rf ~/Library/Caches/ms-playwright 
(shot-scraper) shot-scraper % shot-scraper http://www.example.com/ -o /tmp/example.png
Traceback (most recent call last):
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/bin/shot-scraper", line 33, in <module>
    sys.exit(load_entry_point('shot-scraper', 'console_scripts', 'shot-scraper')())
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/Users/simon/Dropbox/Development/shot-scraper/shot_scraper/cli.py", line 30, in shot
    browser = p.chromium.launch()
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/sync_api/_generated.py", line 11417, in launch
    self._sync(
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/_impl/_sync_base.py", line 111, in _sync
    return task.result()
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/_impl/_browser_type.py", line 90, in launch
    Browser, from_channel(await self._channel.send("launch", params))
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 39, in send
    return await self.inner_send(method, params, False)
  File "/Users/simon/.local/share/virtualenvs/shot-scraper-sQHOtKI2/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 63, in inner_send
    result = next(iter(done)).result()
playwright._impl._api_types.Error: Executable doesn't exist at /Users/simon/Library/Caches/ms-playwright/chromium-965416/chrome-mac/Chromium.app/Contents/MacOS/Chromium
╔═════════════════════════════════════════════════════════════════════════╗
║ Looks like Playwright Test or Playwright was just installed or updated. ║
║ Please run the following command to download new browsers:              ║
║                                                                         ║
║     playwright install                                                  ║
║                                                                         ║
║ <3 Playwright Team                                                      ║
╚═════════════════════════════════════════════════════════════════════════╝

from shot-scraper.

simonw avatar simonw commented on September 28, 2024

Here's how datasette install works: https://github.com/simonw/datasette/blob/dd94157f8958bdfe9f45575add934ccf1aba6d63/datasette/cli.py#L309-L316

    args = ["pip", "install"]
    if upgrade:
        args += ["--upgrade"]
    args += list(packages)
    sys.argv = args
    run_module("pip", run_name="__main__")

from shot-scraper.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.