Comments (6)
Update Bose by execute the following command: python -m pip install bose --upgrade
.
In the browser configuration, utilize the "close on crash" and "undetected" options:
class Task(BaseTask):
task_config = TaskConfig(
close_on_crash=True,
use_undetected_driver=True,
)
Also, selenium is crashing in Docker due to low memory increase it's memory and it should work.
Also, Could you share your dockerfile?
from botasaurus.
No, I have 2.0.8 - it's not too old version.
Yeah, I found this option (close_on_crash), thanks!
But I guess important thing to notice somewhere in guides that webdriver required shm and if you run it with docker, you have to specify --shm-size. Because shm was reason of the InvalidSessionException.
from botasaurus.
@goforbroke1006 Was interested to see a selenium Docker file for learning purposes. Could you show it?
from botasaurus.
Yeah, sure!
This config fits to my purposes:
FROM debian:bookworm-slim
RUN apt update && apt upgrade -y
RUN apt install -y curl unzip
RUN apt-get install python3.11 python3-pip python3.11-venv -y
RUN python3 --version
# https://packages.debian.org/sid/chromium
ARG CHROME_VERSION='114.0.5735.198-1'
ARG CHROMIUM_DEB_VERSION="${CHROME_VERSION}~deb12u1"
# http://chromedriver.storage.googleapis.com/
ARG CHROMEDRIVER_VERSION='114.0.5735.90'
RUN apt install -y \
chromium-common=$CHROMIUM_DEB_VERSION \
chromium-sandbox=$CHROMIUM_DEB_VERSION \
chromium=$CHROMIUM_DEB_VERSION
RUN mkdir -p /code/build/
RUN curl -O -L http://chromedriver.storage.googleapis.com/${CHROMEDRIVER_VERSION}/chromedriver_linux64.zip
RUN unzip ./chromedriver_linux64.zip -d /code/build/
RUN chmod -R 0777 /code/build/
ENV PYTHONUNBUFFERED=1
ENV PYTHONIOENCODING=utf-8
ENV PYTHONLEGACYWINDOWSSTDIO=utf-8
ENV ENV=production
WORKDIR /code/
ADD requirements.txt /code/requirements.txt
RUN python3 -m venv ./venv && . venv/bin/activate && pip3 install -r requirements.txt
COPY ./src /code/src
COPY ./launcher.py /code/launcher.py
COPY ./main.py /code/main.py
RUN echo '#!/bin/bash \n\
\n\
args=$* \n\
\n\
. venv/bin/activate \n\
python3 main.py ${args} \n\
' > /entrypoint.sh
RUN chmod +x /entrypoint.sh
ENTRYPOINT [ "/entrypoint.sh" ]
from botasaurus.
And compose like this:
version: "3.9"
services:
.base-task: &base-task
image: docker.io/goforbroke1006/my-awesome-project:latest
volumes:
- ./output:/code/output:rw
- ./profiles:/code/profiles:rw
- ./tasks:/code/tasks:rw
- ./local_storage.json:/code/local_storage.json:rw
- ./profiles.json:/code/profiles.json:rw
shm_size: "512Mb"
task1-scan-someting:
<<: *base-task
command: someting-one
task2-scan-someting:
<<: *base-task
command: someting-two
from botasaurus.
Thanks
from botasaurus.
Related Issues (20)
- JSONEncoder.default() missing 1 required positional argument: 'o' HOT 1
- Feature requests with regards to file writes HOT 1
- Add support for SOCKS proxies HOT 1
- Node proxy-chain require timeout when using as celery worker task HOT 6
- Migration guide HOT 1
- Please add method to select multiple elements HOT 1
- PeritmeterX / HUMAN Captcha Fails in Headless Mode HOT 3
- Is it possible to run botasaurus_server with multiple workers? HOT 1
- add_arguments error HOT 1
- circular import botasaurus 4.0.24 HOT 3
- Having error while adding user_agent or window_size due to add_argument is none. HOT 3
- Fresh installation + first example errors out with "Unable to connect to browser" HOT 7
- Unresolved attribute reference 'get_attribute' for class 'Element' HOT 5
- Element.query_selector_all() missing 1 required positional argument: 'node_name' HOT 1
- Not all elements are clicked HOT 5
- zombie processes HOT 10
- Profile did not work HOT 2
- error with request HOT 1
- HTTP server ListenAndServe: listen tcp: lookup tcp/: invalid argument HOT 1
- Botasaurus Backend Service and Browser
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from botasaurus.