Comments (1)
Starting with 0.4.0, Browsertrix Crawler now includes support for an 'interactive' profile creation mode where you can log-in to sites and then interact with an embedded browser. You can use this to click on any 'Accept Cookies' messages on any site.
Many sites have such 'accept cookies' dialog boxes and detecting them automatically can be tricky (though ideally can be done). The interactive solution should work well for now.
See: https://github.com/webrecorder/browsertrix-crawler#interactive-profile-creation for more info on this feature.
from browsertrix-crawler.
Related Issues (20)
- Unable to playback vimeo video
- Video on kiwix.org homepage is not retrieved HOT 1
- Update README.md to mention Brave instead of Chrome
- Parameter diskUtilization is ignoring input HOT 2
- Inconsistent Tweet archiving HOT 4
- Cloudflare interstitial wait isn't working HOT 3
- Any way to save seed urls into separate collections? HOT 2
- make browsertrix-crawler runnable in serverless environments HOT 3
- how configurable is the Automated Profile Creation feature
- Add request initiator to WARC? HOT 6
- [Bug]: no warc-info header in any warc file included in a wacz
- SOCKS proxy username and password parameters missing
- Crawl JS and CSS HOT 3
- RCE Vulnerability in puppeter-core HOT 1
- Generate 'pageinfo' resource records with summary of all page resources. HOT 1
- Unable to run multiple crawls in a single bash session HOT 1
- Add option to write pages to queue in Redis
- Brave Default Setting Improvements HOT 1
- Change path in seedFile example in readme.md HOT 3
- Handle seed redirects
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from browsertrix-crawler.