This repository contains an overview of the publicly available resources of my bachelor thesis, such as source code or results.
- Thesis title: Design and implementation of a stealthy OpenWPM web scraper (link to PDF)
- Supervisor: Dr. Ir. Hugo Jonker
- University and program: Radboud University, NL, computing science
- Stealth version of OpenWPM (on the git branch
issue-448-webdriver_attr
; access exact version used in the thesis*)- The main addition of the stealth version is spoofing a revealing JavaScript property. This spoofing functionality is also available as a stand-alone Firefox extension (access exact version used in the thesis).
- OpenWPM measurement version (on the git branch
measurement
; access exact version used in the thesis*) - webdriver testpage (access exact version used in the thesis): A simple test page to test whether the original value of spoofed
window.navigator
objects can be revealed inside iframes.
*: Note after changing the branch it is necessary to rebuild OpenWPM's instrumentation extension (run build-extension.sh
) to resemble the code version of the changed branch.
results/BrowserBasedBotFP
contains the data captured with the BrowserBasedBotFP tool.
For easy inspection and comparisons, rename a database tofpdb.db
and use BrowserBasedBotFP's analysis functionality.fpdb[measurement-chapter].db
: Data from the experiment in the measurement chapter
results/jstemplate
contains the data captured with the JavaScript template attack tool.
For easy inspection and comparisons, rename a measurement file todata.json
and open it with the JavaScript template framework.data[measurement-chapter].json
: Data from the experiment in the measurement chapter
analysisOfHTTPHeader.pcapng
contains Wireshark captures of different browser and automation combinations to investigate the attribute order inside the HTTP packets.
- BrowserBasedBotFP: Framework to measure bot detection related fingerprinting properties (corresponding paper)
- JavaScript template attacks: Framework to measure JavaScript differences betweeen browser environments (corresponding paper)
- BDScanner (fork): Framework using OpenWPM to analyze URLs for their bot detection activity (corresponding paper)
(In my fork, the software dependencies are updated to the newest version)
BrowserBasedBotFP
and JavaScript template attacks
run on the same ports but that can be changed to run them at the same time.