This project scrapes information from a specific webpage and extracts data from the table. The extracted information includes post date, quest number, category code, bid/request name, and bid closing date.
This Python project utilizes Selenium and BeautifulSoup to scrape data from a webpage and extract relevant information from the table. The extracted data can be used for various purposes, such as analysis or tracking upcoming bids.
-
Clone the repository:
git clone https://github.com/vatsrounak/PrimeNumTech_ScrapingProject
-
Change to the project directory:
cd Scrapping_PrimeNumTech
-
Create a virtual environment:
python3 -m venv .venv
-
Activate the virtual environment:
-
On Windows:
.venv\Scripts\activate
-
On macOS/Linux:
source .venv/bin/activate
-
-
Install dependencies:
pip install -r requirements.txt
Run the following command to execute the scraper script:
python scrape.py
The script will print the extracted information from the table.
- beautifulsoup4==4.10.0
- pandas==1.3.3
- selenium==3.141.0
If you'd like to contribute to the project, follow these steps:
- Fork the repository.
- Create a new branch:
git checkout -b feature-name
- Make your changes and commit them:
git commit -m 'Description of changes'
- Push to the branch:
git push origin feature-name
- Submit a pull request.
This project is licensed under the MIT License.