ultralytics / flickr_scraper Goto Github PK

View Code? Open in Web Editor NEW

209.0 9.0 62.0 80 KB

Simple Flickr Image Scraper

Home Page: https://ultralytics.com

License: GNU Affero General Public License v3.0

Python 100.00%

flickr flickr-api scraper image-scraper image-scraping photos image-downloader image-downloader-python

flickr_scraper's Introduction

🚀 Introduction

Flickr scraper is a Python tool designed to help you gather images from Flickr to create datasets for YOLO training. Given your search criteria, this tool simplifies the process of collecting relevant images for various computer vision tasks.

🌟 Key Features

Search image on Flickr using keywords.
Download images directly for dataset assembly.
Streamline the process of collecting training data for YOLO models.

🔧 Requirements

Ensure you have Python 3.7 or later installed. The required dependencies can be installed using:

pip install -U -r requirements.txt

Key packages include:

flickrapi

🛠 Install

To set up the Flickr scraper, follow these steps:

git clone https://github.com/ultralytics/flickr_scraper
cd flickr_scraper
pip install -U -r requirements.txt

⚙️ Run

Before you begin:

Obtain a Flickr API key here.
Insert your API key and secret into flickr_scraper.py:

# Replace with your Flickr API key and secret
key = "YOUR_API_KEY"
secret = "YOUR_API_SECRET"

Execute the script with your search criteria. Specify the number of images to fetch (--n) and use --download to save the images locally. Downloaded images are saved to flickr_scraper/images. Keep in mind Flickr's rate limits and terms of use. Learn more from the Flickr API documentation.

Example command:

python3 flickr_scraper.py --search 'honeybees on flowers' --n 10 --download

You will see output similar to:

0/10 https://live.staticflickr.com/21/38596887_40df118fd9_o.jpg
...
9/10 https://live.staticflickr.com/1770/43276172331_e779b8c161_o.jpg
Done. (4.1s)
All images saved to /Users/glennjocher/PycharmProjects/flickr_scraper/images/honeybees_on_flowers/

Images are then available in the specified folder, ready for further processing or training.

📜 Cite

If our project assists in your research or work, please consider citing it:

🤝 Contribute

We welcome contributions from the community! Whether you're fixing bugs, adding new features, or improving documentation, your input is invaluable. Take a look at our Contributing Guide to get started. Also, we'd love to hear about your experience with Ultralytics products. Please consider filling out our Survey. A huge 🙏 and thank you to all of our contributors!

©️ License

Ultralytics is excited to offer two different licensing options to meet your needs:

AGPL-3.0 License: Perfect for students and hobbyists, this OSI-approved open-source license encourages collaborative learning and knowledge sharing. Please refer to the LICENSE file for detailed terms.
Enterprise License: Ideal for commercial use, this license allows for the integration of Ultralytics software and AI models into commercial products without the open-source requirements of AGPL-3.0. For use cases that involve commercial applications, please contact us via Ultralytics Licensing.

📬 Contact Us

For bug reports, feature requests, and contributions, head to GitHub Issues. For questions and discussions about this project and other Ultralytics endeavors, join us on Discord!

flickr_scraper's People

Contributors

Stargazers

Watchers

flickr_scraper's Issues

Script won't pull photos

I attempted to use the command suggested and have tried to pull other things however I always get this error. It creates the necessary folders, but won't grab any of the photos. Not quite sure how to fix this. Any help would be much appreciated!

`flickr_scraper` % python3 flickr_scraper.py --search 'honeybees' --n 10 --download           
Traceback (most recent call last):
  File "/Users/fah/Projects/flickr_scraper/flickr_scraper.py", line 62, in <module>
    get_urls(search=opt.search,  # search term
  File "/Users/fah/Projects/flickr_scraper/flickr_scraper.py", line 31, in get_urls
    for i, photo in enumerate(photos):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/flickrapi/core.py", line 690, in data_walker
    photoset = rsp.getchildren()[0]
AttributeError: 'xml.etree.ElementTree.Element' object has no attribute 'getchildren'

ERROR: "unrecognized arguments" when the search term has multiple words

Hey! First of all: congrats for the api. Well, every time I try to search something with more than one word, like 'Albert Eistein' I'm having the issue below:

\flickr_scraper-master>python flickr_scraper.py --search 'flat design' --n 10 --download
usage: flickr_scraper.py [-h] [--search SEARCH] [--n N] [--download]
flickr_scraper.py: error: unrecognized arguments: design'

It doesn't happen when the search term is like 'Beatles'.

images

Images for README are hosted here.

raise FlickrError

Hi, when I run following script:

python flickr_scraper.py --search 'honeybees on flowers' --n 10 --download

an error occurred.

Traceback (most recent call last):
  File "flickr_scraper.py", line 68, in <module>
    download=opt.download)  # download images
  File "flickr_scraper.py", line 33, in get_urls
    for i, photo in enumerate(photos):
  File "C:\Users\nanwa\anaconda3\lib\site-packages\flickrapi\core.py", line 688, in data_walker
    rsp = method(page=page, **params)
  File "C:\Users\nanwa\anaconda3\lib\site-packages\flickrapi\call_builder.py", line 38, in __call__
    return self.flickrapi_object.do_flickr_call(self.method_name, **kwargs)
  File "C:\Users\nanwa\anaconda3\lib\site-packages\flickrapi\core.py", line 337, in do_flickr_call
    **params)
  File "C:\Users\nanwa\anaconda3\lib\site-packages\flickrapi\core.py", line 407, in _wrap_in_parser
    return parser(self, data)
  File "C:\Users\nanwa\anaconda3\lib\site-packages\flickrapi\core.py", line 297, in parse_etree
    raise FlickrError(six.u('Error: %(code)s: %(msg)s') % err.attrib, code=code)
flickrapi.exceptions.FlickrError: Error: 100: Invalid API Key (Key has invalid format)

I tried both on windows and ubuntu, reports the same error

Order of photos downloaded

Hello,

Thanks for the great script. It really helps to quickly build datasets for training. But I have a question about the images that are being downloaded.

Are these sorted in a particular order before being downloaded? If I download 50 photos and then download 50 more. Will these be the same photos or are they randomly chosen? Say I later want to add more photos to a category.

Kind regards,

Jaimy van Schelven

ModuleNotFoundError: No module named 'flickrapi'

I get the following when i try to run it

e:\flickr_scraper-master>python3 flickr_scraper.py --search 'cars' --n 10 --download
Traceback (most recent call last):
File "e:\flickr_scraper-master\flickr_scraper.py", line 7, in
from flickrapi import FlickrAPI
ModuleNotFoundError: No module named 'flickrapi'

e:\flickr_scraper-master>

Status code 500 received

@glenn-jocher
I ran this command, the pictures were downloaded without problems, and all 20 pictures were downloaded. But this problem appeared in the console:

python flickr_scraper.py --n 20 --download

AttributeError: 'xml.etree.ElementTree.Element' object has no attribute 'getchildren' when running python3 flickr_scraper.py

Traceback (most recent call last):
  File "/Users/cjgetty/Desktop/flickr_scraper/flickr_scraper.py", line 66, in <module>
    get_urls(search=opt.search,  # search term
  File "/Users/cjgetty/Desktop/flickr_scraper/flickr_scraper.py", line 31, in get_urls
    for i, photo in enumerate(photos):
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/flickrapi/core.py", line 690, in data_walker
    photoset = rsp.getchildren()[0]
AttributeError: 'xml.etree.ElementTree.Element' object has no attribute 'getchildren'

LOL