Git Product home page Git Product logo

web-crawler's Introduction

πŸš€ web-crawler

μ†Œκ°œ

  • μž…λ ₯된 μ£Όμ†Œλ₯Ό ν¬λ‘€λ§ν•˜μ—¬ 좜λ ₯ν•˜λŠ” μ–΄ν”Œλ¦¬μΌ€μ΄μ…˜μž…λ‹ˆλ‹€.
  • URL을 μž…λ ₯ν•˜μ—¬ 좜λ ₯λ‹¨μœ„λ₯Ό μž…λ ₯ν•˜μ—¬ μ œμΆœν•˜λ©΄, ν¬λ‘€λ§ν•œ κ²°κ³Όλ₯Ό μ˜μ–΄,숫자λ₯Ό κ΅μ°¨ν•˜μ—¬ μ˜€λ¦„μ°¨μˆœ 좜λ ₯ν•©λ‹ˆλ‹€.
  • μ˜΅μ…˜μ„ ν†΅ν•˜μ—¬ HTML을 μ œκ±°ν•  것인지 μ œκ±°ν•˜μ§€ μ•Šμ„ 것인지 선택할 수 μžˆμŠ΅λ‹ˆλ‹€.

싀행방법

  • λ³„λ„μ˜ μ„€μΉ˜λŠ” ν•„μš”ν•˜μ§€ μ•Šκ³  ./gradlew bootrun으둜 μ‹€ν–‰κ°€λŠ₯ν•©λ‹ˆλ‹€.
  • μ‹€ν–‰ν›„ http://localhost:8080/swagger-ui/index.html 으둜 μ ‘μ†ν•˜μ—¬ ν…ŒμŠ€νŠΈ κ°€λŠ₯ν•©λ‹ˆλ‹€. μŠ€ν¬λ¦°μƒ·

μ–΄λ–»κ²Œ κ΅¬ν˜„ν–ˆλ‚˜μš”?

  • 크둀링 라이브러리λ₯Ό μ΄μš©ν•˜μ—¬ κ²€μƒ‰ν•˜κ³ μž ν•˜λŠ” μ‚¬μ΄νŠΈμ˜ HTML 정보λ₯Ό λ°›μ•„μ˜΅λ‹ˆλ‹€.
  • 검색 μš”κ±΄μ΄ TEXT인지 HTML인지에 따라 κ²€μƒ‰λœ 결과에 ν—ˆμš©λ˜μ§€ μ•ŠλŠ” 값을 필터링 ν•©λ‹ˆλ‹€.
  • ν…μŠ€νŠΈλ₯Ό μ •λ ¬ν•©λ‹ˆλ‹€.
  • 좜λ ₯ 쑰건에 따라 좜λ ₯ 기쀀을 λ‚˜λˆ•λ‹ˆλ‹€.

web-crawler's People

Contributors

etff avatar

Watchers

avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.