Git Product home page Git Product logo

gh-automation-funda's Introduction

gh-automation-funda

GH_Automation_:: version

License MIT Python 3.11 Poetry Checked with mypy Code style: black Linting: Ruff

Funda.nl automation using GitHub Actions and Aiven for PostgreSQL.

See gh-automation-base for more information.

Usage

1. Create a new Google Sheet

This Google Sheet is used to control the dynamic settings of the automation. This way, you can change the settings at any time without having to change the code.

It should contain a sheet with the following content (for example):

area object_type price_min price_max days_old
["Amsterdam"] ["apartment"] 500000 1000000 3
["Almere", "Utrecht"] ["house", "apartment"] 300000 500000 3

Do not include the backticks in the actual sheet.

In the above example, the automation will search for apartments in Amsterdam that are between 500,000 and 1,000,000 euros and are no older than 3 days, as well as houses or apartments in Almere and Utrecht that are between 300,000 and 500,000 euros and are no older than 3 days.

To be noted that the days_old is only important for the first iteration of the automation. If the automation runs every N minutes, then most of the properties will be redundant, as they will have been found in the previous iteration. A small value is recommended, to avoid parsing a lot of useless properties.

2. Add your Google Sheet to the automation

If the URL of your Google Sheet is

https://docs.google.com/spreadsheets/d/12345678901-abcdefghijklmnopqrstuv1234_56789/edit#gid=9876543210

then the SHEET_ID is 12345678901-abcdefghijklmnopqrstuv1234_56789 and the GID is 9876543210.

In .env, add the following:

GH_AUTO_GOOGLE_SHEETS_1_SHEET_ID=12345678901-abcdefghijklmnopqrstuv1234_56789
GH_AUTO_GOOGLE_SHEETS_1_GID=9876543210
GH_AUTO_GOOGLE_SHEETS_1_MODEL=gh_automation_funda.persistence.settings.FundaSetting

Database structure

Entity-relationship diagram

erDiagram
    PROPERTY ||--|| PROPERTY_FUNDA_DATA : "has"
    PROPERTY_FUNDA_DATA ||--o{ PROPERTY_FUNDA_IMAGE : "has"
    PROPERTY ||--o| PROPERTY_CADASTRAL_DATA : "has"
    PROPERTY ||--|| PROPERTY_CADASTRAL_WOZ : "has"
    PROPERTY_CADASTRAL_WOZ ||--o{ PROPERTY_CADASTRAL_WOZ_ITEM : "has"
Loading

Full diagram in er_diagram.md (here).

gh-automation-funda's People

Contributors

mulugruntz avatar

Watchers

 avatar  avatar

gh-automation-funda's Issues

Filter out the properties that are already in DB

Two cases:

  • either the property was still available last time. We check that it's still the case. This can lead to an update.
  • if we already know that the property was sold, we know there won't be any other updates (we ignore the true selling price, disclosed 6 months later, for now)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.