Web-aggregator for whisky prices in ukrainian whisky shops
Copyright © 2016 mishadoff
Distributed under the Eclipse Public License either version 1.0
Scrapping prices from Ukrainian online whisky shops
License: Eclipse Public License 1.0
Make sure goodwine scrapping is stable.
This needed for consumers of priceous service (like bots) and FE for design prototypes
Epic task to release first minimum viable product which allows users to track prices for whisky for most popular online whisky shops in Ukraine.
For example if we process category "whiskey" we already know the type of spirit and can append it to solr document
Add solr collection with stats, where we can gather history stats for each scrapping process
Basic stats shall include
Sale description currently shown in larger font and red text. Do something more pleasent for eyes
Default links from the search page route to the store, which is ok. But we should provide some icon to open a consolidated product page on our side. It should show description and other props as well.
In 2019 it's a must. Users don't want to see Not Secure near your logo.
We need to handle russian words by enabling proper stemming algotithm
We have experienced a number of ways sites utilize paging for the products.
Let's implement the most common strategies, which can be reused by providers.
Would be good to track price changes for the specific products. For example how much was the bottle of Glenfiddich 12 year ago?
Tracking can be registered two ways:
Result of tracking should be stored to separate solr collection and it should be a block at the stats page
Would be nice to have an integration with cocktaildb in one way or another:
For example: get all alcohol drinks, starting with cheapest, that fit a given recipe. Or get all recipes, in which a given drink participates.
Show simple page like "Database unavailable, try again later"
A lot of people confused about search because there is no search button. Investigate google/duckduckgo search panels and do something similar.
When accounts available one of the user-specific features could be notify when price changes. For example you want to try out a new whisky but 1000 is a big price for it. You can get email or push (if mobile) about price drop.
When code is updated and merged back to master (either through PR or directly) we should build and redeploy artifacts to the server on OVH. Secrets store in Github Secrets
Query results can be cached for the same queries. When new data is available, we should invalidate cache.
Since we have a lot of stores which sold items without excise it should be clear to the user what he is buying. Restaurants and Cafes require excise on the alcohol
Flux is abandoned solr client, which prevents to migrate to latest clojure 1.10.
Some stores work better with thread pool with size 20, some with 10, some do not allow more than 3 parralled connections, so would be good to have this configuration per provider.
When scrapping is finished we get an email with scraping results and possible errors. Would be good to have the email formatted as a table with a list of stores and their results.
Whiskybase is a popular resource for whisky lovers which provides ratings, description, etc.
Would be good to provide options for users to see the whiskybase score, and possibly a link
It's kinda unclear do we need it or not but it's good for testing to have random button "Мне повезет"
Right now we writing to SOLR, which is obvlious, but would be good to produce CSV or XLS with daily updates
Make sure winetime scrapping is stable
Non-distractive background image for search
Sometimes we can see something like this:
"Найдено 1 товаров за 0.0 секунд."
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.