Comments (8)
@caesar0301 , have you implemented this? Or is it wontfix
? Anyway, I have already started working on it.
from apd-core.
@KOLANICH Hi, I didn't notice that you are working on it. If so, that is great! Your idea is similar to what is considered in project https://frictionlessdata.io/. When you finish your idea, we can start a new service to host your API calls.
from apd-core.
@KOLANICH Can you introduce your method in details? Maybe I can give you some help. In my view, there are two potential ways to get the direct links of machine-processable sources. 1. Manually appended into the yaml files; 2. Use crawling tech to to that automatically.
I am fulfilling it by the second. Briefly, use data crawling tech to index dataset pages and endeavor to find original machine-processable source such as tabular/json/xml data. Then start independent processes to retrieve and update data periodically.
from apd-core.
Hi.
I didn't notice that you are working on it.
Closing issues without any hints (s.a. a message, or referencing in a commit solving the issue) about why it has been closed is just impolite.
Your idea is similar to what is considered in project https://frictionlessdata.io/. When you finish your idea, we can start a new service to host your API calls.
I have already read their specs. But their specs are vendor-hosted metadata. We need a standalone one. You know it's pain to deal with people just closing ignoring ones issues and pull requests or even worse just ignoring them because they have no time for them.
Can you introduce your method in details? Maybe I can give you some help. In my view, there are two potential ways to get the direct links of machine-processable sources. 1. Manually appended into the yaml files; 2. Use crawling tech to to that automatically
I think we should start from manually-created ones (with a minor dumb script (already implemented) infering columns types from data). When we have enough large dataset of links to datasets with their hand-crafted descriptions, we can staft training models deriving columns types from textual descriptions.
https://gitlab.com/KOLANICH/SurvivalDatasets
https://github.com/KOLANICH/SurvivalDatasets
is my current draft. Not very finished though and unsuitable to be used for anything for now (I currently use it to debug some survival analysis code).
from apd-core.
daviddiazvico/scikit-datasets#10
scikit-learn/scikit-learn#11619
openml/OpenML#876
are strongly related
from apd-core.
@KOLANICH My apologies! I made a mistake to think that this issues was opened by myself long time ago and without any active response, while your response didn't address my point. Actually, this is my fault. My opened issue was here (awesomedata/awesome-public-datasets#262)
from apd-core.
You could use DCAT( or DCAT-AP) to describe dataset metadata, at distribution level you have accessURL and download URL
from apd-core.
Thanks for letting me know.
from apd-core.
Related Issues (12)
- CALL FOR NEW DATA HOT 13
- Fix github's bug in rendering ftp link in markdown
- Automatic tools to enrich the data entries HOT 1
- Persian News Dataset
- Add fixme link to broken link
- Fix broken link to UK-DALE energy dataset
- cooldatasets.com no longer online HOT 2
- The homepage link to REDD Dataset does not work.
- Add howto for adding a "Complementary Collection" HOT 1
- GNU GPL is not very suitable for data HOT 1
- Code of conduct HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from apd-core.