smutt / icann-dl Goto Github PK
View Code? Open in Web Editor NEWA script to find and download PDF publications from www.icann.org
License: GNU General Public License v3.0
A script to find and download PDF publications from www.icann.org
License: GNU General Public License v3.0
board-cor broken for 2012,2013
icann_cor is by far the largest directory, and it is also the one least likely to be of interest. Maybe add an example of how to exclude it from rsync:
rsync -av --exclude="icann_cor/*" icann-pdf.depht.com::pub .
Right now we're not getting updates on OCTO publications. We catch new OCTO publications, but when already existing OCTO publications are updated and a new one published we don't register them as new. They reuse filenames and replace the old publication with the new one.
We'll need to do a HTTP HEAD and check the size. Not perfect, but the best we can do.
We'll also need to develop a naming scheme since they use the same filename for every version.
We don't want to grab stuff like this.
https://www.icann.org/board-advice-status-current.pdf
We want the actual/redirected filename, not the displayed name.
This goes for .xlsx and .pdf files the same. We need to look at Content-Disposition header to find actual filename.
https://stackoverflow.com/questions/6881034/curl-to-grab-remote-filename-after-following-location
We're missing some early SSAC publications like SAC005. Maybe just download them by hand as one offs.
The RSS feed is hopelessly broken and probably needs a complete rewrite. Depressing that there is no decent library for Atom or RSS in Python3.
Consider downloading blog posts from icann.org.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.