atmdata / atmdata Goto Github PK

ICAO data for aircraft type and manufacturer as in

• https://www.icao.int/publications/DOC8643/Pages/Search.aspx
• https://www.icao.int/publications/DOC8643/Pages/Manufacturers.aspx

can be scraped as follows (in R, similarly in other languages 😄):

library(httr)
library(jsonlite)
library(dplyr)
library(stringr)
library(readr)
library(tidyr)
library(here)

httr::POST("https://www4.icao.int/doc8643/External/AircraftTypes") %>%
  httr::content(as = "text") %>%
  fromJSON() %>%
  dplyr::as_tibble() %>%
  # strip white spaces
  dplyr::mutate_if(is.character, .funs = stringr::str_trim) %>%
  readr::write_csv(here::here("data-raw", "aircraft-type_ICAO.csv"))

POST("https://www4.icao.int/doc8643/External/Manufacturers") %>%
  httr::content(as = "text") %>%
  httr::fromJSON() %>%
  dplyr::as_tibble() %>%
  tidyr::unnest(cols = "Names") %>%
  dplyr::mutate_if(is.character, .funs = stringr::str_trim) %>%
  dplyr::distinct() %>%
  readr::write_csv(here::here("data-raw", "manufacturer_ICAO.csv"))

There is no mention on forbidden/restricted use and robots.txt seems not to forbid too:

User-agent: *
Disallow: /Meetings/Registration/*
Disallow: /ESAF/reg/*
Disallow: /Transfer
Disallow: /demo
Disallow: */Search/*
Disallow: */_layouts/*
Disallow: */Forms/*
Disallow: */mobile/*
Disallow: /CZIR/*
Disallow: /Security/mrtd/*
Disallow: /secretariat/PostalHistory/*
Disallow: /CAFICS/*

There is also https://doc8643.com/ (and a more comprehensive one [but in Russian] http://airwar.ru/) with pictures, technical data, minimalistic performace table and drawing.

For the list of RMAs (Regional Monitoring Agencies) see "States and designated RMAs" see
https://www.eurocontrol.int/service/european-regional-monitoring-agency

Some of them could be used to scrape hex, regno, ao; for example
https://www.eurocontrol.int/rmalive/operatorList.do

The Middle East Monitoring Agency's Approvals page is well done and data seems easy to get:

• https://midrma.com/en/rvsm

Others have medieval sites and/or data (or no public access)

It could be useful to be able to extract data from OSM via the Overpass API

Things like gates/stands, runway (thresholds' coordinates, width, length), for example
https://www.openstreetmap.org/#map=19/51.47123/-0.46122&layers=TD

great idea.
Here are a few possible (possibly open) source of airport data:

• https://ourairports.com/
• openstreatmap (some efforts to extract pure data, but rwy, arp, stands, gates et al. could be extracted; see example from Xavier's traffic library)
• https://www.wikidata.org/ IMHO a trove of data also for aviation in general

I was thinking, in line with issue #2, that it could be cool to present code snippets to retrieve some of the data we list in the sources.
Many examples may be available using Xavier's traffic library...

For me the goal would ultimately be to be able to provide usable datasets directly...so let's collect the recipes!