microsoft / roaddetections Goto Github PK

Its not a valid geojson file so i cannot open it any GIS software.

Any simpler way to convert downloaded tsv file into valid geojson file ?

Hi, I work for DoltHub, the maintainers of Dolt, a version controlled database (think Git and MySQL had a baby).
This is shameless self-promotion, but we took the liberty of importing the data into Dolt and posting it on DoltHub.

By importing to Dolt, I was able to address the following issues:

1. #6
2. #11

The data is also easier to query, update, and modify.

My dream would be for you all to move this project to DoltHub, but short of that maybe put a link in your README to the Dolt version?

Any chance that the approach used here could generate estimates of additional attributes for road segments? I'm particularly interested in road widths as a starter, important for priortising locations for sustainable transport interventions, e.g. new cycleway protected from potentially fast moving motor traffic with light separation (wands, bollards etc).

Many thanks!

Nice work! Would you be kind enough to share the codes for Geometry Generation? I will only use it for academy purpose. Thanks in advance.

Is the code available for the process of geometry generation from semantic segmentation? In other applications I use watershed segmentation and other morphological operations to do the same. I'd be interested to experiment with your approach.

This repository is currently missing a LICENSE.MD file outlining its license. A license helps users understand how to use your project in a compliant manner. You can find the standard MIT license text at the Microsoft repo templates LICENSE file: https://github.com/microsoft/repo-templates/blob/main/shared/LICENSE.
If you would like to learn more about open source licenses, please visit the document at https://aka.ms/license.

I tried to bring into qgis as is, wouldn't open. renamed file extension to .geojson, still wouldn't open.

replaced all lines starting with AIA and trailing spaces with '' and it worked.

I don't know if these dumps are full region coverage as expected though. Can anyone assist?

The records are marked with JAP instead of JPN.

The USA data here has the following issue:

• Unlike all other regions' data, of the 54,484,737 GeoJSON entries in the USA data 204,789 are invalid as they consist of single-value zero-length LineString rather than Point geometry elements which causes issues when trying to process the files

There are then the following inconsistencies when compared to the other region data files

1. The region file name is USA.zip where all other region files are called <region>-Full.zip, for example the East Africa region is AfricaEast-Full.zip
2. The file-name in the zip-archive is _USA.tsv where all other region files are called <region>-Full.tsv, for example the East Africa region is AfricaEast-Full.tsv
3. Unlike all other regions' data, the first column in the USA region TSV file does not contain a three-alpha country code, for example GBR for Great Britain

Outwith this, thank you for making this excellent data set available

Oceania-Full.zip is 282 MB at the moment. If its GeoJSON file was partitioned by country and sorted the ZIP file would be 244 MB instead. This would allow people to download the ZIP file faster. They would also use less space picking out the countries they're interested in. The GeoJSON would open right away in QGIS and other GIS software without first needing to ETL the TSV.

$ vi a.sh

sort AUS.geojson > AUS.sorted.geojson
sort NZL.geojson > NZL.sorted.geojson
sort PNG.geojson > PNG.sorted.geojson
sort VUT.geojson > VUT.sorted.geojson
sort FJI.geojson > FJI.sorted.geojson
sort SLB.geojson > SLB.sorted.geojson
sort TON.geojson > TON.sorted.geojson
sort WSM.geojson > WSM.sorted.geojson
sort FSM.geojson > FSM.sorted.geojson
sort KIR.geojson > KIR.sorted.geojson
sort PLW.geojson > PLW.sorted.geojson
sort MHL.geojson > MHL.sorted.geojson
sort TUV.geojson > TUV.sorted.geojson
sort NRU.geojson > NRU.sorted.geojson

$ cat a.sh | xargs -n1 -P4 -I% bash -xc '%'
$ zip -9 Oceania.sorted.zip \
    AUS.sorted.geojson \
    NZL.sorted.geojson \
    PNG.sorted.geojson \
    VUT.sorted.geojson \
    FJI.sorted.geojson \
    SLB.sorted.geojson \
    TON.sorted.geojson \
    WSM.sorted.geojson \
    FSM.sorted.geojson \
    KIR.sorted.geojson \
    PLW.sorted.geojson \
    MHL.sorted.geojson \
    TUV.sorted.geojson \
    NRU.sorted.geojson

$ unzip -l Oceania.sorted.zip

Archive:  Oceania.sorted.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
1071521607  2023-04-10 18:58   AUS.sorted.geojson
185466598  2023-04-10 18:57   NZL.sorted.geojson
 28007237  2023-04-10 18:57   PNG.sorted.geojson
  6470562  2023-04-10 18:57   VUT.sorted.geojson
  5832797  2023-04-10 18:57   FJI.sorted.geojson
  4423195  2023-04-10 18:57   SLB.sorted.geojson
  1047604  2023-04-10 18:57   TON.sorted.geojson
  1066450  2023-04-10 18:57   WSM.sorted.geojson
   307308  2023-04-10 18:57   FSM.sorted.geojson
   190892  2023-04-10 18:57   KIR.sorted.geojson
   242639  2023-04-10 18:57   PLW.sorted.geojson
   119872  2023-04-10 18:57   MHL.sorted.geojson
    44300  2023-04-10 18:57   TUV.sorted.geojson
    38006  2023-04-10 18:57   NRU.sorted.geojson
---------                     -------
1304779067                     14 files

$ unzip Oceania.sorted.zip NZL.sorted.geojson

For some of the largest datasets, like Canada and Japan, the 3-letter country identifier is redundant since every record in those ZIPs are for their respective countries.

Hello,

There's a large number of invalid linestrings in the _USA.tsv file that only contain a single point or are self-referencing.

Only about 88% of the rows in the US file are valid line strings.

• _USA.tsv = 54,484,737
• Valid geometries = 48,067,790
• Invalid geometries (or points): 6,416,947 (according to @turf/boolean-valid)

ST_IsValid returns false for an additional: 211,991 rows on top of that

Additionally, providing the file as a WKT representation per line would be easier for many folks to load into their database and is much smaller on disk.

Thanks,
Jeff

Is the data for Mexico and China to be published? I see North America has been broken down into countries, but there are only files for the USA and Canada.

A lot of roads on the Latvian side of the border with Estonia are marked as Estonian.

Most of Monaco's roads are marked as French. A very small part of Monaco is marked properly.

Spanish data cuts off 2KM before Gibraltar leaving almost no data for La Línea de la Concepción.

Roads well inside of Spain are marked as Portuguese.

Austria, Liechtenstein and Switzerland all have overlapping issues.

There is a stepping pattern from the French side as it overlaps much of western Switzerland.

Luxemburg and its neighbors.

Would it be possible to access the semantic segmentation tiffs for specific locations? I would like to use it as an input in predicting neighboring objects in other EO segmentation tasks.