Below is the state-wise summary of habitation dataset.

• '# habitations' - number of habitations found in each state's shapefile
• '# district codes' - unique number of district codes used in shapefile
• '# block codes' - unique number of block codes used in shapefile
• '#h missing district_id' - no of habitations without district attribute
• 'master_district_ids' - unique number of district codes used in master data file
• 'master_block_ids' - unique number of block codes used in master data file

So, we have missing attributes for some habitations.

More importantly some states have more district codes than what master file is mentioning. There are 3 such states.

Kerala-Tamil Nadu diff sample

roads_diff.gpkg.zip

• Punjab from Habitation
• 13 files from Proposals (either no proposals or issue with file generation)

Starting this thread to discuss how we can fix the habitation errors in the existing data but also to add missing habitations in our datasets. The habitation data is important to plan for any future connectivity program and might be needed in the near future.

We are currently exploring the below two approaches to find missing habitations in our database:

• Using geo-tagged data from government schemes (SBM, Nal Jal etc)
• Using clustering on facebook pop-data

Apart from that what could be useful?

• SHRUG (not sure what all they have in GIS, but I am planning to write to Sam/Paul.
• OSM (Settlements in OSM but not in PMGSY dataset)
• Census 2011 EBs (I wonder if they are geo-tagged)
• Any other sources?

Known issues with our habitation data:

• Duplicate points
• Ghosts (no settlement underlying it)
• Double counted population
• Missing habitations

1. Same road representation and intersection with other OSM objects - in some cases both OSM and PMGSY represent the same road but don't follow the same path. Since both these sources have been heavily derived from satellite imagery this might be due to alignment errors between the different sources. This might lead to the undesirable result that PMGSY roads intersect with other OSM objects like buildings.
   Examples:

• A location - 31.604613, 76.5811358
• B location - 31.6274521, 76.8230259

2. Overlapping PMGSY roads - Roads from PMGSY might overlap each other.
   Examples:

• A location - 31.6643515, 76.6984829

3. Roads/ways in PMGSY are not continuous. PMGSY data might contain roads which end "in the air", although a clear connection to the nearest way seems reasonable or is visible on the underlaying satellite imagery.
   Examples:

• A, location - 31.5947871, 76.5972274

• B, location - 31.6297787, 76.5892687

• C, location - 31.6563769, 76.6388655

• D, location - 31.5864645, 76.6567495

4. Intersection of roads in PMGSY not represented by common node. To represent the intersection of roads, nodes of ways must intersect, i.e., have a common node. In PMGSY, roads often end in the non-node part of a way.

5. Improper breaks in roads. For purposes of routing, it is important that major roads or ways are broken at intersection with other roads. This might not be the case with PMGSY data.

6. Intersection with other features - Roads in PMGSY do not inform of bridges, thus leading to conflicts with water streams/rivers.
   77.6257746,31.1208713
   

7. Sharp angles - Roads in PMGSY are often sharp around hair-pin bends, necessitating manual intervention.

8. Road not visible in satellite imagery. Some roads in PMGSY are not visible in openly available satellite imagery available via OSM. This prevents double-checking of their existence or their path.

Examples:

• A, location 30.2971615, 78.9484727

9. PMGSY roads are wrongly marked. Roads are clearly wrongly marked compared to the satellite imagery
   Examples:

• A, location 30.2997043, 78.9569806

I'd like to start cleaning the block data - can we do this in an effort and back apply the updates to the official data?

• Needs mapping to LGD data. So it can be used in a meaningful way.
• https://lgdirectory.gov.in/
• https://thejeshgn.com/2021/03/20/mapping-local-government-directory-to-wikidata/

For convenience to explore the data in qgis, would be useful to add a simple bash script that will compile the individual zips into a single gpkg

We need to put up a FAQ document on the portal similar to what we did when we released the PMGSY-III facilities: http://omms.nic.in/Home/PMGSYRuralDataset/

I am currently collating the common questions here: https://docs.google.com/document/d/1EbzAfG_NGmolsuqMZgF0-y3tVuPZf9BAvepRZvU97bI/edit?usp=sharing

Please add them as comments on the document, and if relevant, we'll try to answer them. Eventually, will put the document on the website for larger consumption.

Started a conversation with Facebook Roads Team to get road data back to PMGSY with a more usable License than ODBL -

facebookmicrosites/Open-Mapping-At-Facebook#21

Should we move to git lfs or split >100mb files into chucks? What's the limit on repo size? cc @thejeshgn @planemad

There has been an update at the source, from a preliminary binary comparison, the following files need to be updated -

• Habitation

  • Chhattisgarh.zip

• Bound_Block

  • JammuAndKashmir.zip
  • Ladakh.zip

• Proposals

  • All of them