Git Product home page Git Product logo

nyc-planimetrics's Introduction

nyc-planimetrics's People

Contributors

ekamptner avatar jdolanskyag avatar jwileman1 avatar mattyschell avatar mlipper avatar morgenhealy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nyc-planimetrics's Issues

Remove building footprints and street_centerline from planimetrics metadata

Buildings and centerlines are delivered in planimetrics primarily for internal agency use. We do not publish these on NYC Open data. Building footprints and centerlines on NYCMaps ArcGIS online are current building footprints that we are maintaining, not planimetrics.

Though these two are in the planimetrics capture rules and in the delivered data we should excise them from this repository to avoid confusion. They are documented under the nyc-geo-metadata repository.

add NYC Open Data item names

Layer names on open data can be different from the capture rules and from the delivered and published data. For example Plaza is "Public Plazas" on open data.

Add either a comprehensive crosswalk or an "aka on open data" under each layer where necessary.

Tag the 2022 release

We post-hoc tagged the documentation for the 2016 release when we started on 2022. Let's tag the 2022 release when we release 2022. This is the way.

Decide how to truncate column names in shapefiles

Planimetrics data must be provided to NYC Open Data as shapefiles. The maximum column name length in shapefiles is 10 characters.

In the past we created custom column names instead of letting the software auto-truncate. For example

FEATURE_CODE -> FEAT_CODE
SUB_FEATURE_CODE -> SUB_CODE.

Nice. But more work.

Probably we will want to create some sort of custom mapping. Here's a list of columns from prior releases that exceed the length limit.

Column Name Length
CONSTRUCTION_YEAR 17
SE_ANNO_CAD_DATA 16
SUB_FEATURE_CODE 16
LAST_STATUS_TYPE 16
LAST_STATUS_DATE 16
GROUND_ELEVATION 16
LAST_MODIFY_DATE 16
LAST_MODIFY_BY 14
FEATURE_CODE 12
GEOM_SOURCE 11
HEIGHT_ROOF 11
DOB_JOB_NUM 11
DESCRIPTION 11
STREET_NAME 11
BLOCKFACEID 11

Capture water tanks

Capture all water tanks as polygon features. All building over 6 stories are required to have a water tank. It's estimated that there are 12,000 to 17,000.

How was this dataset created?

Hi,

I'm trying to understand how this dataset was made? I would be interested in applying that methodology to other cities especially with now available high-resolution aerial data from Google Maps.

Thank you for your time.

Some layers might not follow OGC Standard? Issues in QGIS

Hi there - I was so excited to hear that the 2022 Planimetric data are available! I've been starting to explore/work with the data (from here and this is fantastic!

An issue I've encountered (the only one) is that at least some do not render correctly when loaded into QGIS. I have not given a detailed look at all of the layers, but have seen that for both the Roadbed and Shoreline layers, they will render okay when zoomed out, but fail to render when I zoom in. For both of these layers, running the "Fix Geometries" algorithm using the "Linework" repair methods seems to fix this issue best I can tell (with the caveat that I haven't given a good look in ArcPro to compare, but the original layers seem to perform well there).

Here's a screenshot of that menu for easy reference:
image

For the Roadbed layer, the "Force right-hand-rule" tool also seems to work (but I believe that is only applicable to polygons).

Given the issues I'm seeing, and what fixes work, it seems like maybe these are not necessarily following the OGC standard geometry definition, but rather the Esri standard? I wonder if these issues could be addressed in the data available through official channels to support a broad base of users who may use a wide variety of tools.

Happy to connect directly or do additional testing if helpful. And in case it's helpful, here is the info on my QGIS install:
image

Thanks!
Mike

Correct cooling towers documentation

Cooling towers are different from water tanks. Confusion about this (important!) distinction is in the original capture rules and some of it has made it into this repository.

Review the text carefully. This should be corrected under cooling towers.

Building Information Number of the structure the water tank is located on. Obtain from building footprints polygon feature that contains the water tank shape.

Split features for cartographic purposes

Split pavement edge, curbline and roadbed where they are above or below a building or other roadbed (e.g., bridge raised roadway). Attribute the feature as above, at grade or below.

For roadbed:

  • split polygons where they are intersected by other structures (e.g., roadbeds, transportation, buildings). In the example below the two roadbed polygons (white) on either side of the skybridge would be separate polygons coded as 'at grade'. The roadbed polygon under the skybridge would be coded as ' below structure'

pavement_below

In the example below (Lincoln tunnel), the tunnel would be coded as 'below grade'
pavement_below2

housekeeping - tags and branches

Get organized for the 2022 updates.

  1. Create a branch called next
  2. Fork next for all work
  3. Add a tag planimetrics_2014
  4. Use the planimetrics_2014 tag permalink in the documentation. It should say something like "looking for older capture rules - planimetrics_2014 (url here)"

Add Z dimension to "geometry type" where 3d data was captured

Several feature classes are collected with z values. When published in the projection "NAD83 / New York Long Island (ftUS)" (epsg:2263) these z values don't make any sense. Most publishing endpoints flatten these datasets to 2D.

We should at least indicate the z values were collected as part of the capture rules. Add it to the "Geometry Type" under the feature classes where necessary.

Potential Source of Confusion - Park Boundary along Parkways

Flagging this as a potential source of confusion:
Parkways in NYC (e.g., Belt Parkway in Brooklyn/Queens and Korean War Vets Parkway in Staten Island) are listed in the NYC DPR Properties Layer, though they don't really provide access to people. Thus, while medians and boundary vegetation along these roads are designated in the Park/Open Space dataset as "Park Boundary," it might be helpful to define this intricacy somewhere. Or perhaps categorize Accessible vs In-Accessible Open Space/Park Land, which would be more broadly useful.

Holler with any questions on this! Thanks!
mike treglia

create feature view images where missing

All of the subtypes have a "feature view" graphic that outlines examples of the feature on aerial or oblique imagery. One exception is cantilevered buildings which are new in planimetrics 2022.

Return to this at a later date and add the feature view for cantilevered buildings. Oblique imagery probably makes sense.

distinguish between jetties and groins

Is it too in the weeds to note that groins are being included in the hydro structure - jetty subtype?

Engineers build jetties at the mouth of a river or harbor to keep the channel open.

Engineers build shorter, fatter groins perpendicular to the shoreline, commonly to prevent beach erosion.

Most, if not all, of the jetties captured in planimetrics are in actual fact groins. Here is one possible exception.

image

These are all groins.
image

Inconsistent with MapPLUTO

I am trying to intersect the Footprints map with MapPLUTO. But it appears that the polygons in these two datasets are not perfectly consistent: for many buildings, a small proportion of them are outside the corresponding tax lots. Is this a known issue? And is there a method to deal with this?

Use natural language names

The feature class names requested in the capture rules sometimes differ from what is delivered. And in both cases the names inconsistently follow natural language.

For example, retaining wall is two words. A person searching for info on retaining walls will type retaining wall into a search. However the feature class name delivered is smushed into RETAININGWALL.

Column names and subtypes should also consistently follow natural language.

capture rules delivered natural language
MISC STRUCT POLY MISC_STRUCTURE_POLY miscellaneous structure polygon
OPEN SPACE OPEN_SPACE_NO_PARK open space not park
PAVEMENT EDGE CARTO PAVEMENTEDGE_CARTO pavement edge cartographic
RETAINING WALL RETAININGWALL retaining wall
SIDWALK CENTERLINE SIDEWALK_LINE sidewalk centerline
UNDER CONSTRUCTION UNDER_CONSTRUCTION_UNKNOWN under construction unknown

Updating names to match natural language may be ill-advised if it breaks linkages to prior releases and names.

Clarify Agency Name

The Department of Information Technology and Telecommunications (DoITT) is now the Office of Technology and Innovation (OTI).

@timcala37 suggests:

add a line to the Introduction section something like: "As part of a reorganization of City government agencies, DoITT was renamed as 'Office of Technology & Innovation (OTI)'. DoITT and OTI are equivalent terms for the same city agency."

Consider updating this repository for planimetrics released in 2023

This repository has been left adrift. The NYC Office of Technology and Innovation will release new planimetrics data in fall of 2023.

I think the documentation here is outstanding. I use it often. As a user I would like to see the documentation continuously updated.

I would also like to be able to interact with the data experts here in a public forum. I always learn a lot about this data both from the internal experts at the agency and from other users like me. I anticipate having questions about the new release and would like to make suggestions for future planimetrics captures.

clarify Z values

When the vendor delivers planimetrics to CUNY and OTI it is in the form of an ESRI file geodatabase. All spatial data is in spatial reference id 2263 aka "New York State Plane Lambert Conformal Conic." Some of the feature classes in the geodatabase have Z values. These are noted in the repository with geometry type Polyline Z or Point Z (see #49).

Speaking personally here, as a mid-career civil servant with a similarly mid understanding of coordinate reference systems, this is nonsense. Lambert conformal conic projections are flat surfaces. At best the Z values are maybe related to the NAD 83 datum where elevations exist. But then why deliver Z values in an X/Y projection? And more importantly, is this even true?

My colleagues usually look for the "stop generating" button at this point in the conversation. Sometimes an example helps.

If one coordinate of a curb at the top of a hill in Central Park has a Z value of 10, what does this mean? Is this point 10 feet above the surface of the hill? Is it 10 feet above some general contour of the topography of Central Park? Is it 10 feet above mean sea level? Is it 10 feet above the geoid?

image

Much of the time Z values don't matter. But bread and butter GIS operations like nearest neighbor will return different results when the third dimension is included. Not to mention the fact that collecting, accounting for, and documenting the Z values in each format requires a fair amount of effort for data points that provide seemingly no use.

Remove unused name columns

Consider removing the following:
● MEDIAN.STREET_NAME
● OPEN_SPACE_NO_PARK.NAME
● RAILROAD_STRUCTURE.NAME
● TRANSPORT_STRUCTURE.NAME

Update previous captures download links

@mrahman-doitt I'm assigning this open data question to you. Let me know when you have a final answer and URLs and I will update the repository.

These NYC Open Data file geodatabase download links should be verified and updated.

image

I think the plan is for 2022 to continue to point to the existing "latest" NYC Planimetrics:

https://data.cityofnewyork.us/Transportation/NYC-Planimetrics/wt4d-p43d

And the 2016 delivery will need to be archived under a new 4x4 like this:

image

Better define medians

Considerations:

  • Traffic islands without continuous medians (along the block) are not medians, medians are midblock treatments and traffic islands are intersection treatments (NYCDOT).
  • Determine thee difference between traffic islands and median and attribute as necessary.

clarify pavementedge_carto delivery

The capture rules say that only 2 fields will be delivered for pavementedge_carto. They are VISIBILITY and STATUS.

VISIBILITY
Update from imagery, provided reference data
Description:
Only pavement edges that cross other pavement edges or building footprints need attribution. All others can be left as NULL.
Values will be either:
“Show” – for top pavement edge feature representing the roadway that is fully visible in the aerial imagery.
“Hide” – for pavement edge feature that is obscured by either pavement edge or building footprint feature that is above it.

STATUS
Calculated
Description
Field indicating the feature status as it fits into one of the following categories:
a) NEW. A feature captured for the first
time during the 2014 planimetrics update project.b) UPDATED. The feature existed previously but has been updated during the 2014 planimetrics update project.
c) UNCHANGED. The feature is unchanged
from the source planimetrics database.

What was actually delivered is Z_CENTER, Z_START, Z_END, and pos_yitz. pos_yitz was our internally generated column from 2016 planimetrics that we used to symbolize the NYC basemap. It looks like the vendor went ahead and updated this column correctly and pos_yitz is useable for cartographic symbolization in 2022 despite its odd name.

image

update layer names to match delivery

The capture rules have been a little sloppy with names. The documentation in this repository should match what is actually delivered and published.

These should be updated.

misc_struct_poly
misc_structure_poly

Pavement_Edge_Carto
pavementedge_carto

retaining_wall
retainingwall

sidewalk_centerline
sidewalk_line

under_construction
under_construction_unknown

Decide what to do with "comments" and other working columns

The planimetrics 2022 delivery includes several columns that probably shouldn't be published. This gets confusing because in most cases we are saying that the delivered data is gospel even if it doesn't match the capture rules.

If we decide not to publish columns like "comments" we'll need to revise the planimetrics 2022 documentation in this repository.

Misc Issues/Comments

1.) Within ground elevation for Buildings ... "A bare-earth surface is generated from the 2010 LiDAR by first removing any points that fall within a building polygon" ... Is 2010 still correct?

2.) Don't forget to update the URL to 2022 file gdb. "Previous Capture" table needs to be updated with this download URL ... when ready to publish ...

3.) These feature classes exist in file gdb, but I did not include in metadata because I think NYC does not publish these ... just making sure:
-Deleted_line
-Deleted_point
-Deleted_polygon
-NYCPlanimetric_Topology
-Updated_line
-Updated_point
-Updated_polygon
4.) These feature classes exist in file gdb and were newly added to metadata:
-Curb Cut
-Pavement Edge carto
-Street Centerline
-Water Tank

Units of Measurement

Please update the documentation to indicate the units of measurement for each data point. For example, "inches", "feet" or "miles".

Building_footprint notes- fyi

Building_footprint notes- fyi (not an issue ... just so you know)

2022 Capture Rules 'requested' these Building footprint subtypes be captured but they are not in planimetric data. They were all listed as "New domain value":

GAS_STATION_CANOPY
STORAGE_TANK
PARKING
AUXILIARY_STRUCTURE

TEMPORARY_STRUCTURE

Note : CANTILEVERED_BUILDING was also a new domain value, but it was captured in the planimetric data

just so you know ...

Numeric feature codes

Are the numeric feature_codes used elsewhere (just checked a couple feature classes so far - but seems they are not used in file gdb)? ex: Building_footprint// Skybridge = 2110

If not, probably ought to remove from metadata ... just don't want to start removing 'good' info with checking ...

Looks like here is what exists:
Fior example in the feature class called BUILDING_FOOTPRINT. The FEATURE_CLASS and SUB_FEATURE_CODE are both the subtype name ( numeric code not used in either). Ex: FEATURE_CLASS = Building ... and SUB_FEATURE_CODE = Building

In the metadata, there are subtype names and codes:

Subtype Feature Code
Building 2100
Sky Bridge 2110
Building Under Construction 5100
Garage 5110

I can 'clean' this up by removing numeric codes but wanted to double check first. Remove numeric codes?

Descriptions Needed for Attributes/Columns

I added quite a few missing or new attributes. I could not find descriptions for these in the Capture Rules Doc. I took an educated guess on some, other I left blank.

Here is the list that needs to be updated or verified:
1.) Park - need descriptions for LANDUSE and SYSTEM

2.) Pavement Edge Carto Attributes - need descriptions for:
Z CENTER
Z START
Z END
POS YITZ
By the way, this feature class does not have the 'standard' 4 attributes

3.) Street Centerline:
-please check all of them for accuracy- they are all newly added to metadata and some were not in capture rules document
-two attributes that are in data were not documented in metadata- created_date and modified_date (assumed NYC did not publish them ... let me know otherwise)
-many are longer than 10 characters- please provide 'truncated' column name
-this feature class does not have the 'standard' 4 attributes ... just fyi

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.