The three featured projects for CFPB are great and appreciated! But could we include https://github.com/cfpb/cfgov-refresh, the main consumerfinance.gov site, under which college costs and owning a home both live?

It's a small thing but makes for a tidier and more newb-friendly repo.

My company has completed several federally funded projects using open source assets and I'd like to link to some of the public github repos that contain those projects. I'd also like to include the narrative of the projects to explain what they do, etc. Will this feature be available on your site?

Also do you need help building out features like this? I could volunteer some team time if we have appropriate coding skills. I have a great team certified in Drupal here in the US. Thanks!

I saw some discussion about CONTRIBUTING.md in #34. It would be great to pull in the .github files along with any README and LICENSE into the pages on code.gov.

Can an SPDX file or reference identifier for the tracking of opensource software package licenses be included? Including the SPDX standard for tracking opensource packages that may be used within code repositories could provide for streamlined 3rd party license auditing or analysis.

About SPDX

The Software Package Data Exchange® (SPDX®) specification is a standard format for communicating the components, licenses and copyrights associated with software packages.

The SPDX standard helps facilitate compliance with free and open source software licenses by standardizing the way license information is shared across the software supply chain. SPDX reduces redundant work by providing a common format for companies and communities to share important data about software licenses and copyrights, thereby streamlining and improving compliance.

The SPDX specification is developed by the SPDX workgroup, which is hosted by the Linux Foundation. The grass-roots effort includes representatives from more than 20 organizations—software, systems and tool vendors, foundations and systems integrators—all committed to creating a standard for software package data exchange formats.

https://spdx.org/learn

Clear guidance, and examples, of how to ensure that 20% is valuable would be helpful.

Every project has a lot of support code being developed that can be clipped out as libraries, plugins or services.

By pulling these capabilities out and documenting them reasonably well, agencies can benefit from further development on this functionality by the community and other agencies.

For example, src/assets/img/logos/press/fcw.jpg is 1021x483 pixels, 106K, but is shown as a small thumbnail on the front-page and shouldn't need to be larger than a few K. 106K is 17% of your initial page load total for me (Chrome, desktop). Other images in src/assets/img/logos/press/ seems to suffer from the same problem.

I am assuming that the intention is to list every open source repository created by Federal agencies on this website.

If not, I would really like to see a list of all of them and prefer that you feature ones that you think deserve special recognition.

Right now, seeing all the repos for any given agency is a 4-5 step process otherwise: click on the agency, click on the project, click on the link to the repo, click on the acknowledgement that I clicked a link, click on the organization name on GitHub.

I know this was just recently launched (🎉 !), so please take my feedback as someone coming in completely cold to the project.

@okamanda says:

I'm trying to create a table of where executive agencies are hosting repos (e.g., github, bitbucket, etc), and I'm going by the list defined by Clinger-Cohen. I have the following, but I know I'm missing some. Do we have a current list?:

Agency Name ACRONYM
Department of Agriculture USDA
Department of Commerce DOC
Department of Defense DOD
Department of Education ED
Department of Energy DOE
Department of Health and Human Services HHS
Department of Housing and Urban Development HUD
Department of Interior DOI
Department of Justice DOJ
Department of Labor DOL
Department of State DOS
Department of Transportation DOT
Department of Treasury TRE
Department of Veterans Affairs VA
Environmental Protection Agency EPA
National Aeronautics and Space Administration NASA
Agency for International Development AID
Federal Emergency Management Agency FEMA
General Services Administration GSA
National Science Foundation NSF
Nuclear Regulatory Commission NRC
Office of Personnel Management OPM
Small Business Administration SBA

Thank you all so much for the feedback you've given on the draft schema and for the great discussion. It's been invaluable and is baked into the alpha version.

You'll find our alpha version of the project metadata schema below. We welcome your feedback as we iterate on it. We've been working to come up with something carefully considered and easy to comply with as soon as possible so that agencies can have the most time possible to prepare.

The schema needs to describe a vast and diverse universe of software, though we aren't the first folks to think about how to describe a software package. Some of you have mentioned other schemas that aim to solve similar problems. We've looked at them and have found one that we feel best matches the projects we aim to describe here.

Code for DC and DC employees extended a project schema created by BetaNYC and have created their civic.json. We are extending (and slightly modifying) it to create our own schema. They have schema creation and validation tools that we hope to leverage as well. Thanks to the team for sharing their great work!

code.json:

{
    "agency": "DOABC",
    "projects": [
        {
            "status": "Alpha",
            "vcs": "git",
            "repository": "https://github.com/presidential-innovation-fellows/mygov",
            "homepage": "https://agency.gov/project-homepage",
            "downloadURL": "https://agency.gov/project/dist.tar.gz",
            "name": "mygov",
            "description": "A Platform for Conecting People and Government",
            "tags": [
              "platform",
              "government",
              "connecting",
              "people"
            ],
            "languages": [
              "java",
              "python"
            ],
            "contact": {
              "email": "[email protected]",
              "name": "Project Coordinator Name",
              "twitter": "https://twitter.com/projectname",
              "phone": "2025551313"
            },
            "partners": [
                {
                    "name": "DOXYZ",
                    "email": "[email protected]"
                }    
            ],
            "license": "https://path.to/license" OR null,
            "openSourceProject": 1,
            "governmentWideReuseProject": 0,
            "exemption": null,
            "updated": {
                "lastCommit": "2016-04-30",
                "metadataLastUpdated": "2016-04-13",
                "sourceCodeLastModified": "2016-04-12"
            }
        }
    ]
}

You can see our working document here, with some discussion that's already taken place.

Required fields:

• Agency - The agency's acronym
• Project:
  • Name - The project name
  • Description - A description of the project
  • License - null or a URL to the project license
  • Open Source Project: 0, indicating a closed source codebase, or 1, indicating an open source codebase
  • Government-wide Reuse Project: 0 indicates that the project is not developed for government-wide reuse, 1 indicates that it is
  • Tags: A list of keywords that describe the project
  • Contact:
    • Email: Preferably an address dedicated to the project, though any address where a project contact can be reached

Optional fields:

• Project:
  • Status: The list of accepted options can be found in the civic.json specification: "Ideation", "Alpha", "Beta", "Production", "Archival"
  • VCS: The Version Control System that the project uses
  • Repository: The project repository URL
  • Homepage: The project homepage URL
  • Download URL: The URL where a distribution of the project can be found
  • Languages: A list of languages used in the codebase
  • Contact
    • Name: The name of a contact for the project
    • Twitter: The URL of the project Twitter account
    • Phone: The phone number of the project contact
  • Partners: A list of the acronyms of partner agencies involved in the project
  • Exemption: The exemption that excuses the project from government-wide reuse, a number 1-5 corresponding to the exemptions listed here: https://sourcecode.cio.gov/Exceptions/
  • Updated:
    • Last Commit: A timestamp of the last commit made on the project. Would ideally be populated dynamically.
    • Metadata Last Updated: The date that the inventory.json was last updated for this codebase
    • Source Code Last Modified: A field intended for closed-source software and software outside of a VCS. The date the source code or package was last updated

Note:

We differ with civic.json in our implementation of License. civic.json accepts only a URL. We also accept null in the event that there is no license.

Remaining questions:

• Are any of these fields likely to be overly burdensome for agencies to collect or difficult to interpret?
• Are there any reasons we shouldn't accept two different data types for License: null and a string?
• Should the Tags field accept any alphanumeric tag freeform, or should we define a set of tags? The former provides less clean data but maximum flexibility, while the latter keeps data clean and searches simpler, but is rigid.
• Are there other fields we should include in Contact?
• The value of Exemption would be a number between 1-5, corresponding to the exemptions in the policy. Is it a good idea to tie the Exemption field to the content of the policy? I assume this policy will change?
• What format should we collect the Updated field's timestamps in?
• Is there any other information we should collect about partner agencies?

Should there be an identifier for a 'project' similar to how it is in open data? We have data assets where the name has changed, but the identifier has remained the same. I imagine the same happening with 'projects' for the open source initiative.

The GitHub events in the "Activity" section of a repo on code.gov appear to be mostly empty and without context:

Pulling in some additional context here would be useful, although I might suggest eschewing the activity feed in favor of showing open issues or recently merged pull requests.

Not sure what your plans are with regards to this feature, so just take these suggestions as you will.

Section 7.1 of the policy states that:

in support of the objectives and requirements of this policy, agencies should strengthen internal capacity to efficiently and securely deliver OSS as part of regular operations. Additional information on this topic will be provided on Code.gov

Section 7.5 of the policy states that:

when agencies release custom-developed code as OSS, they shall append appropriate OSS licenses to the source code. Additional information on licensing will be available on Code.gov

Hi! I believe the Google Tag Manager code should be placed immediately after the opening body tag - not before the closing body tag. https://developers.google.com/tag-manager/quickstart

I noticed a src directory that contains only images, which raised this question. If I am a contributor, should I edit the HTML and CSS files in dist in my pull request? If so, it may be worth noting this in the README as dist typically represents the outputted code rather than the source files.

Thanks! I'm excited to see this site take shape!

Following up on #5, it looks like our color scheme is just shy of the suggested 4.5:1 ratio:

Add a code of conduct for this repository. 18F has a good one that may be appropriate to adopt more or less directly but will need a quick read through.

Section 5.1 of the policy states that:

Agencies should calculate the percentage of source code released using a consistent measure—such as real or estimated lines of code, number of self-contained modules, or cost—that meets the intended objectives of this requirement. Additional information regarding how best to measure source code will be provided on Code.gov

Section 7.4 of the policy states that:

Accessible, buildable, version-controlled repositories for the storage, discussion, and modification of custom-developed code are critical to both the Government-wide reuse and OSS pilot program sections of this policy. Agencies should utilize existing code repositories and common third-party repository platforms as necessary in order to satisfy the requirements of this policy.36 Code.gov will contain additional information on this topic.

Hi,

A Chrome engineer here. I noted a twitter discussion about this project and decided to trace the loading of the site on a moderate connection (emulated 3G) on moderate hardware (a ~$200 Moto G4).

The resulting chrome:inspect trace shows that an inordinate amount of script is running, delaying first load considerably: https://chromedevtools.github.io/timeline-viewer/?loadTimelineFromURL=https://dl.dropboxusercontent.com/u/6792555/code.gov/code_gov_3G_moto_g4_first_load_TimelineRawData-20161104T161147

The site appears to load ~270K of JS (~1.1MB unzipped) which blocks rendering for several seconds in the trace above. This seems, perhaps, excessive.

As part of the Open Source Pilot, Section 5.2.A requires the following:

Leverage Existing Communities. Whenever possible, teams releasing custom-developed code to the public as OSS should appropriately engage and coordinate with existing communities relevant to the project. Government agencies should only develop their own communities when existing communities do not satisfy their needs.

For discussion: what are things agencies should keep in mind when working with existing communities related to an open source project? For example, what do agencies need to know about contributing to a community maintained project? What should an agency that is developing a new project in a space where a robust community is already doing work?

Part of the Federal Source Code Policy requires that federal agencies make available an inventory of metadata describing their custom software. We’re exploring ways for agencies to provide their inventories. We want to implement a solution that works well for agencies and we need your help to do that.

The Federal Source Code Policy describes code.gov as “the primary discoverability portal for custom-developed code intended both for Government-wide reuse and for release as OSS.” The inventory data that agencies provide will be made available through code.gov. The data we collect should make it possible for agencies to find projects relevant to their needs.

There are two primary areas we see where decisions need to be made: the data format and what data is collected.

Data Format

The two options we are considering are CSV and JSON. The assumed benefit to a CSV-based approach is that it is easier for agencies to create and maintain a CSV than JSON. With this approach, we might create a system for agencies to submit their inventory CSV.
With a JSON-based approach, we might ask agencies to make the “inventory.json” available on their website and we would have a system to retrieve inventories as they change. One drawback to JSON is that it is more effort to maintain, takes specialized knowledge, and we may need to provide a tool to build the JSON. On the other hand, JSON is easy to work with programmatically and matches what Data.gov does, meaning many agencies have some familiarity with the process that inventory updating would entail.

The unanswered questions on data format are:

• Which data format is the best fit: CSV or JSON?
• Is it best to retrieve or ask agencies to submit their inventories?

Collected Data

In either data format, we need to determine what data we will collect. Below is a list of fields we are considering accepting.

Proposed required fields:

• Project Name: The name of the project
• Project Description: A description of what the software does
• Point of Contact Email Address: Email address for the project point of contact

Proposed optional fields:

• Version Control System: the VCS that the project uses
• Repository URL: The URL of the upstream project repository, if applicable
• Project URL: The URL of the project homepage
• Project Tags: Tags that will help Code.gov users find projects based on their needs
• Languages: Which programming languages are used in the software
• Last Updated: Timestamp of when the project codebase was last updated
• License: The type of license that the source code of the project is released under
• Open Project Status: Whether or not this project is open source
• Government-wide Reuse Project Status: Whether this project is designed for reuse across government
• Exemption: Which exemption, if any, is being relied upon for keeping the source code private

For an idea of what the data might look like, we have an early draft of a schema with example content: (https://gist.github.com/theresaanna/a82bfb39b64362bca04e4644706b0ce4)

The questions that we are looking to answer here are:

• Is this the right information to collect? Have we missed something?
• What should we consider on the part of agencies when implementing this data schema?

Thanks for your feedback! It’s crucial for us in meeting our goal of providing a system and schema that are easy to use and meets agencies' needs.

We need to build a backlog of content that will be helpful for folks at agencies who are implementing the sourcecode policy. There are number of explicitly required topics, which I've already opened issues for individually and marked as "Required Content:"

However, there are likely a number of other topics that agencies could use more elaboration on. We need to build a backlog of that content to start writing. We'll do that in part through user research but can use input here as well.

Because PHP.

I work for the Climate Prediction Center (DOC/NOAA/NWS/NCEP), and I have a few open source repos that I'd love to showcase in the initial code.gov launch. However, as per NOAA's IT Security Awareness Course it's currently against NOAA's IT policy to use a non-accredited cloud service (specifically calling out GitHub.com), making it impossible to showcase our work (and impossible to comply with the FSCP). To quote the course:

NOAA-Accredited Services are those for which the NOAA Acquisition and Grants Office has issued a contract award and a NOAA Senior Manager, such as the NOAA CIO, has reviewed the cloud service's security posture and made a risk-based decision to allow the use of the services "as is" or with mitigation from an independent assessor's findings.

Non-accredited cloud services have not been issued a NOAA contract award and/or a NOAA Senior Manager has not reviewed the service's security posture and made a risk-based decision to allow the use of the cloud services.

Non-accredited (less secure) cloud services include but are not limited to: iCloud, Dropbox, Box.net, Evernote, github, Constant Contact, and Springpad.

You may not store government data in non-accredited cloud services.

Questions:

1. How can I still showcase some open source repos? A URL to a tarball would kill my soul a little...
2. How do we push management to reverse such policies in a timely fashion so we can start complying with the FSCP?

I see that you have the 90 and 120 day due dates in your milestones, but it would be nice for those less-Github savvy to see this info listed on the current homepage.

Section 5.1 of the policy states that:

Within 120 days of the publication date of this policy, OMB shall develop metrics to assess the impact of the pilot program. Additional information on these topics will be available on Code.gov.

Many users do not have the correct mail client open with mailtos (osx user's pop up Mail.app when they use gmail or yahoo, win users have outlook).

When you try to manually email, you're prompted with (the -- is first line of my signature):

> --
Unknown command - "--". Try HELP.

Didn't take long to figure out I had to send SUBSCRIBE CODE but I suggest something like a ? icon that when hovered or clicked shows instructions like:

"Send SUBSCRIBE CODE to [email protected] to join mailing list.

https://github.com/USDepartmentofLabor/Swift-Federal-Data-SDK includes a URL in the repo description, but that doesn't show up in https://code.gov/#/explore-code/agencies/DOL

Recommendation: Schema.org for source code metadata model

Schema.org The Schema.org is the web data model used by the largest web search engines (Google, Yahoo, Microsoft Bing, etc.) to index and search all structured data on the web. Using schema.org will thus maximize the discoverability of all software artifacts published on the web. See: https://schema.org

Specifically for code.gov source code

• https://schema.org/SoftwareSourceCode

Background

Federal agencies and knowledge organizations such as IBM, Google (Knowledge Graphs) and National Library of Congress have adopted the W3C Linked Data standard (RDF) as their metadata standard to provide master data management for all their large data resources, making them compliant with all web search engines and technologies. VA, for example, is projecting all of its data assets using Schema.org JSON-LD to surface all 35 years of its data:

http://vistadataproject.info

JSON-LD. This is the W3C Linked Data standard for structured data on the web, also known as the Resource Description Framework (RDF). The JSON-LD serialization of RDF is the most web-programmer friendly form of RDF, and supplements the JSON standard with the "@ context" attribute. See:

• http://json-ld.org/
• https://developers.google.com/schemas/formats/json-ld

Section 7.2 of the policy states that:

Within 120 days of the publication date of this policy, each agency must update—and thereafter keep up to date—its inventory of agency information resources to include an enterprise code inventory that lists custom-developed code for or by the agency after the publication of this policy. [..] Agencies shall fill out this information based on a metadata schema that OMB will provide on Code.gov.

A button should be added under Read the Policy for Join the Discussion or akin

• deprecate sourcecode.cio.gov
• include the policy content in code.gov
• work with community to move related discussion (i.e. on sourcecode.cio.gov issues) to this repository and related channels
• redirect/deprecation notice on sourcecode.cio.gov for any current users.

There's a number of significant requirements in the source code policy for agencies to implement by the 90 and 120 day mark (below).

Though the policy does not require Code.gov until the 90 day mark, it has been rightly stood up immediately so that it can be built in the open, as an open source project. Given the important work that agencies are facing in the 90 and 120 day timeframes, it definitely makes sense to outline a sense of what help code.gov is going to offer them as soon as possible and to try to deliver that help ahead of the 90 day mark as much as possible so that agencies can factor it into their work and benefit from it.

Specifically, I'd suggest:

• creating an issue here to begin outlining what will go on code.gov
• standing up placeholder issues or files for those deliverables
• beginning a conversation here about which deliverables are more urgently needed by agencies that may struggle with any of the 90/120 day requirements
• inviting the open source community to contribute to ^^^

The 90 and 120 day mark is going to come quickly for agencies and it's important that code.gov provide useful support early and often. :)

Disclaimer: I'm really excited about Code.gov.

• "Within 120 days of the publication date of this policy, each agency must update—and thereafter keep up to date—its inventory of agency information resources to include an enterprise code inventory that lists custom-developed code for or by the agency after the publication of this policy."
• "Within 90 days of the publication date of this policy, each agency’s CIO—in consultation with the agency’s CAO—shall develop an agency-wide policy that addresses the requirements of this document."
• "Moreover, within 90 days of the publication date of this policy, each agency’s CIO office must correct or amend any policies that are inconsistent with the requirements of this document, including the correction of policies that automatically treat OSS as noncommercial software."

Section 5.2.B of the Source Code Policy requirest that as part of the Open Source Pilot, agencies:

Engage in Open Development: Software that is custom-developed for or by agencies should, to the extent possible and appropriate, be developed using open development practices. These practices provide an environment in which OSS can flourish and be repurposed. This principle, as well as the one below for releasing source code, include distributing a minimum viable product as OSS; engaging the public before official release;30 and drawing upon the public’s knowledge to make improvements to the project.

For discussion: what do agencies, in particular those with less history/expertise with open source, need to know about developing in a truly open way? What are key examplse of project and organizations that have gotten this right, inside or outside of government?

Section 5.2.C of the Source Code Policy, as part of the Open Source Pilot, requires that agencies:

Adopt a Regular Release Schedule: In instances where software cannot be developed using open development practices, but is otherwise appropriate for release to the public, agencies should establish an incremental release schedule to make the source code and associated documentation available for public use.

For Discussion: what should agencies keep in mind when establishing a regular release schedule for a project that is not being developed in the open in order to maximize public benefit?

When I click on the link to go to your repo, I know I'm clicking a link that leaves code.gov. You even say so:

Hijacking links feels like a dark pattern on the web, where hyperlinks matter. If I mouse over a button and see the link, I expect to be taken to that link.

The world's largest search engines (Google, Yahoo, Microsoft, Yandex) have collaborated on creating a Linked Data (semantic web) based schema such that all search engines can index and structure all data on the web using a single common schema.

This is at https://schema.org

Recommendation: adopt the W3C Linked Data standard (JSON-LD) for the code.gov software catalog using the Google/Microsoft/Yahoo schema.org metadata model for Software Application:

https://schema.org/SoftwareApplication

Note that the U.S. Department of Veterans Affairs, U.S. National Library of Congress, and many other federal agencies and knowledge organizations have already adopted the W3C Linked Data standard and JSON-LD as their metadata standard, making them fully W3C compliant for web search engines.

I can see this happening across agencies and Departments and feel very proud to contribute to source code as it builds trust,collaboration,team work,knowledge sharing across industry and its a good policy to have.It good to start with a small percentage and once we are able to gauge the impact we can always the increase the code release percentage.Win-Win for everyone.I am happy as this a huge step in the right direction not only for the United States but also as an example for the rest of the world to follow.

We need content to support a variety of paths for agencies, but for some, GitHub will be a good fit. Given it's popularity in government (and the wider world), it'll be useful to have a short page of guidance focused on it.

https://pages.18f.gov/open-source-program/setting-up-github/ is a first pass at this.

Hello! I'm excited to see this project out in the world.

I have a small request that # be removed from the URL structure. I know this is an Angular convention, but it makes the URLs less legible and memorable. Also avoiding technology dictated URLs will make it easier to switch frameworks at a future date while preserving the links.

@afeld @rossdakin @konklone I am not sure if url is correct for the discussing the source policy as it takes you to the WhiteHouse/source-code-policy instead of presidential-innovation-fellows/code-gov-web
is this correct.Can someone please check.The White House Source Code Policy discussion has already ended if I am correct just needed to clarify.Thank you

For those of us that aren't as familiar with modern web development tools, would it be possible to add a short "Getting Started" section to the README to stand up a local instance of the site?

I think that it's npm install, but I'm new to Node.js as of today so I'm not sure that is right...

Thanks!

We need an approach to tagging versions of the schema for agencies to conform to, given that agency data calls can take several months and the schema may change in the interim. Tagging commits is useful for the technical team but less so for agencies.

Considerations:

• strategy at the file level for snapshotting releases while keeping the main file (and its commit history in tact
• ux - presenting historical versions of the schema
• need to add a "schema version" dimension to the schema itself

We should consider the current state the first snapshot - agencies are already building their inventories against it.

I had to hard refresh to get the latest release of code.gov, compared to the splash-page only version.

What cache expiration settings are you using? Do they need to be shorter / do you need to implement cache busting?

First of all, it is super awesome that code.gov launched with HTTPS and HSTS. Thank you so much!

Even better would be to get code.gov's subdomains taken care of for perpetuity, which can be done by updating the HSTS header to be:

Strict-Transport-Security: max-age=31536000; includeSubDomains; preload

More details can be found here, but this will let code.gov get preloaded into browsers, and will ensure that any subdomain gets enforced as HTTPS from the get go.

It's much easier to do this now than later, so I encourage swapping out the header before the site grows large. Thank you again for prioritizing HTTPS/HSTS support for launch!

We're building out a list of developer-recommended tools that can be used to help make sure that code being pushed to public repositories is high quality, secure, complies with any relevant policies.

For example, the Consumer Financial Protection Bureau developed Clouseau to scan repos for offending patterns using regex.

Two questions:

1. What subcategories of tools should we be thinking about here?
2. What tools should we include in the list?

Tools don't necessarily have to be open source or free as in beer. They just have to be great and make developers happy.

Per input on Twitter:
https://twitter.com/paulschreiber/status/763065525526269952
https://twitter.com/paulschreiber/status/763066024279306240

Relabel the mailing list button from "listserv" to something like "mailing list" or other less jargony language.

To be sent out at ~noon on Monday.

• updates
• call to action