Add content on the data page explaining the difference between our data on GCS (BigStore) and BigQuery

htmlproofer is failing on some pages. We should determine the cause of the failures and fix the root causes. We generally should not be disabling htmlproofer.

One example is in .travis.yml:
- bundle exec htmlproof ./_site --only-4xx --check-html --file-ignore "/mlab_observatory/" --href-ignore "http://projectbismark.net/signup.html"

We should remove the ignores. Are there others?

We have an international audience, so our site should support translations into different languages. There are multiple ways to do this in Jekyll. We should identify which strategy we want to use and implement that.

About, Contact, Privacy, Terms
FAQ, Blog, RSS
others ???

The linter rule is 'MD001' and the failing page is /publications.

The header levels are actually correct when generated by Jekyll, but the linter is just looking at the markdown file and not the generated page.

1. Should look into seeing possibilities to restructure this in Jekyll, or
2. Look at adding custom check in htmlproofer that checks this instead of the markdown linter since it seems like it is more important that the generated html has the correct header levels.

The default (and current) TTL for objects in the CloudFront cache is 1 day, which means that users could be seeing day-old content. We need to implement a way to invalidate the cache after each production build, but no on other builds, like ones for PRs.

• Drop "Subscribe for..." text
• Add twitter and think about our communications strategy for twitter
• Update icons for these links

The bullets aren't showing in the Data section of some of the pages:

https://www.measurementlab.net/tools/ndt/
https://www.measurementlab.net/tools/npad/
https://www.measurementlab.net/tools/paris_traceroute/

example from NDT:

Data collected by NDT is available…
in raw format at https://storage.cloud.google.com/m-lab/ndt
via an SQL interface (see https://github.com/m-lab/mlab-wikis/blob/master/BigQueryMLabDataset.md).
Visualized in Public Data Explorer.
Please cite this dataset as follows: The MLab NDT Dataset, . http://measurementlab.net/tools/ndt

Overall, the formatting is a little easier to read on OONI Probe & Glasnost, but I'll wait to see the CSS changes that @shredtechular is working on before I file more issues :)

The linter rule is 'MD026' and the failing pages currently are:

• /about page - has a couple headers with "?" at the end. I think this is okay.
• /page/gsoc_2014/ - has 1 header with an "!" at the end. I think this is okay?
• /tools/pathload2/ - has 1 header with an "." at the end.

We can modify this rule so that some punctuation is allowed and others are not. By default, the linter doesn't accept the following punctuation at the end of headers: ' . , ; : ! ? '.

I don't think the h2 is very semantically correct on /tools/pathload2/, so probably just need a decision on if that should really be an h2 or if so, then can we remove the "." at the end?

@critzo @georgiamoon @mtlynch can you provide the business rule for what punctuation you would like to allow at the end of header levels? It seems like I could make a case for allowing ' : ! ? ' but maybe not ' . , ; '.

We should add a Travis check to make sure that files in this repo do not have trailing whitespace (at the very least .md and .yml files).

This can either a simple grep for "\s+$" or can be a more thorough check using a Markdown linter (bonus points if we can smoothly integrate a markdown linter into Travis/git hooks, especially one that reflows lines to break at 80 columns)

Update the content on the contribute page

We should add a build check that checks Jekyll files for the strings http://measurementlab.net or http://www.measurementlab.net and flags the user to convert these to {{ site.baseurl }}

The two icons and links in the header, "contribute" and "research wiki", are either duplicative (contribute) or increasingly non-essential (we are migrating content from the wiki)

See here:

and here:

I think that these are just css issues with what is probably a <ul> element.

Replace the "Infrastructure" tab with a "Data" tab (The Infrastructure page currently accounts for only ~0.3% of our visits) that gives an introduction to our data, move Infrastructure content to the bottom of "Tests" page.

There are several links on the tests/ndt page which still point to the now defunct code.google.com/p/ndt repository. These should be updated to point to github.com/ndt-project/ndt.

Just a note that we should probably create a custom error page. Currently 404s (and other errors) look like this:

http://www.measurementlab.net/lol

A custom error document can be set in the CloudFront configuration.

We'd like to make it as easy as possible for third parties to:

• take an M-Lab produced NDT widget and tweak it
• create their own NDT widget from scratch without forking the NDT client

To do this, we have to solve the following problems:

• Implementers currently have to import five different scripts to use the NDT backend. We should reduce this to just one script import, maybe with a minified script option.
• At least two of the five are UI libraries that the implementer shouldn't have to deal with if they're implementing their own UI (jQuery and gauge)
• Implementers currently have to set two global JS variables (CANV_GAUGE_FONTS_PATH and ndtServer). We should figure out another way of passing this information to NDT explicitly so that it's hard to omit accidentally and it doesn't clutter the global namespace.

If any of these changes require patches to https://github.com/ndt-project/ndt, we can submit a PR to them.

I'd like us to make it as easy as possible for third parties to drop in the HTML5 NDT test onto their own sites in the form of an iframe.

This involves fixing the following sub-issues:

• Make the widget self-contained so that it has M-Lab's logo and a link back to M-Lab's site and brief text explaining that it's a bandwidth test. It shouldn't require surrounding context of the page to explain what the widget is.
• Add a button to the widget that generates the HTML required for embedding
• The client currently generates a JS warning: Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/. It's likely that mlab-ns isn't going to delay the page load much, but it'd be nice to avoid this. I think we can build more intelligence into the widget so that it the widget handles waiting for the mlab-ns response.

If any of these changes require patches to ndt-project/ndt, we can submit a PR to them.

For publication links like:

http://m-lab.github.io/publications/isp-interconnection-impact.pdf

Can we remove the extension so it's just:

http://m-lab.github.io/publications/isp-interconnection-impact

The linter rule is 'MD036' and the failing pages currently include: /about, /tools/pathload2/, /tools/windrider/, /tools/neubot/, and /page/gsoc_2014/.

Will need guidance on how best to handle these items.

We serve the M-Lab site on www.measurementlab.net, and we want to redirect requests for measurementlab.net to that domain. However, we need to point the root domain to an IP address (an A record). CloudFront and S3 don't give you IP addresses to point to (that I know of), but simply domain names, and they manage and muck the underlying IP addresses as they need to. A lot of DNS hosting services provide a redirection service like this, include Amazon's own Route 53. Since we host our own DNS for measurementlab.net, we're going to need to figure out how to redirect requests to measurementlab.net to www.measurementlab.net. As a stop-gap, I pointed measurementlab.net to our Linode Nagios VM, and Apache on that machine redirects (301) all requests to www.measurementlab.net. While this works fine, we won't ultimately want the Linode VM having anything to do with the core website. We can find some service will provide this redirection for us, or we could deploy the redirection on some other server we manage, or we could move our DNS to another provider (like Google Cloud DNS or Amazon Route 53) who provides this sort of redirect.

Additionally the above temporary redirection mechanism does not work for HTTPS at the moment.

When a page has menu two levels like "Data > Overview" the menu gives a visual indication that "Overview" is the selected page, but there's no indication that "Data" is the selected top level menu item.

This is true even on pages with no subpages.

We should give some visual indication on the menu buttons of which menu item is selected.

The linter rule is 'MD032'.

I think this may be able to be fixed once the rest of the CSS/SASS is optimized so we don't have to add specific classes to lists.

Not sure how! But it traveled.

To reproduce:

• Click on "Tests" [Test page loads]
• Click on a Test (e.g. NDT, Sidestream, BISMark, etc) [Test page loads]
• Click on "Tests" in the menu [Tests page does NOT load]

review and update content on pages in the visualization section for consistency, update language, links to new content, etc.

We had to temporarily disable htmlproofer's external link checking in #124 because ISC's web server was misconfigured. It does seem weird that an external site's temporary misconfiguration should stop us from deploying updates, but I feel like this is in general a good check because we don't want to accrue dead external links and we will if we don't have automated checks.

Maybe we can investigate ways of manually overriding the external link checks without actually making changes to code (like some sort of way we can re-run a Travis build with an extra environment variable or something).

On the current tests page, after each test description we say "Data collected by [xxxx] is available...". Replace these with links to the "Data" page when content on the data page is ready

At the bottom of the homepage we list two short summaries and read more links to the two latest blog posts, yet the title of this section is singular. Minor thing, but easy to correct.

The "Run NPAD" link on our NPAD page is broken. This was a known issue, but for some reason the ticket was closed without a good explanation. Any update on this?

On pages with accordion organizers, we should work to make it easier for the author to edit content and not have to manually manage accordion logic. On PR #120 we ran into a lot of issues where the task of writing content was made more difficult by the task of managing accordion logic.

This is currently most problematic on the FAQ page:

https://github.com/m-lab/m-lab.github.io/blob/master/_pages/faq.md

Ideal would be on all of these pages if:

• Content author can author content using only standard markdown and the backend page logic figures out how to organize it into accordion groupings for proper rendering
• Content author does not need to do any kind of metadata management (declaring up front the names or numbers of headings)

We have a draft document that will refresh the content of the FAQ.

There is inconsistency with our URLs for tools and tests. The tab is labeled tests but the URL for NDT is:

https://www.measurementlab.net/**tools**/ndt/

But then the URL for the index of all tests is:

https://www.measurementlab.net/**tests**/

Suggest changing all URLs to tests and adding redirects. At the very least, we should be consistent.

After pulling markdown linting into the new-data-docs branch, it's flagging issues on several pages:

https://travis-ci.org/m-lab/m-lab.github.io/builds/116247912

The current link to run NPAD on its test page is to M-Lab NS, and results in an error.
http://mlab-ns.appspot.com/npad?format=redirect

Update the link to run NPAD to a correct, working URL

The linter rule is 'MD022' and the failing pages currently include: /publications, /about, and /faq.

This should be able to be fixed once the rest of the CSS/SASS is optimized so we don't have to add specific classes to headers.

We should make it easier to find permalinks to blog posts. Currently, the only way I can figure out how to do it is to go to the per-month category, then copy the "Read more" link, but users should be able to find permalinks from within the blog post itself. The title is an obvious place to make a permalink, but I'm open to other ideas.

When we moved content into the new site, we moved PDFs to new URLs. We need to create an alias to support the old URLs so they automatically redirect to the new ones.

For example this PDF:
http://www.measurementlab.net/static/observatory/M-Lab_Interconnection_Study_US.pdf

is now at this URL:
http://www.measurementlab.net/publications/isp-interconnection-impact.pdf

@mtlynch @shredtechular Any thoughts on how to accomplish this?

Any ideas on how this is happening?

cc @shredtechular @critzo

Update Creative Commons license to most recent (International 4.0).

On the BigQuery Schema page, there are a few style issues. See here:

https://mtlynch.github.io/m-lab.github.io/data/bq/legacymigration/

1. The tables are too wide and use an unnecessary amount of empty space
2. The tables don't have sufficient padding/margins separating them from subsequent text:
   

(1) might be an issue that's unique to this page, so it's fine if the change only applies to that page. (2) is probably something we want to fix globally.

We have discussed and planned a new IA, restructuring the site pages, menus and content. Review the proposed site IA and move the site content over to the new IA when ready.

This file is minified:

https://github.com/m-lab/m-lab.github.io/blob/master/js/desktop.min.js

It's unclear if we have the un-minified source anywhere, but minified JS should not be in source control. We should de-minifiy this using a JS formatter that can apply the formatting rules of the Google JS style guide.

Need to have the CSS from @shredtechular first, but then we should double check the markdown and make sure everything is consistent