Measurement Lab Source Code
This is the source code of the Measurement Lab website built using Jekyll.
Contributing
To learn about how to setup your environment, and our standards and practices for contributing, please see CONTRIBUTING.md.
Source code of the Measurement Lab website built using Jekyll
Home Page: https://www.measurementlab.net
License: Apache License 2.0
This is the source code of the Measurement Lab website built using Jekyll.
To learn about how to setup your environment, and our standards and practices for contributing, please see CONTRIBUTING.md.
Add content on the data page explaining the difference between our data on GCS (BigStore) and BigQuery
htmlproofer is failing on some pages. We should determine the cause of the failures and fix the root causes. We generally should not be disabling htmlproofer.
One example is in .travis.yml
:
- bundle exec htmlproof ./_site --only-4xx --check-html --file-ignore "/mlab_observatory/" --href-ignore "http://projectbismark.net/signup.html"
We should remove the ignores. Are there others?
We have an international audience, so our site should support translations into different languages. There are multiple ways to do this in Jekyll. We should identify which strategy we want to use and implement that.
About, Contact, Privacy, Terms
FAQ, Blog, RSS
others ???
The linter rule is 'MD001' and the failing page is /publications.
The header levels are actually correct when generated by Jekyll, but the linter is just looking at the markdown file and not the generated page.
htmlproofer
that checks this instead of the markdown linter since it seems like it is more important that the generated html has the correct header levels.The default (and current) TTL for objects in the CloudFront cache is 1 day, which means that users could be seeing day-old content. We need to implement a way to invalidate the cache after each production build, but no on other builds, like ones for PRs.
The bullets aren't showing in the Data section of some of the pages:
https://www.measurementlab.net/tools/ndt/
https://www.measurementlab.net/tools/npad/
https://www.measurementlab.net/tools/paris_traceroute/
example from NDT:
Data collected by NDT is available…
in raw format at https://storage.cloud.google.com/m-lab/ndt
via an SQL interface (see https://github.com/m-lab/mlab-wikis/blob/master/BigQueryMLabDataset.md).
Visualized in Public Data Explorer.
Please cite this dataset as follows: The MLab NDT Dataset, . http://measurementlab.net/tools/ndt
Overall, the formatting is a little easier to read on OONI Probe & Glasnost, but I'll wait to see the CSS changes that @shredtechular is working on before I file more issues :)
The linter rule is 'MD026' and the failing pages currently are:
We can modify this rule so that some punctuation is allowed and others are not. By default, the linter doesn't accept the following punctuation at the end of headers: ' . , ; : ! ? '.
I don't think the h2 is very semantically correct on /tools/pathload2/, so probably just need a decision on if that should really be an h2 or if so, then can we remove the "." at the end?
@critzo @georgiamoon @mtlynch can you provide the business rule for what punctuation you would like to allow at the end of header levels? It seems like I could make a case for allowing ' : ! ? ' but maybe not ' . , ; '.
We should add a Travis check to make sure that files in this repo do not have trailing whitespace (at the very least .md and .yml files).
This can either a simple grep for "\s+$" or can be a more thorough check using a Markdown linter (bonus points if we can smoothly integrate a markdown linter into Travis/git hooks, especially one that reflows lines to break at 80 columns)
Update the content on the contribute page
We should add a build check that checks Jekyll files for the strings http://measurementlab.net
or http://www.measurementlab.net
and flags the user to convert these to {{ site.baseurl }}
The two icons and links in the header, "contribute" and "research wiki", are either duplicative (contribute) or increasingly non-essential (we are migrating content from the wiki)
Replace the "Infrastructure" tab with a "Data" tab (The Infrastructure page currently accounts for only ~0.3% of our visits) that gives an introduction to our data, move Infrastructure content to the bottom of "Tests" page.
There are several links on the tests/ndt page which still point to the now defunct code.google.com/p/ndt repository. These should be updated to point to github.com/ndt-project/ndt.
Just a note that we should probably create a custom error page. Currently 404s (and other errors) look like this:
http://www.measurementlab.net/lol
A custom error document can be set in the CloudFront configuration.
We'd like to make it as easy as possible for third parties to:
To do this, we have to solve the following problems:
CANV_GAUGE_FONTS_PATH
and ndtServer
). We should figure out another way of passing this information to NDT explicitly so that it's hard to omit accidentally and it doesn't clutter the global namespace.If any of these changes require patches to https://github.com/ndt-project/ndt, we can submit a PR to them.
I'd like us to make it as easy as possible for third parties to drop in the HTML5 NDT test onto their own sites in the form of an iframe.
This involves fixing the following sub-issues:
Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/.
It's likely that mlab-ns isn't going to delay the page load much, but it'd be nice to avoid this. I think we can build more intelligence into the widget so that it the widget handles waiting for the mlab-ns response.If any of these changes require patches to ndt-project/ndt, we can submit a PR to them.
For publication links like:
http://m-lab.github.io/publications/isp-interconnection-impact.pdf
Can we remove the extension so it's just:
http://m-lab.github.io/publications/isp-interconnection-impact
The linter rule is 'MD036' and the failing pages currently include: /about, /tools/pathload2/, /tools/windrider/, /tools/neubot/, and /page/gsoc_2014/.
Will need guidance on how best to handle these items.
We serve the M-Lab site on www.measurementlab.net, and we want to redirect requests for measurementlab.net to that domain. However, we need to point the root domain to an IP address (an A record). CloudFront and S3 don't give you IP addresses to point to (that I know of), but simply domain names, and they manage and muck the underlying IP addresses as they need to. A lot of DNS hosting services provide a redirection service like this, include Amazon's own Route 53. Since we host our own DNS for measurementlab.net, we're going to need to figure out how to redirect requests to measurementlab.net to www.measurementlab.net. As a stop-gap, I pointed measurementlab.net to our Linode Nagios VM, and Apache on that machine redirects (301) all requests to www.measurementlab.net. While this works fine, we won't ultimately want the Linode VM having anything to do with the core website. We can find some service will provide this redirection for us, or we could deploy the redirection on some other server we manage, or we could move our DNS to another provider (like Google Cloud DNS or Amazon Route 53) who provides this sort of redirect.
Additionally the above temporary redirection mechanism does not work for HTTPS at the moment.
When a page has menu two levels like "Data > Overview" the menu gives a visual indication that "Overview" is the selected page, but there's no indication that "Data" is the selected top level menu item.
This is true even on pages with no subpages.
We should give some visual indication on the menu buttons of which menu item is selected.
The linter rule is 'MD032'.
I think this may be able to be fixed once the rest of the CSS/SASS is optimized so we don't have to add specific classes to lists.
To reproduce:
review and update content on pages in the visualization section for consistency, update language, links to new content, etc.
We had to temporarily disable htmlproofer's external link checking in #124 because ISC's web server was misconfigured. It does seem weird that an external site's temporary misconfiguration should stop us from deploying updates, but I feel like this is in general a good check because we don't want to accrue dead external links and we will if we don't have automated checks.
Maybe we can investigate ways of manually overriding the external link checks without actually making changes to code (like some sort of way we can re-run a Travis build with an extra environment variable or something).
On the current tests page, after each test description we say "Data collected by [xxxx] is available...". Replace these with links to the "Data" page when content on the data page is ready
At the bottom of the homepage we list two short summaries and read more links to the two latest blog posts, yet the title of this section is singular. Minor thing, but easy to correct.
The "Run NPAD" link on our NPAD page is broken. This was a known issue, but for some reason the ticket was closed without a good explanation. Any update on this?
On pages with accordion organizers, we should work to make it easier for the author to edit content and not have to manually manage accordion logic. On PR #120 we ran into a lot of issues where the task of writing content was made more difficult by the task of managing accordion logic.
This is currently most problematic on the FAQ page:
https://github.com/m-lab/m-lab.github.io/blob/master/_pages/faq.md
Ideal would be on all of these pages if:
We have a draft document that will refresh the content of the FAQ.
There is inconsistency with our URLs for tools and tests. The tab is labeled tests but the URL for NDT is:
https://www.measurementlab.net/**tools**/ndt/
But then the URL for the index of all tests is:
https://www.measurementlab.net/**tests**/
Suggest changing all URLs to tests
and adding redirects. At the very least, we should be consistent.
After pulling markdown linting into the new-data-docs branch, it's flagging issues on several pages:
https://travis-ci.org/m-lab/m-lab.github.io/builds/116247912
The current link to run NPAD on its test page is to M-Lab NS, and results in an error.
http://mlab-ns.appspot.com/npad?format=redirect
Update the link to run NPAD to a correct, working URL
The linter rule is 'MD022' and the failing pages currently include: /publications, /about, and /faq.
This should be able to be fixed once the rest of the CSS/SASS is optimized so we don't have to add specific classes to headers.
We should make it easier to find permalinks to blog posts. Currently, the only way I can figure out how to do it is to go to the per-month category, then copy the "Read more" link, but users should be able to find permalinks from within the blog post itself. The title is an obvious place to make a permalink, but I'm open to other ideas.
When we moved content into the new site, we moved PDFs to new URLs. We need to create an alias to support the old URLs so they automatically redirect to the new ones.
For example this PDF:
http://www.measurementlab.net/static/observatory/M-Lab_Interconnection_Study_US.pdf
is now at this URL:
http://www.measurementlab.net/publications/isp-interconnection-impact.pdf
@mtlynch @shredtechular Any thoughts on how to accomplish this?
Update Creative Commons license to most recent (International 4.0).
On the BigQuery Schema page, there are a few style issues. See here:
https://mtlynch.github.io/m-lab.github.io/data/bq/legacymigration/
(1) might be an issue that's unique to this page, so it's fine if the change only applies to that page. (2) is probably something we want to fix globally.
We have discussed and planned a new IA, restructuring the site pages, menus and content. Review the proposed site IA and move the site content over to the new IA when ready.
This file is minified:
https://github.com/m-lab/m-lab.github.io/blob/master/js/desktop.min.js
It's unclear if we have the un-minified source anywhere, but minified JS should not be in source control. We should de-minifiy this using a JS formatter that can apply the formatting rules of the Google JS style guide.
Need to have the CSS from @shredtechular first, but then we should double check the markdown and make sure everything is consistent
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.