The EdFi standards are a national set of standards around transmitting data. We can parse our data to feed into this standard.

Tech Docs:
https://techdocs.ed-fi.org/

Tech Community Guidelines:
https://techdocs.ed-fi.org/display/ETKB/Technical+Community+Guidelines

Ed-Fi Data Standard:
https://techdocs.ed-fi.org/pages/viewpage.action?pageId=12158835

GitHub:
https://github.com/Ed-Fi-Alliance/Ed-Fi-Standard

API:
https://techdocs.ed-fi.org/display/ODSAPI20

ODS:
https://github.com/Ed-Fi-Alliance/Ed-Fi-ODS

Dashboards:
https://techdocs.ed-fi.org/display/ETKB/Ed-Fi+Dashboards

Exchange:
http://exchange.ed-fi.org/

This will greatly help us share the site with new developers.

https://docs.google.com/a/quill.org/spreadsheets/d/1VnPNinkE6h2lHiiW36up_fEORu_O2zq8HQ2spaWRlG8/edit#gid=0

All of these issues should be in the repo here as ideas to pursue.

What I’m using is the old unix ‘style’ command. Basically, I’m saving the user’s text to a temp file, and running the command, parsing and returning the results to the user in a clean way.

For example, to install this on ubuntu:

$ sudo apt-get install diction

Then, let’s say I save your paragraph in the email below to a temp file, this is what it looks like from the command line:

$ diction tmp.txt
readability grades:
Kincaid: 11.7
ARI: 12.0
Coleman-Liau: 12.1
Flesch Index: 46.7/100
Fog Index: 14.3
Lix: 53.4 = school year 10
SMOG-Grading: 12.7
sentence info:
594 characters
119 words, average length 4.99 characters = 1.66 syllables
6 sentences, average length 19.8 words
33% (2) short sentences (at most 15 words)
0% (0) long sentences (at least 30 words)
1 paragraphs, average length 6.0 sentences
0% (0) questions
66% (4) passive sentences
longest sent 29 wds at sent 3; shortest sent 14 wds at sent 5
word usage:
verb types:
to be (5) auxiliary (1)
types as % of total:
conjunctions 8% (9) pronouns 13% (15) prepositions 10% (12)

See what I’m saying, I just parse that and display it.

You can find information about this at http://www.gnu.org/software/diction/diction.html

It’s old, but still works. Nothing really to open source since it’s already open source.

We should set up automated follow up emails shortly.

We can give the teacher the ability to make something like this with pivot tables.

https://docs.google.com/a/quill.org/spreadsheets/d/1nyCoAsLLxjAtAfgbqNZfTtughAGjHHj2Yqhih5Y-8WI/edit?usp=sharing

Demonstrating Mastery

• We want to move to an authentic mastery based system, where students have either mastered or not yet mastered the materials. Right now we simply assign a score to students, rather than showing how the students have passed a mastery threshold.
• The danger is that in each app the learning experience is different, so something tailored to grammar would not work elsewhere.
• The core goal of an authentic system is to count up points earned for each activity. Students earn points for answering a question correctly. They get bonus points based on the difficulty of the question.
• Earning enough points constitutes mastering the material. Students will get a multiplier for getting multiple questions correct in a row, and we’re ideally looking for a string of correct answers to constitute mastey.

Change Activity Status Names
Activities have 3 states: proficient, near proficient, not proficient. We want to be able to say that students are either in a mastery or non-mastery state. To do so, we should potentially re-name these activities to something like “Great, Close, and Nope” (placeholder names) to separate them from proficient.

Top of the Fold

It is probably too much info to feature the playing animation at the top. Instead we could feature some text that states:

With Quill Writer
Learn proofreading skills for free.

With Quill Grammar
Learn grammar concepts for free.

Each time we swap out the app name and what the app does in a queue.

Challenges

We should feature, either before or after the apps, a set of three challenges that features a unit of our best content. These pages will link into our "discover > learners" section, where we list each type of learner we have content for.

Social Proof

We need to include social proof in the form of usage: users, number of questions answered (perhaps real-time?), press we've received, and support we've received. The social proof might need to be at the top, but the other two could go at the bottom.

Features + Animations

The animations can possibly go in the middle of the landing page, where we explain how each app works. We could also feature 3 "generic" features about Quill, such as the common core content, personalized learning, engaging activities for students, and the fact that these are built by a nonprofit, open source community.

The store that shows the activities grouped together.
The dedicated page for each one - all of the details plus the reviews - quill.org/goals/153
The preview version of each one - the individual activity vs. the goal page - quill.org/activities/153
The final page each for each one
The review page for each app's final product.
The settings page for modifying the settings.
The set up page to set up the initial default settings.

The goal for the activity
A preview of what is involved
the author of the activity
the challenge rating
the comments
additional meta-data:
How many questions are there?

How long does it take?

How many written words produced? How much text read?

How many concepts does it cover? I.e. one idea, a shuffle of 10 different concepts?

What is the mechanism by which you answer question? Drag and drop, multiple choice,

free answer?

What is the question to time ratio?

What knowledge level does it cover? I.e. tagged to a standard against other lessons.

What are similar lessons?

How difficult is it? Use a combination of reviews and machine learning to suss out what

the success to failure rate is.

Feedback from First Time Users

hey, it runs smoothly and I didn't find anything buggy apart from that I clicked discover first and got lost around those pages, community etc.

Didn't understand what I was looking at so went back. Saw a green red and brown screen that flicked from right to left. Noticed you had two sign up buttons then the play now. Clicked play now, that was all fine.

Went back to play another game then noticed the home red and brown screens don't always load every time when they flick through.

Now this may surprise you but because of the size of the red green and brown panels and my safari browser is on a 13inch screen I didn't see that I could scroll down. I only just noticed after 20 minutes on your site that the home screen is scroll down. I didn't see OUR PURPOSE Write Your Own Stories until 20 minutes into surfing your website.

You may want to see how the site runs on safari. safari scroll bars disappear and I thought your website was a big red screen.

Create a generic excel template where the CSV data is automatically parsed into totals for users. This data can live on a separate page from the raw data. The idea is that teachers load the CSV into the template.

The title in the progress report should be sticky, so that a user can see the titles no matter how for down the page they are.

Goal: Set up a page, called Quill.org/discussion that embeds the UserVoice application.

Location: Quill.org/Discussion. This should go under the static page views.

• Create the page www.quill.org/discussion
• Embed the page: https://quill-discussion.uservoice.com/
• rework the Top Menu bar so that it shows the following options:

  "About Quill", "Our Community", "Discussion & Support" "Social Media Links".

The About Quill tab is a drop down that links to the following pages:
About Quill:
Lessons
Mission

Our Community is a drop down that links to the following pages:
About Us
News
Develop

First Option: Without the Share Bar:

Second Option: With the Share Bar.

• Create a textbox that includes a submit button. This is a mirage box. We actually are going to use the UserVoice to enable the feedback.
• The user voice box should be modified so that it spans the length of the textbox. It should not be it's normal square shape.
• Click on the button causes the Uservoice Widget to Fade in on top.
• The textbox / form field should have a blink cursor in it.

From @RyanNovas on June 24, 2014 21:54

Goal: Design a homepage for partners to display our activities in isolation.

How: Make the wireframe below more pretty.

Copied from original issue: empirical-org/Empirical-Core#245

Set up a stable staging server for app developers to test the integration process without interfering with our work.

Right now the user has to select very specific, granular activities within the LMS. For example, the user selects Lose vs. Loose as a lesson.

A better system here would be to let users select a 1 level concept, such as Commonly Confused Words. Teachers could select the 1 level concepts they want to focus on.

Why Wait?

The reason why it is not worth doing this now is that certain 1 level concepts can have a lot of content in them. Rather than thinking of an activity as a 10 minute chunk, an activity could take 40-50 minutes. In this environment students work on the activity, quit out, and then continue on it later. We need to introduce the notion of an activity being in progress.

Note: Grammar hero will explore this territory. We can play around with the idea of letting users select content from different buckets (standard level, individual standard, 2 level concept, 1 level concept, 0 level concept) and use this to test out how granular we want to make the interface.

Genuine Mastery Based Learning

We can use Scott’s interface where each student has either completed a 1 level concept or is working on it. Students have to master all of the concepts to move on. Instead of simply having a score, students have a breakdown of individual concepts they need to master within an activity. Once they master all of the concepts, the student masters the activity. The LMS needs to know how each student has performed on each concept. Once a student has mastered a concept, it moves on to the next concept on the tree.

In App: How do you sequence the questions?

Option 1 - 10 questions per 0 level concept.

Students start the 1 level activity. They work on the stuff. There goal is to get 10 questions correct for each level 0 concept within the level 1 concept. They can quit out half way through, and it’s registered as in progress on the LMS. Once they have gotten 10 correct it is passed.

Option 2 - Diagnose and Follow up

Within the activity, you can either show 3 random level 0 questions all from all of the level 1 concepts. If a student gets those concepts wrong, the student then gets 10 follow up questions. If the student gets those questions correct, the student is marked as correct for that 0 level concept. The underlying idea here is that we present students with a wide range of questions, 3 from each concept, and if students know those answers they are marked as having the material and move on to the next set. If they get it wrong, they then focus on practicing those materials.

Painting a Picture: What this looks like on the LMS

As it stands right now, you might have a unit called “Commonly Confused Words” which has 10 activities in it about the individual word pairings.

In this new system you might have 10 1 level concepts, such as Commonly Confused Words, Relative Pronouns, Infinitive Tense, etc. all under the unit “Language Standards” and that would be your unit.

Question: How to handle duplicated units?

One of the problems here is that students will get duplicated activities, where they will practice the same thing they have already mastered. One way in which we can hack this is by taking greater control over the assigning process. Essentially we think about it like this: we provide the student a diagnostic, and then we create a unit of activities for the student based on the student’s performance on the content and the grade level. Once those student has mastered those materials we provide the next set. In this world the teacher really only specifics how much time they want to spend on grammar, and we will then fill that time with a series of activities. By generating the activities ourselves, we won’t create duplicates.

Maybe we make screenrecording gifs to show tutorials in uservoice using: http://www.cockos.com/licecap/ in addition to the pictures.

The Tour

We give teachers a tour of the key terms in our app. It is like a dictionary, where you flip through the pages, and each page has a key term and the definition of that term. This allows teachers to quickly map out the key aspects of our vocabulary.

Another way of doing the tour - we introduce Teachers to all of the quill terms:

• Activity, Unit, Passage, Quill Grammar, Quill Writer
• On each page is a word and a definition
• It feels somewhat like a dictionary.
• When you press next, half the page flips over, like turning a book.

The most important thing we can do now to get new teachers started is to make it very easy for teachers to select their first playlist. We also want to (1) explain to them our 3 key terms (apps, activities, units) and that each app has a specific experience.

Three things to know about Quill:

We provide you with a toolbox full of tools.

1. Pick the apps you want to use with our class.

2. Select the playlist.

Select a playlist for one group of students.

Don’t worry you can more playlists later for separate groups of students.

Native English speakers.

Select grade

or select a playlist to jump right in.

3. Each activity is timed to be 10 minutes in length.

A warm up or end of class.

Show the activities selected the average time length for each.

Select the students, or create the student accounts now.

Currently teachers have to remember what they have assigned or flip between views. We could just have a little checkbox or something and a column that says "Previously assigned"

Discover Follow up:

• apps page - showcase each of the apps, describe how they work.
• Showcase all of the activities via showing a version of the activity planner where users can preview activities.
• For the activity planner, let teachers preview the lessons - we need to get the preview button in there somewhere.
• Embed each of the apps in an iframe under our apps so users can play them.
• Add the sign up & sign pages tweaked - new design + choose your team.
• Add underline to discover, community and support.
• Think about how we can add a nice header image between the tabs nav and the top nav.

One problem with the behavior of clicking on an activty to load it is that on a tablet users will not be able to view the tooltip. Ideally tablet users could click on a row to bring up the tooltip, and from there they could click on the tooltip to load the activity. To do so we will add a link on the tooltip called view activity.

For desktop users we should also add the view activity option. The key thing here is that when you hoover over the tooltip, we need to make sure that you can navigate from the box to the tooltip to be able to click on view activity. At the moment the tooltip will close as soon as you leave the box.

Similar to Chrome Store app. Could use Swift and a Safari view controller. Another alternative is to user http://electron.atom.io/ and distribute anywhere.

Submit the site to UI review sites to get feedback on how we can improve it.

https://docs.google.com/document/d/1iHJjuzoSnOV_E0C-GOPkirqJeYmBecpnOGWb-mw_tG0/edit

We set up a table: for every concept, here are all of the results across all of the students.

School administrators will be able to manage classrooms (delete), transfer students between classrooms, view student learning data, and export student learning data. This dashboard will be available by December.

Canvas has great API documentation. We should review it to see what details we should include in our own.

https://canvas.instructure.com/doc/api/quiz_questions.html

Have a professional complete a security audit of the infrastructure.

Third party apps at the moment cannot create their own concept tags. At some point in the future we will enable app developers to create their own concept tags.

This is something we will want to think about. It may be the a user proposes a tag, and then the editor reviews it and authorizes it. For now, we will maintain the list of tags as a CSV.

There a number of UX review services available here:
http://openstrate.gy/

From the teacher portal teachers want the standard, students on the Y axis, activities on the X axis view. We should provide this as an option to teachers. This works well when everyone does the same activity.

Like codio.com, Terminal provides a remote developer environment. However, Terminal seems to be a better, more reliable tool.

Set up an interactive tutorial for Empirical Core as well as the apps. We use a standardized help button in the top right corner of the apps. It launches the tutorial that guides the user through the pages.

We should also have a link called Quill.org/profile#tutorial that automatically triggers the tutorial state.

Rubocop enforces Ruby style conventions. As an open source project, it's important that users follow our conventions. This will be a good way of keeping our style consistent.

bbatsov/rubocop

Activities page updates without requiring a refresh (Daniel had to repeatedly tell the students if the activities weren't there to refresh the page.)

This should be using some sort of live refresh functionality.

Setup CircleCI for our code review platform.

From a teacher:

is it possible to have a 'hide units' option in the 'my units' section. This is so I can still save my older units for research purposes and yet not have to scroll down everytime i want to look at the new units i have created for this year.

This plugin enables highlighting of SLIM for VIM. It should help developers who aren't familiar with SLIM.

https://github.com/slim-template/vim-slim

We have a whole bunch of UI elements, sorted into a sprawling CSS folder. We need to package and standardize all of these elements into a single UI Kit.

When a user signs in, if they skip school selection we do not know their city or state. We could use their IP to add this info into the database.

WHAT YOU CARE ABOUT

Packages are king

IN PRE-PACKAGED UNITS YOU SEQUENCE THE CONTENT ALREADY. THE STANDARDS ARE EXTERNAL VALIDATORS - THEY CORRESPOND WITH WHATEVER SYSTEM PEOPLE ARE USING TO CHECK THE BOXES. THE CONCEPTS OUR OWN HIERARCHY. THEY TELL US WHICH CONCEPTS STUDENTS HAVE LEARNED

IN A PACKAGE YOU GET ALL OF THE CONCEPTS, AND WHEN AN ACTIVITY IS COMPLETED YOU GET THE BREAKDOWN OF THE CONCEPTS. The standards and concepts are interesting once the activity is completed.

WHAT’S GOT TO GO - Moving away from One Standard / One Concept per activity.

EACH ACTIVITY CAN HAVE MULTIPLE ACTIVITIES AND MULTIPLE STANDARDS ASSOCIATED WITH IT. THEY ARE TREATED LIKE META-DATA IN THE SYSTEM, AND THEY AREN’T GIVEN PRIORITY WHILE SEARCHING.

IT’S NOT CLEAR WHAT SHOULD REPLACE THIS (AUTHORS PERHAPS? OUR CHALLENGE SCORE?)

THE NOTION OF ONE CONCEPT AND ONE STANDARD WILL HAVE TO GO. THEY WILL GET REPLACED BY THE NEW MULTI-CONCEPT, MULTI-STANDARD SYSTEM, WHERE THE MAIN UNIT OF MEASUREMENT IS THE PACKAGE, WHICH HAS ALL OF THE GOODIES IN IT.

Hi Peter, please make it possible to the same due date to multiple lessons in the same checkbox fashion. Also, please allow us to open individual student accounts to see more detail.

From @petergault on November 10, 2013 22:37

{
"user": "gault8121",
"bio": "Peter is designing the Quill learning experiences and coordinating the team. With instant feedback each student can be challenged at her or his own level.",
"role": "Product Developer, Team Coordinator",
"startDate": "3/7/13",
"teams": "homepage,lesson_design,contributors_page",
"skills": "UX,lessons,games"
}

Copied from original issue: empirical-org/Empirical-Core#24

Discover
(Our Apps)
(Browse All Activities / Explore Library)
(Quill Writer)
(Quill Grammar)
(Quill Proofreader)
About
Mission (http://www.quill.org/mission)
Community Development (http://www.quill.org/develop)
Meet the Team (quill.org/about)
FAQ (quill.org/faq)
Impact
Statistics http://www.quill.org/impact
(Testimonials)
(Tagboard)
( Blog / News )
( Press )
( Educator Events )
( Developer Events )

We can use RailsAdmin to automatically build our GUI based on our tables, rather than manually creating it.

We can visualize a massive data set using Keen for teachers. For example, we can show the number of activities completed per day across a large number of users. Keen works particularly well when dealing with large data sets, such as when you have thousands of users or tens of thousands of activity sessions.

Long Term Idea - Quill Summary
We want to show a summary of all of the student’s accomplishments on Quill. To do so we want to focus on how much the student has accomplished. Since we have many different apps, here is how we can do so:

We want to have a nice, appealing one page summary of what the student has learned. This is the easily shareable, 1 page PDF, that distills everything down to its core.

Top level Summary: Quill - 37 Activities Completed
Quill Proofreader - 7 successful passages proofread (Goal: 5)
ComicDrop - 5 successful comics drawn (Goal: 3)
Sentence Shuffle: 11 successful sets of sentences reorganized. (Goal: 6)

This is a nice overview of all of the stuff the student did. What’s nice about this is that each app has its own verb, it’s own experience, and this display captures those verbs. The other thing that is nice about this is that while we don’t fall back on using a percentage to show that the kid is successful (which restricts the range of apps), we can still show success by comparing the number completed to a goal. Since these are successful sessions, they’re all passing.

From @petergault on February 27, 2015 16:45

What: When teachers are assigning activities, the site is creating multiple activity sessions. This bug has cropped up multiple times, and it is a serious bug. The question is whether we can somehow either: (1) monitor the app to see when multiple instances of the same activity session are being created at the same moment or (2) preventing the system from creating more than one activity session at a time (i.e. if a teacher is batch assigning 5 activities to 10 students, it should only create 50 activity sessions. If it creates 100 sessions or 250 sessions or any number that is not 50, the site should prevent this from happening).

How: Here are some potential options for how we can perhaps monitor this behavior. These might not be the right tools, and perhaps the solution is to limit the behavior rather than monitor the errors:

https://www.runscope.com
Runscope verifies that your APIs are returning the right data, even for complicated multi-step workflows and chained requests.

http://apimetrics.io/
Set customizable failure alerts for badly formed JSON or other unexpected results. Configure sequences of tests from our online tool for the key scenarios you depend on, then run them from anywhere in the world.

Copied from original issue: empirical-org/Empirical-Core#579

TenMarks provides teachers with a generated PDF that contains all of the student login info for all of the students. We could generate something like this, and enable teachers to print and share login info.

https://helloreverb.com/developers/swagger/
http://raml.org/index.html
http://apiary.io/
http://apiblueprint.org/

Use this to track user behavior, provide support, solicit feedback, and provide automated emails.