scalacenter / course-management-tools Goto Github PK

Sometimes, the only use of a linearized master repository is to inspect diffs between subsequent exercises. In that case, the build definition should be the absolute bare minimum as not to be more distractive than necessary.

To accomplish this, an option should be added to the linearize command (this could also be a CMT setting) that reduces the build def to the minimum.

Currently nextExercise and pullSolution overwrite entire files. It would be cool if they would support patching an existing file with - for example - the diff between the file in the current and the previous exercise. So that students can apply one particular exercise solution to their files, without overwriting their entire file.

Original Issue: https://github.com/typesafe-training/fast-track-akka-scala/issues/9
Originally reported by @alejandrolujan

The following instructions suggested the creation of a private field, where none was needed or expected:

Create a private guest actor without a name.
Use a createGuest factory method that creates a Guest.
Instead, I suggest the following:

Use a createGuest factory method that creates a Guest actor without a name.

I usually use IntelliJ to present exercises in a course delivery. The new version 2017.2 includes an “SBT shell” with a nice visual completion feature.

However it does not play nice with our sbt tasks:

• Typing man e is challenging. My workaround was to add a space after the e.
• None of our tasks, such as nextExercise are offered in completion.

I don’t know if there is something we can do to address these issues, but it would be nice.

While adding two new commands to a course master repo for a Lagom project, it has become clear that the corresponding code (located under the project folder) should be manageable from the course-management-tools. If not, managing the code base becomes too difficult and people will be working with outdated code or missing sbt commands on course master repos.
The idea of this enhancement is twofold:

• add an admin command to the course management tools, that will allow updating all code in a course master repo from a central location
• add a template generation command that will allow one to get a sample project tuned for Scala/Java/Play and Lagom projects. In this way, the existing template for a Scala project can be removed, which will simplify maintenance of the tools in general

Opposite of installing a course, basically.

As the creator of a course I want to be able to take my completed course, studentify it and then publish that to a repository (or other public location from which students can install the course).

Creating a new repository?
git push?

Another alternative is to have the studentified courses be distributed as 'release' artifacts. Having a 'release' of a main repo result in the zip'd studentified directory being added to the releases of a the main repo would allow for releases courses to be versioned (semantically?)

The drawback would be more complication on the install side - where we couldn't use git clone for pulling and updating courses anymore, we'd have to download and unzip.

At that point we'd have the option of storing multiple versions of the same course. This would complicate things slight (would require maintaining a mapping of the current version of a course being used, as well as the course name as a key. I'm not sure of the benefits of maintaining multiple versions of a course locally, though. For the course creator to do some comparisons yes, but there should be more students for a course than creators and adding the necessary complexity (rather than just overwriting a course) as well as taking up space on their machines for the sake of this functionality doesn't seem worth it.

The project (CMT) has been lacking proper testing since it started. Adding testing is not straightforward, since the real test lies in verifying the functionality of the artifacts such as a studentified or a linearized version of a master repository created by the tools or the running of the mainadm command.

The approach taken in the Moving from Scala 2 to Scala 3 course can be applied here though (implemented via Github Actions)

The idea would be to add a number of CMT master repositories to this repo, say under a folder named validation. For each of these master repositories, CI can generate & execute the testscript generated by the mainadm -t command.
The different validation repositories would verify certain project settings or combinations thereof.

Our released zip archives contain the Jenkinsfile that we use in our CI pipeline. This is unnecessary. These Jenkinsfiles should be stripped out by the Studentify process.

Currently the 'student' tooling is configured through the SSettings.scala file that gets generated by the studentify command.

We want to get rid of this generated Scala file in favour of a plain properties configuration file; which is usable for different student tools.

Let's develop the contents of the configuration file in this issue.

The studentified version of a course repo ships a number of source files along the project directory which are not relevant to the actual course.

eg:

Man.scala
Zip.scala
Navigation.scala
StudentCommandsPlugin.scala
...

It would be nice if we could create an sbt plugin containing all student commands and just add it inside plugins.sbt of the studentified artifact.

Once pullSolution has been used, there is no way to get back to the initial state of the exercise, it seems

For historic reasons, mainadm, studentify, linearize and delinearize are separate executables. We want to merge them into a single executable, with different subcommands.

At the same time, we want to split the code in two modules: a library with purely the operations, and an SBT part that integrates the library into SBT. This allows the library to also be used outside sbt, for example by the detached mode student client.

This will also reduce duplicate code.

There's currently a PR #33 that implements exercise renumbering. A new command, masterAdm is added for this purpose. Even though it has full functionality (renumber all exercises starting from a given offset (default = 0) and a delta between consecutive exercises (default = 1), it should be made more specific like adding an insertExerciseBeforeExerciseNr <nr>, deleteExerciseNr <nr>, etc...

Original Issue: https://github.com/typesafe-training/fast-track-akka-scala/issues/14
Originally reported by @alejandrolujan

Why do we have this in CoffeeHouse.scala:

guestBook += guest -> (guestBook(guest) + 0)

instead of this:

guestBook += guest -> guestBook(guest)

?

Currently, there's no checking of the structure of a course master, e.g. 'is there a .courseName file in the project root folder?', 'Do all exercise folders have a README.md file in src/test/resources?', 'do the README.md files have the minimally expected structure?', etc...

In some scenarios, an lingering, [almost] empty exercise folder will cause the linearize command to fail with the following output:

Cloning into 'Pi-Akka-Cluster'...
done.
Cleaned master repo: /var/folders/gc/pmt7y3s91wz7ch_k91f6d1yw0000gn/T/sbt_50196839/Pi-Akka-Cluster
Setting student repository bookmark to exercise_000_initial_state
Error in removeExercisesFromCleanMaster, bailing out

As a short term workaround, look for the offending folder(s) and remove it.
long term fix would be to automatically remove the troublesome folders.

Note: the empty folders come into existence when switching from a branch that contained them. For example, in such a branch, a new exercise was created and run (creating a target folder in it). When switching to another branch that doesn't contain the new exercise, the exercise folder is retained. This is because the git configuration is most project excludes the target folder, and thus a non-empty target folder is retained after the branch switch. The 'empty' exercise is subsequently incorrectly picked up by linearize command.

Ideally, a studentified repo should be self-contained (with the exception of the JVM).
sbt 0.13.15 now has support for offline installation which is a crucial feature to make self-contained student repos possible.

The existing studentify tool is specialized to generate student repos for 'regular' Scala or java projects (sources under src/main, test code under src/test and multi-jvm tests under src/multi-jvm).
Projects based on the Play framework or the Spark workshop utilize different location as the one mentioned above to store source files. This enhancement would add for this. Selection of the project style would be done via a new command line option.

Leaving out the trailing underscore should work. However, I would allow it only if there’s no trailing part on the name after the 3 digits. E.g. step_003 is ok. So is step_003_bla, but step_003bla isn’t.

mainadm should get -g option when the -t option is present. This is needed for validating a studentified repository that has been generated as initialised for git.

Lower priority, but could be interesting to track Dotty's evolution

In the current implementation of studentify, the solution code (man code + tests) is stored in a hidden folder (.cue). IntelliJ actually sees all files, including .scala files. In general, this isn't a problem, but in some specific cases, it can cause a problem.
For example, in the Fast Track to Scala course, in exercise Use packages, if the student uses the Refactor feature of the IDE, the latter will try to refactor stuff in files in the hidden folder. Obviously this is undesirable...
A relatively simple solution exists: instead of storing the solution for an exercise as-is under the hidden folder, a zipped version of the folder is stored. When a particular operation needs access to the content of an exercise solution, the corresponding zip file is un-zipped, the operation is executed (un-modified from the original version) and finally, the unzipped folder is removed.

When loading the project with sbt or IntelliJ on a case sensitive file system (as is the default in Linux), you'll get an error message:

[error] java.io.FileNotFoundException: .../lightbend-training/LAS-E-lightbend-akka-for-scala-expert/.courseName (No such file or directory)

The file in the repository is called .coursename, but sbt looks for .courseName

Course management Tools customize sbt prompt through StudentCommandsPlugin

The following shell prompt is generated usually.

man [e] > Course Name > exercise name >

StudentCommandsPlugin fails to generate the shell prompt if used along with projects already set shellPromt using their own sbt plugin.
Play Framework is an example, it uses its own sbt shell prompt.

man e command fails to render the README on individual exercises on a master repository.
This happens on projects without an src directory (Play Framework)

The idea to use the course management tools to manage Maven based projects has been explored:

• Use a master project with different exercises in the same was as for a Java/Scala project.
• Use cmt-studentify to create studentified artifacts where the exercise base folder holds a Maven build definition.
• Students use an sbt session (as in a Java/Scala project) to 'drive' the exercises (using nextExercise, pullSolution, etc.

This approach works, but it has a few inconveniences due to the way the exercises navigation is managed in the studentified repo.:

• pullSolution swaps the complete exercise code state for the complete solution state. This also wipes out tolling files and folders (for example, in IntelliJ, the project .iml folder). In general, this requires a student to reload the project which can be done by a single click, but it is a waste of time).
• nextExercise [and friends] works fine, with one potential inconvenience: if the Maven build def changes between exercises, the modified build def will not be loaded by the command

This issue proposes to enhance the tooling to:

• allow for a list of folders in the exercise folder that should be left as-is by pullSolution. This list should be defined via configuration using the existing course-management.conf mechanism.
• make nextExercise, prevExercise, gotoExercise also pull the Maven build def file. This may actually be as simple as adding it to the studentify.test-code-folders config setting.

As a bonus, it may be worth considering dropping the passing of config settings via the Settings.scala mechanism and replace it with regular configuration parsing: the course-management.conf can be added to the studentified repo as-is (or transformed to another config format if that is easier) and should be read when starting up the sbt project.

Right now, the code base contains a series of regular expressions that are the same or at least very similar. This should be cleaned-up.

Add a CI step that checks there's a license header in all the files that should have a license hader

Cause is that, after initialisation, validataStudentRepo.sh moves to the next exercise and then runs the test on the selected exercise. Hence, the tests on the first exercise on a repo are never run.

As masteradm -b <masterRepo> doesn't look at the config file, it doesn't know that folders might have a different naming scheme (eg. step_).

Overview

Currently, when executing any of the cmtc commands it's necessary to pass a "studentified repo" path which points to where the command will be executed.

This requires, of course, for the studentified version of a course to be present on the machine where the command is being executed.

What if the studentified version of the course could be downloaded automatically by the course management tools?

What if, once it has been downloaded, the course could be set as the default target for the cmtc commands?

Implementation

What is proposed, then, is for the addition of 'package management' operations for the course management tools:

Install

The install command would allow a user to specify the course to be downloaded and have the course management tools automatically get the course from wherever it is hosted and place it on the local machine.

This leads to two questions:

1. Where are the courses stored remotely?
2. Where is the downloaded course stored locally?

In an initial implementation i propose to have courses stored in Github repositories and use git clone to download the files (allowing for them to be easily kept up-to-date).

So, the install command could just provide the username/project convention, for example:

cmtc install thinkmorestupidless/studentified-test

Would clone the thinkmorestupidless/studentified-test repository into some local location... where?

Following the conventions of other tools it would make sense to have the notion of a CMT_HOME directory which defaults to ~/.cmt

In ~/.cmt would be located a *.conf file which can store any global configuration necessary for the tool operations (which can be loaded when the tool is executed) and a ~/.cmt/courses sub-directory could store the courses themselves...

Giving this layout:

Context

Once a course has been installed we need to be able to tell the course management tools which course in the ~/.cmt/courses directory we want to have the course management tools operate on (that is, which course to talk to when we issue next-exercise, pull-solution, etc)

By have *-context commands we can have the course management tools write to a the ~/.cmt/cmt.conf file and read from that file to set a current-context property which points to the location of the course we want to operate on.

set-context

So, for example:

cmtc set-context studentified-test

would result in ~/.cmt/cmt.conf looking like:

cmt {
  current-context: "~/.cmt/courses/thinkmorestupidless/studentified-test"
}

We can then load ~/.cmt/cmt.conf when the cmtc is executed and have the context available as a default (which could also be overloaded if a path is supplied by any of the cmtc commands.

# executes on the course specified in `cmt.current-context`
cmtc next-exercise

# executes on the course located at /my-course
cmtc next-exercise /my-course

get-context

To know the current context (which course i'm currently using) we can then add cmtc get-context which will just print the value of cmt.current-context to the screen

cmtc get-context
>> /Users/trevorburtonmccreadie/.cmt/courses/thinkmorestupidless/studentified-test

We could add an extension to the student tooling to serve a web ui that students can use to navigate through the exercises, preview solutions, etcetera.

Would it be useful to have commands dispatch events and record them for display later?

Use Cases

A course could be delivered and students may not complete all exercises on-site. They may go back and look at some parts later.

A tutor would be able to see when they have completed them - also when students in the room haven't completed things.

So, as a tutor you could have a view on the current students in the course - seeing how much everyone is doing.

Historically, the exercise instruction file were located in the src/test/resources folder. After the introduction of the possibility to put these files in each exercise project's root folder, the validation in mainadm wasn't adapted.

Currently, a course has at least two git repositories. In the future there may even be more (for example, a trainers manual might be in yet another repo).
Ideally, everything should be packaged in a single repo.
Where needed, new tools like the ones already available in the course management tools, should be created to generate use-case specific distributions (such as the trainers manual)

Original Issue: https://github.com/typesafe-training/fast-track-akka-scala/issues/11
Originally Reported by @alejandrolujan

Some ending single quotes are misplaced, so the following:

Log Guest {guest} added to book at info.
Log Guest {guest} caffeine count incremented. at info.
Log Sorry, {guest}, but you have reached your limit. at info.
Override the postStop hook to log Goodbye! at info.
should be replaced by:

Log Guest {guest} added to book. at info.
Log Guest {guest} caffeine count incremented. at info.
Log Sorry, {guest}, but you have reached your limit. at info.
Override the postStop hook to log Goodbye! at info.

Several of the current commandline arguments to the authoring tools seem to fit better in the configuration file.

• multiJVM
• isADottyProject
• autoReloadOnBuildDefChange
• addMainCommands

Note that these are also specific to either the SBT student tooling, or SBT projects in general, so should probably be subkeys of some sbt specific configuration key.

An issue was originally reported in FTTAS: https://github.com/typesafe-training/fast-track-akka-scala/issues/17

Suggests that the colour scheme may not be appropriate in some cases. We should check if this is a problem with Studentify and try to remedy it if possible.

When a master repo has exercises in a subfolder (specified via the studentify.relative-source-folder setting), the .gitignore file in a linearized artifact (or a studentified artifact generated with the studentify -g command) should be a copy of the .gitignore in the master repository's root folder.

At present, this is not that is not the case.

This would make it possible to execute CMT commands studentify, linearize, ... without having to launch them from sbt

I'm trying this out, but I cannot get the project to compile. The dependency to org.scala-sbt %% io % "0.13.12" cannot be found.

I have a plain sbt setup without extra repos. I use coursier (but also tried without coursier)

coursier.ResolutionException: Encountered 1 error(s) in dependency resolution:
  org.scala-sbt:io_2.11:0.13.12:
    not found:
      /Users/jgordijn/.ivy2/local/org.scala-sbt/io_2.11/0.13.12/ivys/ivy.xml
      https://repo1.maven.org/maven2/org/scala-sbt/io_2.11/0.13.12/io_2.11-0.13.12.pom
      http://repo.typesafe.com/typesafe/releases/org/scala-sbt/io_2.11/0.13.12/io_2.11-0.13.12.pom
[error] (*:coursierResolution) coursier.ResolutionException: Encountered 1 error(s) in dependency resolution:
[error]   org.scala-sbt:io_2.11:0.13.12:
[error]     not found:
[error]       /Users/jgordijn/.ivy2/local/org.scala-sbt/io_2.11/0.13.12/ivys/ivy.xml
[error]       https://repo1.maven.org/maven2/org/scala-sbt/io_2.11/0.13.12/io_2.11-0.13.12.pom
[error]       http://repo.typesafe.com/typesafe/releases/org/scala-sbt/io_2.11/0.13.12/io_2.11-0.13.12.pom
[error] Total time: 2 s, completed Nov 4, 2016 2:34:22 PM

In the current state, commands are very chatty (especially linearise and delinearise). This is mainly due to git processes that write to stdout during execution. This could be suppressed by in default operation and re-enabled when a yet to be added verbosity option is specified during command execution.
Proper logging should be added and all phases during command processing should be logged at DEBUG level. Goal of this would be to simplify and accelerate troubleshooting in case of problems.

Until now, each CMT project has a common project who's content is aggregated into each of the exercise projects. Even though there are good use cases for this (see the akka-pi-cluster project for an example), there's no need for this in most projects.

The proposal is to make this configurable (via a setting in CMT configuration). For backwards compatibility, the default will be to configure for a common project.

Original Issue: https://github.com/typesafe-training/fast-track-akka-scala/issues/13
Originally reported by @alejandrolujan

Two of my students made the same mistake: not forwarding the message from the CoffeeHouse to the Barista, but instead just using tell.

This was an error, the system did not work properly, since the Barista was replying to the CoffeeHouse, while the intention was to reply to the Waiter.

However, the tests did not detect this. It would be nice if they did - students trusted the tests and did not catch the error until a later exercise.

Original Issue: https://github.com/typesafe-training/fast-track-akka-scala/issues/8
Originally reported by @alejandrolujan

The current instruction:

Log CoffeeHouse Open at debug.
Should be:

Log "CoffeeHouse Open" at debug when the actor is created.

When running cmt-mainadm on a project that has common-project-enable set to false, it still reports that there is no common project in the repo.

Original Issue: https://github.com/typesafe-training/fast-track-akka-scala/issues/10
Originally reported by @alejandrolujan

Clarify the following instruction:

In this exercise, we will implement state by tracking a Guest actor's favorite Coffee.
With the following:

In this exercise, we will implement state by tracking a Guest actor’s coffee count and favourite Coffee.

Currently, no comprehensive set of CMT master repositories exist. This makes it particularly difficult for anyone who has no experience with the tooling to start using it in anger. At best, one finds an existing CMT project that is modified to suit the needs at hand. Not ideal...

#91 provides a good foundation to fill this gap: the validation master repositories can be created in such a way that they can serve as CMT master repository templates.

Also, we can imagine that we start issuing releases that contain these CMT templates readily usable by anyone wishing to start with CMT.

Alternatively, add a new CMT command, or an option -init <template name> to mainadm.

Original Issue: https://github.com/typesafe-training/fast-track-akka-scala/issues/12
Originally reported by @alejandrolujan

The last 4 tests on CoffeeHouseSpec assume the maps has a default value of zero, but this was never requested. The entries in the guest book are created when the CreateGuest message is processed. However, the tests assume that a map with a default value of zero is used.

If an approval request comes in for a guest that is not part of the bar, we should reject it, should we not?

I think we should either:

Change the tests to register the guest before the drink is ordered (my preference), or
Change the instructions to specify we want a default value of zero for guests not registered on the Coffee House

The generated test script doesn't include linearize command. It would be nice if we run tests on a linearized repo.
We are already testing the main repository iterating through each projects, hence running test on the last exercise can validate the linearize command