For an at-a-glance preview of how the whole workflow is going. Pages running horizontally, output of each job running vertically. Maybe per-workflow?

scaling seems to be wrong - crop job crops too much.

This seems to be the next job on our to-do list (other than DJVU binarisation, possibly). Filing it here for reference.

If we're going to be using the RGB salzinnes images then we need to either:

• Convert RGB to greyscale before threshold binarise
• Do DJVU binarise with defaults and no frontend
• Do DJVU with frontend

so that the workflow can run

Putting in just an email, no u/p causes http500

E.g., if we crop, we need the old/new offset so that we can show the old image along with the coordinates generated by the new image.

Future additions might be to allow 2 different cropped images: 1 small for recognition, 1 large but without colour bars for showing in diva.

It would be nice to have more fine-grained controls for the segmentation frontend. For example, if we could delete points with the delete key, and click+drag to highlight multiple points, etc.

Other things:

• When you drag one corner of a box, it should drag the two adjacent corners as well (to preserve perpendicularity) - maybe only when holding control.
• Make the canvas slightly larger than the image so that you have more of a "buffer" when moving points. Right now, if I move a point too close to the edge of the image, I lose control of it, which is undesirable.
• What else?

Make this work really easily by making a new despeckle job that fits in there.

Because segmentation is based on staff detection. We should at least make it a little higher and lower. Will need to investigate to get optimal values.

Could handle things like

• Passing in the relevant model instances (Project, Page, Job, etc) from the slug/id
• Rendering the template (with the context dictionary being the return value of the function)
• Automatically generating breadcrumbs (when relevant)

To be considered. Low priority (maybe after the July 1st deadline).

As suggested by Ich (just storing here for future reference if needed):

1. During workflow creation, inspect the result of a process on a whole bunch of pages (e.g. see that binarise value x is good for all these pages)
2. When it runs, don't bring up the binarise interface, just automatically run it
3. Way to go back and tweak value for a job on a single page, then run from there again. (Save results from the last run?)

The staff removal task takes quite a long time (around 80-100 seconds is what I see on susato). It would be great if this could be profiled and then optimised, if possible.

so that people aren't inclined to click the submit button

Filler text

These would be JPEG images of the same size as the 400px-wide thumbnail (or another one) but not scaled down (so only a small section would be visible).

Useful for despeckle and maybe others.

What part of the image should be shown?

It needs to be possible to edit parameters and also decide if the step should be automatic or not. This will require some minor changes to the model infrastructure, and to the way job views are handled to cut down on code duplication.

It should also be possible to edit parameters and restart a job that has already completed.

from a quick look at the console, it seems that there is something undefined - no image is being loaded

• Pyramid tiff
• Make sure divaserve works
• Install diva, hook up to solr

If you need to change a box and it's got lots of points then it's very hard. Make a button that just chooses the left/right/top/bottom most corners and makes a rectangle for easy resizing.

e.g.

• crop
• rotate
• binarise
• recognise

Be able to remove rotate without removing recognise & binarise first. Only if the output of crop can be used as input to binarise

For @brianstern and @antonkhelou

Instead of storing the scaling factor in the database (when creating thumbnails), we can just send the original image size in the JSON that's passed to the frontend.

Make the json look something like this:

{
  "width": 4000,
  "height": 5000,
  "points": [list,of,points]
}

then you can calculate the scale factor in the frontend based on the size of the thumbnail and these sizes.

Or at least parts of it. Neon uses it so why not. At least the buttons will be prettier.

What's going on with this? Does a frontend exist?

If the first job is an automatic one, it must be kicked off as soon as the workflow has been created.

Also, account for multiple workflows for one page (does everything take that into account? e.g. when getting results)

e.g. if you do /project/1/crop and there's no crop jobs available, it gives 404. Maybe it should redirect to /projects/1 with a flash message that you can't do that job?

takes about 15s to compute results for nothing

So that the server is responsive after you upload something. Also useful for if you give a directory and ingest all files.
Need to make sure that things that require pages don't show these "uploaded" pages until thumbs are ready. Can't make workflow, etc.

Neon needs images that are ~1200px wide to show as a background. It also needs a webservice call to ask for the original size of that image for scaling purposes.

To show on the project page.

• How many pages are done
• Total %age complete
• How many things are waiting for people (queue size per job, or just 1 global queue?)
• How fast jobs are being done?

Should be started right after the previous job finishes (in the @rodan_task decorator).

Also, should take the workflow-specific defaults into account (param). The job-specific defaults are the backup.

@brianstern, can you put up your work on the interfaces? You don't have to put them in the repository or worry about making them Django templates, even just putting it up as a gist and then linking it would be fine. I just need to see how you've done it so I can cleanly integrate it with what I have already.

... and add to a different set of pages

I'm working on this now. All the result and result file stuff should be handled outside of the task-specific code. Here's a sneak preview:

@task
@do_stuff(input_filetype='tiff')
def simple_binarise(image_filepath, **kwargs):
    output_img = gamera.core.load_image(image_filepath)
    output_img = output_img.threshold(kwargs['threshold'])

    return {'tiff': output_img}

The control for rotating the image in the rotation task could be resized or floated next to the canvas. Brought up by Andrew this morning. Filed here for reference.

The border removal plugin (github:DDMAL/gamera-lyric-extraction) works now, and it would be great if we could integrate it into the workflow system.

Using it is pretty straightforward:

from gamera.toolkits import lyric_extraction

# requires an image of type GREYSCALE

im = load_image("blah.tiff")
im = im.to_greyscale()

# border_removal returns a mask.
mask = im.border_removal(**use the defaults**)

result = im.mask(mask)

The result should be a greyscale file with the borders (incl. colour bars) removed.

The technique behind it is pretty tricky, so not sure if we can do this interactively with JS, but the default settings seem to work fine, so this can probably be a non-interactive task.

When a segmented staff is split into multiple parts (generally by an ornate letter), only the leftmost staff is removed.

For Ich's demo, we need a script to run through on the prod site, as well as a video showing the same thing (<5 minutes)

Use model.__class__.__name__

Sometimes a celery task will crash. We need to make sure it recovers if possible and restarts, or show this information somewhere so that someone can go in and fix it

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/celery/execute/trace.py", line 181, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/wliu/rodan_source/rodan/jobs/utils.py", line 55, in real_inner
    outputs = f(*input_paths, **kwargs)
  File "/home/wliu/rodan_source/rodan/jobs/pitch_finding.py", line 25, in pitch_find
    recognized_glyphs = aomr_obj.run(glyphs, poly_list)
  File "/home/wliu/rodan_source/rodan/jobs/aomr_resources/AomrObject.py", line 104, in run
    self.find_staves(poly_list)
  File "/home/wliu/rodan_source/rodan/jobs/aomr_resources/AomrObject.py", line 242, in find_staves
    diff_lo = avg_lines[3]-avg_lines[2]
IndexError: list index out of range

Found 1 staff. The JSON produced by staff-finding looks like this:

[[[[1084, 215], [1086, 215], [1164, 215], [1170, 215]]], [[[701, 1833], [776, 1833]]]

as discussed, d155dcf gets image width by opening the image as the page is loaded. If this is a large tiff then we don't want this delay. We need to store these params somewhere.

Because they're awesome

For dynamically updating the job grid and stuff.

!!!!

There are a lot of them, and they are messy and don't all work properly.