academysoftwarefoundation / opencue Goto Github PK

There a few hardcoded email addresses in the cuebot code. For example:
cuebot/src/main/java/com/imageworks/spcue/service/EmailSupport.java
We should move these to configs.

Define best practices, especially in a cloud environment. Cue specifically RQD expect that usernames are consistent (but not UIDs, right?)

The database and Cuebot will continue to run on Linux but all other pieces - PyCue, PyOutline, CueGUI, and RQD - should run on Windows.

RQD should in theory run on all platforms but there is some OS-specific code in there especially around system metrics collection.

Ensure that all "public" methods in pycue have clean appropriate docstrings.

By default we only have the #IFRAME# replacement token, which limits our ability to do frame chunking in the application (executing a single command with a frame range, so we only incur the application startup cost once).
It would be nice to have a start and end frame token exposed by default in pyoutline making this functionality at least work for Maya and Nuke.

Regardless of the column selected for sorting, the job table will rearrange itself apparently randomly.

Both Python and Java implementations of FrameRange/FrameSet assume that all frames are whole numbers - ints are used throughout. Possibly other places in the code make this assumption as well.

Ensure that subframes are supported and fix as necessary.

Pyoutline is the one library that is snake case. For consistency, we should move it to be camel case.

We can construct the changelist from the list of commits in master since the last release.

Existing Dockerfiles and setup.py use hardcoded placeholder version number. This should come from the build system.

Scripts/docker images/other tools for packaging and deploying the various OpenCue components.

Currently OpenCue assumes a secure network between components and uses an unsecured gRPC channel.

Create a test deployment of OpenCue that makes use of self-signed TLS certs for gRPC, make any code changes necessary, and document how users can set this up on their own.

Plus: build it into the Docker images so a default deployment uses secure channels already.

Standalone app, as well as Maya and Nuke plugins.

spi_cue.Cue3.wrappers.layer defines a function unbookProcs. This method signature does not have any match in ICE or on the server. Verify that it is ok to remove.

The old C++ implementation has smart string representations - sequences of frames are grouped by range and step. I believe this only really affects visual representations of the frame range; the backend deals with only flattened lists of frames.

See the old libFileSequence test code[1] for examples of how the grouping works.

Our existing code uses very simple string representations - flat comma-separated lists of frames contained within. By Python and Java implementations should be switched to match the old C++ library.

[1] https://github.com/imageworks/OpenCue/blob/165d4c54b5e9ad55ec84416d8fdc79761174cabf/libFileSequence/test/FileSequenceTest.cpp#L800

There were two files, cue3.yaml and etc/default.yaml, which have similar contents including some SPI-specific stuff in there.

This has been condensed to a single etc/default.yaml but still needs cleanup and documentation of the values available.

This involves removing the existing Point and TrackIt concepts and replacing them with a redesigned general-use API. Goals will be to implement an API that meets the existing Trackit setup's requirements, as well as can support other popular production management tools e.g. Shotgun.

They are all listed individually by name now.

Per Google OSS guidelines.

This file will spell out exactly how potential contributors can contribute code changes, including signing a CLA as well as the Pull Request process itself.

Update all code and docs to use the new name.

Frame logs are currently written out to a shared file system that all the clients need to also have mounted. We should continue to support this configuration, but also create an interface for exporting to a logging service.

This flag doesn't currently do anything, so the process always runs in the foreground.

Currently most of servants return the generic grpc error (equivalent of returning a 500). Have the servants catch errors and send appropriate onError messages using grpc's own error messaging.

For example:

    public void findJob(JobFindJobRequest request, StreamObserver<JobFindJobResponse> responseObserver) {
        try {
            responseObserver.onNext(JobFindJobResponse.newBuilder()
                    .setJob(whiteboard.findJob(request.getName()))
                    .build());
            responseObserver.onCompleted();
        } catch (EmptyResultDataAccessException e) {
            responseObserver.onError(Status.NOT_FOUND
                    .withDescription(e.getMessage())
                    .withCause(e)
                    .asRuntimeException());
        }
    }

Currently many of the WhiteboardDaoTests in cuebot simply call the method they are testing but do not check that the returned value is correct.
We should go through and update these tests to make sure that the returned value is accurate.

RQD runs unit tests, but that's about it.

PyCue, PyOutline, CueAdmin, CueGUI, and CueSubmit all have tests in various forms that aren't being run automatically.

It looks like FOLDER.f_order in the database was mislabeled as a float. It is currently being stored as an integer so we should relabel it to be FOLDER.int_order.

After the grpc switch, it appears that enums don't display in the GUI correctly - only numerical values.

The old healthcheck used a simple ICE test client, and needs to be replaced.

com.imageworks.spcue.VirtualProc is an example of ThreadMode using the value of the gRPC enum. Instead host.threadMode should simply use the enum.

The old ICE layer had caching for individual instances of each object type. We should add a similar cache to fit somewhere within the new architecture - at the database layer is a likely candidate.

If you look at ManageJob.launchSpecAndWait, this method is probably doing too much logic in the servant class. We should have jobLauncher.launch return a list of Jobs so that we don't have to iterate over the spec and call findJob inside the servant.

The logo needs to be added to the submit widget and cuegui splash screen.

Cuebot and RQD use somewhat different logging setups and RQD at least needs some cleanup - in various places it uses standing logging as well as print directly.

Clean up the logging code in both, using standard logging libraries, and document how logs can be written to either a) a configurable path on disk, or b) to a logging service such as Cloud Logging.

rqutil.py has a function, checkAndCreateUser that only works with linux. Add mac and windows support.

Right now in pyoutline the launch command will only return the first job submitted. It should return all jobs submitted and let the caller handle what to do with them.
See pyoutline/src/outline/backend/cue3.py : launch()

Currently a number of rqd variables are hardcoded in rqconstants.py. Some of these are proper constants but others need to be dynamic, such as:

• Cuebot hostname
• Port numbers
• Logging config

These should be modified to support a config file / environment vars so that the application code doesn't need to be modified by the end user.

Cuebot sends email in a variety of situations e.g. job completion, job failure. Some of code hardcodes SPI-specific things like imageworks email addresses, and other code assumes all users use a @imageworks.com email domain.

OpenCue contains a concept called "mcp" which is basically a term for local temporary storage. It is monitored as a resource in the same way that cores or RAM is, to ensure that render nodes have enough scratch storage to complete a job.

It originally had a common mountpoint of /mcp on Linux, hence the name.

Rename this to be something a bit more clear - "tmp" or something like that - and make the mount point a config value as its hardcoded to /mcp in a few places particularly RQD.

https://github.com/imageworks/OpenCue/pull/139 adds some defaults when Show/Shot aren't set as env vars.

However when testing this, I filled some values into the Show and Shot fields in CueSubmit and it looks like the defaults are still being used. If the show defined by default_show doesn't exist, the job submission fails with a generic exception. The shot doesn't seem to affect much, and the job submission succeeds, but when I pull up the job attributes in CueGUI I do see the shot from the config file rather than what I filled into the CueSubmit window.

Ice provided a way to log the number of

• requests for data
• requests for function
• error requests

The SystemStats call would retrieve this info. We need to reimplement this functionality in the absence of ICE.
See TODO in com.imageworks.spcue.servant.CueStatic.

Code is stored under a src/ subdir unlike other components, and exposes most of its functionality in a toplevel namespace via wildcard imports.