jobteaser / job_tomate Goto Github PK

Was removed with update to v0.2.0.

We should notify the #maintenance channel if there is too many issues in the maintenance board. Define appropriate levels and send notification on appropriate events (Events::JIRA::IssueCreated for example).

When an issue changes to "Functional review" status:

• for issues with a "Feature owner", set him/her as assignee,
• otherwise, set the reporter.

Toggl reports from users who do not need to have their timespent added to JIRA issues are currently remaining in the pending status. Instead, they should be set to ignored.

Instead of using user accounts to create worklogs, use the shared user (JobTomate). This removes the need to set clear-text password in JobTomate's DB, and simplifies the addition of a new team member.

It is possible to add comments to worklogs, so take the opportunity to indicate who's the associated team member in the comment.

• Create worklogs using JobTomate user (credentials in environment variables)
• Add a comment to the worklog indicating who did the worklog

Send alerts about Toggl over Slack:

• If no report received after N hours
• If more than 10 hours have been logged on 1 day

Never worked well, not easy to understand, and not working at all (I think). Also help with using it by implementing an help command.

Toggl is not used by the team anymore. Remove all related code.

• If an issue is moved to a status after development (e.g. review, functional review, released)
• And it is a bug
• And it has an empty bug cause
• Notify the developer

• PR events not added to JIRA issues
• Assignee update on issue status change not done

Bug raised by @wRadion, to be confirmed.

This is intended for humans, so please use the correct link!

https://jobteaser.atlassian.net/rest/api/2/issue/22900 => https://jobteaser.atlassian.net/browse/JT-XYZ

So we can drop JIRA user passwords from the db

For example, make a tutorial on how to add a new workflow by using StoredWebhook and StoredRequest to build fixtures, write tests and implement a new workflow in TDD.

Logs of services triggered by a task do not have a transaction id:

May 31 13:31:47 job-tomate app/scheduler.6370:  I, [2017-05-31T20:31:46.592164 #4]  INFO -- : JobTomate::Commands::Toggl::FindEntry.run transaction='' - START 
May 31 13:31:47 job-tomate app/scheduler.6370:  I, [2017-05-31T20:31:46.644489 #4]  INFO -- : JobTomate::Commands::Toggl::FindEntry.run transaction='' - END (5.2291737e-05s) 
May 31 13:31:47 job-tomate app/scheduler.6370:  I, [2017-05-31T20:31:46.644980 #4]  INFO -- : JobTomate::Triggers::Tasks::FetchTogglReports.run transaction='' - END (0.033110937645s) 

Add a transaction ID for those too.

• Workflows should not be processed synchronously anymore. Instead, they should be processed through Sidekiq jobs.
• Ensure Sidekiq interface may be used to detect failed jobs, inspect the triggering event and restart the job.

Example:

Ticket jira: https://jobteaser.atlassian.net/browse/JT-4208
PR: https://github.com/jobteaser/jobteaser/pull/3841

In the message Your tests passed on CircleCI! it would be great to have a link pointing to the PR.

An Even better option:

Your tests passed on CircleCI! View on: JIRA Github

• Ensure code coverage is calculated and reported by the CI test build
• Add the badge to the README

Pending Toggl entries should not accumulate. If this happens, this means something is not working as expected (maybe some users have not been created in the database, thus preventing JobTomate from knowing how to process their worklogs).

We may imagine a scheduled workflow, running everyday, that would send a Slack message, e.g. to the CoreTech channel, to warn them from the issue.

• log the stored webhook's ID
• remove the payloads
• remove the stacktraces

Instead of generating a new transaction ID, re-use the request ID.

Only necessary if asynchronous is restored (disabled to fix #61). Do with #52.

(NB: Hard one)

On creation of a new maintenance issue, get some information about the issue and try to determine the appropriate developer. We may use words in the issue title / description and figure out with simple rules, check if a link to a bug tracker is included, check if another issue was mentioned...

• If an issue is moved to review or functional review
• And the issue's field for feature env is empty
• Notify the developer

JobTomate is not a critical service and it's well tested. A big limitation to its greater use in the team is that it is not easily deployed (plus it's not easy to start coding with it).

Setting up automated deployment will remove a limitation by enabling changes to be quickly deployed to production without having to ask someone else (generally really busy!) to deploy it.

Send a Slack DM if an issue is an a given threshold of its due date (3 days before, on due date, every day after due date)

Seems not to be working anymore. Raised by @giniouxe.

Mar 21 16:56:51 job-tomate app/web.1: 2017-03-21 15:56:51 - NoMethodError - undefined method []' for nil:NilClass: Mar 21 16:56:51 job-tomate app/web.1: /app/lib/job_tomate/values/github/status.rb:30:in author_github_user'
Mar 21 16:56:51 job-tomate app/web.1: /app/lib/job_tomate/values/github/status.rb:34:in author_user' Mar 21 16:56:51 job-tomate app/web.1: /app/lib/job_tomate/actions/slack_notify_on_github_status_update.rb:16:in run'
Mar 21 16:56:51 job-tomate app/web.1: /app/lib/job_tomate/support/service_pattern.rb:26:in run' Mar 21 16:56:51 job-tomate app/web.1: /app/lib/job_tomate/events/github/status_updated.rb:14:in run'
Mar 21 16:56:51 job-tomate app/web.1: /app/lib/job_tomate/support/service_pattern.rb:26:in run' Mar 21 16:56:51 job-tomate app/web.1: /app/lib/job_tomate/triggers/webhooks/github.rb:43:in process_status_event'
Mar 21 16:56:51 job-tomate app/web.1: /app/lib/job_tomate/triggers/webhooks/github.rb:31:in run_events' Mar 21 16:56:51 job-tomate app/web.1: /app/lib/job_tomate/web.rb:36:in block (3 levels) in class:Web'
Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1610:in call' Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1610:in block in compile!'
Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:974:in block (3 levels) in route!' Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:993:in route_eval'
Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:974:in block (2 levels) in route!' Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1014:in block in process_route'
Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1012:in catch' Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1012:in process_route'
Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:972:in block in route!' Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:971:in each'
Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:971:in route!' Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1084:in block in dispatch!'
Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in block in invoke' Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in catch'
Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in invoke' Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1081:in dispatch!'
Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:906:in block in call!' Mar 21 16:56:51 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in block in invoke'
Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in catch' Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in invoke'
Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:906:in call!' Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:894:in call'
Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/xss_header.rb:18:in call' Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/path_traversal.rb:16:in call'
Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/json_csrf.rb:18:in call' Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in call'
Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in call' Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in call'
Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/rack-1.6.4/lib/rack/nulllogger.rb:9:in call' Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/rack-1.6.4/lib/rack/head.rb:13:in call'
Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:181:in call' Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:2021:in call'
Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1486:in block in call' Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1795:in synchronize'
Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1486:in call' Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/puma-2.15.3/lib/puma/configuration.rb:79:in call'
Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/puma-2.15.3/lib/puma/server.rb:541:in handle_request' Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/puma-2.15.3/lib/puma/server.rb:388:in process_client'
Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/puma-2.15.3/lib/puma/server.rb:270:in block in run' Mar 21 16:56:52 job-tomate app/web.1: /app/vendor/bundle/ruby/2.3.0/gems/puma-2.15.3/lib/puma/thread_pool.rb:106:in block in spawn_thread'

Ping the issue's reporter, developer (and/or reviewer, functional reviewer, depending on the status of the issue) if the issue has not been updated for a long time (e.g. 3 days?).

PR validated in review (:+1: in the comments)

• add a comment in JIRA and change the issue's status to "Functional Review".
• add a "development review ok" Github label

PR declined in review (:-1: in the comments)

• add a comment in JIRA (including the comment on the PR) and change the issue's status to "In Dev"
• remove any "development review ok" label on Github
• add "modification required" label on Github

On deployment of an issue, add the link to the deployed feature environment in a comment on the corresponding JIRA issue.

As seen in pairing / code review with @marinados, some parts of the architecture clearly need to be better explained to reduce the knowledge to be shared to comply with the intended design!

Propositions:

• To document workflows, use diagrams instead. Indicate clearly what event in the workflow triggers an action.
• Clarify that only Actions are intended to implement business logic, not Events. Maybe even better by renaming Action to Workflow, would prevent to think that some action is really performed (what could induce adding a condition before calling an Action!). Another possible refactoring to fix this is to move to an event bus pattern.
• Explain that actions must not be dependent. Actions should be runnable concurrently and dependance between actions is handled through JIRA that will trigger other webhooks after a change. JobTomate should be stateless in this regard.
• Clarify the overall architecture by creating an high-level diagram.
• Give indications on what is necessary to contribute and how to get access (e.g. Heroku permissions...)
• Write some contributing howtos, e.g. how to add a new workflow

Commentaires review JobTomate.pdf

Only the Product Owner is taken into account to assign a JIRA card when it is moved to Functional Review.
The Reporter should be used instead to assign the JIRA card to the proper user when the ticket is a Bug.

Ask the functional reviewer a feedback on the JIRA issue when the functional review is accepted.

Send a Slack message to the developer's manager or tech lead if an issue makes more than 2 returns to "In Dev" status after review or functional review

On a new JIRA issue, send Slack alerts to the #maintenance channel depending on the number of issues in the maintenance board

Github notifies of the status update of a pull request through its webhook. Notify the pull request's submitter when the status is received.

Build is working locally, but not on TravisCI

If you check this JIRA you'll see that the worklog is said to be started at 7:30 PM. In fact, the worklog started at about 5:00 PM and ended at 5:30 PM. It was then added by JobTomate at 7:30 PM.

Adjust what is logged to JIRA so that the start time in JIRA matches the real start time.

If one deletes a Toggl report, we don't diff with the stored TogglEntry records to identify which one(s) and fix JIRA worklogs.

When i create a pull request, i receive "Code Climate is analyzing this code." message from tomato.

I think this message is useless, it's possible to remove it ?

Send a Slack DM to the developer if the JIRA is changed to status "In Review" without a PR in the comments.

Continues #22.

Processing Toggl report for an user that is not yet registered in JobTomate. The corresponding TogglEntry is correctly remaining in pending status, but this raises an exception (when performing the request to JIRA) because the response is unexpected.

Expectations

• The request to JIRA should not be done if we don't know the user performing the worklog.
• We should make so that the JIRA client correctly handles such a response (probably some error).

Seems to cause the following exception:

» 25 May 2016 11:31:18.645 248 <190>1 2016-05-25T09:31:18.437215+00:00 app scheduler.2306 - - /app/vendor/bundle/ruby/2.2.0/gems/httparty-0.13.7/lib/httparty/response.rb:67:in `slice': wrong number of arguments (3 for 1..2) (ArgumentError) from /app/vendor/bundle/ruby/2.2.0/gems/httparty-0.13.7/lib/httparty/response.rb:67:in `method_missing' from /app/lib/job_tomate/commands/jira/support/client.rb:46:in `merge_paginated_responses' from /app/lib/job_tomate/commands/jira/support/client.rb:23:in `block in exec_request' from /app/lib/job_tomate/commands/jira/support/client.rb:22:in `loop' from /app/lib/job_tomate/commands/jira/support/client.rb:22:in `exec_request' from /app/lib/job_tomate/commands/jira/add_worklog.rb:54:in `create' from /app/lib/job_tomate/commands/jira/add_worklog.rb:35:in `run' from /app/lib/job_tomate/support/service_pattern.rb:26:in `run' from /app/lib/job_tomate/events/toggl/helpers.rb:48:in `create_worklog' from /app/lib/job_tomate/events/toggl/helpers.rb:26:in `create_worklog_and_update_entry' from /app/lib/job_tomate/events/toggl/new_report.rb:26:in `run' from /app/lib/job_tomate/events/toggl/new_report.rb:17:in `run' from /app/lib/job_tomate/triggers/tasks/fetch_toggl_reports.rb:42:in `create_entry' from /app/lib/job_tomate/triggers/tasks/fetch_toggl_reports.rb:33:in `block in process_reports' from /app/lib/job_tomate/triggers/tasks/fetch_toggl_reports.rb:30:in `each' from /app/lib/job_tomate/triggers/tasks/fetch_toggl_reports.rb:30:in `process_reports' from /app/lib/job_tomate/triggers/tasks/fetch_toggl_reports.rb:26:in `run' from /app/lib/job_tomate/support/service_pattern.rb:26:in `run' from bin/run_task:11:in `<main>' Context

Add the PR's branch name in the JIRA comment about a new PR to help with testing in in development, easier checkout.

Currently, the only way of updating the configuration is to connect to the production Rails console. This requires Heroku access and some time on the terminal. Plus you need to know the correct command to run to add an user. Not really user-friendly, and this led to the configuration not being updated.

Selected solution

Google Sheets integration

Implementation

To limit the cost of implementation, the following solution will be implemented:

• The current User model stored on MongoDB is kept so it will be usable as a local copy of the data, in case the Google doc is not accessible (e.g. token expired)
• A script is implemented to perform the one-way synchronization from Google Sheets to the local DB. The script will be run through a simple scheduler.

Steps

• Integrate with Google Sheets API
• Create a file with the appropriate content
• Add a valid token to authenticate the API calls
• Implement local caching to ensure the application continues to work if access to the Google Sheet is broken
• Add specific documentation
  • how to update the token if needed
  • which user is used to provide the permission

Possible solutions

ActiveAdmin-like interface

This would require to manage authentication and I'd like to keep JobTomate simpler than that for now. In the future, we may rely on the IDP so including authentication would be easier.

JSON file on AWS

Simple, easy, but not really adequate to manage accesses to update the configuration either.

Google Sheets integration

We would have a Google Sheets document containing the documentation. This would allow easy edition of the configuration, with versioning, authentication, traceability. Integration seems relatively easy in Ruby, see this documentation.

Objectives

Update the JIRA issue's roles (developer, reviewer, feature owner) if empty when an user changes the issue status.

Example requirements

Moving issue to "In Development" as a developer

As a user that is a developer
When I move a JIRA issue from "Open" to "In Development"
And if the JIRA issue has no "developer" yet
Then I should be set to the "developer"
And I should be assigned to the issue

• Indicate what is the change: new issue, issue moved to In Review, etc.
• Applause for solved issues!

We should use security tokens provided by Github and JIRA to secure the received webhooks.

When a pull request gets a new comment, the pull request's submitter is notified of the comment in Slack.