Git Product home page Git Product logo

triagebot's Introduction

Triagebot

This is the triage and team assistance bot for the rust-lang organization.

Please see the forge for our documentation, and feel free to contribute edits if you find something helpful!

How triagebot works

Triagebot consists of a webserver with several endpoints. The /github-hook and /zulip-hook endpoints receive webhook notifications from the respective services. Triagebot can then respond to those notifications to perform various actions such as adjusting labels.

The Triagebot webserver also includes several other endpoints intended for users to access directly, such as https://triage.rust-lang.org/agenda.

Triagebot uses a Postgres database to retain some state. In production, it uses RDS.

The server at https://triage.rust-lang.org/ runs on ECS and is configured via Terraform. Updates are automatically deployed when merged to master.

Installation

To compile the Triagebot you need OpenSSL development library to be installed (e.g. for Ubuntu-like Linux distributions sudo apt install libssl-dev).

Run cargo build to compile the triagebot.

Running triagebot

It is possible to run triagebot yourself, and test changes against your own repository. Some developers may settle with testing in production as the risks tend to be low, but the more intrepid may find it easier to iterate separately.

The general overview of what you will need to do:

  1. Create a repo on GitHub to run tests on.

  2. Configure a database

  3. Configure webhook forwarding

  4. Configure the .env file:

    1. Copy .env.sample to .env
    2. GITHUB_TOKEN: This is a token needed for Triagebot to send requests to GitHub. Go to GitHub Settings > Developer Settings > Personal Access Token, and create a new token. The repo permission should be sufficient. If this is not set, Triagebot will also look in ~/.gitconfig in the github.oauth-token setting.
    3. DATABASE_URL: This is the URL to the database. See Configuring a database.
    4. GITHUB_WEBHOOK_SECRET: Enter the secret you entered in the webhook above.
    5. RUST_LOG: Set this to debug.
  5. Run cargo run --bin triagebot. This starts the http server listening for webhooks on port 8000.

  6. Add a triagebot.toml file to the main branch of your GitHub repo with whichever services you want to try out.

  7. Try interacting with your repo, such as issuing @rustbot commands or interacting with PRs and issues (depending on which services you enabled in triagebot.toml). Watch the logs from the server to see what's going on.

Configure a database

To use Postgres, you will need to install it and configure it:

  1. Install Postgres. Look online for any help with installing and setting up Postgres (particularly if you need to create a user and set up permissions).

  2. Create a database: createdb triagebot

  3. In the .env file, set the DATABASE_URL:

    DATABASE_URL=postgres://eric@localhost/triagebot

    replacing eric with the username on your local system.

Configure webhook forwarding

I recommend at least skimming the GitHub webhook documentation if you are not familiar with webhooks. In order for GitHub's webhooks to reach your triagebot server, you'll need to figure out some way to route them to your machine. There are various options on how to do this. You can poke holes into your firewall or use a proxy, but you shouldn't expose your machine to the the internet. There are various services which help with this problem. These generally involve running a program on your machine that connects to an external server which relays the hooks into your machine. There are several to choose from:

gh webhook

The gh CLI is the official CLI tool which I highly recommend getting familiar with. There is an official extension which provides webhook forwarding and also takes care of all the configuration. See cli/gh-webhook for more information on installing it.

This is super easy to use, and doesn't require manually configuring webhook settings. The command to run looks something like:

gh webhook forward --repo=ehuss/triagebot-test --events=* \
  --url=http://127.0.0.1:8000/github-hook --secret somelongsekrit

Where the value in --secret is the secret value you place in GITHUB_WEBHOOK_SECRET in the .env file, and --repo is the repo you want to test against.

ngrok

The following is an example of using https://ngrok.com/ to provide webhook forwarding. You need to sign up for a free account, and also deal with configuring the GitHub webhook settings.

  1. Install ngrok.

  2. Run ngrok http 8000. This will forward webhook events to localhost on port 8000.

  3. Configure GitHub webhooks in the test repo you created. In short:

    1. Go to the settings page for your GitHub repo.

    2. Go to the webhook section.

    3. Click "Add webhook"

    4. Include the settings:

      • Payload URL: This is the URL to your Triagebot server, for example http://7e9ea9dc.ngrok.io/github-hook. This URL is displayed when you ran the ngrok command above.
      • Content type: application/json
      • Secret: Enter a shared secret (some longish random text)
      • Events: "Send me everything"

License

Triagebot is distributed under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE and LICENSE-MIT for details.

triagebot's People

Contributors

apiraino avatar camelid avatar chazkiker2 avatar dependabot-preview[bot] avatar dependabot[bot] avatar dtolnay avatar ehuss avatar elinvynia avatar giraffate avatar jackh726 avatar johntitor avatar joshtriplett avatar jyn514 avatar kellda avatar kobzol avatar kraai avatar leseulartichaut avatar llandy3d avatar mark-simulacrum avatar mibac138 avatar nikomatsakis avatar obmarg avatar pietroalbini avatar pitaj avatar rylev avatar sladyn98 avatar spastorino avatar tmandry avatar urgau avatar wafflelapkin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

triagebot's Issues

Dependabot can't resolve your Rust dependency files

Dependabot can't resolve your Rust dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

    Updating crates.io index
error: failed to select a version for the requirement `core-foundation-sys = "^0.7.1"`
  candidate versions found which didn't match: 0.8.0, 0.7.0, 0.6.2, ...
  location searched: crates.io index
required by package `core-foundation v0.7.1`
    ... which is depended on by `security-framework v0.4.4`
    ... which is depended on by `native-tls v0.2.4`
    ... which is depended on by `postgres-native-tls v0.3.0`
    ... which is depended on by `triagebot v0.1.0 (/home/dependabot/dependabot-updater/dependabot_tmp_dir

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Escape angle brackets in issue titles

GitHub issues with < > in the title, such as rust-lang/rust#69163, emit html on the notifications page that contains raw angle brackets, which end up treated by the browser as html tags. We should emit these characters as &lt; and &gt; and similar for any other html characters that need escaping.

Undecided on whether the description given in add commands should be escaped or should allow html tags. I think I would lean toward ideally angle brackets in add commands should also get escaped.

Highfive integration

Consider adopting some of the functionality of @rust-highfive into triagebot.

Things like:

  • Greeting new contributors
  • Heuristics based assignment of PRs to a random reviewer
  • Reassigning PRs in response to r? @username

These are related to things triagebot already does.

Yaahc workflow user story

Per discussion with @Mark-Simulacrum in the triagebot zulip stream I'm documenting my current workflow for managing active work items and notifications and what features I'd like to see out of triagebot.

Current Workflow

Currently I use github notifications beta to manage my notifications via the web ui, and I have email notifications completely turned off.

For me this is sufficient for keeping track of my notifications but it does not handle the following issues.

  • Keeping track of current work items
  • Prioritizing current work items
  • Making my work queue / notifications queue public to other users on github

For tracking my work items I created a github project board in an otherwise empty repository that I use to track current issues/prs that I'm working on or want to work on. https://github.com/yaahc/prs/projects/1

This has the advantage of being public, and letting me track my current work items, but its painfully manual and its not visible / people rarely find their way to that page to see what I'm working on.

Ideal Workflow

Ideally for me I'd continue to use github notifications ui for all my notifications, but I'd like my current notifications to be made public for other users to consume, though I'd only want notifications from public repos to be visible, for me having the triage bot notifications page be a publicly visible copy of my current notifications would be idea.

For tracking work items I'm not particularly happy with the current state of my project board, mostly because of how manual it is, I often forget to put issues or prs into the board. Ideally a replacement via triagebot would support the following features.

  • Track my currently in progress work
  • Automatically add newly opened PRs and issues that are assigned to me to my work queue
  • Automatically move closed PRs and issues to a done list or remove them from the board entirely
  • Link this work queue in places for other contributors / users to find

To me it would be extra cool if all of this worked by integrating with my existing project board, though that's probably a bit of a pipe dream. The most important thing for me is an ordered list of work that is visible enough for other people to see and comment on, but that mostly manages itself so I don't have to constantly go out of my way to remember what I'm working on and add it to the list.

"meta <idx> <metadata...>" affects index of item

I sent "meta 6 something or other" to triagebot on Zulip and it correctly set the bullet text for that item, but in the process that item ended up at index 5 instead of remaining at 6 as I would expect.

Assuming someone else didn't run "as dtolnay move ..." at the same time. :)

Include help in comment trailer

One of the cool UX things about dependabot is that it includes the help commands at the end of every response hidden under a summary, which lets you easily look at the available commands without leaving the github thread. It would be nice if triagebot also did this or something similar.

Example

Screenshot 2020-06-15 at 16 36 37

Screenshot 2020-06-15 at 16 36 47

Triagebot on Zulip: "Failed to query team API: could not get team data"

I sent a "meta" command to triagebot on Zulip and got back:

Failed to query team API: could not get team data

Caused by:
0: error sending request for url (https://team-api.infra.rust-lang.org/v1/zulip-map.json): connection error: Connection reset by peer (os error 104)
1: connection error: Connection reset by peer (os error 104)
2: Connection reset by peer (os error 104)

I resent the same command ~5 seconds later and it worked. Maybe this needs some retry logic?

Don't re-assign user if they're already assigned

Example:

  1. Comment by @user:

    @rustbot claim
     
    [some other text]
    

    Bot assigns @user (directly or indirectly) and edits original issue comment if needed.

  2. @user edits their comment:

    @rustbot claim
    
    -[some other text]
    +[some different text]

    Bot unassigns @user (directly or indirectly) and edits original issue comment if needed, then assigns @user (directly or indirectly) and edits original issue comment if needed.

rust-lang/rust example

Perhaps the bot should not do anything in 2, if possible. Especially if the target user is already assigned.

Add command for relabeling issues from Zulip

As suggested in today's Prioritize WG meeting, being able to relabel issues from Zulip could be helpful, e.g. for prioritizing issues as per the prioritize-WG or labeling during meetings, without leaving the Zulip chat.

The command syntax could probably follow the existing @rustbot modify labels on GitHub, but may need to take an additional link-to-the-issue parameter, e.g. @rustbot modify labels for #XXXXX: .... If absent, it may look at the topic name and get the link from there (e.g. all topics names for I-prioritize issues have the link to the issue inside them).

This issue has been assigned to @LeSeulArtichaut via this comment.

Add 'desposition-merge' command

A workflow I often run into is where someone has opened a PR, I've reviewed it, but before I merge I want to cc a group or individual to look over it, but I don't want to block merging the PR on their review if they're busy. It would be nice if you could say @rustbot merge 2 days to have rustbot merge automatically if there's been no review in the time given.

Error sending request: connection closed before message completed

I got this in response to a move 12 1 command on Zulip:

Failed to parse movement, expected move <from> <to>: failed to get RDS cert

Caused by:
0: error sending request for url (https://s3.amazonaws.com/rds-downloads/rds-ca-2019-root.pem): connection closed before message completed
1: connection closed before message completed.

I tried the same command again and it worked.

Possibly same root cause as #332?

Ping teams for marker teams

Right now we have rustbot ping everyone individually (cc @ghost) in a marker team, this isn't scable as GitHub mentions stop after 50 people. This might seem hard to reach but ICE-Breakers Cleanup Crew has 24 members and ICE-Breakers LLVM has 14 members. possible that we get more than 50 members, or if there are issues we want to tag multiple marker teams, we would easily hit the over 50 limit.

We could add logic to triagebot to only a select 50 people at a time, but then we'd need to have logic to ensure everyone gets an equal amount of notifications, and would add significant complexity. It would be simpler to have them as github teams. This also has the added benefit of the notification going into the team-mentions section in the new GitHub notifications rather than mention. Since these issues don't require that individuals specific attention rustbot should respect that and ping the team.

@pietroalbini mentioned in #3 reservations about granting marker team members with a GitHub Team since they would be granted "member" giving some kind of perceived authority. I don't think we should be applying any kind of authority to GitHub's structure. We already have members on GitHub that don't match with our current membership. rust-lang/team should be considered the normative structure of the Rust teams.

rust-lang/rust#69225 (comment)

React when someone applies `I-prioritize`

Having to leave a @rustbot prioritize comment is less convenient than just applying I-prioritize along with the other labels, so it would be great if triagebot could notice the label being applied and do its magic (creating the Zulip stream, etc.)

Warn if beta-nominated label is added without team label

More abstractly, we should have a list of "team required" labels (e.g., I-nominated, beta-nominated, S-waiting-on-team...) in the configuration for rust-lang/rust, as well as a "team label" list ("T-" and "WG-" maybe).

The bot would detect labels in the first list being added without an accompanying team label being added. There is a proposal of some sort of "if not added within 1 minute" or something, but I don't think that's easy to implement and probably best to just not do that since it adds complexity to the implementation (we have no timers today).

Auto-assignment

Highfive currently auto-assigns people to review PRs based (I believe) on some match between the directory where the changes occur and its JSON config containing assignees. It would be great if we could not implement auto-assignment in triagebot, but also make some improvements to the auto-assignment system.

Requirements

  • Instead of using a JSON config, assignees should taken from data in rust-lang/team.

Potential Improvements

  • On zulip there was a discussion around how to improve review latency, or at least the review experience. One proposed idea was place a limit how many open S-waiting-on-review PRs you as a rust-lang member could be assigned, and if all members of a team are assigned a new S-waiting-on-reviewer is given to the PR. Which would also allow us to properly communicate a team's current bandwidth.

Unresolved Questions

  • How would people set their limit?
    • It would be easiest (I would guess) on the implementation side if this limit was also set in team, but team can take a some time to get a PR merged.
    • Because of the above I feel it would be better suited as a bot command on Zulip.
    • The solution could also be both, a user sets through a bot command which updates the team repo.

Notifications for new issues and PRs

In serde-rs repos and my personal repos, usually when someone makes an issue or PR they don't ping my username. But I would still like to treat those as a notification because I am the one responsible.

It would be great to set via some config, whether local to the repo or global to the user or org, that says who wants to be notified on new issues/PRs.

Release week regression labeling

If a regression-from-stable-to-beta label is added in the release week, label with T-compiler, T-release, I-nominated and ping the release and compiler teams.

Split the `prioritize` feature in smaller parts

As of today, the prioritize feature has 3 roles:

  • when the @rustbot prioritize is used, label the issue with the prioritization label
  • when certain labels are added, label the issue with the prioritization label
  • when the prioritization label is added, create a conversation on Zulip

I would propose to split those 3 features. This will allow for more flexibility, e.g. to define multiple triggers for auto-labeling. This is needed for the automation the Prioritization WG would like to set up in the near future:

  • Create a topic per stable/beta nominations, i-nominated and prs-waiting-on-team
  • Create a topic per P-critical and P-high regressions

I propose to have configuration like:

[prioritize]
label = "I-prioritize"

[autolabel."I-prioritize"]
trigger_labels = ["regression-from-stable-to-stable", "regression-from-stable-to-beta", "regression-from-stable-to-nightly"]
exclude_labels = [
    "P-*",
    "T-infra",
    "T-release",
]

[notify-zulip."I-prioritize"]
zulip_stream = 227806
topic = "I-prioritize #{number} {title}"
message_on_add = "@**WG-prioritization** issue #{number} has been requested for prioritization."
message_on_remove = "Issue #{number}'s prioritization request has been removed."

cc @Mark-Simulacrum @spastorino

Auto-hide comments that only contain a label command

When using the 'rustbot modify labels' command, the comment containing the command is essentially just noise. The GitHub UI already shows label changes in a more compact and readable format.

It would be useful for triagebot to auto-hide the comment containing the label command (assuming that the message has no other content).

According to isaacs/github#1480 (comment), this can be done through the GitHub GraphQL API.

Exclusive labels

We might want to add the ability to define some label patterns as "exclusive", so that there can only be a single label matching a pattern on an issue.

For example, with S-* marked as exclusive a PR with the S-waiting-on-review will have S-blocked after someone types the command:

@rustbot modify labels: S-blocked

Add ability to ping WGs members using triagebot

We've agreed on Zulip that is going to be great if we can have a way of pinging WG members (which are not part of rust-lang github org) using triagebot.

People is already able to sign up by themselves to WGs, just by sending a PR to the team repo on github. We would stick with that idea and encourage people to do so.

This issue is about being able to reach them out. We want a command like @rustbot icebreaker llvm that would add a github comment reaching out all the members of that working group with a text that looks like ...

Hey LLVM icebreakers! This bug has been identified as a good
"LLVM ice-breaking candidate". In case it's useful, here are some
[general instructions](link) for tackling these sorts of bugs. Maybe take a look?

cc @foo @bar @baz ...

Where, the first part of that text is a predefined text that's specific to the working group. Every working group has a very similar text but it's not exactly the same. Specifically the link to instructions is going to be different.

And then, in the cc part, the bot figures out the list of members it needs to mention.

Don't require the final `.` when parsing command

Currently @rustbot expects the final . to terminate a modify labels command. Leaving out the . leads to a cryptic error message like rust-lang/rust#58208 (comment):

Error: Parsing label command in comment failed: ...-on-author|error: a label delta at >|...

The error message should be improved, or even better, treat the end-of-line as another terminator.

Add command/workflow to go from S-waiting-on-author to S-waiting-on-review

Currently the status labels for PRs are mostly manually updated by triagers, it would be nice if authors of PRs could more easily update the status when it's S-waiting-on-author when they believe they have completed the necessary changes.

Potential Solutions

  • Detect when the author has requested a review using GitHub's request review feature and automatically update labels.
  • Add some kind of @rustbot review command.

I think ideally we would have both, as @rustbot review is easy to teach, but more people are using GitHub's review features as it better integrates with GitHub notifications, and the @rustbot review command could be implemented so that it uses GitHub's request review feature.

The assignment has failed silently

See this comment.
Triagebot tried to edit the issue's description and assign them but it seems its work has failed.
It's nice for the bot to tell us what's happening, i.e. show an error message.

Ping aliases for `@rustbot ping the_alias`

Instead of having to remember @rustbot ping icebreakers-cleanup-crew it would be great if we could add a memorable alias for the group so that I could e.g. write @rustbot ping cleanup. We would configure this by adding a list of aliases under a ping section.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.