alexellis / derek Goto Github PK
View Code? Open in Web Editor NEWReduce maintainer fatigue by automating GitHub
Home Page: https://github.com/alexellis/derek/blob/master/USER_GUIDE.md
License: MIT License
Reduce maintainer fatigue by automating GitHub
Home Page: https://github.com/alexellis/derek/blob/master/USER_GUIDE.md
License: MIT License
I'm a little stuck on the "Install Derek as a GitHub app" section as I can't find it on the marketplace as most "Github apps" are.
So then I'm guessing I have to create my own Github app? If so could you help with what these fields will needs to be filled with.
I already have a publicly facing openfaas gateway that I can use.
(Also might be an idea to link to the Github app registration page.)
https://github.com/settings/apps/new
ย
A PR was raised to enable users to invoke Derek via another prefix such as "/"
in #56, but the code was not signed-off and the patch could not be merged.
This is a suggestion for third-party usage of Derek outside of the OpenFaaS project. I think this is a good beginner issue.
Anyone can host their own Derek and also validate their own customer list
The code will validate against our list.
There is a workaround though - just disable customer validation via the existing env-var flag.
I'd suggest we allow the customer list URL to be supplied via env-var too.
Related #51
Addition of a new feature to check the contents of a PR description for a referenced issue number. See: https://help.github.com/articles/closing-issues-using-keywords/ for a full explanation.
A PR is raised with an issue reference but without a closure keyword. This results in manual activity to keep repo issues in sync with project state.
Similar to the sign-off check. Test the incoming detail for GitHub closure keywords:
close
closes
closed
fix
fixes
fixed
resolve
resolves
resolved
alongside an issue reference. Further detail on the format of these is available at the aforementioned link.
On receipt of a PR without a closure Derek should apply a label, such as no-issue
, unlinked
, no-closure
and post a comment to provide the user with the information required to help them update the description.
Not a bug
Enabling the ability for Derek to apply more of a project's contributing guide through contributor education & prompting.
N/A
So now that we have milestone support it would be good to have Derek help compile the description for releases by reacting to the event fired and then doing a quick search and populating the text i.e.
Changelog:
- PR x by username
This would be an optional feature.
If we implement Roles 2.0 #22 then we could add "Derek rebase" and set up permissions for who can run this command and what the validation is.
i.e.
I have a PoC that works, but it has no policy beyond being in the curators
section.
Following the README should be sufficient to deploy and run your own Derek instance.
The generated JWT tokens are invalid.
Replace the hardcoded App ID in jwt_auth.go
with one sourced from the env.
Unable to get own Derek instance running, positive side tho is that I now know a lot more about Githubs App auth ๐
Docker version docker version
(e.g. Docker 17.0.05 ):
17.06.1-ce
Are you using Docker Swarm or Kubernetes (FaaS-netes)?
Swarm
Operating System and version (e.g. Linux, Windows, MacOS):
Ubuntu 17.04 on DigitalOcean
Link to your project or a code example to reproduce issue:
n/a
We need to be able to specify a "contributing_url" field for when Derek is used across an organisation or many repos. The "redirect" flag now means many Derek files are just redirecting to another repo for rules - at least in OpenFaaS we don't need separate contributing guides.
I create a Markdown page with a link to the openfaas/faas contributing guide but sometimes forget and that's not a great experience because new developers get linked to a 404.
Add field contributing_url
and use that when given, otherwise use the existing behaviour.
I noticed that a contributor had Derek installed on one of his repos. He had a PR arrive without sign-off and Derek added the no-dco label.
The developer removed the label then merged the PR.
I would suggest that we prevent Derek from responding to "Derek remove label: no-dco" for the above reason.
Those with write-access can still remove it via the UI controls.
I would like to move Derek to the OpenFaaS golang-http-template so that the process can stay alive for longer without re-forking. This would allow access tokens etc to be cached in memory to reduce API requests and will increase memory under load.
It is also a step towards #62 (break out SDK) so that Derek can be run on other platforms or by other processes other than the OpenFaaS watchdog to enable wider use.
Each request runs in a new process meaning that panic
and os.Exit()
are fair-game to appear anywhere in the code. This prevents the above, so part of the work is refactoring any exiting and error handling.
Template:
golang-http via: https://github.com/openfaas-incubator/golang-http-template
The initial work will be moving to the new template signature, this should be minimal, then from there testing that we never call panic/os.Exit.
The current Dockerfile is unnecessary at this point, since it could be replaced by the original Golang template. https://github.com/alexellis/derek/blob/master/Dockerfile
Needs more thought - but I can't always remember what milestone number to pick when doing this on mobile for instance.
Derek set milestone: 0.5.4
vs:
Derek set milestone: next
Which could be: 0.5.5
Derek add milestone
-> Might just add the latest one available.
There are other tools that can generate change logs but this seems like a simple feature to explore more.
Derek already supports manipulation of Labels. The GitHub API for Milestones is very similar. Milestones are used to group issues that belong together as a feature or a deliverable. See here for the distinctions: https://guides.github.com/features/issues/
Derek set milestone: 4.8
should add the named milestone to the current issue
Derek remove milestone: 4.8
should accordingly remove the milestone
Milestones aren't supported at all
Add code analogous to the Label
handling.
Subsurface is starting to use Milestones to group issues that we want to address for upcoming releases and it would be very useful if more people were able to set/remove milestones from issues, without being Collaborators on the project
or
Add a new feature so that when a contributor opens a PR and refences an issue, or multiple issues a label is automatically applied to the issue(s).
This could help contributors filter out at the top level those issues which are currently in progress / being addressed.
Course of action to reproduce:
Derek add label: fred
<-- label added
Derek remove label: fred
<-- label removed
Delete comment 1 <-- label added again
Potential fix would be to inspect the POSTed JSON to see if "action": "deleted"
is set and cause Derek to refuse to act within AddLabel
and RemoveLabel
in this instance. Within the code this would be req.Action
Add an autolabels
section to DEREK.yml
so that any issues raised containing the words within this section are automatically labelled:
e.g.
autolabels:
- proposal
- question
- enhancement
Also add - autolabels
to the features
section to enable/disable
Right now secrets need to be built into a local/private image or bind-mounted. Derek should read secrets from the OpenFaaS /run/secrets/
folder which happens to be the same on Kubernetes and Swarm alike.
Secrets:
Issue identified by @MrTinD
A secret for derek-secret-key (for HMAC) should be read from a file when needed i.e. docker secret create derek-secret-key ./derek-secret-key
When creating the HMAC secret via docker secret create derek-secret-key ./derek-secret-key
we get an extra new-line in the data we read and that makes the validation of HMAC fail because in the GitHub UI we can only enter a single line of text.
File:
my-secret\n
GitHub:
my-secret
Work-around - use echo -n | docker secret create derek-secret-key -
to suppress the new-line terminator in the file.
Change code to always strip down any new-line characters when reading the HMAC secret.
Caused frustration for user submitting patch, we need to make this better.
or
docker version
(e.g. Docker 17.0.05 ): Version: 18.05.0-ce
We're using Docker Swarm but this may apply to Kubernetes too.
OK, this is possibly my own fault for missing the forrest for the trees, but I can't seem to find documentation of the config file syntax and especially which features are actually supported. There are a couple of examples, but... is there documentation somewhere that I just didn't see?
An easy to find section in the documentation that tells me more about the syntax of the config file and the supported features.
I looked on GitHub, I git-grep-ed through the sources, I couldn't find it - and am very worried that @alexellis will respond to this issue saying: look, there...
A new user of the bot might benefit from easy to find documentation...
or
Upon submission of a PR Derek should check the originating branch name, and if he finds master
the PR closed with a message advising the user that they should re-submit from a non-master branch.
Also, check that pull request is against master / default.
In a conversation with @alexellis we saw an issue in Derek using the latest master version of openfaas.
failed to report status github.RepoStatus{State:"success", TargetURL:"http://<domain>:8080/function/alexellis-hallo", Description:"function successfully deployed as: alexellis-hallo", Context:"DEPLOY"}, error: unable to read private key path: /run/secrets/derek-private-key, error: open /run/secrets/derek-private-key: no such file or directory
This is due to openfaas/faas#692
Update the path to search /var/openfaas/secrets
Scenario: impatient contributor deletes whole PR template message and decides it doesn't apply to them.
Result: PR is decorated with a comment telling user to fill out PR template.
Opt in via feature in .DEREK.yml file pr_template_checking
#
in repo/.github/PULL_REQUEST_TEMPLATE.md
must be present in body of PR@rgee0 @dirkhh @johnharris85 thoughts?
Would it be useful to provide Derek as an SDK?
The GitHub auth that we've written has already been useful for the GitHub App in OpenFaaS Cloud and could be useful as a separate package.
Due to forking of Derek over one user not wanting to follow DCO process, would it be advantageous for us to release Derek as an SDK so the functionality can be invoked via other processes such as a regular HTTP server or as suggested in another issue - an AWS Lambda function? #13
If Derek was packaged as an SDK a few things would need to change:
log.Fatal*
lines may need to be removedAny chance derek could delete the command comments in the issue/PR after the command is executed?
It would be great to be able to send multiple values in the same action, i.e:
Derek add label: UX, enhancement, proposal
(,
as separator, more natural)
or
Derek add label: UX || enhancement || proposal
(||
as separator)
I can work on this if the proposal is accepted
Awesome project! Was thinking about taking a crack at #17 and wandered how you wanted to handle more features being added in the code? Checks like this one assume only one feature for handling PRs, and this might need to be refactored if more are being added?
Also the code in handlePullRequest may need some refactoring to run through different features if they're enabled / disabled etc...
#17 is just a single addition so can fit into the current model, but just wanted to check on direction before I do anything, or if you think refactoring for potentially more scope later on is the way to go first?
Happy to have a go at either, thanks!
Add Derek set title: This is the new title
functionality so that maintainers can painlessly manage imprecise or ambiguous issue titles.
In order to monitor and observe usage of Derek we should offer the ability to send messages to a Slack channel.
All usage of Derek via comments / PRs is public so I do not see any concerns with this going to one place.
It will give an overview at a glance whether there is any recent activity, by which installations and which users.
Example of status messages sent to Slack:
This would use the incoming web hooks feature rather than any specialist library and could be swapped for any other HTTP endpoint. Slack messages have a basic format of:
{
"text": "Message to channel goes here"
}
GitHub supports a .github
directory to reduce the amount of clutter in a projects root directory for issue and contribution docs. It would be nice if DEREK.yml could also be placed there.
A potential downside is that GitHub owns the .github
namespace. Alternately a directory named .automation
would be nice for this type of configuration data, but pointless if only Derek uses it.
Threads in PRs and Issues should be lockable by issuing Derek lock
and unlock-able by issuing Derek unlock
I think Derek can help us with commit messages - as a sort of "linter"
Some may argue a pre-commit hook could do this, but it would have to live in every git repo - Derek can automate some of this.
This post by Chris Beans seems to be well-regarded in the larger community and is also referenced vey the Moby project: https://chris.beams.io/posts/git-commit/
This would be a "feature" and it would be up to the project maintainers as to what they do with the information.
If the rules are invalidated a label could be added with a comment. When the rules are valid the label will be removed. You could get the label more than once but not whilst the "label" is still in place.
Label could be: review/commit-msg
We looked into this before in a previous issue, but I think we could leverage GitHub Checks or commit statuses to give feedback on the DCO and other things Derek checks.
We post a label and a comment when one of the commits in a PR isn't signed off, but this still allows merging through the UI and some newer contributors don't understand that every commit needs to be signed-off-by - not just the last one.
Prototype working with a commit status / check where one of several commits in a PR is detected as invalid to see if this can fail the whole check. The feedback given before by @stealthybox said only the final commit in a series determines the pass/fail of a PR (but we should check again)
This would be change to go from alexellis/derek to openfaas/derek.
Any time we cut a new release of Derek we should be publishing a new Docker image to the Hub.
See openfaas/faas or openfaas/faas-netes for an example in .travis.yml and any other Makefiles.
Derek should allow Maintainers to close and reopen issues and PRs.
Derek close: mandatory reason text
Derek reopen: mandatory reason text
We can now remove the hmac.go code and vendor it:
https://github.com/alexellis/hmac
Good beginner issue
Alex: "Derek apply label skill/beginner"
Derek: Sorry you're not in the MAINTAINERS file.
Derek: Sure
Derek added label "skill/beginner" to the issue
The .DEREK.yml file should take over the place of the MAINTAINERS file.
This will support roles / actions and mapping users to roles, but initially will look like this:
maintainers:
- alexellis
- rgee0
features:
- dco_check
- comments
See 2.0 extensions - #22
A setting should be made available to .DEREK.yml which means Derek will redirect and pull his config from another location. We should probably enforce some basic validations here and I'm open to suggestions.
We maintain separate config files per repo within the organization of openfaas / openfaas-incubator
Add a tag to the YAML which overrides the behavior to load the current file.
OpenFaaS contributors have required this
or
This is a suggestion to help with first impressions and helping new contributors to a project/repo.
To help users identify and give help to new contributors to a repository Derek could add a label such as derek/first-time
or derek/new-contributor
This metadata may be represented in the metadata already of a GitHub push/PR - https://developer.github.com/v3/activity/events/types/#pushevent
Alex
It would be useful to be able to enable Derek to automatically announce releases to an organisation's twitter account, for instance.
Can we make development and testing easier by automating the creation/config of the Github App via the GitHub API?
Ideal flow:
derek --provision-app --name="Derek PR 68"
When authenticated we have 50 requests to the API / hour. When we are authenticated we have 5k per hour +/-.
It's unlikely that we will burn up 5k in an hour on the OpenFaaS org, but larger projects may hit this and this should be observable.
What do we do?
How do we track this?
Version 2.0 of .DEREK.yml would be:
roles:
- role: curator
actions:
- labels
- close_issue
- open_issue
- role: maintainer
actions:
- close_issue
- open_issue
users:
- name: alexellis
roles:
- maintainer
- name: rgee0
roles:
- maintainer
- curator
features:
- dco_check
- comments
The config redirect which means a local repo can have just a single entry "redirect_url: https://" rather than a complete .DEREK.yml file has been really useful across the dozen or so OpenFaaS repos. There are times where a local override is needed for a specific repo.
I would see this working as an intersection with the local override taking precedence. This would also be useful for the rebase feature in #91.
There may be a generic config set up in the "main" repo - all other repos may point there defining "@alexellis has rebase access" - then the CLI repo for instance may have @rgee0 and @johnmccabe as having rebase access, so you'd end up with the following for the CLI repo: @alexellis @rgee0 and @johnmccabe without having to specify that @alexellis had access to this feature.
Possible, but with duplication and maintenance is required for separate files.
If redirect_url is present, fetch that config file then overlay it with the file being read afterwards.
At the moment Derek cannot assign reviewers to a pull request.
Support comments like:
Derek assign reviewer: username
Good option is to make it work with a list of users:
Derek assign reviewer: user1, user2
Derek should be able to assign and unassign issues and PRs, with the target me
as a shortcut for the maintainer who issued the request.
Derek assign: me
Derek assign: alexellis
Derek unassign: me
Derek unassign: alexellis
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.