checkupjs / checkup Goto Github PK
View Code? Open in Web Editor NEWA health checkup for your project.
Home Page: https://checkupjs.github.io/
License: MIT License
A health checkup for your project.
Home Page: https://checkupjs.github.io/
License: MIT License
walksync (what we use to get paths) uses minimatch under the hood, which by default will not match dotfiles. there is an option dot
, which is false by default. Spoke to @rwjblue, and will surface dot
as a top level option to walksync.
Discussed potentially surfacing all minimatch options as top level to walksync instead, waiting on response from @rwjblue about what he prefers in terms of next steps.
Once that option is surfaced to walksync top level (one way or another), set as true to calls to walksync in get-paths
, so dotfiles get excluded appropriately.
Example of the problem :
Right now node_modules/.bin
is being included even though we are passing in **/node_modules/**
as a path to be excluded
Need to collect simple project repo info
meta
category (once #174 is merged)src/tasks/index.ts
(it's unnecessary)register-task.ts
import to single line import from direct task file (no longer importing from index)Summary
At the bottom of the checkup report in console (also be pulled into a separate section for JSON results) we want to show a list of Action Items
so a dev running checkup can know what requires immediate attention. Tasks can be highlighted as urgently actionable, phrased in a way that makes it clear to a dev what they are meant to do to fix. Once an Action Item
is complete, it will no longer show up in the Action Items
list (unlike tasks, which are purely informational, not prescriptive, and will show up with zero results). For example, if you have zero skipped tests, that result would show up. But if there was an Action Item
configured to reduce number of skipped tests, and you had zero, it wouldn't show up as an Action Item
for you - since you have already done it.
We want to make it possible for both task authors and those configuring checkup to add items to the Action Items
list.
Examples
Examples of tasks that would benefit from having Action Items
configured by authors:
^^^^ The theme emerging here is maybe "Project Prerequisites" or things related to the setup/toolchain of your app.
Examples of why those configuring checkup may want to set up custom Action Items
:
lint-disables
, we may want to configure a threshold such that prioritizing removing lint-disables
would come up as an Action Item
- so if your app has more than x lint-disables
, it would show up as an Action Item
Action Item
, such that if you have fewer than x percent of your tests as unit tests, show an Action Item
to prioritize refactoring your test suite^^^^^ theme emerging here is problematic patterns (potentially blocking other work) and overall health of an area of the app.
Requirements
Action Items
are conditional (by default in a boolean manner - i.e number of problems > 0, but can be configured more explicitly as defined by the task)Action Items
can be turned on by default in a task by the task authorAction Items
by default can be overridden by configAction Items
by default can still have an ActionItemMessage
, in the event that they do become Action Items
. That message is also overridable by configAction Items
by default can become Action Items
by turning them on as such in the config.Action Items
(maybe 5?). If more than that are turned on by default given the plugins a user has enabled, 5 are chosen at random (we should also supply a warning message saying that we dropped Action Items
because there are more than 5 enabled by default). Config will validate that no more than 5 can be passed in.Changes to be made
export interface TaskMetaData {
taskName: TaskName;
friendlyTaskName: TaskName;
taskClassification: TaskClassification;
}
will become
export interface TaskMetaData {
taskName: TaskName;
friendlyTaskName: TaskName;
taskClassification: TaskClassification;
actionItemData?: ActionItemData;
}
export interface ActionItemData {
isActionItem: boolean;
actionItemMessage: string;
actionItemConditions?: ActionItemCondition[];
}
export interface ActionItemCondition {
key: string;
condition: Function;
value?: number;
}
Task authors can choose to expose any number of actionItemConditions
to be configured to turn on/off.
Examples:
For the eslint-disable
task, an ActionItemCondition
could be:
let condition: ActionItemCondition = {
key: 'numEslintDisables';
condition: (numEslintDisables, value = 0) => numEslintDisables > value;
value: 20;
}
^^ so if there are more than 20 'numEslintDisables', it would show up as an action item.
The config to make eslint-disables
an Action Item
with its default message would look like this:
{
actionItems: {
'eslint-disables': true,
}
}
The config to make eslint-disables
an Action Item
with a custom error message would look like this:
{
actionItems: {
'eslint-disables': {
message: 'Stop disabling eslint!!!!!!!!!'
}
}
}
^^^ if you supply a custom message, it is turned on as an Action Item
by default. If this is confusing, we can also add an enabled
property, but i dont see why someone would supply a custom message without turning it on.......
The config to conditionally make eslint-disables
an Action Item
with a custom error message would look like this:
{
actionItems: {
'eslint-disables': {
message: 'Stop disabling eslint!!!!!!!!!',
condition: 'numEslintDisables',
value: 20
}
}
}
This change would require verifying config upfront to ensure that the conditions being used in the config are in fact exposed by a given task.
I am open to the idea that adding this level of flexibility may be more complexity that it is worth, and would only be used by power users. I think it is quite powerful and can see many uses for it in my app, but I can ack that it is complicated. We can potentially reduce the scope here by just making action items toggleable on/off in a boolean manner, and maybe adding the custom message as well, without the conditional piece (basically removing ActionItemCondition
from ActionItemData
).
Looking forward to feedback :)
The package.json should generate the latest version of the @checkup packages.
Can use yarn info <package> version --json
for this
Write a task that returns the number mixins remaining to migrate, with links to documentation on the "dangers of mixins". For each mixin, we should also return the number of time the mixin is being used (to help approximate scope/effort to remove)
Adding a new checkup plugin to configuration can be handled via generators.
$ checkup config:add plugin checkup-plugin-ember
This would do the following:
checkup-plugin-ember
entry to the plugins
array in the configcheckup-plugin-ember
package to the package.json's devDependenciesI feel like we should start thinking about naming tasks more what they do, like what eslint does. In that sense, we should think about naming this FindTodosInCommentsTask and the file would be find-todos-in-comments-task.ts. This makes it much easier to understand what the task does just by reading the filename. We should also go through each current task and make sure its name makes sense.
Rename all tasks to be verbs that describe what the tasks are doing.
It's hard to understand what Eslint Overrides" means. Does it mean config overrides, rule overrides? I would honestly not have attributed this to
eslint-disable` annotations. I think we should rename this to make it more clear.
Use case: if developers want to run checkup as part of a script, they shouldn't need to shell out to do so. There should be a way to programmatically invoke checkup directly to make this process cleaner
We should have a consistent pattern for handling errors across the CLI. Errors that occur should follow the following rules:
Write a task that returns the number of lint overrides (broken down by type of override - eslint, stylelint, templatelint, etc)
Write a task that returns the types of tests in the application, with some information about module length (average module length, number of modules over x lines long, etc)
I am trying to add a local plugin for ember-template lint that looks something like the following;
const TEMPLATE_LINT_CONFIG: TemplateLintConfig = {
plugins: [
{
name: 'local-rules',
rules: {
'hello-world': require(path.resolve(__dirname, '../../template-lint-rules/hello-world')),
},
},
],
rules: {
'hello-world': 'error'
},
};
Looking at the types for ember-template-lint config it looks like it is missing some fields.
checkup/packages/core/src/types/ember-template-lint.ts
Lines 29 to 33 in 9fe60a9
ember-template-lint doesn't expose types you can see the top level properties that are allowed here:
I am using @checkup/[email protected]
The current stdout infra via the ui module is fairly disjointed and doesn't lead to consistency for task authors. We should add some standardized ui ouput functions that provide consistency when outputting a task.
The following improvements/changes need to be made to file searching primitives:
file-searcher-task
file-searcher.ts
to just export functions, and remove base classNeed a meta task that outputs
NaN
being returned for migration tasks when there are no files found in association to ESLint migration tasks@ember/component
.Write a task that defines what are "Ember core deps" and returns the version of each. Potential enhancement - return latest of each of these deps so people can see the difference between where they are and where they should be
Currently, checkup can only be run on an entire application, beginning at the root which is passed in via (the only) command line args. We need to know the root for a number of reasons, one of which being it is where checkups goes to look for the checkup config.
Checkup needs to be able to operate on a subset of the files in a directory, primary use case being large applications with shared ownership. In order to provide detailed and targeted insights to the right people, we need to account for shared ownership of an application as a first class citizen, and provide separate checkup reports for each of the separate ownership "groups".
To accomplish this, the following changes need to be made:
--cwd
flag, defaulting to .
.Implementation details of task related changes:
Right now there are 4 types of tasks we support - the following changes will need to be made to each type to allow for this functionality.
file-searcher.ts
will need to be modified to account for this functionality.Extensions:
Should include:
any reason this chunk isn't using the no-object-extend
rule defined here?
https://github.com/scalvert/eslint-plugin-ember-es6-class
this is what our team used to track our native class migration
Oclif plugins are identifiable via the oclif-plugin
keyword added to the plugin's package.json. We should add a similar designation for checkup plugins.
checkup-plugin
When loading plugins, we should validate that the keyword is present. We should also update the plugin generator to include this keyword.
plugin-ember-octane
We should consider exposing security issues via https://www.npmjs.com/package/@npmcli/arborist and https://www.npmjs.com/package/npm-audit-report#basic-usage-example
None of these are my ideas, they all come directly from the book Software Design X Rays: Fix Technical Debt with Behavioral Code Analysis by Adam Tornhill :)
But it would be super cool to have analysis of what are the “hot” files, or where do we have lots of different people touching the same files which can be broken down to more granular modules, where do we have related changes always happening in tandem indicating tight coupling, among other things.
Write a task that returns the number of skipped tests
Should include:
Write a task that returns the number of functions without JSDocs. This can be extended to also check the number of Components without JSDocs defining the interface, etc
When running checkup generate plugin foo ./lib
I get the following error:
› Error: No valid generator found for plugin. Valid generators are base-generator.d,
› base-generator.d.ts.map, base-generator.js, base-generator.js.map, config.d, config.d.ts.map,
› config.js, config.js.map, plugin.d, plugin.d.ts.map, plugin.js, plugin.js.map, task.d,
› task.d.ts.map, task.js, task.js.map
When I log out what this.validGenerators
I get:
[
'base-generator.d',
'base-generator.d.ts.map',
'base-generator.js',
'base-generator.js.map',
'config.d',
'config.d.ts.map',
'config.js',
'config.js.map',
'plugin.d',
'plugin.d.ts.map',
'plugin.js',
'plugin.js.map',
'task.d',
'task.d.ts.map',
'task.js',
'task.js.map'
]
Write a task that returns the number of stateless services (aka the number of services that shouldn't be services)
Write a task that runs all of the linting defined in the project being "checked up on" and returns a summary of the current violations.
There should be multiple breakdowns of data here - number of violations by filetype (eslint v templatelint v stylelint), breakdown of violations by rule, etc.
Need to determine number of TODOs (TODO, todo) in comments
Need a total of ember addons/engines
Write a task that breaks down the different types of constructs in an Ember app
I'd like to propose a change to the aggregated result structure - the structure we ultimately end up with when we merge task results.
First, I propose we change the categories to the following:
When we process the task results, we'd 'lift' the meta tasks out to a top level meta
bucket, leaving the remaining tasks that were categorized as either Insights
or Migrations
in a sibling results
bucket. This would allow us greater control over the meta
output, as it pertains to reporting.
Here's the proposed format:
{
"meta": {
"project": {
"name": "my-project",
"version": "0.0.1",
"repository": {
"activeDays": "0 days",
"age": "0 days",
"totalCommits": 0,
"totalFiles": 0
}
},
"checkup": {
"configHash": "9b74c9897bac770ffc029102a200c5de",
"version": "0.0.1"
}
},
"results": [
{
"meta": {
"taskName": "mock-task",
"friendlyTaskName": "Mock Task",
"taskClassification": {
"category": "insights",
"priority": "medium"
}
},
"result": {
"mockTask1": 5
}
},
{
"meta": {
"taskName": "mock-task2",
"friendlyTaskName": "Mock Task 2",
"taskClassification": {
"category": "insights",
"priority": "high"
}
},
"result": {
"mockTask2": 10
}
},
{
"meta": {
"taskName": "mock-task3",
"friendlyTaskName": "Mock Task 3",
"taskClassification": {
"category": "insights",
"priority": "low"
}
},
"result": {
"mockTask2": 10
}
},
{
"meta": {
"taskName": "mock-task4",
"friendlyTaskName": "Mock Task 4",
"taskClassification": {
"category": "migrations",
"priority": "low"
}
},
"result": {
"mockTask2": 10
}
}
]
}
We need to figure out a generic way to output the meta data in a nice way for reporting, but it shouldn't be too complicated.
Please comment with your thoughts.
checkup generate plugin foobar
, it will create checkup-plugin-foobar
as your plugin namecheckup generate plugin checkup-plugin-foobar
, it will generate as ischeckup-plugin-*
, it will generate the files inside that directory instead of creating a new top level dirFollowing the setup process defined at https://github.com/checkupjs/checkup/blob/master/packages/cli/README.md it notes to generate a config file via running
$ checkup generate config
However that produces an error
So I did this instead:
$ checkup generate config.js
Following the prompts, I selected Ember
and JSON
yet the resulting file is .checkuprc
which does not match any of the options that I chose when generating the file.
I would like to see the docs be updated to match the correct syntax for file generation (or the file generation to be fixed to match the docs) and then the resulting file to match the options passed .checkuprc.json
in this case or .checkuprc.js
if I chose JS or .checkuprc.toml
if I choose TOML
Write a task that breaks down dependency freshness for all deps in the repo, and organizes them by most -> least out of date
Similar to how eslint provides the ability to provide options to rules, we need to be able to provide options to tasks.
Based on the current config format, we should allow options to be passed to Tasks via the task
object.
type CheckupConfig = {
plugins: string[];
tasks: {
[fullyQualifiedTaskName: string]: "on" | "off" | ["on" | "off", unknown];
};
}
The key format is
ember/ember-test-task
ember
- short plugin name (the plugin name with the checkup-plugin
segment removed)
/
- delimiter between plugin name and task name
ember-test-task
- task name
The value format is
boolean
- indicates whether a task is enabled or disabled. When enabled, this key/value will not be present. When disabled, will be false
.
object
- when an object is provided, this constitutes the options passed to the task.
The checkup config will be passed to each task, which should have a getter in the base task to read the specific tasks' config values from the checkup config.
We may want to consider allowing tasks to be skipped up front - TBD
Write a task that returns the project info
The current plan with the CLI included the ability to output to an HTML generated report. While the use of an HTML report is still a viable path, I propose we disconnect it from the CLI itself.
This will do the following:
type
in the result object will allow you to hint to the system how you want your result output. This is effectively the path to marrying the structure we were moving towards with result data, and the looseness with which we currently allow outputting to the console/json. The end result should be simpler for consumers in the frequent case (most people just want to gather some data and have it output in a consistent fashion), but allow for extension for custom/complex cases (you want to create a task result yourself, and specify the output).I propose we
The current config setup is quite complicated. We can simplify the overall loading and generation of configs by using some simple functions:
Should include:
Add checkup-plugin
keyword to package.json
REmove src/types/index.ts
Update @checkup/core
and @checkup/test-helpers
to ^0.0.1
Update tsconfig.json to not inherit
{
"compilerOptions": {
"declaration": true,
"importHelpers": true,
"module": "commonjs",
"strict": true,
"target": "es2017",
"allowSyntheticDefaultImports": true,
"sourceMap": true,
"outDir": "lib",
"rootDir": "src",
"types": ["jest", "node"]
},
"include": ["src/**/*"]
}
Update package.json with @types/jest and @types/node
Currently @checkup/core is missing the direct dependency on a few libs.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.