checkmarx-ltd / cx-flow Goto Github PK
View Code? Open in Web Editor NEWCheckmarx Scan and Result Orchestration
License: Apache License 2.0
Checkmarx Scan and Result Orchestration
License: Apache License 2.0
JIRA severity based due dates
Investigate IAST Integration Flows.
Introduce a UI for managing the configuration of CxFlow in a user-friendly way
Issue link from the previous repo: https://github.com/checkmarx-ts/cx-flow/issues/39
The logic used to determine the link to the exact line of source in within the repository will break for BitBucket when the branch contains a forward slash (i.e. feature/branch).
Currently ADO Workitems only support issue description.
Enhance functionality to allow for custom fields in ADO WorkItems - introduce the same functionality as JIRA.
Create pipeline of automated integration testing for validating various 3rd party APIs are working as desired
When running Instance Batch mode, the file name substitution logic does not allow for Checkmarx team to be used in filename with the [TEAM] format.
Note: CxFlow supports 3 batch modes through CLI
Project based: The results for a single project is retrieved and published as per the defined feedback channel (file based, ticket/JIRA based).
Team based: The latest results for each project for a given team are returned and published as per the defined feedback channel)
Instance based: The latest results for each project for the entire instance of Checkmarx is returned and published as per the defined feedback channel.
For Team and Instance using a file based feedback (i.e. Json/CxXml/Csv) there is the ability to substitute parts of the filename per project ([REPO], [PROJECT]). The team ([TEAM] block is only substituted when Team batch mode is used, the team is not available to reference (at the moment) to substitute in the Instance batch mode).
Example [TEAM] substitution for Json/Xml/Csv outputs. Team is not currently available when running against the entire instance of checkmarx.
`json:
file-name-format: "[NAMESPACE]-[REPO]-[BRANCH]-[TIME].json"
data-folder: "D:\tmp"
cx-xml:
file-name-format: "[NAMESPACE]-[REPO]-[BRANCH]-[TIME].xml"
data-folder: "D:\tmp"
csv:
file-name-format: "[TEAM]-[PROJECT]-[TIME].csv"
data-folder: "D:\tmp"`
Introduce the capability of externalizing credential injection. Some examples are:
Issue link from previous repo : https://github.com/checkmarx-ts/cx-flow/issues/40
Currently CxFlow publishes/updates findings in JIRA. The ticket that is generated however does not get reflected in Checkmarx.
CxFlow should associate the ticket with the finding(s) on the Checkmarx side to allow for reconciliation and for a reference point within the Checkmarx console.
Allow for external Script (Groovy) for JIRA custom fields
Introduce Configuration as Code capability to override CxFlow settings
Issue link from previous repo : https://github.com/checkmarx-ts/cx-flow/issues/36
Checkmarx version: 8.9.0 HF13 (SDLC)
CX-flow version: 1.5.4
When having a project already created in Checkmarx SAST, and CXflow is setup to generate scans on Pull Requests, the Checkmarx preset specified in the Pull Request Webhook parameter is never used. Instead, the preset defined in the application.yml is always used.
Use case:
Say that we have a CXFlow instance listening to Github PRs that triggers scans. The "scan-preset" property in application.yml not specified (intentionally), so that one can override the preset in the WebHook Parameter (as specified in GitHubController.Java in method:
public ResponseEntity pullRequest
Observations:
the "override" is never used (greyed out by the IDE)
No matter what preset I specify in the WebHook parameter, the Checkmarx Default is always used
While debugging through the code, I noticed that in file FlowService.java, method
private Integer determinePresetAndProjectId(ScanRequest request, String ownerId, String projectName ) {
it seems that the check is wrongly done (unless I understood #the code logic wrong), and the method fetches and sets the project's preset if the request has the scanPresetOverride as false.
image
By removing the negation before the "request.isScanPresetOverride()", it works as expected and the scan will use the project's defined Preset (that was set from within the Web Interface).
I might have not understood the code logic, or maybe this is just a bug and the negation needs to be removed.
Add ServiceNow Support for Defect Tracking.
Allow for external script (Groovy) for Issue Descriptions
Where do we envision the configuration for this Groovy script to be?
I suggest under cx-flow or/and per bug tracker implementation to allow different issue descriptions per service (the second would take priority). This would be an optional property.
The per implementation is more for cases where organisations have multiple different services which require possibly their own description format for each service and don't need two separate CxFlow instances to achieve this. This could also be achieved if we pass in the bug tracking system into the Groovy data binding as run-time.
Proposed Example:
cx-flow:
// ...
// The name of the property is subject to change
script-issueDescription: "IssueDescription.groovy"
Add ability to link issues in JIRA
Introduce Checkmarx Project Branching capability within CxFlow
Currently, a new project is created for every repo/branch combination unless project naming overrides are leveraged. This Enhancement is to leverage a base project and branch it within Checkmarx instead of creating new each time.
Issue link from previous repo: https://github.com/checkmarx-ts/cx-flow/issues/31
Allow for the ability to configure defect tracking against policy violations in M&O
Investigate and implement additional capabilities within GitLab Pull Request feedback
https://wiki.jenkins.io/display/JENKINS/Violation+Comments+to+Bitbucket+Server+Plugin
Kerberos authentication support would be nice for organisations that don't allow username and passwords to be used to authenticate to JIRA.
Introduce support for direct links to CB lessons based on findings.
Currentl CxFlow allows for a global training link that is referenced within the body of a ticket/issue, which can be the high level URL of Codebashing if desired. CxFlow should allow for deeper integration with Codebashing to introduce a link to the exact lesson that would aid in developers fixing/learning about the issue.
Enhance CxFlow to integrate with Jira in a 2-way manner. i.e. Change events within Jira are then reflected in Jira.
This would need to be built from WebHook integration in Jira that reflect changes to the issues in Checkmarx.
A listener would need to be developed for the Issue Updated webhook within JIRA:
https://developer.atlassian.com/server/jira/platform/webhooks/
Things to consider:
What can be updated? State / transitions, comments, labels?
How much can/should be configurable/customizable?
Leverage the GitHub Wiki to document CxFlow with public-facing details.
Add the ability to leverage templating for Defect / Issue description
Investigate the use of Velocity template language:
https://velocity.apache.org/
When the scan completes and 0 vulnerabilites of any severity are found, IF the breakbuild flag is set to true in the config file, we get an error message (Code 10).
If breakbuild=false, then we get past that error message. Please fix so when breakbuild=true AND there are 0 results, the process completes successfully.
Publish compiled versions of CxFlow via standard locations (such as Maven Central).
i.e. OSS Repository Hosting provided by Sonatype:
https://central.sonatype.org/pages/ossrh-guide.html
Reflect scan details in Pull/Merge feedback within already existing comments if a scan has triggered a second time. This is applicable to all repository types.
Sample implementation:
https://github.com/danger/danger/blob/56286aab4d3b9644eb6da5e81d0ab5b32575e9d4/lib/danger/comment_generators/github_inline.md.erb#L3
In addition, I think there should be some options for configuring how additional commits added to an existing PR are handled. Right now, if a PR is created, every time additional commit(s) are pushed to the PR HEAD, a new "scan submitted" comment is added and, more importantly, a new scan is queued. The latter means that the scan queue can very quickly become completely consumed by scan requests for a single PR of a single project, especially if the project/PR consists of a large code base and/or requires a full scan. In our current production environment, a single project has one PR that has resulted in 30 additional scans in the queue; this is made worse in our specific case because this particular project has > 1M lines of code and can only be handled by a small subset of our scan engines.
I'd propose handling this by updating CxFlow's PR processing behavior to do one or more of the following:
If a scan request exceeds the configured timeout in CxFlow, optionally cancel the actual scan itself.
Allow setting a configurable per-project webhook queue limit. Behavior might be that the queue is handled such that existing scan closest to completion/greatest progress % is left in queue, and then remaining queue size is allocated with most recent scan request prioritized. In other words, attempt to minimize time-to-results (by ensuring first/closest to completion scan is not canceled) while also ensuring most recently committed changes are added to the scan queue and older scans that have been queued but not started are canceled to make room for newest commits.
Add Dashboard Integration Support for Result Ingestion
Investigate project cleanup strategy. Possibilities include:
Allow for external Script (Groovy) for JIRA custom fields
Investigate and implement additional capabilities within ADO Pull Request feedback
https://wiki.jenkins.io/display/JENKINS/Violation+Comments+to+Bitbucket+Server+Plugin
Add the ability to block merge based on scan results filter for:
Archer is a commonly used Risk Management Platform that we could push vulnerabilities into natively from CxFlow. I'm unfamiliar with the customization's that the platform has but if organisations have different requirements we should be able to support them.
Add capability to scan repository for code content and finger print the application to help determine a preset/scan ruleset to use automatically
Issue link from previous repo : https://github.com/checkmarx-ts/cx-flow/issues/37
Customers using CxFlow in CLI mode through GitLab CI integration would like to leverage the Security Dashboard capabilities.
Create bug tracker implementation forGitLabDashboard, which is a specific JSON output format.
https://docs.gitlab.com/ee/user/application_security/security_dashboard/
When ADO PR Payload is sent to the Push (/ado/push) endpoint, CxFlow has an NPE.
2020-02-19 13:27:31.565 ERROR 2276 --- [io-11111-exec-7] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.NullPointerException] with root cause
java.lang.NullPointerException: null
at com.checkmarx.flow.controller.ADOController.pushRequest(ADOController.java:322) ~[classes!/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_211]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_211]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_211]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_211]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:888) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) ~[tomcat-embed-websocket-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:108) ~[spring-boot-actuator-2.2.4.RELEASE.jar!/:2.2.4.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at com.checkmarx.flow.filter.SecurityFilter.doFilter(SecurityFilter.java:41) ~[classes!/:na]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1598) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_211]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_211]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_211]
Allow for automated project creation to include assigning a Policy from M&O
Investigate and implement additional capabilities within BitBucket Pull Request feedback
https://wiki.jenkins.io/display/JENKINS/Violation+Comments+to+Bitbucket+Server+Plugin
->We have integrated Cx-Flow scan with Gitlab CI/CD pipeline. Even after marking few HIGH vulnerabilities as Not exploitable in Cx Console, Gitlab CI/CD pipeline fails. We are using Cx-Flow version of 1.5.4 (latest version).
->We also added flag break-build=true (as suggested by Support Engineer) and tested it again. But its still failing.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.