Environment:

1. Insights Version - 4.1 / Platform Service binary created from latest master code.
2. OS - RHEL 7.5
3. All services and tools up and runnning
4. Registration of agents - Online
5. Bitbucket Agent configured via UI.

Issue:

After selecting the Artifactory option from the drop down in the "Agent Management" tab, we are able to see an "artifactory" folder get created in the unzip path mentioned in the server-config.json file.
However, as soon as the register option is invoked, the service fails to get registered.

We could also observe that the "artifactory" folder is not getting created in the "PlatformAgents" path, where a folder presently exists for the bitbucket agent configured via the UI.

it would be great if the build system is not maven, but gradle.

Hi,
While trying to configure sonar tool, Neo4j is found to get the sonar data as follows

And, in RabbitMQ, under the Queue CODEQUALITY_SONAR_DATA, we get the following

For rendering the graph in grafana for the same sonar data,we used the query from the neo4j as follows

But, we get some error as

It would be better if we are provided with a clear picture in rendering the graph and also with the queries that needs to be used, to get the data from neo4j.
Regards,
Swetha

I have updated the latest artifacts PlatformService.war, InSightsUI.zip & Agent 2.0 configurations.
Tomcat is not starting after the update. Attaching the logs catalina.out & log.out.
catalina.txt
log.txt

The download scripts for Insights were not available at https://github.com/CognizantOneDevOps/Insights/blob/master/Deployment/RHEL7/

The URL was returning a 404 ERROR. Could you please look into it? Thanks in advance! 👍

Problem statement:

1. All agents running and collecting data.
2. Neo4j, Rabbitmq and ES DB up and running.
3. Data collected is not getting posted from the Neo4j DB via the graphaware plugin to ES.

Log file for Neo4j log attached.
Error.log.txt

1. Lead Time to Change
2. Deployment Frequency
3. Change Failure Rate
4. Mean time to Restore
5. Squad Throughput
6. Availability
7. Canary Deployments
8. Code Coverage

Hi,
For running the Insights Inference-Spark Jobs, it is given that the jar "PlatformInsights-0.0.1-SNAPSHOT-jar-with-dependencies.jar" has to be run. But, the path in which the Spark Jar could be manually installed, "%WORKSPACE%\Insights\PlatformInsights" doesn't exist.Please help me to resolve this issue.
Regards,
Swetha.

@vishwajitmankar
The data from Insights for the Rally Agent is having issues at the moment. The data that is getting pushed from the agent is queuing up at Rabbitmq. However, the subsequent push to the DBs via the Engine is not happening. We are able to see that many messages are waiting for acknowledgement from the server side. Once we stop the agent from pushing data, we can see that over time the data is getting pushed to the DBs. One of the main issues we have been facing is that the size of the queued up messages goes up drastically and we miss a lot of data. Also, after a while there are no consumers found for Rally in Rabbitmq and the insights engine logs have a connection reset by peer error.

As you can see from the screenshot, the messages are getting queued up. This was not observed on any of the other agents. Please advice on how this can be tackled for Rally? Thanks in advance.

Environment details:

Insights v1.4
OS: Oracle Linux 7
I have used the scripts from below URL to setup insight
https://github.com/CognizantOneDevOps/Insights/tree/master/Deployment/RHEL7/reference-DocRoot-Scripts

I'm able to log into the UI dashboard with the PowerUser. But the admin tab is not enabled.
PowerUser was configured as Grafana Admin in Grafana.

Attaching the PlatformService logs
log.out.txt

Please help.

currently the code cannot be built with java-12 as the dependencies which dropped out in later java versions are not specified explicitely.

Followed the correlation steps. Updated in serverconfig.json & correlation.json but not seeing the relationship type in Neo4j.

I am trying to new installation on new server. It seems that there is a new version of Insights 3.8.1 released, however the installation scripts are still pointing to V1.0 . I checked the confluence as well, but there is no reference on how to get installable for the latest version. Please update the script locations always point to latest release version or atleast update documentation with correct links

Current implementation expects at least 10 records to upload.
Also if there is only one BU, user not able to expand BU structure.

Hi Team,

We are currently using Insights 4.2 and we are trying to come up with a backup or failover (if need be) instance for our datasources - Neo4j and Elastic Search.
We do see an option to backup data for Elastic Search & Neo4j from the documentation, but in our case we are looking for an instance that we can switch to in case our existing datasource is impacted.
Please let us know if you have any suggestion for the same.
Can we achieve this by just backing up the data directory (NEO4J_HOME/neo4j-community-3.3.0/data/) for Neo4j and the Index directory (/var/lib/elasticsearch/) for elastic search.
Please share your thoughts regarding the same.

After installation of insights, i try to register an agent from the UI. But the registration fails with a message "Agent register fail".

The version of insights installed is the latest one and the artifacts are taken from https://platform.cogdevops.com/insights_install/installationScripts/latest/RHEL/artifacts/ as per the installation scripts.

All the other components are up and running. There is seen to be an issue with the Platform Engine from the logs. PFA a copy of the logs.

Also, from RabbitMQ it can be seen that the agent queues have no consumers.

Please help rectify the issue. Thanks in advance. @vishwajitmankar

EngineLog.txt

The sslverify option is not showing up via the UI. Addition of the ssl verify option via the config.json file gets overridden every time the agent is opened via the edit option in the "Agent Management" section.

Subsequent agent start does not work with an SSL exception.

$ ping infra.cogdevops.com
PING infra.cogdevops.com (54.166.124.44) 56(84) bytes of data.
...

Referenced in Dockerfile:

https://infra.cogdevops.com:8443/repository/docroot/insights_install/installationScripts/latest/RHEL/python/get-pip.py

1. is this server down permanently?
2. shouldn't we get the latest pip scripts from their original source, rather than from cogdevops.com?

Is there anyway I can dynamically configure new Dashboards via REST API or any CLI options ?

To run PlatformInsights jar for Insights Inference, getting an error stating ERROR ErrorLogger: Job (iSightSpark.SparkJobExecutorModule threw an exception. org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: invalid resource given; expecting [index]/[type] - received ]

Please help by giving a resolution to this exception.

After configuring the Insights application setup using the default scripts, we were unable to view the "Admin" / "Agent Management" tab in the UI. Also, the user roles are not getting populated in the UI.

hi,

I had installed all the components required for insight application. But am not sure on how to collect the jenkins data and render graph for the same. I tried configuring jenkinsAgent as shown below and running JenkinsAgent() python script , gives me an error

PFB log_jenkinsAgent and config file

log_JenkinsAgent.log

config.txt

The script for downloading the platform agent zip file is pointing to a directory without the zip.

The path and the directory have been highlighted

Where would we have to set the credentials for elastic search in the Insights setup? How can the data push from Neo4j to elastic search using the graph aware plugin access the set credentials?

There are some deployment issues with Tomcat 8 on RHEL7.4. It works good with tomcat 7.
error - WAR not getting deployed because of hibernate validation issues.

Thanks,
-Mayank Gangwal

Many a time after entering correct credentials Insights gives login failure message. This might be because of cookie issue. Need further investigation.

Hi,
I'm trying to access the wiki page on Tools Configuration
https://onedevops.atlassian.net/wiki/spaces/OI/pages/220367/Tools+Configuration
Any alternate guide for tools configuration?

I have presently set the runSchedule field in the config.json for my agent to 1440 to make it run at an interval of 24 hours. However, the agent is not getting restarted at the same time on a daily basis. Is the time for restarting the agent measured from the time it starts the data collection or does runSchedule refer to the time after the collection is complete?

With latest check-in, because of check-in issue, package structure will show error after importing code in eclipse for PlatformService. Build will be successful though. Will be fixed in upcoming release.

Team,
I am trying to install Insights on RHEL7, and I referred to https://onedevops.atlassian.net/wiki/spaces/OI/pages/211638/Installation+of+InSights for the instruction. The recommendation is to use 2 machines, but there is no clarity below points:

1. What all packages should be installed on machine1 and machine 2
2. What setting to change in each server to make the integration possible

Thanks
Vaibhav

Hi,
I have tried to set up Insights with Oracle Linux 7 by following the below link
https://onedevops.atlassian.net/wiki/spaces/OI/pages/211638/Installation+of+InSights
I have completed the setup using insights_all option.
Agent Management Tab is missing in UI.
Am I using the right jar/binaries file? Please help.

hi Team,

I have installed all the tools required for Insight, and able to view the login page of insight app. I was not able to login as it requires an admin credentials. Is there any default admin credentials to login? or point me on how to configure ldap credential?

kindly help me to resolve this issue.

Regards,
Manoj.

Insights latest release v3.8, UI build is failing.

Does the Insights architecture have a purging mechanism to take care of data duplication?

Also is there a targeting mechanism we can you during purging to get rid of specific indices / data from both Neo4j and Elastic Search?

How can we remove / change the default admin / admin credentials for the InSights UI application?

From what i have understood, the credentials being used are the ones from the Postgres DB for Grafana. How can i add another set of credentials for the same? Could you please share the query or the script to be run on the postgres DB?

it would be much nicer if the documentation would be with the code, not in a decoupled wiki.

Hi,
We are able to collect the data from neo4j and render the graph in grafana.
But, we want some information on fetching the data from Elastic Search datasource.

• The Spark Job jar is configured and is running.
• PlatformEngine.jar is running too and data is added to the RabbitMQ Queue.
  But,we are not sure if data is indexed in Elastic Search.
  Please give a clear picture on how to validate the data in ES,how to render the graph in grafana,the graphs that supports ES and also about the queries that needs to be used in the graph.
  Regards,
  Manoj

hi,
I wanted to collect sonar data, configured config.json and followed below steps for the same

i see no data is being collected and got an issue 'No json object could be decoded' . PFB config.json and log file. Please help me to resolve this issue
config.txt
log_SonarAgent.log

Regards,
Manoj.

the code does not build on arch linux with the following error, maven and npm is installed:

[INFO] --- frontend-maven-plugin:1.6:npm (npm run build) @ PlatformUI3 ---
[INFO] Running 'npm run build' in /run/media/rt/1tb-home/backup/src/Insights/PlatformUI3
[INFO] 
[INFO] > [email protected] build /run/media/rt/1tb-home/backup/src/Insights/PlatformUI3
[INFO] > ng build --prod --base-href=/app/
[INFO] 
[ERROR] sh: ng: command not found
[ERROR] npm ERR! file sh
[ERROR] npm ERR! code ELIFECYCLE

We have configured a Jenkins agent via the Insights UI (v4.1). The agent is unable to read the data due to an SSL verification error.

The UI did not have an option to pass the username and the password. We hard coded the credentials in the python script for the rest facade. This took care of the authentication failure error.

Presently, even after giving sslVerify = False in the BaseAgent.py and also setting the (verify = False) for all the requests.get() methods, we are facing the ssl verification failure issue.

Please help in getting it resolved.

OS - RHEL 7.5
Python: 2.7.5
Insights Version: 4.1
Jenkins Agent version: 4.1

Registration of New Agent is failing due to issues with Queue creation in RabbitMq

hi,

how can one derive application/product quality metrics where an application has 10 services & each services has multiple components, is there are way we can aggregate the quality metrics related to build, deployment, code quality etc across different sprints ?

Regards,
Manoj.

After following the steps mentioned in the official documentation at https://onedevops.atlassian.net/wiki/spaces/OI/pages/160202755/Insights+Architecture , there was a step for running a query against the DB in the application server to register the Daemon agent. However, there were no relations in the Postgres database and we were unable to find any mentions of a DB Script to create such a table.

Issue description:

There are multiple versions of org.apache.poi:poi in PlatformRegressionTest. As shown in the following dependency tree, according to Maven's "nearest wins" strategy, only org.apache.poi:poi:4.1.2 can be loaded, org.apache.poi:poi:4.1.0 will be shadowed.

However, several methods defined in shadowed version org.apache.poi:poi:4.1.0 are referenced by client project via org.apache.poi:poi-ooxml:4.1.0 but missing in the actually loaded version org.apache.poi:poi:4.1.2.

For instance, the following missing method(defined in org.apache.poi:poi:4.1.0) are actually referenced by PlatformRegressionTest, which will introduce a runtime error(i.e., "NoSuchMethodError") into PlatformRegressionTest.

org.apache.poi.poifs.crypt.EncryptionVerifier: org.apache.poi.poifs.crypt.EncryptionVerifier clone() is invoked by PlatformRegressionTest via the following path:

Invocation path------
<com.cognizant.devops.platformregressiontest.test.common.XLUtils: java.lang.String getCellData(java.lang.String,java.lang.String,int,int)> Insights\PlatformRegressionTest\target\classes
<org.apache.poi.xssf.usermodel.XSSFRow: org.apache.poi.xssf.usermodel.XSSFCell getCell(int)> Repositories\org\apache\poi\poi-ooxml\4.1.0\poi-ooxml-4.1.0.jar
<org.apache.poi.xssf.usermodel.XSSFRow: org.apache.poi.xssf.usermodel.XSSFCell getCell(int,org.apache.poi.ss.usermodel.Row$MissingCellPolicy)> Repositories\org\apache\poi\poi-ooxml\4.1.0\poi-ooxml-4.1.0.jar
<org.apache.poi.xssf.usermodel.XSSFRow$1: void clinit()> Repositories\org\apache\poi\poi-ooxml\4.1.0\poi-ooxml-4.1.0.jar
<org.apache.poi.ss.usermodel.CellType: org.apache.poi.ss.usermodel.CellType[] values()> Repositories\org\apache\poi\poi\4.1.2\poi-4.1.2.jar
<org.apache.poi.poifs.crypt.agile.AgileEncryptionVerifier: java.lang.Object clone()> Repositories\org\apache\poi\poi-ooxml\4.1.0\poi-ooxml-4.1.0.jar
<org.apache.poi.poifs.crypt.agile.AgileEncryptionVerifier: org.apache.poi.poifs.crypt.agile.AgileEncryptionVerifier clone()> Repositories\org\apache\poi\poi-ooxml\4.1.0\poi-ooxml-4.1.0.jar
<org.apache.poi.poifs.crypt.EncryptionVerifier: org.apache.poi.poifs.crypt.EncryptionVerifier clone()>

Suggested fixing solutions:

1. Upgrade dependency org.apache.poi:poi-ooxml from 4.1.0 to 4.1.2.Because the newer version org.apache.poi:poi-ooxml:4.1.2 does not invoke the above missing methods, such changing cn solve the problem.
2. Change direct dependency org.apache.poi:poi from 4.1.2 to 4.1.0. Because version 4.1.0 includes the above missing methods and is compatible with other versions of org.apache.poi:poi in the project.
   This repair will introduce the following new conflicts, but these conflicts are benign (will not cause runtime exceptions).
   Conflict: commons-codec:commons-codec versions: [1.12, /1.13, /1.9]

Please let me know which solution do you prefer? I can submit a PR to fix it.

Thank you very much for your attention.
Best regards,

Dependency tree----

[INFO] com.cognizant.devops:PlatformRegressionTest:jar:7.6
[INFO] +- org.apache.poi:poi:jar:4.1.2:compile
[INFO] \- org.apache.poi:poi-ooxml:jar:4.1.0:compile
[INFO]    \- (org.apache.poi:poi:jar:4.1.0:compile - omitted for conflict with 4.1.2)