Running in VMWARE about openidp HOT 19 CLOSED

When you say "config server failed", you mean during health check? If that is the case, please ignore it as for me also, health check never passed but when checked each container logs separately after few minutes, all services get started normally. If no, please share the config server logs.

Also, i would suggest to allocate 16 gb. For me, i have 16 gb vm running this application alone. And it occupies 12 gb all the time. You probably going to need more than that if you don't want your machine gasping for ram all the time.

P.s. Lack of ram is one of the reason for config server failing to start. If logs does not make much sense, do allocate more memory. I learnt this hard lesson after debugging this issue on 8gb ram.

from openidp.

Thank you for the quick help. I am running it in a VM on a machine with only 16 GB total memory, but next week I am going to try it on a standalone with 32 GB of memory. I cant push the VM much more past 12GB for now.

from openidp.

@xxxkrogoth Since you are trying to run in vmware, i assume you are using internal ip/hostname for cluster installation or have forwarded all the required ports to your host machine (in case of NAT) for using host ip.

As @manishbansal8843 said that you can also check if cluster is running even if health check fails which is true in case you have slow machine where cluster is struggling for resources.

To verify this replace health_check.sh in you setup directory with this new file and try to give it a spin. If machine slowness is issue it will pass all health check but if you have configured the network bit off which obstructs the mirco-services handshake then you need to correct the Vmware nodes network setup first as it actually means cluster is not setup correctly.

from openidp.

Thanks! I will give it a shot today. I already modified the original build.sh to allow more time to run the health check, but lets see if this helps. Also, 3 types of Linux types are mentioned in the beginning. CentOS 7, Ubuntu 16.x and RHEL 7.x. Any particular distribution work better than others?

from openidp.

idp/idpdeployment:latest is not running when I do Docker service list. Any ideas how to manually start this? I am really close on this. I have no idea why my paste has strikouts in it, but thats the Docker situation.

krogoth@ubuntu:~$ sudo docker service list

ID NAME MODE REPLICAS IMAGE PORTS
bntv86oc5mub IDP_app replicated 1/1 idp/idpapp:latest *:8080->8080/tcp
orbdghglf1c2 IDP_config replicated 1/1 idp/idpconfig:latest *:8888->8888/tcp
pn1xilh6xy1j IDP_dashboard replicated 1/1 idp/idpdashboard:latest *:8184->8184/tcp
xhlkv83htanl IDP_eureka replicated 1/1 idp/idpeureka:latest *:8761->8761/tcp
lwyepbgfyz4n IDP_grafana replicated 1/1 grafana/grafana:5.2.1 *:3000->3000/tcp
xcx64kqdjk4k IDP_jenkins replicated 1/1 idp/jenkins:latest *:8085->8080/tcp, *:50000->50000/tcp
sdheda1m9oq8 IDP_kafka replicated 1/1 wurstmeister/kafka:0.11.0.1
iskhr8xrrdpc IDP_keycloak replicated 1/1 jboss/keycloak:4.0.0.Beta1 *:8989->8080/tcp
xswqq7ur5lv6 IDP_oauth replicated 1/1 idp/idpoauth:latest *:8181->8181/tcp
ejkf7pr1lqpb IDP_postgres replicated 1/1 postgres:9.6.1 *:5432->5432/tcp
ketutcucpzs9 IDP_scheduler replicated 1/1 idp/idpscheduler:latest *:8222->8222/tcp
kuqk98f5ilc9 IDP_services replicated 1/1 idp/idpservices:latest *:8889->8889/tcp
tl7e0k0ifzow IDP_subscription replicated 1/1 idp/idpsubscription:latest *:8090->8090/tcp
pneyjbwdztm9 IDP_zookeeper replicated 1/1 zookeeper:3.5 *:2181->2181/tcp
krogoth@ubuntu:$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6112e3c255e0 jboss/keycloak:4.0.0.Beta1 "/opt/jboss/docker-e…" 45 minutes ago Up 45 minutes 8080/tcp IDP_keycloak.1.jrqhlp6zxoawqzzo77od63uis
79d964accb8d grafana/grafana:5.2.1 "/run.sh" 45 minutes ago Up 45 minutes 3000/tcp IDP_grafana.1.wxj5wnf5pirjd3hfawl9gyxg7
bfc19bdaaf8b wurstmeister/kafka:0.11.0.1 "start-kafka.sh" About an hour ago Up About an hour 0.0.0.0:9092->9092/tcp IDP_kafka.1.qy045k5mi1goxx5s0m0tszoko
820a674ac0da zookeeper:3.5 "/docker-entrypoint.…" About an hour ago Up About an hour 2181/tcp, 2888/tcp, 3888/tcp IDP_zookeeper.1.t02avgzidmn63xjeuyjz16o3i
47f9cca10ea2 postgres:9.6.1 "/docker-entrypoint.…" About an hour ago Up About an hour 5432/tcp IDP_postgres.1.zb2evwof6gc792q6ymcsmwins
0a76163f37e0 idp/idpservices:latest "/bin/sh -c /idprest…" About an hour ago Up About an hour 8889/tcp IDP_services.1.05hi2wun89jinpt71e7ant6f0
11d5426ef782 idp/idpconfig:latest "/bin/sh -c /idpconf…" About an hour ago Up About an hour 8888/tcp IDP_config.1.fcyjixcre3y8r0hrn2fi244m6
67d36a843fbd idp/idpeureka:latest "/bin/sh -c /idpeure…" About an hour ago Up About an hour 8761/tcp IDP_eureka.1.nz3dac8ivlvpn8t16ns7l3bgf
080fe970226a idp/idpscheduler:latest "/bin/sh -c /idpsche…" About an hour ago Up About an hour 8222/tcp IDP_scheduler.1.289mmvpfu01gcyxntdyrqekgg
00a05cad1d8d idp/idpapp:latest "/bin/sh -c /idpapp/…" About an hour ago Up About an hour 8080/tcp IDP_app.1.y94nb6b2yuemcizt6rtrqguk6
60206ef71c4c idp/jenkins:latest "catalina.sh run" About an hour ago Up About an hour 8080/tcp, 8085/tcp, 50000/tcp IDP_jenkins.1.wo72kufdbvdx6s38qxkpv84zk
8419c6fd41a5 idp/idpoauth:latest "/bin/sh -c /idpoaut…" About an hour ago Up About an hour 8181/tcp IDP_oauth.1.wr6dbj88q7d38lcotz8ih8kii
49dbc380b854 idp/idpsubscription:latest "/bin/sh -c /idpsubs…" About an hour ago Up About an hour 8090/tcp IDP_subscription.1.hzv0k7izwc9f1acx9vh32316c
5f8052307a4b idp/idpdashboard:latest "/bin/sh -c /idpdash…" About an hour ago Up About an hour 8184/tcp IDP_dashboard.1.sfujdfbk60v6qwmabasxlje0j
krogoth@ubuntu:$

from openidp.

@xxxkrogoth That component "idpdeployment" is in beta testing hence not released. I understand your confusion from pictures in Wiki page. You can ignore that. We will fix those images.

Your cluster seems to be running correctly. Are you able to hit the url ?

from openidp.

No I cannot get into the IDP website on 8080. I can get into Keycloak and base Apache Tomcat pages and do the written modifications for Keycloak:8989
. Here is the port report-

Not shown: 65524 closed ports
PORT STATE SERVICE
2181/tcp open unknown
2377/tcp open unknown
3000/tcp open ppp
5432/tcp open postgresql
7946/tcp open unknown
8085/tcp open unknown
8761/tcp open unknown
8888/tcp open sun-answerbook
8989/tcp open unknown
9092/tcp open unknown
50000/tcp open ibm-db2

I also see a lot of things saying it needs to load manually. Here is the load log.

Compiling UI .......
npm WARN @angular/[email protected] requires a peer of @angular/common@^2.2.3 but none is installed. You must install peer dependencies yourself.
npm WARN @angular/[email protected] requires a peer of @angular/core@^2.2.3 but none is installed. You must install peer dependencies yourself.
npm WARN @angular/[email protected] requires a peer of zone.js@^0.7.2 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @angular/core@>=6.0.0-rc.5 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @angular/common@>=6.0.0-rc.5 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @angular/core@^2.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @angular/common@~2.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @angular/compiler@~2.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @angular/core@~2.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @angular/forms@~2.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @angular/core@^2.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @angular/http@^2.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @angular/common@>=6.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @angular/core@>=6.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of @angular/forms@>=6.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

Your global Angular CLI version (6.2.1) is greater than your local
version (1.7.4). The local Angular CLI version is used.

To disable this warning use "ng config -g cli.warnings.versionMismatch false".

from openidp.

@xxxkrogoth these are warning message for optional dependencies. Not used. You can ignore the warning. We have established till now that cluster is complied and running on you VM. Only thing pending is handshake across services.

Can you get us the output for following commands on your VM running IDP ?

docker service inspect IDP_oauth
docker logs -f $(docker ps | grep oauth | awk '{ print $1 }')

from openidp.

docker service inspect IDP_oauth

[
{
"ID": "1pvxnf1ladu0cavdyujjhc46y",
"Version": {
"Index": 429
},
"CreatedAt": "2019-01-08T15:38:34.496573446Z",
"UpdatedAt": "2019-01-08T15:38:34.498652433Z",
"Spec": {
"Name": "IDP_oauth",
"Labels": {
"com.docker.stack.image": "idp/idpoauth",
"com.docker.stack.namespace": "IDP"
},
"TaskTemplate": {
"ContainerSpec": {
"Image": "idp/idpoauth:latest",
"Labels": {
"com.docker.stack.namespace": "IDP"
},
"Env": [
"CONFIG_HOSTNAME=ubuntu",
"CONFIG_PASSWORD=idpadmin@123",
"CONFIG_PORT=8888",
"CONFIG_USERNAME=idpadmin",
"DASHBOARD_HOSTNAME=ubuntu",
"DASHBOARD_PORT=8184",
"EUREKA_HOSTNAME=ubuntu",
"EUREKA_PORT=8761",
"GRAFANA_HOSTNAME=ubuntu",
"GRAFANA_PORT=3000",
"IDPAPP_HOSTNAME=ubuntu",
"IDPAPP_PORT=8080",
"JENKINS_HOSTNAME=ubuntu",
"JENKINS_PASSWORD=650f0ef465ef16b5a8e1bdcbd461973c",
"JENKINS_PORT=8085",
"JENKINS_USERNAME=idpadmin",
"KAFKA_HOSTNAME=kafka",
"KAFKA_PORT=9092",
"KEYCLOAK_HOSTNAME=ubuntu",
"KEYCLOAK_PASSWORD=admin",
"KEYCLOAK_PORT=8989",
"KEYCLOAK_USER=admin",
"OAUTH_HOSTNAME=ubuntu",
"OAUTH_PORT=8181",
"POSTGRES_DEFAULT=keycloak",
"POSTGRES_HOSTNAME=postgres",
"POSTGRES_PASSWORD=root",
"POSTGRES_PORT=5432",
"POSTGRES_USER=postgres",
"PROFILE=paas",
"SCHEDULER_HOSTNAME=ubuntu",
"SCHEDULER_PORT=8222",
"SERVICES_HOSTNAME=ubuntu",
"SERVICES_PORT=8889",
"SSL_ENABLED=false",
"SUBSCRIPTION_HOSTNAME=ubuntu",
"SUBSCRIPTION_PORT=8090",
"TZ=Asia/Kolkata",
"ZOOKEEPER_HOSTNAME=zookeeper",
"ZOOKEEPER_PORT=2181"
],
"Privileges": {
"CredentialSpec": null,
"SELinuxContext": null
},
"StopGracePeriod": 10000000000,
"DNSConfig": {},
"Isolation": "default"
},
"Resources": {},
"RestartPolicy": {
"Condition": "any",
"Delay": 5000000000,
"MaxAttempts": 0
},
"Placement": {},
"Networks": [
{
"Target": "6oan3ihrta9od8ya1ute1srul",
"Aliases": [
"oauth"
]
}
],
"ForceUpdate": 0,
"Runtime": "container"
},
"Mode": {
"Replicated": {
"Replicas": 1
}
},
"UpdateConfig": {
"Parallelism": 1,
"FailureAction": "pause",
"Monitor": 5000000000,
"MaxFailureRatio": 0,
"Order": "stop-first"
},
"RollbackConfig": {
"Parallelism": 1,
"FailureAction": "pause",
"Monitor": 5000000000,
"MaxFailureRatio": 0,
"Order": "stop-first"
},
"EndpointSpec": {
"Mode": "vip",
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 8181,
"PublishedPort": 8181,
"PublishMode": "ingress"
}
]
}
},
"Endpoint": {
"Spec": {
"Mode": "vip",
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 8181,
"PublishedPort": 8181,
"PublishMode": "ingress"
}
]
},
"Ports": [
{
"Protocol": "tcp",
"TargetPort": 8181,
"PublishedPort": 8181,
"PublishMode": "ingress"
}
],
"VirtualIPs": [
{
"NetworkID": "bnulpjbi989xm2gov5rvrgma3",
"Addr": "10.255.0.73/16"
},
{
"NetworkID": "6oan3ihrta9od8ya1ute1srul",
"Addr": "10.0.2.23/24"
}
]
}
}
]

docker logs -f $(docker ps | grep oauth | awk '{ print $1 }')

root@ubuntu:~/openIDP# docker logs -f $(docker ps | grep oauth | awk '{ print $1 }')
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....
Waiting for Config Server to start. Sleeping for 5 sec .....

from openidp.

@xxxkrogoth Its clear from the logs that the hostname i.e. "ubuntu" of your VM is not getting resolved in containers. Hence handshake not occuring.
Network setup is not working. Suggest you to either fix hostname resolution for VM or use the machine IP for setup by giving that value to HOSTNAME variable in script as mentioned here.

In both case, please ensure you have required ports forwarded from VM to host OS via bridge setup or use NAT for direct access, unless you are planing to run X-server/GNOME session in your ubuntu installation in that case you can use the browser of ubuntu installation itself to check since port will be local in that case.

from openidp.

Should I use a FQDN for it in the script after I set up the hostname on the unit. Like test.ubuntu.server for instance? Also, for now I am using it locally with a minimal LXDE and Firefox and still cannot get to it in the VM Nat. Once I can get into it with the NAT in the VM I will test if I can get to it outside the virtual setup.

from openidp.

Also, is their a way to uninstall openIDP completely, or disable it?

from openidp.

@xxxkrogoth Yes you can use FQDN but bear in mind in that case its accessible only within your unit unless you replicate the DNS setup.

So as per your statement this is what i understood, that you have setup IDP in a VM and trying to access it from VM only. Correct me if i am wrong.
Because in that case all you need to do is replace hostname variable in build script with 127.0.0.1 or ip address of your virtual ethernet port setup in VM which can be found by

ifconfig -a

Also for uninstalling IDP, its simple 2 steps.

1. Fire the following command to remove the docker stack
   docker stack rm IDP

2. Delete the openIDP folder

and its clean, no trace of IDP on your system.

from openidp.

Yes, trying to connect to it only inside the VM. Did the local host IP, didn't work. Did the Hostname -I (IP) and that didn't work as well. updated hosts file and did all the other stuff to try and get it going and still cannot get to it from the browser.

from openidp.

Jaskirit, can you try and duplicate this on VMWare workstation? It would be beneficial to have this able to run in a VM environment.

from openidp.

Can anyone try and get this to run in a VM like VMWARE WORKSTATION?

from openidp.

We will try to give it a shot. Please share the network configuration you used for your VM plus what was the Host OS ?

from openidp.

The network configuration is standard NAT and the OS was CentOS

from openidp.

jaskirat8, any updates on this?

from openidp.