megamsys / gulp Goto Github PK

View Code? Open in Web Editor NEW

6.0 6.0 17.0 76.73 MB

Agent which controls the lifecycle of app in cloud

Home Page: https://www.megam.io

Makefile 1.31% Go 97.92% Shell 0.59% Lua 0.01% Batchfile 0.16%

gulp's People

Contributors

Stargazers

Watchers

Forkers

rajthilakmca radhakrishnana kishoreneelamegam thomasalrin morpheyesh vijaykanthm28 rajesh-rajagopal muthukumarhelios rathishvbr virtengine suganyakaliyamoorthy lakshmi2110 ranjithamca jaeko44

gulp's Issues

onclick images components onclick show false for hadoop

time="2016-09-08T20:28:48Z" level=debug msg="Carton id: ASM4775029357889529119\nname: white-shadow-7854\ncartonsid: AMS7138117788851423507\ntosca: vertice.analytics.hadoop\nimageversion: ""\ncompute:\n cpushare: 1 Core\n memory: 1 GB\n swap: ""\n hdd: 24 GB\nssh:\n user: root\n prefix: vk-007\ndomainname: megambox.com\nprovider: one\npublicip: ""\nboxes:\n- id: COM5822444808342579057\n cartonsid: AMS7138117788851423507\n cartonid: ASM4775029357889529119\n cartonname: white-shadow-7854\n name: aged-wood-2644\n level: 5\n domainname: megambox.com\n inputs:\n cpu: 1 Core\n domain: megambox.com\n hdd: 24 GB\n ipv4private: "true"\n ipv4public: "false"\n ipv6private: "false"\n ipv6public: "false"\n keypairoption: "1"\n lastsuccessstatusupdate: 08 Sep 16 21:28 BST\n provider: one\n ram: 1 GB\n region: sydney\n resource: s1.micro\n sshkey: vk-007\n status: vnchostupdated\n storage_hddtype: HDD\n version: ""\n tosca: vertice.analytics.hadoop\n imageversion: ""\n compute:\n cpushare: 1 Core\n memory: 1 GB\n swap: ""\n hdd: 24 GB\n ssh:\n user: root\n prefix: vk-007\n publicip: ""\n repo:\n type: image\n source: vertice\n oneclick: false\n url: ""\n hook: null\n status: vnchostupdated\n provider: one\n commit: ""\n envs: []\n address: null\n operations: []\nstatus: vnchostupdated\n"

Set admin email for all bitnami VMs

Jonathan:
Also if u can preset the admin email using bitnami make sure u use the Virtengine emailSent . So that admin can get email, say for password reset etc.

Code review - Move the bootstrap code outside the service.

Kindly move the bootstrap code outside service. in a folder boot

Bootstrapper - boot
In boot, move the code for SSH outside

I will review more when we move the common code out of gulp in the next rel.

localhost is not reachable

Needs to change the conf file, by replacing localhost with loopback ip(127.0.0.1)

when stop app

time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;36;10mSTOP CARTON ASM1302335199497027584\x1b[0m"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;37;10m==> pipeline [3]\x1b[0m"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;32;10m => step 0: update-status-riak action\x1b[0m"
time="2016-01-22T16:45:40+01:00" level=debug msg=" set status[COM1302335199505416192] of machine (meade.megambox.com, stopping)"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;34;10m > [riak] open \x1b[0m[192.168.1.105:8087] success"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;34;10m > [riak] fetch\x1b[0m (assembly, ASM1302335199497027584)"
time="2016-01-22T16:45:40+01:00" level=debug msg="meade.megambox.com_log:{22 Jan 16 16:45 CET --- stop box (meade.megambox.com) gulp canal COM1302335199505416192}"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;34;10m > [riak] close\x1b[0m"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;34;10m > [riak] open \x1b[0m[192.168.1.105:8087] success"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;34;10m > [riak] store\x1b[0m (assembly, ASM1302335199497027584)"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;34;10m > [riak] close\x1b[0m"
time="2016-01-22T16:45:40+01:00" level=debug msg=" set status[COM1302335199505416192] of machine (meade.megambox.com, stopping)"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;34;10m > [riak] open \x1b[0m[192.168.1.105:8087] success"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;34;10m > [riak] fetch\x1b[0m (components, COM1302335199505416192)"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;34;10m > [riak] close\x1b[0m"
time="2016-01-22T16:45:40+01:00" level=debug msg="meade.megambox.com_log:{22 Jan 16 16:45 CET update status for machine (meade.megambox.com, stopping) gulp canal COM1302335199505416192}"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;34;10m > [riak] open \x1b[0m[192.168.1.105:8087] success"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;34;10m > [riak] store\x1b[0m (components, COM1302335199505416192)"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;34;10m > [riak] close\x1b[0m"
time="2016-01-22T16:45:40+01:00" level=debug msg="\x1b[1;32;10m => step 1: stop-box action\x1b[0m"
time="2016-01-22T16:45:40+01:00" level=debug msg="meade.megambox.com_log:{22 Jan 16 16:45 CET stop for box (meade.megambox.com) gulp canal COM1302335199505416192}"
ERR 705 (103.56.92.4:4150) IO error - EOF
time="2016-01-22T16:45:40+01:00" level=debug msg="meade.megambox.com_log:{22 Jan 16 16:45 CET meade.megambox.com --> ([java stop]) gulp canal COM1302335199505416192}"
ERR 706 (103.56.92.4:4150) IO error - EOF
time="2016-01-22T16:45:42+01:00" level=debug msg="meade.megambox.com_log:{22 Jan 16 16:45 CET Error: Could not find or load main class stop gulp canal COM1302335199505416192}"
ERR 707 (103.56.92.4:4150) IO error - EOF
time="2016-01-22T16:45:42+01:00" level=debug msg="\x1b[0;33;10m => step 1: stop-box action error - exit status 1\x1b[0m"
time="2016-01-22T16:45:42+01:00" level=debug msg="\x1b[1;31;10m => step 0: update-status-riak action\x1b[0m"
time="2016-01-22T16:45:42+01:00" level=debug msg=" set status[COM1302335199505416192] of machine (meade.megambox.com, error)"
time="2016-01-22T16:45:42+01:00" level=debug msg="meade.megambox.com_log:{22 Jan 16 16:45 CET stop for box (meade.megambox.com) failed. gulp canal COM1302335199505416192}"
time="2016-01-22T16:45:42+01:00" level=debug msg="\x1b[1;34;10m > [riak] open \x1b[0m[192.168.1.105:8087] success"
time="2016-01-22T16:45:42+01:00" level=debug msg="\x1b[1;34;10m > [riak] fetch\x1b[0m (assembly, ASM1302335199497027584)"
time="2016-01-22T16:45:42+01:00" level=debug msg="meade.megambox.com_log:{22 Jan 16 16:45 CET exit status 1 gulp canal COM1302335199505416192}"
time="2016-01-22T16:45:42+01:00" level=debug msg="\x1b[1;34;10m > [riak] close\x1b[0m"
time="2016-01-22T16:45:42+01:00" level=debug msg="\x1b[1;34;10m > [riak] open \x1b[0m[192.168.1.105:8087] success"
time="2016-01-22T16:45:42+01:00" level=debug msg="\x1b[1;34;10m > [riak] store\x1b[0m (assembly, ASM1302335199497027584)"
time="2016-01-22T16:45:42+01:00" level=debug msg="\x1b[1;34;10m > [riak] close\x1b[0m"
time="2016-01-22T16:45:42+01:00" level=debug msg=" set status[COM1302335199505416192] of machine (meade.megambox.com, error)"
time="2016-01-22T16:45:42+01:00" level=debug msg="\x1b[1;34;10m > [riak] open \x1b[0m[192.168.1.105:8087] success"
time="2016-01-22T16:45:42+01:00" level=debug msg="\x1b[1;34;10m > [riak] fetch\x1b[0m (components, COM1302335199505416192)"
time="2016-01-22T16:45:42+01:00" level=debug msg="\x1b[1;34;10m > [riak] close\x1b[0m"
time="2016-01-22T16:45:42+01:00" level=debug msg="\x1b[1;34;10m > [riak] open \x1b[0m[192.168.1.105:8087] success"
time="2016-01-22T16:45:42+01:00" level=debug msg="\x1b[1;34;10m > [riak] store\x1b[0m (components, COM1302335199505416192)"
time="2016-01-22T16:45:42+01:00" level=debug msg="\x1b[1;34;10m > [riak] close\x1b[0m"
time="2016-01-22T16:45:42+01:00" level=error msg="error on execute stop pipeline for box meade.megambox.com - exit status 1"
time="2016-01-22T16:45:42+01:00" level=error msg="Unable to stop the box exit status 1"

step4: failed to update riak

chef-solo run completed.
VM-name : glibbest.megambox.com
but gulp tries for untiesmegambox.com

time="2015-12-10T15:27:08+01:00" level=debug msg="\x1b[1;32;10m  => step 4: update-status-riak action\x1b[0m" 
time="2015-12-10T15:27:08+01:00" level=debug msg="\x1b[1;34;10m  > [riak] open  \x1b[0m[192.168.1.105:8087] success" 
time="2015-12-10T15:27:08+01:00" level=debug msg="\x1b[1;34;10m  > [riak] fetch\x1b[0m (components, COM1286739496121008128)" 
time="2015-12-10T15:27:08+01:00" level=debug msg="\x1b[1;34;10m  > [riak] close\x1b[0m" 
time="2015-12-10T15:27:08+01:00" level=debug msg="\x1b[0;33;10m  => step 4: update-status-riak action error - Failed to fetch structure from riak.\t--> all nodes down\x1b[0m" 
time="2015-12-10T15:27:08+01:00" level=debug msg="\x1b[1;31;10m  => step 3: deploy action\x1b[0m" 
time="2015-12-10T15:27:08+01:00" level=debug msg="\x1b[1;31;10m  => step 2: prepare-box-repository action\x1b[0m" 
time="2015-12-10T15:27:08+01:00" level=debug msg="\x1b[1;31;10m  => step 1: prepareConfig action\x1b[0m" 
time="2015-12-10T15:27:08+01:00" level=debug msg="\x1b[1;31;10m  => step 0: prepareJSON action\x1b[0m" 
time="2015-12-10T15:27:08+01:00" level=error msg="error on execute create pipeline for box untiesmegambox.com - Failed to fetch structure from riak.\t--> all nodes down" 
time="2015-12-10T15:27:08+01:00" level=error msg="Unable to deploy box : Failed to fetch structure from riak.\t--> all nodes down"

chef-repo is not cloned by gulp inside vm

Chef-repo is not presenting anywhere in the vm. It needs to be cloned under MEGAM_HOME. For temporary fix, i have chef-repo in image itself.

Mirror issue 1254

https://gitlab.com/megamsys/nilavu/issues/1254

When scm is empty, differentiate app and core in tosca_type

I get condinous emails from mail gun with same application

I have launched that Java application on 2016-10-13 06:17:47 +0000
even i removed the VM from opennebula i still receive the email from mailgun
Vertice still receive message for silent-rain-4164.megambox.com

Gulp just says "Error". We need to improve it via events.

Improvements to say specific errors via events

Rabbitmq send timeout error when log overload for centos

source code not found in vm

I think gulp needs to clone the Source code of the BYOC_APP. But after launching vm, source code is not cloned in the vm.

If gulp won't clone it, send scm_url to cookbook via chef-solo.json

no need to run chef-solo in torpedo launch

before chef-solo run notify user via log, that application is being deploy

6 Mar 16 11:08 IST --- deploy box (plaits.megambox.com, image:megam)
16 Mar 16 05:40 UTC --- torr (/var/lib/megam/gulp/0.96.tar.gz)
16 Mar 16 05:46 UTC �[32m- execute start nodejs�[0m
16 Mar 16 05:46 UTC �[37m+#�[0m
16 Mar 16 05:46 UTC �[37m+# Licensed under the Apache License, Version 2.0 (the "License");�[0m
16 Mar 16 05:46 UTC �[37m+#�[0m
16 Mar 16 05:46 UTC �[37m+#�[0m
16 Mar 16 05:46 UTC (skipped due to action :nothing)
16 Mar 16 05:46 UTC �[37m+# Unless required by applicable law or agreed to in writing, software�[0m
16 Mar 16 05:46 UTC �[37m+# Copyright 2012, Riot Games�[0m
16 Mar 16 05:46 UTC �[37m@@ -1 +1,67 @@�[0m
16 Mar 16 05:46 UTC * template[/etc/chef/ohai_plugins/nginx.rb] action create
16 Mar 16 05:46 UTC �[37m--- /etc/chef/ohai_plugins/nginx.rb 2016-03-16 05:46:58.574087999 +0000�[0m
16 Mar 16 05:46 UTC [2016-03-16T05:46:58+00:00] INFO: template[/etc/chef/ohai_plugins/nginx.rb] updated file contents /etc/chef/ohai_plugins/nginx.rb

assembly status other than launched like (vnchostupdated) skips bootrapping actions

WHERE id = ? ALLOW FILTERING [COM5822444808342579057]
time="2016-09-08T20:28:48Z" level=debug msg="\x1b[1;34;10m > [scylla] Connection close\x1b[0m"
time="2016-09-08T20:28:48Z" level=debug msg="Carton id: ASM4775029357889529119\nname: white-shadow-7854\ncartonsid: AMS7138117788851423507\ntosca: vertice.analytics.hadoop\nimageversion: ""\ncompute:\n cpushare: 1 Core\n memory: 1 GB\n swap: ""\n hdd: 24 GB\nssh:\n user: root\n prefix: vk-007\ndomainname: megambox.com\nprovider: one\npublicip: ""\nboxes:\n- id: COM5822444808342579057\n cartonsid: AMS7138117788851423507\n cartonid: ASM4775029357889529119\n cartonname: white-shadow-7854\n name: aged-wood-2644\n level: 5\n domainname: megambox.com\n inputs:\n cpu: 1 Core\n domain: megambox.com\n hdd: 24 GB\n ipv4private: "true"\n ipv4public: "false"\n ipv6private: "false"\n ipv6public: "false"\n keypairoption: "1"\n lastsuccessstatusupdate: 08 Sep 16 21:28 BST\n provider: one\n ram: 1 GB\n region: sydney\n resource: s1.micro\n sshkey: vk-007\n status: vnchostupdated\n storage_hddtype: HDD\n version: ""\n tosca: vertice.analytics.hadoop\n imageversion: ""\n compute:\n cpushare: 1 Core\n memory: 1 GB\n swap: ""\n hdd: 24 GB\n ssh:\n user: root\n prefix: vk-007\n publicip: ""\n repo:\n type: image\n source: vertice\n oneclick: false\n url: ""\n hook: null\n status: vnchostupdated\n provider: one\n commit: ""\n envs: []\n address: null\n operations: []\nstatus: vnchostupdated\n"
time="2016-09-08T20:28:48Z" level=debug msg="\x1b[1;36;10mBOOT CARTON ASM4775029357889529119\x1b[0m"
time="2016-09-08T20:28:48Z" level=debug msg="\x1b[1;36;10mwhite-shadow-7854.megambox.com\x1b[0m in (\x1b[1;32;10m13.714µs\x1b[0m)\n\x1b[0;33;10m skip boot for box (white-shadow-7854.megambox.com)\n boot for box (white-shadow-7854.megambox.com) OK\n\x1b[0m"
time="2016-09-08T20:28:48Z" level=debug msg="white-shadow-7854.megambox.com_log:{08 Sep 16 20:28 UTC skip boot for box (white-shadow-7854.megambox.com) gulp aged-wood-2644 COM5822444808342579057}"
time="2016-09-08T20:28:48Z" level=debug msg="white-shadow-7854.megambox.com_log:{08 Sep 16 20:28 UTC boot for box (white-shadow-7854.megambox.com) OK gulp aged-wood-2644 COM5822444808342579057}"

Gulp code review.

Lets review tomorrow after bind is done.

bitnami generate solo.json with username,password,url and tosca_type:bitnami

notify user, that log is completed.

After the chef-solo run, notify that, log is completed.

04 Feb 16 12:20 IST ---- Destroying old machine tom11.megambox.com ----
04 Feb 16 12:20 IST ---- Destroying old machine tom11.megambox.com ----
04 Feb 16 17:18 IST logs nirvana ! machine tom11.megambox.com 
04 Feb 16 12:50 CET --- download (/var/lib/megam/gulp/chef-repo)
04 Feb 16 12:50 CET --- download (/var/lib/megam/gulp/chef-repo) OK
04 Feb 16 12:50 CET change state of machine from (tom11.megambox.com, bootstrapped)
04 Feb 16 12:50 CET append authorized keys for box (tom11.megambox.com) OK
04 Feb 16 12:50 CET append authorized keys for box (tom11.megambox.com) OK
04 Feb 16 12:50 CET --- kickofff chefsolo box (tom11.megambox.com)
04 Feb 16 17:20 IST ---- Added route to machine (tom11.megambox.com, 103.56.92.64)
gobbling logs gobbling logs...

ubuntu torpedo launch error

time="2015-10-19T15:10:04+02:00" level=debug msg="Clone chef cookbooks" 
time="2015-10-19T15:10:04+02:00" level=debug msg="github Initialized" 
time="2015-10-19T15:10:04+02:00" level=debug msg="chefsolo initialized" 
time="2015-10-19T15:10:04+02:00" level=info msg="chefsolo provisioner reports the following:\n    chef-solo provisioner initiated. " 
time="2015-10-19T15:10:04+02:00" level=debug msg="New bucket: assembly" 
time="2015-10-19T15:10:04+02:00" level=debug msg="[riak] open ([192.168.1.105:8087]) SUCCESS" 
time="2015-10-19T15:10:04+02:00" level=debug msg="[riak] closing..." 
time="2015-10-19T15:10:04+02:00" level=debug msg="[pipeline] running the Forward for the update-ip-riak action" 
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0xa0 pc=0x557354]

goroutine 1 [running]:
github.com/megamsys/gulp/carton.(*Ambly).SetIPAddress(0x0, 0xc8201474d0, 0xc, 0x0, 0x0)
    /home/thomas/code/megam/workspace/packager/megamgulpd/build/trusty/gulp/src/github.com/megamsys/gulp/carton/assembly.go:229 +0xc4
github.com/megamsys/gulp/subd/gulpd.glob.func5(0x0, 0x0, 0xc820147380, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0)
    /home/thomas/code/megam/workspace/packager/megamgulpd/build/trusty/gulp/src/github.com/megamsys/gulp/subd/gulpd/actions.go:83 +0x12e
github.com/megamsys/libgo/action.(*Pipeline).Execute(0xc82000f0c0, 0xc820147380, 0x1, 0x1, 0x0, 0x0)
    /home/thomas/.go/src/github.com/megamsys/libgo/action/action.go:141 +0x7da
github.com/megamsys/gulp/subd/gulpd.(*Service).updateStatusPipeline(0xc820114c00, 0x0, 0x0)

Send running mail before full process complete

Jun 27 02:29:21 crimson-meadow-4281.megam.cloud gulpd[2438]: 2017/06/27 02:29:12 shell[sh] 70500K .......... .......... .......... .......... .......... 73% 29.5M 6s
Jun 27 02:29:21 crimson-meadow-4281.megam.cloud gulpd[2438]: 2017/06/27 02:29:12 shell[sh] 70550K .......... .......... .......... .......... .......... 73% 6.10M 6s
Jun 27 02:29:21 crimson-meadow-4281.megam.cloud gulpd[2438]: 2017/06/27 02:29:12 shell[sh] 70600K .......... .......... .......... .......... .......... 73% 13.1M 6s
Jun 27 02:29:21 crimson-meadow-4281.megam.cloud gulpd[2438]: 2017/06/27 02:29:12 shell[sh] 70650K .......... .......... .......... .......... .......... 73% 2.75M 6s
Jun 27 02:29:21 crimson-meadow-4281.megam.cloud gulpd[2438]: 2017/06/27 02:29:12 shell[sh] 70700K .......... .......... .......... .......... .......... 73% 2.21M 6s
Jun 27 02:29:21 crimson-meadow-4281.megam.cloud gulpd[2438]: 2017/06/27 02:29:12 shell[sh] 70750K .......... .......... .......... .......... .......... 73% 19.0M 6s
Jun 27 02:29:21 crimson-meadow-4281.megam.cloud gulpd[2438]: 2017/06/27 02:29:12 shell[sh] 70800K .......... .......... .......... .......... .......... 73% 1.15M 6s
Jun 27 02:29:21 crimson-meadow-4281.megam.cloud gulpd[2438]: 2017/06/27 02:29:12 shell[sh] 70850K .......... .......... .......... .......... .......... 73% 1.74M 6s
Jun 27 02:29:21 crimson-meadow-4281.megam.cloud gulpd[2438]: 2017/06/27 02:29:12 shell[sh] 70900K .......... .......... .......... .......... .......... 74% 19.2M 6s
Jun 27 02:29:21 crimson-meadow-4281.megam.cloud gulpd[2438]: 2017/06/27 02:29:12 shell[sh] 70950K .......... .......... .......... .......... .......... 74% 5.99M 6s

Github clone chef-repo error

Check 1: Download release file of chef-repo(tar.gz)
Check 2: Check for chef-repo directory
else clone chef-repo

To find a ip private or public using following conditions

           static VALUE

addrinfo_ipv4_private_p(VALUE self)
{
uint32_t a;
if (!extract_in_addr(self, &a)) return Qfalse;
if ((a & 0xff000000) == 0x0a000000 || /* 10.0.0.0/8 /
(a & 0xfff00000) == 0xac100000 || / 172.16.0.0/12 /
(a & 0xffff0000) == 0xc0a80000) / 192.168.0.0/16 */
return Qtrue;
return Qfalse;
}

Gulpd - bind service needs to be added.

@rajthilakmca Lets discuss the bind parms, just so we could fix nilavu ( @thomasalrin fyi) in a clean way.

Then request @vijaykanthm28 to fix it. what do ya say ?

code review

log data needs the correct format

JSON doesn't parse well in the UI if it the Message isn't in JSON format.

"Message":"{\"Source\":\"Deploying\",\"Type\":\"Info\",\"Message\":\" update status for machine (still-silence-869.megambox.com, launching)\"}"

Messages that need a change.

"Message":"--- torr (/var/lib/megam/gulp/0.96.tar.gz) OK"
"Message":" update status for machine (still-silence-869.megambox.com, bootstrapping)"
"Message":"--- download (/var/lib/megam/gulp/chef-repo)"
"Message":"--- torr (/var/lib/megam/gulp/0.96.tar.gz)"
"Message":" boot for box (still-silence-869.megambox.com)"
"Message":" update status for machine (still-silence-869.megambox.com
"Message":"--- download (/var/lib/megam/gulp/chef-repo)
"Message":" update status for machine (still-silence-869.megambox.com
"Message":" update status for machine (still-silence-869.megambox.com
"Message":" update status for machine (still-silence-869.megambox.com
"Message":" update status for machine (still-silence-869.megambox.com, bootstrapping)"
"Message":" update status for machine (still-silence-869.megambox.com, chefconfigsetup_starting)"
"Message":" update status for machine (still-silence-869.megambox.com, chefconfigsetup_starting)" "Message":" update status for machine (still-silence-869.megambox.com, chefconfigsetup_starting)"

defaultsudo always setted as true

The public ip shall be stored under a publicip key

Change ip to publicip

//put virtual machine ip address in riak
func (a *Ambly) SetIPAddress(status string) error {

    a.Outputs = append(a.Outputs, NewJsonPair("ip", status))

    if err := db.Store(ASSEMBLYBUCKET, a.Id, a); err != nil {
        return err
    }
    return nil
}

panic: runtime error: invalid memory address or nil pointer dereference

time="2015-10-28T06:17:28+01:00" level=debug msg="return ipnet.IP.String [103.56.92.49]"
time="2015-10-28T06:17:28+01:00" level=debug msg="put virtual machine ip address in riak [103.56.92.49]"

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x558293]

goroutine 1 [running]:
github.com/megamsys/gulp/carton.(*Ambly).SetIPAddress(0x0, 0xc8201277f0, 0xc, 0x0, 0x0)
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/carton/assembly.go:232 +0x7f3
github.com/megamsys/gulp/subd/gulpd.glob.func5(0x0, 0x0, 0xc820127570, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0)
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/subd/gulpd/actions.go:84 +0x1c2

CI upgrade Carton throws error

time="2016-02-03T07:03:09+01:00" level=debug msg="\x1b[1;36;10mUPGRADE CARTON ASM1306541044795572224\x1b[0m"
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x20 pc=0x4b043c]

goroutine 2829 [running]:
fmt.Fprintf(0x0, 0x0, 0xa51800, 0x19, 0xc8205c75d8, 0x1, 0x1, 0x2, 0x0, 0x0)
/usr/local/go/src/fmt/print.go:189 +0xac
github.com/megamsys/gulp/carton.(_Upgradeable).opsBuild(0xc820436300, 0x0, 0x0)
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/carton/upgrade.go:106 +0x166
github.com/megamsys/gulp/carton.(_Upgradeable).(github.com/megamsys/gulp/carton.opsBuild)-fm(0x0, 0x0)
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/carton/upgrade.go:41 +0x2e
github.com/megamsys/gulp/upgrade.run(0x0, 0x0, 0xc8204e50c0, 0x12, 0xc8204e4780, 0x1, 0x4, 0x7f280447d1a8, 0xc8204e5040, 0x0, ...)
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/upgrade/operation.go:143 +0x45c
github.com/megamsys/gulp/upgrade.Run(0x0, 0x0, 0xc8204e50c0, 0x12, 0xc8204e4780, 0x1, 0x4, 0x7f280447d1a8, 0xc8204e5040, 0x0, ...)
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/upgrade/operation.go:129 +0x58
github.com/megamsys/gulp/carton.(_Upgradeable).operateBox(0xc8204e4fc0, 0x7f280447d1a8, 0xc8204e5040, 0x0, 0x0)
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/carton/upgrade.go:85 +0x289
github.com/megamsys/gulp/carton.(_Upgradeable).Upgrade(0xc8204e4fc0, 0x0, 0x0)
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/carton/upgrade.go:62 +0x672
github.com/megamsys/gulp/carton.(_Carton).Upgrade(0xc8204e0100, 0x0, 0x0)
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/carton/carton.go:101 +0x10a
github.com/megamsys/gulp/carton.UpgradeProcess.Process(0xc8204e4100, 0x16, 0xc8204e0100, 0x0, 0x0)
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/carton/megd_process.go:119 +0x2d
github.com/megamsys/gulp/carton.(_UpgradeProcess).Process(0xc8205f40a0, 0xc8204e0100, 0x0, 0x0)
:13 +0xae
github.com/megamsys/gulp/carton.(_ReqOperator).Accept(0xc8205c7f48, 0xc8205c7f18, 0x0, 0x0)
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/carton/req_operator.go:42 +0x170
github.com/megamsys/gulp/subd/gulpd.(_Handler).serveNSQ(0xc82012c200, 0xc82014a050, 0x0, 0x0)
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/subd/gulpd/handler.go:41 +0x190
created by github.com/megamsys/gulp/subd/gulpd.(*Service).processNSQ
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/subd/gulpd/service.go:92 +0xc8

goroutine 1 [chan receive, 21 minutes]:
github.com/megamsys/gulp/cmd/gulpd/run.(_Start).Run(0xc82000ef00, 0xc82000afa0, 0x0, 0x0)
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/cmd/gulpd/run/command.go:81 +0x675
github.com/megamsys/libgo/cmd.(_Manager).Run(0xc8200f33f0, 0x0, 0x0, 0x0)
/home/megam/.go/src/github.com/megamsys/libgo/cmd/cmd.go:184 +0x1ed1
main.main()
/home/megam/code/workspace/go/src/github.com/megamsys/gulp/cmd/gulpd/main.go:52 +0xba

Gulp - CI needs to be fixes

@rajthilakmca We already know the CI parms.

Can we request @vijaykanthm28 to fix it. what do ya say ?

Have to fetch the ssh keys using ssh keys and org_id

If you create a ssh keys with an existing name, gulp to fetch the existing ssh keys so can't login to vm,
change the query using org_id and ssh key.

update same ip two times

'{"key":"public_ipv4","value":"88.99.200.181, 88.99.200.181"}'

Log Formats needs to be corrected,

0001-01-01T00:00:00Z:D
0001-01-01T00:00:00Z:
0001-01-01T00:00:00Z:
0001-01-01T00:00:00Z:
0001-01-01T00:00:00Z:T
0001-01-01T00:00:00Z:i
0001-01-01T00:00:00Z:m
0001-01-01T00:00:00Z:e
0001-01-01T00:00:00Z:
0001-01-01T00:00:00Z:
0001-01-01T00:00:00Z:
0001-01-01T00:00:00Z:
0001-01-01T00:00:00Z:T
0001-01-01T00:00:00Z:i```

rabbitmq ip is not loaded from gulpd.conf

I have 192.168.1.105 as rabbitmq host in gulpd.conf. But gulpd is trying to connect to 127.0.0.1

[10/01/15 12:41:00] [INFO] Gulpd starting at 2015-10-01 12:41:00.188419017 +0200 CEST
[10/01/15 12:41:00] [INFO] verifying rabbitmq
Error: dial tcp 127.0.0.1:5672: connection refused
 Please start rabbitmq service.

Remove config ganglia

Remove/cleanup conf
i Leader
ii election time out

Gulpd uses the down level libgo

time="2015-12-08T15:47:07+01:00" level=debug msg="Notify to [file] Logger "
time="2015-12-08T15:47:07+01:00" level=debug msg="push logs to Logger file"
time="2015-12-08T15:47:07+01:00" level=debug msg="Notify to [queue] Logger "
time="2015-12-08T15:47:07+01:00" level=debug msg="[amqp] dial (amqp://guest:[email protected]:5672/) SUCCESS"
time="2015-12-08T15:47:07+01:00" level=debug msg="[amqp] pub SUCCESS."
time="2015-12-08T15:47:07+01:00" level=debug msg="[amqp] dial (amqp://guest:[email protected]:5672/) SUCCESS"
time="2015-12-08T15:47:07+01:00" level=debug msg="[amqp] pub SUCCESS."
time="2015-12-08T15:47:07+01:00" level=debug msg="push logs to Logger queue"

change owner from '' to 'root'

app and service launch error on stateup carton

ime="2016-01-22T10:19:10+01:00" level=debug msg="\x1b[1;36;10mSTATEUP CARTON ASM1302247294959222784\x1b[0m"
time="2016-01-22T10:19:10+01:00" level=debug msg="\x1b[1;37;10m==> pipeline [5]\x1b[0m"
time="2016-01-22T10:19:10+01:00" level=debug msg="\x1b[1;32;10m => step 0: generate-solo-json action\x1b[0m"
time="2016-01-22T10:19:10+01:00" level=debug msg="\x1b[1;32;10m => step 1: generate-solo-config action\x1b[0m"
time="2016-01-22T10:19:10+01:00" level=debug msg="\x1b[1;32;10m => step 2: clone-box action\x1b[0m"
time="2016-01-22T10:19:10+01:00" level=debug msg="\x1b[0;33;10m => step 2: clone-box action error - fatal error, couldn't locate the repository manager (github)\x1b[0m"
time="2016-01-22T10:19:10+01:00" level=debug msg="\x1b[1;31;10m => step 1: generate-solo-config action\x1b[0m"
time="2016-01-22T10:19:10+01:00" level=debug msg="\x1b[1;31;10m => step 0: generate-solo-json action\x1b[0m"
time="2016-01-22T10:19:10+01:00" level=error msg="error on execute chefsolo pipeline for box unbuckles.megambox.com - fatal error, couldn't locate the repository manager (github)"
time="2016-01-22T10:19:10+01:00" level=debug msg="\x1b[1;36;10munbuckles.megambox.com\x1b[0m in (\x1b[1;32;10m678.098µs\x1b[0m)\n\x1b[0;33;10m--- stateup box (unbuckles.megambox.com)\n--- kickofff chefsolo box (unbuckles.megambox.com)\n generate solo json for box (unbuckles.megambox.com) generate solo json for box (unbuckles.megambox.com) OK\n generate solo config for box (unbuckles.megambox.com)\n generate solo config for box (unbuckles.megambox.com) OK\n clone repository for box (unbuckles.megambox.com) clone repository for box failed.\nfatal error, couldn't locate the repository manager (github)\n\x1b[0m"
time="2016-01-22T10:19:10+01:00" level=error msg="Unable to stateup box : fatal error, couldn't locate the repository manager (github)"
time="2016-01-22T10:19:10+01:00" level=debug msg="unbuckles.megambox.com_log:{22 Jan 16 10:19 CET --- stateup box (unbuckles.megambox.com) gulp shipwreck COM1302247294971805696}"
ERR 19 (103.56.92.4:4150) IO error - EOF
time="2016-01-22T10:19:10+01:00" level=debug msg="unbuckles.megambox.com_log:{22 Jan 16 10:19 CET --- kickofff chefsolo box (unbuckles.megambox.com) gulp shipwreck COM1302247294971805696}"
ERR 20 (103.56.92.4:4150) IO error - EOF
time="2016-01-22T10:19:10+01:00" level=debug msg="unbuckles.megambox.com_log:{22 Jan 16 10:19 CET generate solo json for box (unbuckles.megambox.com) gulp shipwreck COM1302247294971805696}"
time="2016-01-22T10:19:10+01:00" level=debug msg="unbuckles.megambox.com_log:{22 Jan 16 10:19 CET generate solo json for box (unbuckles.megambox.com) OK gulp shipwreck COM1302247294971805696}"
time="2016-01-22T10:19:10+01:00" level=debug msg="unbuckles.megambox.com_log:{22 Jan 16 10:19 CET generate solo config for box (unbuckles.megambox.com) gulp shipwreck COM1302247294971805696}"
time="2016-01-22T10:19:10+01:00" level=debug msg="unbuckles.megambox.com_log:{22 Jan 16 10:19 CET generate solo config for box (unbuckles.megambox.com) OK gulp shipwreck COM1302247294971805696}"
time="2016-01-22T10:19:10+01:00" level=debug msg="unbuckles.megambox.com_log:{22 Jan 16 10:19 CET clone repository for box (unbuckles.megambox.com) gulp shipwreck COM1302247294971805696}"
time="2016-01-22T10:19:10+01:00" level=debug msg="unbuckles.megambox.com_log:{22 Jan 16 10:19 CET clone repository for box failed. gulp shipwreck COM1302247294971805696}"
time="2016-01-22T10:19:10+01:00" level=debug msg="unbuckles.megambox.com_log:{22 Jan 16 10:19 CET fatal error, couldn't locate the repository manager (github) gulp shipwreck COM1302247294971805696}"

Launch torpedo step 2 clone box get panic

time="2016-01-19T12:23:06+01:00" level=debug msg="\x1b[1;37;10m==> pipeline [5]\x1b[0m"
time="2016-01-19T12:23:06+01:00" level=debug msg="\x1b[1;32;10m => step 0: generate-solo-json action\x1b[0m"
time="2016-01-19T12:23:06+01:00" level=debug msg="\x1b[1;32;10m => step 1: generate-solo-config action\x1b[0m"
time="2016-01-19T12:23:06+01:00" level=debug msg="\x1b[1;32;10m => step 2: clone-box action\x1b[0m"
time="2016-01-19T12:23:06+01:00" level=debug msg=" notify pickpockets.megambox.com_log"
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x5685a6]

goroutine 273 [running]:
github.com/megamsys/gulp/provision.(_Box).Clone(0xc820295380, 0x0, 0x0)
/home/megam/code/workspace/packager/megamgulpd/build/trusty/gulp/src/github.com/megamsys/gulp/provision/box.go:226 +0x3a6
github.com/megamsys/gulp/provision/chefsolo.glob.func17(0x0, 0x0, 0xc8202e84f0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0)
/home/megam/code/workspace/packager/megamgulpd/build/trusty/gulp/src/github.com/megamsys/gulp/provision/chefsolo/actions.go:207 +0x271
github.com/megamsys/libgo/action.(_Pipeline).Execute(0xc8202f0180, 0xc8202e84f0, 0x1, 0x1, 0x0, 0x0)
/home/megam/.go/src/github.com/megamsys/libgo/action/action.go:147 +0xc4c
github.com/megamsys/gulp/provision/chefsolo.(*chefsoloProvisioner).kickOffSolo(0xc8200bea80, 0xc820295380, 0x7fa28c246878, 0xc8202f0020, 0x0, 0x0)

To build gulp package the error in download go.skia.org/infra/go/exec

godep: error downloading dep (go.skia.org/infra/go/exec): unrecognized import path "go.skia.org/infra/go/exec"
godep: error downloading dep (go.skia.org/infra/go/gitinfo): unrecognized import path "go.skia.org/infra/go/gitinfo"
godep: error downloading dep (go.skia.org/infra/go/tiling): unrecognized import path "go.skia.org/infra/go/tiling"
godep: error downloading dep (go.skia.org/infra/go/util): unrecognized import path "go.skia.org/infra/go/util"
godep: error downloading dep (go.skia.org/infra/go/vcsinfo): unrecognized import path "go.skia.org/infra/go/vcsinfo"
godep: Error downloading some deps. Aborting restore and check.

conf file needs to be changed

gulpd.conf has riak too

  [meta]
    user = "root"
    riak =["192.168.1.105:8087"]
    nsqd = ["localhost:4150"]
    scylla = ["192.168.1.247"]
    scylla_keyspace = "vertice"

whenever i restart gulpd, it runs all the processes from first

Step1: gulpd run completed(chef-solo run completed)
step2: gulpd restarted
now gulpd tries to clone chef-repo