siemens / fluffi Goto Github PK
View Code? Open in Web Editor NEWFLUFFI (Fully Localized Utility For Fuzzing Instantaneously) - A distributed evolutionary binary fuzzer for pentesters
License: MIT License
fluffi's People
Contributors
Stargazers
Watchers
Forkers
joyride9999 maverickwoo gitttt vinamrabhatia tomsie jack51706 asdyxcyxc 5l1v3r1 rajivraj ufwt hack-payload-collection spormanndavid berdav sandi2382 xorman n0tty jdefrancesco sears-s rakhithjkfluffi's Issues
Links not working without ngnix
mon.fluffi not working -> mon.fluffi:8885 works
Taking Internet offline too early
Some events should trigger "checkHostsAlive"
for better usability trigger "checkHostsAlive"
- web container start
- adding a new system
Changing location after adding first location does not work
missing documentation on how to start a fuzzjob
Its needed to add information on how to actually start a fuzzjob :)
Would help a bit also if a small description is also added regarding on what the 4 steps are doing:
- initial setup
- deploy fluffi
- deploy SUT/Dependency
- configure instances
And also some description regarding the parameters in each tab.
Deploy fluffi reports OK but has error
PLAY [windows] *****************************************************************
TASK [Gathering Facts] *********************************************************
ok: [batrix3]
TASK [initial-windows-Timezone : Set timezone to 'Romance Standard Time' (GMT+01:00)] ***
ok: [batrix3]
TASK [initial-windows-GFlags : Ensure source directory exists] *****************
ok: [batrix3]
TASK [initial-windows-GFlags : Fetch GFlag zip file "ftp://ftp.fluffi/initial/windows/ansible/GFlags/GFlags.zip"] ***
changed: [batrix3]
TASK [initial-windows-GFlags : Unzip GFlags] ***********************************
changed: [batrix3]
TASK [initial-windows-PrepareEnvironment : Create a FLUFFI directory] **********
ok: [batrix3]
TASK [initial-windows-PrepareEnvironment : Set ACL for C:\FLUFFI] **************
ok: [batrix3]
TASK [initial-windows-PrepareEnvironment : Set AllowUnencrypted to True] *******
fatal: [batrix3]: FAILED! => {"changed": true, "cmd": "Set-Item -Path WSMan:\\localhost\\Service\\AllowUnencrypted -Value $true", "delta": "0:00:03.234031", "end": "2020-04-16 09:33:04.872752", "msg": "non-zero return code", "rc": 1, "start": "2020-04-16 09:33:01.638721", "stderr": "Set-Item : WinRM firewall exception will not work since one of the network connection types on this machine is set to \r\nPublic. Change the network connection type to either Domain or Private and try again. \r\nAt line:1 char:65\r\n+ ... ing $false; Set-Item -Path WSMan:\\localhost\\Service\\AllowUnencrypted ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: (:) [Set-Item], InvalidOperationException\r\n + FullyQualifiedErrorId : System.InvalidOperationException,Microsoft.PowerShell.Commands.SetItemCommand", "stderr_lines": ["Set-Item : WinRM firewall exception will not work since one of the network connection types on this machine is set to ", "Public. Change the network connection type to either Domain or Private and try again. ", "At line:1 char:65", "+ ... ing $false; Set-Item -Path WSMan:\\localhost\\Service\\AllowUnencrypted ...", "+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~", " + CategoryInfo : NotSpecified: (:) [Set-Item], InvalidOperationException", " + FullyQualifiedErrorId : System.InvalidOperationException,Microsoft.PowerShell.Commands.SetItemCommand"], "stdout": "", "stdout_lines": []}
PLAY RECAP *********************************************************************
batrix3 : ok=7 changed=2 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
Missing directories when running deploy2FTP.sh
fluffi/build/ubuntu_based/deploy2FTP.sh
Line 31 in 4bb9d49
/srv/fluffi/data/ftp/files/fluffi/linux/{x64,x86,arm32,arm64} exist on Fluffi master. In our deployment, they don't exist after a fresh install.
P.S. It seems reasonable to add mkdir /fluffi/linux/x64 (and three similar commands) to the FTP commands in this shell script to ensure that these directories exist.
Fetch Rest-Instance-Starter-Autostart failed
[WARNING]: While constructing a mapping from /usr/local/lib/python3.6/dist-
packages/polemarch/projects/1/roles/general-windows-
syncToRamdisk/tasks/main.yml, line 25, column 3, found a duplicate dict key
(win_file). Using last defined value only.
[DEPRECATION WARNING]: Instead of sudo/sudo_user, use become/become_user and
make sure become_method is 'sudo' (default). This feature will be removed in
version 2.9. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
PLAY [windows] *****************************************************************
TASK [Gathering Facts] *********************************************************
ok: [batrixwin]
TASK [deployment-windows-fluffi : Ensure directory exists] *********************
changed: [batrixwin]
TASK [deployment-windows-fluffi : Fetch FLUFFI from "ftp://ftp.fluffi/fluffi/windows/x64/fluffi.zip"] ***
changed: [batrixwin]
TASK [deployment-windows-fluffi : Unzip FLUFFI to persistent C] ****************
changed: [batrixwin]
TASK [deployment-windows-fluffi : Parse location to host] **********************
ok: [batrixwin] => (item={'key': 'batrixwin', 'value': 'FLUFFI'})
TASK [deployment-windows-fluffi : Create persistent shortcuts on desktop] ******
changed: [batrixwin] => (item={'key': 'TR', 'value': 'TestcaseRunner'})
changed: [batrixwin] => (item={'key': 'TG', 'value': 'TestcaseGenerator'})
changed: [batrixwin] => (item={'key': 'TE', 'value': 'TestcaseEvaluator'})
changed: [batrixwin] => (item={'key': 'LM', 'value': 'LocalManager'})
TASK [deployment-windows-fluffi : Create ramdisk shortcuts on desktop] *********
changed: [batrixwin] => (item={'key': 'TR', 'value': 'TestcaseRunner'})
changed: [batrixwin] => (item={'key': 'TG', 'value': 'TestcaseGenerator'})
changed: [batrixwin] => (item={'key': 'TE', 'value': 'TestcaseEvaluator'})
changed: [batrixwin] => (item={'key': 'LM', 'value': 'LocalManager'})
TASK [general-windows-syncToRamdisk : Obtain information about ramdisk] ********
ok: [batrixwin]
TASK [general-windows-syncToRamdisk : Check existence] *************************
skipping: [batrixwin]
TASK [general-windows-syncToRamdisk : Delete existing files on ramdisk (x64)] ***
ok: [batrixwin]
TASK [general-windows-syncToRamdisk : Ensure directories exists] ***************
changed: [batrixwin]
TASK [general-windows-syncToRamdisk : Execute copy command for x64] ************
changed: [batrixwin]
TASK [general-windows-syncToRamdisk : Execute copy command for x64/dynamorio] ***
changed: [batrixwin]
TASK [general-windows-syncToRamdisk : Execute copy command for x86] ************
changed: [batrixwin]
TASK [general-windows-syncToRamdisk : Execute copy command for x86/dynamorio] ***
changed: [batrixwin]
TASK [initial-windows-deployRestInstanceStarter : Set file path to fetch] ******
ok: [batrixwin]
TASK [initial-windows-deployRestInstanceStarter : Create a FLUFFI directory] ***
changed: [batrixwin]
TASK [initial-windows-deployRestInstanceStarter : Fetch Rest-Instance-Starter from "ftp://ftp.fluffi/restStarter/main.py"] ***
changed: [batrixwin]
TASK [initial-windows-deployRestInstanceStarter : Fetch Rest-Instance-Starter-Autostart from "ftp://ftp.fluffi/restStarter/restInstanceAutostart.bat"] ***
fatal: [batrixwin]: FAILED! => {"changed": false, "dest": "C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\restInstanceAutostart.bat", "elapsed": 0, "msg": "The path 'C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup' does not exist for destination 'C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\restInstanceAutostart.bat', or is not visible to the current user. Ensure download destination folder exists (perhaps using win_file state=directory) before win_get_url runs.", "url": "ftp://ftp.fluffi/restStarter/restInstanceAutostart.bat"}
PLAY RECAP *********************************************************************
batrixwin : ok=17 changed=12 unreachable=0 failed=1 skipped=1 rescued=0 ignored=0
Setting up FUN (unclear doc) (2)
In the setting up Fun there is the following section
"Replace the following dummy textfiles with their respective binaries. A link to obtain these is contained in the dummy files:
/srv/fluffi/data/smb/files/initial/updatesWS2008
/srv/fluffi/data/ftp/files/initial/windows/ansible"
However is not really clear what it should be done here ... what files should be replaced?
/srv/fluffi/data/fluffiweb/app/static/bootstrap does not exist
After the commit cbc2cfd#diff-c65083bf5529e8821f2c7d386c933379, /srv/fluffi/data/fluffiweb/app/static/bootstrap does not exist in a fresh install. However, the generated html page still refers to it.
Examples quoted from view-source:http://web.fluffi:8880/ after a fresh install:
- The
headblock:
<head>
<title>FLUFFI - Home</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<!-- Bootstrap -->
<link href="/static/bootstrap/css/bootstrap.min.css?bootstrap=3.3.7.1.dev1" rel="stylesheet">
<!--[if lt IE 9]>
<script src="/static/html5shiv.min.js"></script>
<script src="/static/respond.min.js"></script>
<![endif]-->
<link href="/static/css/fluffi.css" rel="stylesheet" />
<link href="/static/images/favicon.ico" rel="shortcut icon">
<script src="/static/3rdParty/jquery/jquery-2.1.4.js"></script>
<script src="/static/3rdParty/socket.io-client/socket.io.slim.js"></script>
</head>- After
footertowards the end ofbody:
<footer class="container">
<p>© Siemens AG, Corporate Technology 2020 | FLUFFI 1.0</p>
</footer>
<script src="/static/javascript/fuzzjobNavigation.js" type="text/javascript"></script>
<script src="/static/bootstrap/jquery.min.js?bootstrap=3.3.7.1.dev1"></script>
<script src="/static/bootstrap/js/bootstrap.min.js?bootstrap=3.3.7.1.dev1"></script>
</body>check qemu dependencies
qemu compiles and runs without flex and bison, but we should investigate if not having them introduces any regressions - see #112
polemarch & ansible & adding new systems & variables
While adding new system to FUN, the variables defined in
/srv/fluffi/data/polenext/projects/1/hosts
in [windows:vars] category where not imported in polemarch. Had to add them manually in polemarch for the system to have the status ok.
Maybe i did something wrong along the way... but i'm thinking in the usage section
Tell FLUFFI about the system. To do so you have two options: either add it to ansible's hosts file (persistent), or use the Add System button in FLUFFI's web GUI. When adding it, just use the system's host name without any domain suffix
this category should be expanded a bit and have more description on what should be done also in polemarch.
building on linux RAM
In the section for compiling fluffi on linux it say "please install/allocate at least 6 GB of RAM", however for safety reasons i think should also recommend setting the swap file of 8GB.
Saying this because having ~8gb RAM for the machine and 4 gb swap, it may happen that at some point will run OOM and will crash when compiling!
Initial setup succeeds with stub binaries
Appatently, the initial setup step on windows succeeds in environments in which the binaries in /srv/fluffi/data/ftp/files/initial/windows/ansible were not replaced with actual files.
Please modify the /srv/fluffi/data/polenext/projects/1/roles/initial-windows* tasks to fail when the downloaded binaries are invalid (e.g. to small)
PREPARE_ENV flag
The flag has no effect on windows, it should be possible to have docker machines also on windows...
Add task schedule to run lm script at start up failed
[DEPRECATION WARNING]: Instead of sudo/sudo_user, use become/become_user and
make sure become_method is 'sudo' (default). This feature will be removed in
version 2.9. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
PLAY [linux, odroids] **********************************************************
skipping: no hosts matched
PLAY [windows] *****************************************************************
TASK [Gathering Facts] *********************************************************
ok: [batrixwin]
TASK [initial-windows-Timezone : Set timezone to 'Romance Standard Time' (GMT+01:00)] ***
ok: [batrixwin]
TASK [initial-windows-GFlags : Ensure source directory exists] *****************
ok: [batrixwin]
TASK [initial-windows-GFlags : Fetch GFlag zip file "ftp://ftp.fluffi/initial/windows/ansible/GFlags/GFlags.zip"] ***
changed: [batrixwin]
TASK [initial-windows-GFlags : Unzip GFlags] ***********************************
changed: [batrixwin]
TASK [initial-windows-PrepareEnvironment : Create a FLUFFI directory] **********
ok: [batrixwin]
TASK [initial-windows-PrepareEnvironment : Set ACL for C:\FLUFFI] **************
ok: [batrixwin]
TASK [initial-windows-PrepareEnvironment : Set AllowUnencrypted to True] *******
changed: [batrixwin]
TASK [initial-windows-GroupPolicy : Fetch LGPO from "ftp://ftp.fluffi/initial/windows/ansible/LGPO/LGPO.exe"] ***
changed: [batrixwin]
TASK [initial-windows-GroupPolicy : Copy registry.txt to remote machine via template] ***
changed: [batrixwin]
TASK [initial-windows-GroupPolicy : Execute LGPO.exe to generate group policy registry.pol] ***
changed: [batrixwin]
TASK [initial-windows-GroupPolicy : Execute LGPO.exe to apply group policy registry.pol] ***
changed: [batrixwin]
TASK [initial-windows-pagefile : Disabling pagefiles on windows] ***************
changed: [batrixwin]
TASK [initial-windows-Imdisk : Check architecture for x86 installer] ***********
skipping: [batrixwin]
TASK [initial-windows-Imdisk : Fetch ImDisk installer from "ftp://ftp.fluffi/initial/windows/ansible/ImDisk/ImDiskTk-x64.exe"] ***
changed: [batrixwin]
TASK [initial-windows-Imdisk : Install imdisk by executing ImDiskTk-x64.exe] ***
changed: [batrixwin]
TASK [initial-windows-Imdisk : Create new ramdisk] *****************************
changed: [batrixwin]
TASK [initial-windows-Imdisk : Ensure startup script directory exists] *********
changed: [batrixwin]
TASK [initial-windows-Imdisk : Copy startup script to machine] *****************
changed: [batrixwin]
TASK [initial-windows-Imdisk : Add task schedule to run script at start up] ****
fatal: [batrixwin]: FAILED! => {"changed": true, "msg": "failed to modify scheduled task: (17,8):UserId:"}
RUNNING HANDLER [initial-windows-GroupPolicy : Ensure Windows NTP service is running] ***
RUNNING HANDLER [initial-windows-GroupPolicy : Force Windows NTP time sync] ****
PLAY RECAP *********************************************************************
batrixwin : ok=18 changed=13 unreachable=0 failed=1 skipped=1 rescued=0 ignored=0
Small typo in Setting up Fun
There is a section for setting up file permissions "
Set file permissions correctly:
sudo chown -R 1000:1000 /srv/fluffi/data/{ftp/files,tfpt,smb/files} "
- tfpt is actually tftp
Evaluate the decision to prepend "dev-" to runner systems created using web interface
With current master, a runner system created using the web interface has two "names" that spell differently. Say I have just created a system called fluffi-0228-Linux3 using the web interface.
- In http://web.fluffi:8880/systems, we see a new table row with link
fluffi-0228-Linux3in the browser. But the target of that link ishttp://web.fluffi:8880/systems/view/dev-fluffi-0228-Linux3/linux. - In http://pole.fluffi:8888/#/host, we see a new host with name
dev-fluffi-0228-Linux3, but with the Ansible variableansible_ssh_hostset tofluffi-0228-Linux3.fluffi. - In MariaDB
systemstable, theNamecolumn of the new row isdev-fluffi-0228-Linux3.
This design can be quite confusing to starters and basically require manual editing of the DB content.
Moreover, if we restart the containers (without any modification to the file system or DB content), fluffi-0228-Linux3 won't show up in http://web.fluffi:8880/systems any more because of its "dev-" prefix in the DB. However, adding fluffi-0228-Linux3 again would not work and would result in the error message:
Error adding new system! System already exists (or check database)!
This issue is related to #98.
Configure Instances Tab in Linux group doesn't work
With the master at following commit, http://web.fluffi:8880/systems shows multiple groups: windows, linux and odroids.
On going to Linux group, we have 4 options.
The current state of the system is not reflected in 4] Configure Instances. There were some active fuzzjobs, each with a few TG, TR and TE, but the configurations shows 0 for each of them.
Besides, on changing the number of TG, TR and TE for a fuzzjob, an error pop-up box is displayed with the message "Error: Could not configure Instances!"
The terminal output at the time is the following:
fluffiweb_1 | 'NoneType' object has no attribute 'ID'
fluffiweb_1 | [pid: 16|app: 0|req: 153/226] 10.180.62.198 () {50 vars in 1150 bytes} [Thu Mar 19 11:54:4$
2020] POST /systems/configureSystemInstances/linux => generated 223 bytes in 6 msecs (HTTP/1.1 302) 5 hea
ders in 377 bytes (1 switches on core 0)
fluffiweb_1 | 10.180.62.198 - - [19/Mar/2020:11:54:40 -0400] "POST /systems/configureSystemInstances/linu
x HTTP/1.1" 302 223 "http://web.fluffi:8880/systems/view/linux/linux" "Mozilla/5.0 (X11; Ubuntu; Linux x86
_64; rv:74.0) Gecko/20100101 Firefox/74.0" "-"
fluffiweb_1 | [pid: 16|app: 0|req: 154/227] 10.180.62.198 () {44 vars in 911 bytes} [Thu Mar 19 11:54:41
2020] GET /systems => generated 20860 bytes in 549 msecs (HTTP/1.1 200) 4 headers in 247 bytes (1 switches
on core 0)
fluffiweb_1 | 10.180.62.198 - - [19/Mar/2020:11:54:41 -0400] "GET /systems HTTP/1.1" 200 20860 "http://we
b.fluffi:8880/systems/view/linux/linux" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:74.0) Gecko/20100101 F
irefox/74.0" "-"
Adding systems to FUN
Trying to add a system to FUN i get an message "Missing location!
To add systems to the database is a location necessary."
However don't really know what this represents and there is nothing mentioned in the Usage section!
setting up FUN (unclear)
Would help if there is a bit of continuity between step 2) (compiling) and step 3 (setting up).
-
is this this step "git clone --depth 1 --branch master https://github.com/siemens/fluffi.git ./fluffigit necessary? As this was already done on step 2?
-
would also help if the checkout folder is the same as previous ... here it is used "fluffigit"
-
touch /srv/fluffi/data/dnsmasq/dnsmasq.leases should be done after the rsync command otherwise results in error "touch: cannot touch '/srv/fluffi/data/dnsmasq/dnsmasq.leases': No such file or directory
" -
Would be helpful if examples on how to change the default configuration are provided . E.g. using other ip then 10.66.0.1
Usage documentation unclear
I set up the initial machine in vbox with 10.66.0.1 ip. Compiled everything and docker is running as i can access the web server through http://web.fluffi:8880/
Added a second machine vbox machine to the network with ip 10.66.0.2
Both machine can communicate with each other .. as from the second machine i can ping gm.fluffi and get a response.
(ps : seems that also on the second machine i needed to edit resolv.conf and also stop network-manager service for this to work)
- My understanding is that point 1) from the usage section refers to commands that should be done on the second machine, except the editing of the hosts file. Is this correct?
- as i am not familiar with ansible , an example will help in editing the host file. I'm guessing the user and pass edited there should be the user and pass set on the second machine (10.66.0.2)
- "chmod 777 /initialConfiguration.sh" should be changed to "chmod 777 initialConfiguration.sh" in the docs
- "/bin/bash /initialConfiguration.sh" should be changed to "/bin/bash initialConfiguration.sh" in the docs
- running initialConfiguration.sh ends up with error "sed: can't read /etc/netplan/01-netcfg.yaml: No such file or directory"
- adding a system from the web.fluffi ends up in 502 error bad gateway error. Also is not clear if the hostname of the second machine needs to have ".fluffi" suffix as when adding a new system there is a message "The extension .fluffi to reach the system is set automatically."
Order to start apps on agents
Is not really clear in what order LM, T* should be started ... and it seems one agent is waiting for another
Usage and starting agents
In the docs it is mentioned "Alternatively, you can connect to the system directly (SSH, RDP), and start the agents there." ...
Using only the interface LM, TR, TG and TE did not start automatically after deployment of fluffi ... so it seems to me that they can be started only by remote ssh/rdp
fluffi_fluffiweb_1 container error
Attached log(this is on refactor-and-deployment-and-doc branch):
*** Starting uWSGI 2.0.17 (64bit) on [Tue Apr 7 12:31:48 2020] ***
compiled with version: 6.4.0 on 27 March 2018 12:43:27
os: Linux-5.3.0-46-generic #38~18.04.1-Ubuntu SMP Tue Mar 31 04:17:56 UTC 2020
nodename: b6aec27acd92
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 3
current working directory: /app
detected binary path: /usr/sbin/uwsgi
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 3
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
Python version: 3.6.9 (default, Oct 17 2019, 12:14:22) [GCC 6.4.0]
Python main interpreter initialized at 0x5624d2ab9f40
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 1239640 bytes (1210 KB) for 16 cores
*** Operational MODE: preforking ***
2020-04-07 12:31:49,675 INFO success: uwsgi entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Traceback (most recent call last):
File "./main.py", line 12, in
from app import app
File "./app/init.py", line 21, in
app.config.from_object("config")
File "/usr/local/lib/python3.6/site-packages/flask/config.py", line 170, in from_object
obj = import_string(obj)
File "/usr/local/lib/python3.6/site-packages/werkzeug/utils.py", line 546, in import_string
2020-04-07 12:31:54,332 INFO exited: uwsgi (exit status 22; not expected)
import(import_name)
File "./config.py", line 18, in
+ "/fluffi_gm"
socket.gaierror: [Errno -3] Try again
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. GAME OVER ***
2020-04-07 12:31:55,334 INFO spawned: 'uwsgi' with pid 74
[uWSGI] getting INI configuration from /app/uwsgi.ini
[uWSGI] getting INI configuration from /etc/uwsgi/uwsgi.ini
;uWSGI instance configuration
[uwsgi]
cheaper = 2
processes = 16
ini = /app/uwsgi.ini
module = main
callable = app
wsgi-disable-file-wrapper = true
enable-threads = true
ini = /etc/uwsgi/uwsgi.ini
socket = /tmp/uwsgi.sock
chown-socket = nginx:nginx
chmod-socket = 664
hook-master-start = unix_signal:15 gracefully_kill_them_all
need-app = true
die-on-term = true
plugin = python3
show-config = true
;end of configuration
502 bad gatway Error while access Web fluffi (http://localhost:8880).
- Starting container de-attach mode
sudo docker-compose -f /srv/fluffi/docker-compose.yaml up --force-recreate
Saw error in log as "fluffiweb_1 | (pymysql.err.InternalError) Packet sequence number wrong - got 1 expected 0"
ubuntu@ubuntu1804:/srv/fluffi/data/fluffigm$ sudo docker-compose -f /srv/fluffi/docker-compose.yaml up --force-recreate
Recreating fluffi_smb_1 ... done
Recreating fluffi_fluffiweb_1 ... done
Recreating fluffi_dashsync_1 ... done
Recreating fluffi_grasync_1 ... done
Recreating fluffi_dnsmasq_1 ... done
Recreating fluffi_mon_1 ... done
Recreating fluffi_fluffigm_1 ... done
Recreating fluffi_ntp_1 ... done
Recreating fluffi_polenext_1 ... done
Recreating fluffi_mariadb_1 ... done
Recreating fluffi_ftp_1 ... done
Attaching to fluffi_dnsmasq_1, fluffi_ftp_1, fluffi_smb_1, fluffi_polenext_1, fluffi_dashsync_1, fluffi_grasync_1, fluffi_mariadb_1, fluffi_fluffiweb_1, fluffi_fluffigm_1, fluffi_ntp_1, fluffi_mon_1
dashsync_1 | WebSocket transport not available. Install eventlet or gevent and gevent-websocket for improved performance.
dashsync_1 | * Serving Flask app "main" (lazy loading)
dashsync_1 | * Environment: production
dashsync_1 | WARNING: This is a development server. Do not use it in a production deployment.
dashsync_1 | Use a production WSGI server instead.
dashsync_1 | * Debug mode: on
dashsync_1 | * Running on http://0.0.0.0:4000/ (Press CTRL+C to quit)
dashsync_1 | * Restarting with stat
dashsync_1 | WebSocket transport not available. Install eventlet or gevent and gevent-websocket for improved performance.
dashsync_1 | * Debugger is active!
dashsync_1 | * Debugger PIN: 478-190-491
fluffigm_1 | standard_init_linux.go:211: exec user process caused "no such file or directory"
dnsmasq_1 | dnsmasq: started, version 2.80 cachesize 150
dnsmasq_1 | dnsmasq: compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth no-DNSSEC loop-detect inotify dumpfile
dnsmasq_1 | dnsmasq-dhcp: DHCP, IP range 10.66.1.1 -- 10.66.1.254, lease time 6h
dnsmasq_1 | dnsmasq-dhcp: DHCP, static leases only on 10.66.0.0, lease time 1h
dnsmasq_1 | dnsmasq-tftp: TFTP root is /srv/tftp
dnsmasq_1 | dnsmasq: using local addresses only for domain fluffi
dnsmasq_1 | dnsmasq: reading /etc/resolv.conf
dnsmasq_1 | dnsmasq: using local addresses only for domain fluffi
dnsmasq_1 | dnsmasq: ignoring nameserver 10.66.0.1 - local interface
dnsmasq_1 | dnsmasq: read /etc/hosts - 1 addresses
dnsmasq_1 | dnsmasq-dhcp: read /etc/ethers - 0 addresses
dnsmasq_1 | dnsmasq: query[AAAA] db.fluffi from 172.18.0.7
dnsmasq_1 | dnsmasq: config db.fluffi is NODATA-IPv6
dnsmasq_1 | dnsmasq: query[A] db.fluffi from 172.18.0.7
dnsmasq_1 | dnsmasq: /etc/hosts db.fluffi is 10.66.0.1
fluffiweb_1 | Checking for script in /app/prestart.sh
fluffiweb_1 | There is no script /app/prestart.sh
fluffiweb_1 | /usr/lib/python2.7/site-packages/supervisor/options.py:298: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
fluffiweb_1 | 'Supervisord is running as root and it is searching '
fluffiweb_1 | 2020-04-22 12:21:12,420 CRIT Supervisor running as root (no user in config file)
fluffiweb_1 | 2020-04-22 12:21:12,420 INFO Included extra file "/etc/supervisor.d/supervisord.ini" during parsing
fluffiweb_1 | 2020-04-22 12:21:12,445 INFO RPC interface 'supervisor' initialized
fluffiweb_1 | 2020-04-22 12:21:12,445 CRIT Server 'unix_http_server' running without any HTTP authentication checking
fluffiweb_1 | 2020-04-22 12:21:12,446 INFO supervisord started with pid 1
mariadb_1 | 2020-04-22 12:21:10-04:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.2.31+maria~bionic started.
mariadb_1 | 2020-04-22 12:21:11-04:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
mariadb_1 | 2020-04-22 12:21:11-04:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.2.31+maria~bionic started.
mariadb_1 | 2020-04-22 12:21:12 140164411012800 [Note] mysqld (mysqld 10.2.31-MariaDB-1:10.2.31+maria~bionic) starting as process 1 ...
mariadb_1 | 2020-04-22 12:21:12 140164411012800 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb_1 | 2020-04-22 12:21:12 140164411012800 [Note] InnoDB: Uses event mutexes
mariadb_1 | 2020-04-22 12:21:12 140164411012800 [Note] InnoDB: Compressed tables use zlib 1.2.11
mariadb_1 | 2020-04-22 12:21:12 140164411012800 [Note] InnoDB: Using Linux native AIO
mariadb_1 | 2020-04-22 12:21:12 140164411012800 [Note] InnoDB: Number of pools: 1
mariadb_1 | 2020-04-22 12:21:12 140164411012800 [Note] InnoDB: Using SSE2 crc32 instructions
mariadb_1 | 2020-04-22 12:21:12 140164411012800 [Note] InnoDB: Initializing buffer pool, total size = 56.25G, instances = 50, chunk size = 128M
ntp_1 | 22 Apr 12:21:13 ntpd[1]: ntpd [email protected] (1): Starting
ntp_1 | 22 Apr 12:21:13 ntpd[1]: Command line: /usr/sbin/ntpd -d
ntp_1 | 22 Apr 12:21:13 ntpd[1]: proto: precision = 0.085 usec (-23)
ntp_1 | restrict default: KOD does nothing without LIMITED.
ntp_1 | Finished Parsing!!
ntp_1 | 22 Apr 12:21:13 ntpd[1]: restrict default: KOD does nothing without LIMITED.
ntp_1 | restrict: op 1 addr 0.0.0.0 mask 0.0.0.0 mflags 00000000 flags 000005d0
ntp_1 | restrict: op 1 addr :: mask :: mflags 00000000 flags 000005d0
ntp_1 | restrict: op 1 addr 10.66.0.0 mask 255.255.254.0 mflags 00000000 flags 00000190
ntp_1 | restrict: op 1 addr 127.0.0.1 mask 255.255.255.255 mflags 00000000 flags 00000000
ntp_1 | attempting to open log /var/log/ntp.log
ntp_1 | 22 Apr 12:21:13 ntpd[1]: switching logging to file /var/log/ntp.log
ntp_1 | move_fd: estimated max descriptors: 1048576, initial socket boundary: 16
ntp_1 | 22 Apr 12:21:13 ntpd[1]: Listen and drop on 0 v6wildcard [::]:123
ntp_1 | 22 Apr 12:21:13 ntpd[1]: Listen and drop on 1 v4wildcard 0.0.0.0:123
ntp_1 | 22 Apr 12:21:13 ntpd[1]: Listen normally on 2 lo 127.0.0.1:123
ntp_1 | restrict: op 1 addr 127.0.0.1 mask 255.255.255.255 mflags 00003000 flags 00000001
ntp_1 | 22 Apr 12:21:13 ntpd[1]: Listen normally on 3 eth0 172.18.0.6:123
ntp_1 | restrict: op 1 addr 172.18.0.6 mask 255.255.255.255 mflags 00003000 flags 00000001
ntp_1 | 22 Apr 12:21:13 ntpd[1]: Listening on routing socket on fd #20 for interface updates
ntp_1 | key_expire: at 0 associd 54804
ntp_1 | peer_clear: at 0 next 1 associd 54804 refid INIT
ntp_1 | event at 0 81.7.10.197 8011 81 mobilize assoc 54804
ntp_1 | newpeer: 172.18.0.6->81.7.10.197 mode 3 vers 4 poll 6 10 flags 0x1 0x1 ttl 0 key 00000000
ntp_1 | key_expire: at 0 associd 54805
ntp_1 | peer_clear: at 0 next 2 associd 54805 refid INIT
ntp_1 | event at 0 85.114.128.137 8011 81 mobilize assoc 54805
ntp_1 | newpeer: 172.18.0.6->85.114.128.137 mode 3 vers 4 poll 6 10 flags 0x1 0x1 ttl 0 key 00000000
ntp_1 | key_expire: at 0 associd 54806
ntp_1 | peer_clear: at 0 next 3 associd 54806 refid INIT
ntp_1 | event at 0 188.68.36.203 8011 81 mobilize assoc 54806
ntp_1 | newpeer: 172.18.0.6->188.68.36.203 mode 3 vers 4 poll 6 10 flags 0x1 0x1 ttl 0 key 00000000
ntp_1 | key_expire: at 0 associd 54807
ntp_1 | peer_clear: at 0 next 4 associd 54807 refid INIT
ntp_1 | event at 0 85.25.210.112 8011 81 mobilize assoc 54807
ntp_1 | newpeer: 172.18.0.6->85.25.210.112 mode 3 vers 4 poll 6 10 flags 0x1 0x1 ttl 0 key 00000000
ntp_1 | event at 0 0.0.0.0 c01d 0d kern kernel time sync enabled
ntp_1 | event at 0 0.0.0.0 c012 02 freq_set kernel 0.000 PPM
ntp_1 | event at 0 0.0.0.0 c011 01 freq_not_set
ntp_1 | event at 0 0.0.0.0 c016 06 restart
polenext_1 | 6:C 22 Apr 12:21:09.841 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
polenext_1 | 6:C 22 Apr 12:21:09.841 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=6, just started
polenext_1 | 6:C 22 Apr 12:21:09.841 # Configuration loaded
smb_1 | smbd version 4.11.5 started.
smb_1 | Copyright Andrew Tridgell and the Samba Team 1992-2019
smb_1 | daemon_ready: daemon 'smbd' finished starting up and ready to serve connections
fluffiweb_1 | 2020-04-22 12:21:13,448 INFO spawned: 'nginx' with pid 9
fluffiweb_1 | 2020-04-22 12:21:13,452 INFO spawned: 'uwsgi' with pid 10
fluffiweb_1 | [uWSGI] getting INI configuration from /app/uwsgi.ini
fluffiweb_1 | [uWSGI] getting INI configuration from /etc/uwsgi/uwsgi.ini
fluffiweb_1 |
fluffiweb_1 | ;uWSGI instance configuration
fluffiweb_1 | [uwsgi]
fluffiweb_1 | cheaper = 2
fluffiweb_1 | processes = 16
fluffiweb_1 | ini = /app/uwsgi.ini
fluffiweb_1 | module = main
fluffiweb_1 | callable = app
fluffiweb_1 | wsgi-disable-file-wrapper = true
fluffiweb_1 | enable-threads = true
fluffiweb_1 | ini = /etc/uwsgi/uwsgi.ini
fluffiweb_1 | socket = /tmp/uwsgi.sock
fluffiweb_1 | chown-socket = nginx:nginx
fluffiweb_1 | chmod-socket = 664
fluffiweb_1 | hook-master-start = unix_signal:15 gracefully_kill_them_all
fluffiweb_1 | need-app = true
fluffiweb_1 | die-on-term = true
fluffiweb_1 | plugin = python3
fluffiweb_1 | show-config = true
fluffiweb_1 | ;end of configuration
fluffiweb_1 |
fluffiweb_1 | *** Starting uWSGI 2.0.17 (64bit) on [Wed Apr 22 12:21:13 2020] ***
fluffiweb_1 | compiled with version: 6.4.0 on 27 March 2018 12:43:27
fluffiweb_1 | os: Linux-5.3.0-46-generic #38~18.04.1-Ubuntu SMP Tue Mar 31 04:17:56 UTC 2020
fluffiweb_1 | nodename: 642ed7e20ef4
fluffiweb_1 | machine: x86_64
fluffiweb_1 | clock source: unix
fluffiweb_1 | pcre jit disabled
fluffiweb_1 | detected number of CPU cores: 4
fluffiweb_1 | current working directory: /app
fluffiweb_1 | detected binary path: /usr/sbin/uwsgi
fluffiweb_1 | your memory page size is 4096 bytes
fluffiweb_1 | detected max file descriptor number: 1048576
fluffiweb_1 | lock engine: pthread robust mutexes
fluffiweb_1 | thunder lock: disabled (you can enable it with --thunder-lock)
fluffiweb_1 | uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 3
fluffiweb_1 | uWSGI running as root, you can use --uid/--gid/--chroot options
fluffiweb_1 | *** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
fluffiweb_1 | Python version: 3.6.9 (default, Oct 17 2019, 12:14:22) [GCC 6.4.0]
fluffiweb_1 | Python main interpreter initialized at 0x555e08298f40
fluffiweb_1 | uWSGI running as root, you can use --uid/--gid/--chroot options
fluffiweb_1 | *** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
fluffiweb_1 | python threads support enabled
fluffiweb_1 | your server socket listen backlog is limited to 100 connections
fluffiweb_1 | your mercy for graceful operations on workers is 60 seconds
fluffiweb_1 | mapped 1239640 bytes (1210 KB) for 16 cores
fluffiweb_1 | *** Operational MODE: preforking ***
fluffi_fluffigm_1 exited with code 1
ntp_1 | peer_xmit: at 1 172.18.0.6->81.7.10.197 mode 3 len 48 xmt 0xe24aed7a.0fbdd5a7
ntp_1 | auth_agekeys: at 1 keys 0 expired 0
dnsmasq_1 | dnsmasq: query[A] grafana.com from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
dnsmasq_1 | dnsmasq: query[AAAA] grafana.com from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
dnsmasq_1 | dnsmasq: query[A] grafana.com from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
dnsmasq_1 | dnsmasq: query[AAAA] grafana.com from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
dnsmasq_1 | dnsmasq: query[A] grafana.com.fluffi from 172.18.0.4
dnsmasq_1 | dnsmasq: config grafana.com.fluffi is NXDOMAIN
dnsmasq_1 | dnsmasq: query[AAAA] grafana.com.fluffi from 172.18.0.4
dnsmasq_1 | dnsmasq: config grafana.com.fluffi is NXDOMAIN
dnsmasq_1 | dnsmasq: query[A] grafana.com.localdomain from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
dnsmasq_1 | dnsmasq: query[AAAA] grafana.com.localdomain from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
dnsmasq_1 | dnsmasq: query[AAAA] grafana.com.localdomain from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
dnsmasq_1 | dnsmasq: query[A] grafana.com.localdomain from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
fluffiweb_1 | 2020-04-22 12:21:14,721 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
fluffiweb_1 | 2020-04-22 12:21:14,721 INFO success: uwsgi entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
dnsmasq_1 | dnsmasq: query[A] db.fluffi from 172.18.0.3
dnsmasq_1 | dnsmasq: /etc/hosts db.fluffi is 10.66.0.1
dnsmasq_1 | dnsmasq: query[A] db.fluffi from 172.18.0.3
dnsmasq_1 | dnsmasq: /etc/hosts db.fluffi is 10.66.0.1
dnsmasq_1 | dnsmasq: query[A] ftp.fluffi from 172.18.0.3
dnsmasq_1 | dnsmasq: /etc/hosts ftp.fluffi is 10.66.0.1
dnsmasq_1 | dnsmasq: query[A] mon.fluffi from 172.18.0.3
dnsmasq_1 | dnsmasq: /etc/hosts mon.fluffi is 10.66.0.1
ntp_1 | peer_xmit: at 2 172.18.0.6->85.114.128.137 mode 3 len 48 xmt 0xe24aed7b.0f2cc11d
ntp_1 | event at 2 85.114.128.137 8014 84 reachable
ntp_1 | clock_filter: n 1 off -0.029517 del 0.093327 dsp 7.937501 jit 0.000000
ntp_1 | peer_xmit: at 3 172.18.0.6->188.68.36.203 mode 3 len 48 xmt 0xe24aed7c.0f22c4e0
fluffiweb_1 | WSGI app 0 (mountpoint='') ready in 3 seconds on interpreter 0x555e08298f40 pid: 10 (default app)
fluffiweb_1 | uWSGI running as root, you can use --uid/--gid/--chroot options
fluffiweb_1 | *** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
fluffiweb_1 | *** uWSGI is running in multiple interpreter mode ***
fluffiweb_1 | spawned uWSGI master process (pid: 10)
fluffiweb_1 | spawned uWSGI worker 1 (pid: 15, cores: 1)
fluffiweb_1 | spawned uWSGI worker 2 (pid: 16, cores: 1)
fluffiweb_1 | running "unix_signal:15 gracefully_kill_them_all" (master-start)...
mariadb_1 | 2020-04-22 12:21:16 140164411012800 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1 | 2020-04-22 12:21:16 140099627861760 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
mariadb_1 | 2020-04-22 12:21:16 140164411012800 [Note] InnoDB: Highest supported file format is Barracuda.
ntp_1 | peer_xmit: at 4 172.18.0.6->85.25.210.112 mode 3 len 48 xmt 0xe24aed7d.0f1f1b8f
ntp_1 | event at 4 85.25.210.112 8014 84 reachable
ntp_1 | clock_filter: n 1 off -0.030662 del 0.103061 dsp 7.937501 jit 0.000000
mariadb_1 | 2020-04-22 12:21:17 140164411012800 [Note] InnoDB: 128 out of 128 rollback segments are active.
mariadb_1 | 2020-04-22 12:21:17 140164411012800 [Note] InnoDB: Creating shared tablespace for temporary tables
mariadb_1 | 2020-04-22 12:21:17 140164411012800 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
mariadb_1 | 2020-04-22 12:21:17 140164411012800 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mariadb_1 | 2020-04-22 12:21:17 140164411012800 [Note] InnoDB: Waiting for purge to start
mariadb_1 | 2020-04-22 12:21:17 140164411012800 [Note] InnoDB: 5.7.29 started; log sequence number 1615791
mariadb_1 | 2020-04-22 12:21:17 140100692342528 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
mariadb_1 | 2020-04-22 12:21:17 140100692342528 [Note] InnoDB: Buffer pool(s) load completed at 200422 12:21:17
mariadb_1 | 2020-04-22 12:21:17 140164411012800 [Note] Plugin 'FEEDBACK' is disabled.
mariadb_1 | 2020-04-22 12:21:17 140164411012800 [Note] Server socket created on IP: '::'.
mariadb_1 | 2020-04-22 12:21:17 140164411012800 [Warning] 'user' entry 'root@f70f48c5732e' ignored in --skip-name-resolve mode.
mariadb_1 | 2020-04-22 12:21:17 140164411012800 [Warning] 'proxies_priv' entry '@% root@f70f48c5732e' ignored in --skip-name-resolve mode.
mariadb_1 | 2020-04-22 12:21:17 140164411012800 [Note] Reading of all Master_info entries succeeded
mariadb_1 | 2020-04-22 12:21:17 140164411012800 [Note] Added new Master_info '' to hash table
mariadb_1 | 2020-04-22 12:21:17 140164411012800 [Note] mysqld: ready for connections.
mariadb_1 | Version: '10.2.31-MariaDB-1:10.2.31+maria~bionic' socket: '/var/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
fluffi_fluffigm_1 exited with code 1
dnsmasq_1 | dnsmasq: query[A] usage.influxdata.com from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
dnsmasq_1 | dnsmasq: query[AAAA] usage.influxdata.com from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
dnsmasq_1 | dnsmasq: query[AAAA] usage.influxdata.com from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
dnsmasq_1 | dnsmasq: query[A] usage.influxdata.com from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
dnsmasq_1 | dnsmasq: query[A] usage.influxdata.com.fluffi from 172.18.0.4
dnsmasq_1 | dnsmasq: config usage.influxdata.com.fluffi is NXDOMAIN
dnsmasq_1 | dnsmasq: query[AAAA] usage.influxdata.com.fluffi from 172.18.0.4
dnsmasq_1 | dnsmasq: config usage.influxdata.com.fluffi is NXDOMAIN
dnsmasq_1 | dnsmasq: query[AAAA] usage.influxdata.com.localdomain from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
dnsmasq_1 | dnsmasq: query[A] usage.influxdata.com.localdomain from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
dnsmasq_1 | dnsmasq: query[AAAA] usage.influxdata.com.localdomain from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
dnsmasq_1 | dnsmasq: query[A] usage.influxdata.com.localdomain from 172.18.0.4
dnsmasq_1 | dnsmasq: config error is REFUSED
polenext_1 | Operations to perform:
polenext_1 | Apply all migrations: admin, auth, authtoken, contenttypes, django_celery_beat, main, sessions
polenext_1 | Running migrations:
polenext_1 | Applying contenttypes.0001_initial... OK
polenext_1 | Applying auth.0001_initial... OK
polenext_1 | Applying admin.0001_initial... OK
polenext_1 | Applying admin.0002_logentry_remove_auto_add... OK
polenext_1 | Applying admin.0003_logentry_add_action_flag_choices... OK
fluffi_fluffigm_1 exited with code 1
polenext_1 | Applying contenttypes.0002_remove_content_type_name... OK
polenext_1 | Applying auth.0002_alter_permission_name_max_length... OK
polenext_1 | Applying auth.0003_alter_user_email_max_length... OK
polenext_1 | Applying auth.0004_alter_user_username_opts... OK
polenext_1 | Applying auth.0005_alter_user_last_login_null... OK
polenext_1 | Applying auth.0006_require_contenttypes_0002... OK
polenext_1 | Applying auth.0007_alter_validators_add_error_messages... OK
polenext_1 | Applying auth.0008_alter_user_username_max_length... OK
polenext_1 | Applying auth.0009_alter_user_last_name_max_length... OK
polenext_1 | Applying auth.0010_alter_group_name_max_length... OK
polenext_1 | Applying auth.0011_update_proxy_permissions... OK
polenext_1 | Applying authtoken.0001_initial... OK
polenext_1 | Applying authtoken.0002_auto_20160226_1747... OK
polenext_1 | Applying django_celery_beat.0001_initial... OK
polenext_1 | Applying django_celery_beat.0002_auto_20161118_0346... OK
polenext_1 | Applying django_celery_beat.0003_auto_20161209_0049... OK
polenext_1 | Applying django_celery_beat.0004_auto_20170221_0000... OK
polenext_1 | Applying django_celery_beat.0005_add_solarschedule_events_choices_squashed_0009_merge_20181012_1416... OK
polenext_1 | Applying django_celery_beat.0006_periodictask_priority... OK
dnsmasq_1 | dnsmasq: query[A] db.fluffi from 172.18.0.7
dnsmasq_1 | dnsmasq: /etc/hosts db.fluffi is 10.66.0.1
dnsmasq_1 | dnsmasq: query[AAAA] db.fluffi from 172.18.0.7
dnsmasq_1 | dnsmasq: config db.fluffi is NODATA-IPv6
polenext_1 | Execute: /usr/local/bin/pyuwsgi --enable-threads --master --http=:8080 --daemonize=/tmp/polemarch/logs/polemarch_web.log /usr/local/lib/python3.6/dist-packages/polemarch/web.ini --static-map /static=/usr/local/lib/python3.6/dist-packages/polemarch/static --static-map /static=/usr/local/lib/python3.6/dist-packages/vstutils/static --static-map /static=/usr/local/lib/python3.6/dist-packages/django/contrib/admin/static --static-map /static=/usr/local/lib/python3.6/dist-packages/drf_yasg/static --static-map /static=/usr/local/lib/python3.6/dist-packages/rest_framework/static --attach-daemon2 stopsignal=15,reloadsignal=1,exec=/usr/bin/python -m celery worker --app=polemarch.wapp:app --loglevel=INFO --logfile=/tmp/polemarch/logs/polemarch_worker.log --pidfile=/tmp/polemarch/pid/polemarch_worker.pid --autoscale=8\,1 --hostname=root@%h --beat --queues=celery
polenext_1 | Applying main.0001_v2... OK
polenext_1 | [uWSGI] getting INI configuration from /usr/local/lib/python3.6/dist-packages/polemarch/web.ini
polenext_1 | [uWSGI] getting INI configuration from /usr/local/lib/python3.6/dist-packages/polemarch/main/settings.ini
polenext_1 | *** WARNING: Can't find section "uwsgi" in INI configuration file /usr/local/lib/python3.6/dist-packages/polemarch/main/settings.ini ***
polenext_1 | [uWSGI] getting INI configuration from /etc/polemarch/settings.ini
polenext_1 | [uwsgi-static] added mapping for /static => /usr/local/lib/python3.6/dist-packages/polemarch/static
polenext_1 | [uwsgi-static] added mapping for /static => /usr/local/lib/python3.6/dist-packages/vstutils/static
polenext_1 | [uwsgi-static] added mapping for /static => /usr/local/lib/python3.6/dist-packages/django/contrib/admin/static
polenext_1 | [uwsgi-static] added mapping for /static => /usr/local/lib/python3.6/dist-packages/drf_yasg/static
polenext_1 | [uwsgi-static] added mapping for /static => /usr/local/lib/python3.6/dist-packages/rest_framework/static
fluffi_fluffigm_1 exited with code 1
polenext_1 | Wait until polemarch is available
polenext_1 | Polemarch is now available
polenext_1 | Add Project
fluffi_fluffigm_1 exited with code 1
polenext_1 | Applying main.0002_modules_and_rename... OK
polenext_1 | Applying main.0003_models_optimization... OK
polenext_1 | Applying main.0004_projecttemplate... OK
polenext_1 | Applying main.0005_history_on_delete... OK
polenext_1 | Applying main.0006_migrate_django_22... OK
polenext_1 | Applying sessions.0001_initial... OK
polenext_1 | <Response [201]>
polenext_1 | Add Inventory
dnsmasq_1 | dnsmasq: query[AAAA] db.fluffi from 172.18.0.7
dnsmasq_1 | dnsmasq: config db.fluffi is NODATA-IPv6
dnsmasq_1 | dnsmasq: query[A] db.fluffi from 172.18.0.7
dnsmasq_1 | dnsmasq: /etc/hosts db.fluffi is 10.66.0.1
polenext_1 | Add Periodic Task CheckHostAlive
polenext_1 | <Response [201]>
polenext_1 | Add Periodic Task CleanHistory
polenext_1 | <Response [201]>
polenext_1 | Add Periodic Task ManageAgents
fluffi_fluffigm_1 exited with code 1
polenext_1 | <Response [201]>
polenext_1 | Send Sync
polenext_1 | Initialization Finished!
mon_1 | % Total % Received % Xferd Average Speed Time Time Time Current
mon_1 | Dload Upload Total Spent Left Speed
100 45 100 34 100 11 918 297 --:--:-- --:--:-- --:--:-- 1216
mon_1 | HTTP/1.1 200 OK
mon_1 | Cache-Control: no-cache
mon_1 | Content-Type: application/json
mon_1 | Expires: -1
mon_1 | Pragma: no-cache
mon_1 | Date: Wed, 22 Apr 2020 16:21:38 GMT
mon_1 | Content-Length: 34
mon_1 |
dnsmasq_1 | dnsmasq: query[A] db.fluffi from 172.18.0.7
dnsmasq_1 | dnsmasq: /etc/hosts db.fluffi is 10.66.0.1
dnsmasq_1 | dnsmasq: query[AAAA] db.fluffi from 172.18.0.7
dnsmasq_1 | dnsmasq: config db.fluffi is NODATA-IPv6
polenext_1 | Operations to perform:
polenext_1 | Apply all migrations: admin, auth, authtoken, contenttypes, django_celery_beat, main, sessions
polenext_1 | Running migrations:
polenext_1 | No migrations to apply.
dnsmasq_1 | dnsmasq: query[AAAA] mon.fluffi from 172.18.0.3
dnsmasq_1 | dnsmasq: config mon.fluffi is NODATA-IPv6
dnsmasq_1 | dnsmasq: query[A] mon.fluffi from 172.18.0.3
dnsmasq_1 | dnsmasq: /etc/hosts mon.fluffi is 10.66.0.1
fluffiweb_1 | (pymysql.err.InternalError) Packet sequence number wrong - got 1 expected 0
fluffiweb_1 | (Background on this error at: http://sqlalche.me/e/2j85)
fluffi_fluffigm_1 exited with code 1
polenext_1 | Execute: /usr/local/bin/pyuwsgi --enable-threads --master --http=:8080 --daemonize=/tmp/polemarch/logs/polemarch_web.log /usr/local/lib/python3.6/dist-packages/polemarch/web.ini --static-map /static=/usr/local/lib/python3.6/dist-packages/polemarch/static --static-map /static=/usr/local/lib/python3.6/dist-packages/vstutils/static --static-map /static=/usr/local/lib/python3.6/dist-packages/django/contrib/admin/static --static-map /static=/usr/local/lib/python3.6/dist-packages/drf_yasg/static --static-map /static=/usr/local/lib/python3.6/dist-packages/rest_framework/static --attach-daemon2 stopsignal=15,reloadsignal=1,exec=/usr/bin/python -m celery worker --app=polemarch.wapp:app --loglevel=INFO --logfile=/tmp/polemarch/logs/polemarch_worker.log --pidfile=/tmp/polemarch/pid/polemarch_worker.pid --autoscale=8\,1 --hostname=root@%h --beat --queues=celery
polenext_1 | [uWSGI] getting INI configuration from /usr/local/lib/python3.6/dist-packages/polemarch/web.ini
polenext_1 | [uWSGI] getting INI configuration from /usr/local/lib/python3.6/dist-packages/polemarch/main/settings.ini
polenext_1 | *** WARNING: Can't find section "uwsgi" in INI configuration file /usr/local/lib/python3.6/dist-packages/polemarch/main/settings.ini ***
polenext_1 | [uWSGI] getting INI configuration from /etc/polemarch/settings.ini
polenext_1 | [uwsgi-static] added mapping for /static => /usr/local/lib/python3.6/dist-packages/polemarch/static
polenext_1 | [uwsgi-static] added mapping for /static => /usr/local/lib/python3.6/dist-packages/vstutils/static
polenext_1 | [uwsgi-static] added mapping for /static => /usr/local/lib/python3.6/dist-packages/django/contrib/admin/static
polenext_1 | [uwsgi-static] added mapping for /static => /usr/local/lib/python3.6/dist-packages/drf_yasg/static
polenext_1 | [uwsgi-static] added mapping for /static => /usr/local/lib/python3.6/dist-packages/rest_framework/static
dnsmasq_1 | dnsmasq: query[AAAA] db.fluffi from 172.18.0.7
dnsmasq_1 | dnsmasq: config db.fluffi is NODATA-IPv6
dnsmasq_1 | dnsmasq: query[A] db.fluffi from 172.18.0.7
dnsmasq_1 | dnsmasq: /etc/hosts db.fluffi is 10.66.0.1
polenext_1 | Wait until polemarch is available
polenext_1 | Polemarch is now available
polenext_1 | Polemarch already initialized
polenext_1 | Polemarch not available retrying...
polenext_1 | Polemarch already initialized
polenext_1 | Polemarch not available retrying...
polenext_1 | Polemarch already initialized
polenext_1 | Polemarch not available retrying...
polenext_1 | Polemarch already initialized
polenext_1 | Polemarch not available retrying...
polenext_1 | Polemarch already initialized
polenext_1 | Polemarch not available retrying...
polenext_1 | Polemarch already initialized
polenext_1 | Polemarch not available retrying...
polenext_1 | Polemarch already initialized
polenext_1 | Polemarch not available retrying...
polenext_1 | Polemarch already initialized
polenext_1 | Polemarch not available retrying...
polenext_1 | Polemarch already initialized
polenext_1 | Polemarch not available retrying...
dnsmasq_1 | dnsmasq: query[A] db.fluffi from 172.18.0.7
dnsmasq_1 | dnsmasq: /etc/hosts db.fluffi is 10.66.0.1
dnsmasq_1 | dnsmasq: query[AAAA] db.fluffi from 172.18.0.7
dnsmasq_1 | dnsmasq: config db.fluffi is NODATA-IPv6
polenext_1 | Polemarch already initialized
polenext_1 | Polemarch not available retrying...
polenext_1 | Polemarch already initialized
polenext_1 | Polemarch not available retrying...
polenext_1 | Polemarch already initialized
polenext_1 | Polemarch not availab
Deploy SUT/Dependency->Select Package doesn't work for Linux Machines
With the master at following commit, http://web.fluffi:8880/systems shows multiple groups: windows, linux and odroids.
Clicking on any of the machines in Linux group takes us to a tab which has 4 options. (I guess the 4th option doesn't exist anymore as per the latest master).
For 3] Deplot SUT/Dependency, 'Deploy' option in 'Select Fuzzjob:' doesn't work.
I am also attaching the docker-compose logs when I press the button:
fluffiweb_1 | Traceback (most recent call last):
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2309, in call
fluffiweb_1 | return self.wsgi_app(environ, start_response)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2295, in wsgi_app
fluffiweb_1 | response = self.handle_exception(e)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1741, in handle_exception
fluffiweb_1 | reraise(exc_type, exc_value, tb)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
fluffiweb_1 | raise value
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
fluffiweb_1 | response = self.full_dispatch_request()
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
fluffiweb_1 | rv = self.handle_user_exception(e)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
fluffiweb_1 | reraise(exc_type, exc_value, tb)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
fluffiweb_1 | raise value
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
fluffiweb_1 | rv = self.dispatch_request()
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
fluffiweb_1 | return self.view_functionsrule.endpoint
fluffiweb_1 | File "./app/views.py", line 1511, in viewInstallFuzzjobPackage
fluffiweb_1 | name=selectedFuzzJob).join(models.FuzzjobDeploymentPackages).filter(
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2243, in join
fluffiweb_1 | from_joinpoint=from_joinpoint,
fluffiweb_1 | File "", line 2, in _join
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/base.py", line 220, in generate
fluffiweb_1 | fn(self, *args[1:], **kw)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2422, in _join
fluffiweb_1 | left, right, onclause, prop, create_aliases, outerjoin, full
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2445, in _join_left_to_right
fluffiweb_1 | ) = self._join_determine_implicit_left_side(left, right, onclause)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2578, in _join_determine_implicit_left_side
fluffiweb_1 | "Can't determine which FROM clause to join "
fluffiweb_1 | sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Try adding an explicit ON clause to help resolve the ambiguity.
fluffiweb_1 | [pid: 16|app: 0|req: 137/455] 10.180.62.198 () {50 vars in 1073 bytes} [Mon Apr 6 20:18:04 2020] POST /systems/view/fluffi-0228-Linux1/deployFuzzPackage => generated 0 bytes in 8 msecs (HTTP/1.1 500) 0 headers in 0 bytes (0 switches on core 0)
fluffiweb_1 | 2020/04/06 20:18:04 [error] 12#12: *875 upstream prematurely closed connection while reading response header from upstream, client: 10.180.62.198, server: , request: "POST /systems/view/fluffi-0228-Linux1/deployFuzzPackage HTTP/1.1", upstream: "uwsgi://unix:///tmp/uwsgi.sock:", host: "web.fluffi:8880", referrer: "http://web.fluffi:8880/systems/view/fluffi-0228-Linux1/linux"
fluffiweb_1 | 10.180.62.198 - - [06/Apr/2020:20:18:04 -0400] "POST /systems/view/fluffi-0228-Linux1/deployFuzzPackage HTTP/1.1" 502 173 "http://web.fluffi:8880/systems/view/fluffi-0228-Linux1/linux" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:74.0) Gecko/20100101 Firefox/74.0" "-"
restrict logging repetition
When something goes wrong, FLUFFI generates so many log entries per second that it gets hard to see what is actually going wrong.
Maybe we need to implement some form of rate limiting or backing of if the message is the same one again and again.
ERROR: for fluffigm Cannot start service fluffigm: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"./GlobalManager\": stat ./GlobalManager: no such file or directory": unknown"
While starting container getting error as
"ERROR: for fluffigm Cannot start service fluffigm: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: "./GlobalManager": stat ./GlobalManager: no such file or directory": unknown"
`ubuntu@ubuntu1804:$ cd FLUFFI/srv/fluffi//FLUFFI/srv/fluffi$ sudo docker-compose -f /srv/fluffi/docker-compose.yaml up -d --force-recreate
ubuntu@ubuntu1804:
Removing fluffi_fluffigm_1
Recreating fluffi_mon_1 ...
Recreating fluffi_polenext_1 ...
Recreating fluffi_fluffiweb_1 ...
Recreating fluffi_dnsmasq_1 ... done
Recreating fluffi_polenext_1 ... done
Recreating fluffi_mon_1 ... done
Recreating fluffi_smb_1 ...
Recreating fluffi_fluffiweb_1 ... done
Recreating 81e2bb49e3a6_fluffi_fluffigm_1 ... error
Recreating fluffi_dashsync_1 ... done
Recreating fluffi_grasync_1 ... done
Recreating fluffi_smb_1 ... done
Recreating fluffi_ftp_1 ... done
unkRecreating fluffi_ntp_1 ... done
ERROR: for fluffigm Cannot start service fluffigm: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: "./GlobalManager": stat ./GlobalManager: no such file or directory": unknown
ERROR: Encountered errors while bringing up the project.
ubuntu@ubuntu1804:/FLUFFI/srv/fluffi$ cd data/fluffigm//FLUFFI/srv/fluffi/data/fluffigm$ ls
ubuntu@ubuntu1804:
GlobalManager libmariadb.so.3 libsharedmemipc.so libzmq.so.5
ubuntu@ubuntu1804:/FLUFFI/srv/fluffi/data/fluffigm$ nano GlobalManager/FLUFFI/srv/fluffi/data/fluffigm$ `
ubuntu@ubuntu1804:
As per doc I have already added GlobalManager components into /srv/fluffi/data/fluffigm/
Cannot add system when starting from scratch
When starting with a new database and not configuring any hosts, "Add system" fails with an error message:
Deleting a system from web.fluffi buggy
Trying to delete a system from web.fluffi ends up with error. Sometime happened that i could not see it on web.fluffi... but the system was in the database and could not add it again with the same name.
172.18.0.1 - - [16/Apr/2020:11:53:32 +0300] "GET / HTTP/1.0" 200 9066 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "-"
[pid: 119|app: 0|req: 19/33] 172.18.0.1 () {42 vars in 738 bytes} [Thu Apr 16 11:53:32 2020] GET / => generated 9066 bytes in 33 msecs (HTTP/1.0 200) 2 headers in 81 bytes (1 switches on core 0)
[pid: 130|app: 0|req: 7/34] 172.18.0.1 () {44 vars in 786 bytes} [Thu Apr 16 11:53:47 2020] GET /systems => generated 22078 bytes in 933 msecs (HTTP/1.0 200) 3 headers in 96 bytes (1 switches on core 0)
172.18.0.1 - - [16/Apr/2020:11:53:48 +0300] "GET /systems HTTP/1.0" 200 22078 "http://web.fluffi/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "-"
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2309, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2295, in wsgi_app
response = self.handle_exception(e)
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1741, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
raise value
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
raise value
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "./app/views.py", line 1247, in removeSystemFromPolemarch
removeResult = ANSIBLE_REST_CONNECTOR.removeSystem(hostName)
File "./app/utils/ansible.py", line 156, in removeSystem
2020/04/16 11:53:58 [error] 12#12: *127 upstream prematurely closed connection while reading response header from upstream, client: 172.18.0.1, server: , request: "GET /systems/removeSystem/conan HTTP/1.0", upstream: "uwsgi://unix:///tmp/uwsgi.sock:", host: "localhost:8880", referrer: "http://web.fluffi/systems"
for system in systems:
TypeError: 'NoneType' object is not iterable
[pid: 130|app: 0|req: 8/35] 172.18.0.1 () {44 vars in 831 bytes} [Thu Apr 16 11:53:53 2020] GET /systems/removeSystem/conan => generated 0 bytes in 5025 msecs (HTTP/1.0 500) 0 headers in 0 bytes (0 switches on core 0)
172.18.0.1 - - [16/Apr/2020:11:53:58 +0300] "GET /systems/removeSystem/conan HTTP/1.0" 502 173 "http://web.fluffi/systems" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "-"
Prerequisites needs more info
Would be nice if in prerequisites is enumerated everything that is needed for building to eliminate wait time for rebuilding in case something went wrong or is not installed.
E.g. :
- VS2017 with v141 build tools
- Go with https://github.com/go-sql-driver/mysql driver
- Cygwin with needed packages (make, cmake, gcc-,mingw64-i686-, mingw64-x86_64-*)
- Perl
Fuzzjobs sync systems failed
I keep getting the error
"Sync systems failed!
Something went wrong at the inital synchronization between PoleMarch and the database. Pleasy retry!"
can we somehow allow an anonymously writable ftp root?
otherwise we cannot use ftp to create new subdirectories and files in root, as mentioned in #103
Missing dependencies info "smbclient"
In usage section, there is :
"smbclient '//smb.fluffi/install' -c 'cd initial; get MAC2Host.csv; get initialConfiguration.sh' -U"
At least on ubuntu18.04 desktop, this is not installed by default ... anyway because of the ip changing, and dns resolving stuff by the time i arrived to this part the machine had no internet so had to remake it again with inet :)
Originally posted by @joyride9999 in #112 (comment)
Declare fluffi as local domain in dnsmasq
Somewhere around this line:
we can declare
fluffi to be a local domain:
local=/fluffi/For example, dnsmasq will reply to AAAA queries with NODATA-IPv6 instead of REFUSED after this change. Example output after applying the proposed change:
dnsmasq_1 | dnsmasq: query[A] ftp.fluffi from 10.180.62.166
dnsmasq_1 | dnsmasq: /etc/hosts ftp.fluffi is 10.180.62.169
dnsmasq_1 | dnsmasq: query[AAAA] ftp.fluffi from 10.180.62.166
dnsmasq_1 | dnsmasq: config ftp.fluffi is NODATA-IPv6
As another benefit, a dig www.fluffi would not generate a storm of queries. (Note that www.fluffi does not exist in the dnsmasq hosts file.)
In a private network, whether declaring fluffi as local or not in dnsmasq would give rise to the same behavior because there is no upstream DNS resolver. Therefore, I think the proposed change is safe.
The real benefit of the proposed change is that it ensures the query issuer receives the correct response and can act accordingly. For example, the query issuer could be systemd in a Linux runner. Here is how we currently set up some Linux VM runners so that both *.fluffi and public Internet hostnames resolve:
# Add Fluffi Master as DNS server
sudo mkdir -p /etc/systemd/resolved.conf.d
cat <<HERE | sudo tee /etc/systemd/resolved.conf.d/fluffi.conf >/dev/null
[Resolve]
# Tell systemd to use Fluffi master for DNS queries in the Global scope
DNS=10.180.62.169
# Search .fluffi; this will also adjust "search" in /etc/resolv.conf
Domains=fluffi
HERE
sudo systemctl restart systemd-resolved.serviceP.S. The reason for this runner DNS setup is because we do not intend to run our own apt.fluffi. In fact, a runner in a truly private network cannot pass the "Initial setup" step on the web interface because it requires access to some apt repository to run:
Description for automatically starting LM
In the usage section there is this phrase
"
IMPORTANT: When managing agents with the agent manager, you need to enable the agent manager by setting it to ACTIVE in the Options dialog on the Systems tab."
I would change it to
"IMPORTANT: When managing agents with the agent manager, you need to enable the agent manager by setting it to ACTIVE from the Settings button on the Systems tab"
Deploy fluffi reports OK but has error
PLAY [windows] *****************************************************************
TASK [Gathering Facts] *********************************************************
ok: [batrix3]
TASK [initial-windows-Timezone : Set timezone to 'Romance Standard Time' (GMT+01:00)] ***
ok: [batrix3]
TASK [initial-windows-GFlags : Ensure source directory exists] *****************
ok: [batrix3]
TASK [initial-windows-GFlags : Fetch GFlag zip file "ftp://ftp.fluffi/initial/windows/ansible/GFlags/GFlags.zip"] ***
changed: [batrix3]
TASK [initial-windows-GFlags : Unzip GFlags] ***********************************
changed: [batrix3]
TASK [initial-windows-PrepareEnvironment : Create a FLUFFI directory] **********
ok: [batrix3]
TASK [initial-windows-PrepareEnvironment : Set ACL for C:\FLUFFI] **************
ok: [batrix3]
TASK [initial-windows-PrepareEnvironment : Set AllowUnencrypted to True] *******
fatal: [batrix3]: FAILED! => {"changed": true, "cmd": "Set-Item -Path WSMan:\localhost\Service\AllowUnencrypted -Value $true", "delta": "0:00:03.234031", "end": "2020-04-16 09:33:04.872752", "msg": "non-zero return code", "rc": 1, "start": "2020-04-16 09:33:01.638721", "stderr": "Set-Item : WinRM firewall exception will not work since one of the network connection types on this machine is set to \r\nPublic. Change the network connection type to either Domain or Private and try again. \r\nAt line:1 char:65\r\n+ ... ing $false; Set-Item -Path WSMan:\localhost\Service\AllowUnencrypted ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: (:) [Set-Item], InvalidOperationException\r\n + FullyQualifiedErrorId : System.InvalidOperationException,Microsoft.PowerShell.Commands.SetItemCommand", "stderr_lines": ["Set-Item : WinRM firewall exception will not work since one of the network connection types on this machine is set to ", "Public. Change the network connection type to either Domain or Private and try again. ", "At line:1 char:65", "+ ... ing $false; Set-Item -Path WSMan:\localhost\Service\AllowUnencrypted ...", "+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~", " + CategoryInfo : NotSpecified: (:) [Set-Item], InvalidOperationException", " + FullyQualifiedErrorId : System.InvalidOperationException,Microsoft.PowerShell.Commands.SetItemCommand"], "stdout": "", "stdout_lines": []}
PLAY RECAP *********************************************************************
batrix3 : ok=7 changed=2 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
/srv/fluffi/data/tftp does not exist in a fresh installation
/srv/fluffi/data/tftp is excluded in the rsync step to create or update /srv/fluffli. So we created it by hand for now.
The first point of interest would be:
Line 107 in 9306937
deploy fluffi fails with "connection types on this machine is set to Public"
PLAY [windows] *****************************************************************
TASK [Gathering Facts] *********************************************************
ok: [batrix3]
TASK [initial-windows-Timezone : Set timezone to 'Romance Standard Time' (GMT+01:00)] ***
ok: [batrix3]
TASK [initial-windows-GFlags : Ensure source directory exists] *****************
ok: [batrix3]
TASK [initial-windows-GFlags : Fetch GFlag zip file "ftp://ftp.fluffi/initial/windows/ansible/GFlags/GFlags.zip"] ***
changed: [batrix3]
TASK [initial-windows-GFlags : Unzip GFlags] ***********************************
changed: [batrix3]
TASK [initial-windows-PrepareEnvironment : Create a FLUFFI directory] **********
ok: [batrix3]
TASK [initial-windows-PrepareEnvironment : Set ACL for C:\FLUFFI] **************
ok: [batrix3]
TASK [initial-windows-PrepareEnvironment : Set AllowUnencrypted to True] *******
fatal: [batrix3]: FAILED! => {"changed": true, "cmd": "Set-Item -Path WSMan:\localhost\Service\AllowUnencrypted -Value $true", "delta": "0:00:03.234031", "end": "2020-04-16 09:33:04.872752", "msg": "non-zero return code", "rc": 1, "start": "2020-04-16 09:33:01.638721", "stderr": "Set-Item : WinRM firewall exception will not work since one of the network connection types on this machine is set to \r\nPublic. Change the network connection type to either Domain or Private and try again. \r\nAt line:1 char:65\r\n+ ... ing $false; Set-Item -Path WSMan:\localhost\Service\AllowUnencrypted ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : NotSpecified: (:) [Set-Item], InvalidOperationException\r\n + FullyQualifiedErrorId : System.InvalidOperationException,Microsoft.PowerShell.Commands.SetItemCommand", "stderr_lines": ["Set-Item : WinRM firewall exception will not work since one of the network connection types on this machine is set to ", "Public. Change the network connection type to either Domain or Private and try again. ", "At line:1 char:65", "+ ... ing $false; Set-Item -Path WSMan:\localhost\Service\AllowUnencrypted ...", "+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~", " + CategoryInfo : NotSpecified: (:) [Set-Item], InvalidOperationException", " + FullyQualifiedErrorId : System.InvalidOperationException,Microsoft.PowerShell.Commands.SetItemCommand"], "stdout": "", "stdout_lines": []}
PLAY RECAP *********************************************************************
batrix3 : ok=7 changed=2 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
http://web.fluffi:8880/systems/removeDevSystem/{{system}}/ returns 404
To reproduce, visit http://web.fluffi:8880/systems, then:
-
Create a runner system using the web interface. For this example, say the system name is
fluffi-0228-Linux3. -
Click on the X icon at the end of the row of the newly-created system. The browser will visit http://web.fluffi:8880/systems/removeDevSystem/fluffi-0228-Linux3/ and get a 404.
Audit root directories that should be created in /srv/fluffi/data/ftp/files/
Due to this step:
Line 111 in ab9b81f
it is not possible to create a new root directory in
ftp.fluffi using the FTP protocol. As a consequence, we would create new missing root directories via terminal work on the host of the containers.
So far, we have already discovered three more missing directories. A code audit may identify others. From our current provision script:
mkdir -p \
/srv/fluffi/data/dnsmasq \
/srv/fluffi/data/ftp/files/fluffi/linux/{x64,x86,arm32,arm64} \
/srv/fluffi/data/ftp/files/fluffi/windows/{x64,x86} \
/srv/fluffi/data/ftp/files/{SUT,archive} `# BUG` \
/srv/fluffi/data/ftp/files/initial/linux/activePackages `# BUG` \
/srv/fluffi/data/tftp
deploy fluffi task failed
TASK [deployment-windows-fluffi : Fetch FLUFFI from "ftp://ftp.fluffi/fluffi/windows/x64/fluffi.zip"] ***
fatal: [batrix3]: FAILED! => {"changed": false, "dest": "C:\\FLUFFI\\x64\\fluffi.zip", "elapsed": 0, "msg": "Error requesting 'ftp://ftp.fluffi/fluffi/windows/x64/fluffi.zip'. The remote server returned an error: (550) File unavailable (e.g., file not found, no access).", "status_code": 550, "url": "ftp://ftp.fluffi/fluffi/windows/x64/fluffi.zip"}
PLAY RECAP *********************************************************************
batrix3 : ok=2 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
update copyright header
currently all files start with
Copyright 2017-2019 Siemens AG
This needs to be changed to
Copyright 2017-2020 Siemens AG
Windows shortcuts on runner after deploy of fluffi are wrong
After deploy fluffi action, the shortcuts are created with wrong path.
Also the path seems to be the only difference between "disk" and "ram" shortcuts
ubuntu build error paths!?
Running ./buildAll.sh -PREPARE_ENV=TRUE WITH_DEPS=TRUE i get the error:
cp: cannot stat '/fluffi/dependencies/qemu/Intel80386/qemu-arm': No such file or directory
make[1]: *** [depcopy] Error 1
make: *** [deps] Error 2
Dependencies folder has the parent "core" folder!
Also does the git clone needs to be done on the root folder "/" or it should support relative paths?
AVX_512 handling in DynamoRIO breaks Fluffi execution
On newer machines Fluffi won't start properly, because we inherited following bug of DynamoRIO:
AVX-512 handling breaks DynamoRIO execution on Windows with full AVX-512 support
The described work-around, setting avx512_enabled to false works for now.
http://web.fluffi:8880/systems returns 502
We believe we currently have a Fluffi master VM and two Linux Fluffi runner VMs that are running "as correctly as we know how to". The most important evidence in support of this belief includes:
- The three periodic tasks in PoleMarch all run correctly: Polemarch has seen 0 error/interrupted/delay/offline events.
- The two runner VMs can connect to the rest of the infra just fine: DNS queries, smbclient, etc. all work.
However, http://web.fluffi:8880/ actually shows http://web.fluffi:8880/static/images/load.png on the right half of the page. And if we visit http://web.fluffi:8880/systems, we get a 502 Bad Gateway and the container log is shown at the end of this message. Any idea what we should check next?
fluffiweb_1 | Traceback (most recent call last):
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2309, in __call__
fluffiweb_1 | return self.wsgi_app(environ, start_response)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2295, in wsgi_app
fluffiweb_1 | response = self.handle_exception(e)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1741, in handle_exception
fluffiweb_1 | reraise(exc_type, exc_value, tb)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
fluffiweb_1 | raise value
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
fluffiweb_1 | response = self.full_dispatch_request()
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
fluffiweb_1 | rv = self.handle_user_exception(e)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
fluffiweb_1 | reraise(exc_type, exc_value, tb)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
fluffiweb_1 | raise value
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
fluffiweb_1 | rv = self.dispatch_request()
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
fluffiweb_1 | return self.view_functions[rule.endpoint](**req.view_args)
fluffiweb_1 | File "./app/views.py", line 1066, in systems
fluffiweb_1 | changeAgentStarterModeForm=changeAgentStarterModeForm)
fluffiweb_1 | File "./app/helpers.py", line 26, in renderTemplate
fluffiweb_1 | return _renderTemplate(*args, nav = nav.elems, **kwargs)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/templating.py", line 135, in render_template
fluffiweb_1 | context, ctx.app)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask/templating.py", line 117, in _render
fluffiweb_1 | rv = template.render(context)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/jinja2/asyncsupport.py", line 76, in render
fluffiweb_1 | return original_render(self, *args, **kwargs)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/jinja2/environment.py", line 1008, in render
fluffiweb_1 | return self.environment.handle_exception(exc_info, True)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/jinja2/environment.py", line 780, in handle_exception
fluffiweb_1 | reraise(exc_type, exc_value, tb)
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/jinja2/_compat.py", line 37, in reraise
fluffiweb_1 | raise value.with_traceback(tb)
fluffiweb_1 | File "/app/app/templates/systems.html", line 13, in top-level template code
fluffiweb_1 | {% extends "base.html" %}
fluffiweb_1 | File "/app/app/templates/base.html", line 50, in top-level template code
fluffiweb_1 | {{ message }}
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask_bootstrap/templates/bootstrap/base.html", line 1, in top-level template code
fluffiweb_1 | {% block doc -%}
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask_bootstrap/templates/bootstrap/base.html", line 4, in block "doc"
fluffiweb_1 | {%- block html %}
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask_bootstrap/templates/bootstrap/base.html", line 20, in block "html"
fluffiweb_1 | {% block body -%}
fluffiweb_1 | File "/usr/local/lib/python3.6/site-packages/flask_bootstrap/templates/bootstrap/base.html", line 23, in block "body"
fluffiweb_1 | {% block content -%}
fluffiweb_1 | File "/app/app/templates/systems.html", line 221, in block "content"
fluffiweb_1 | {{ host.InstanceCount }} {% if host.confLM > 0 %} <div class="systems-lm-text">({{host.confLM}} LM)</div> {% endif %}
fluffiweb_1 | jinja2.exceptions.UndefinedError: 'app.utils.ansible. object' has no attribute 'confLM'
fluffiweb_1 | 2020/02/26 19:18:41 [error] 12#12: *89 upstream prematurely closed connection while reading response header from upstream, client: 10.180.62.146, server: , request: "GET /systems HTTP/1.1", upstream: "uwsgi://unix:///tmp/uwsgi.sock:", host: "web.fluffi:8880", referrer: "http://web.fluffi:8880/index"
fluffiweb_1 | 10.180.62.146 - - [26/Feb/2020:19:18:41 -0500] "GET /systems HTTP/1.1" 502 173 "http://web.fluffi:8880/index" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:73.0) Gecko/20100101 Firefox/73.0" "-"
fluffiweb_1 | [pid: 16|app: 0|req: 13/56] 10.180.62.146 () {46 vars in 824 bytes} [Wed Feb 26 19:18:40 2020] GET /systems => generated 0 bytes in 478 msecs (HTTP/1.1 500) 0 headers in 0 bytes (0 switches on core 0)
Covered Blocks field is always 0
With the master at following commit, http://web.fluffi:8880/index shows the list of fuzzjobs.
On going to any particular fuzzjob ( e.g. http://web.fluffi:8880/projects/view/1 will list the properties of the first fuzzed job), the number of covered blocks are reported as 0, even when the fuzzjob has performed a significant number of testcase executions.
PFA a zip SUT for which the issue took place.
running in a vbox problems
- dns does not seem to work, as i cant access using names .. probably configurion is wrong
- trying to access localhost:8880 gives nginx error
Vbox machine is ubuntu 18.04, static ip set to 10.66.0.1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.