mozilla / mig Goto Github PK
View Code? Open in Web Editor NEWDistributed & real time digital forensics at the speed of the cloud
Home Page: http://mig.mozilla.org/
License: Mozilla Public License 2.0
Distributed & real time digital forensics at the speed of the cloud
Home Page: http://mig.mozilla.org/
License: Mozilla Public License 2.0
Checkin mode should not be able to hang, ever. We could use the value of MODULETIMEOUT to kill the agent when the delay is expired.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1155981
Assigned to: Julien Vehent [:ulfr]
On 2015-04-18 08:30:18 -0700, Julien Vehent [:ulfr] wrote:
When an agent executes N operations, each operation is ran in parallel against its own instantiation of a module. Results are aggregated into a JSON array and sent back to the scheduler. Modules don't know about each other and have no way to interact.
The ability to chain the work of modules would open new forensics options. The proposed workflow is to chain operations like unix commands are chained with pipes: the output of a module could become the input of another module.
To achieve this, we need a data transformation layer that convert the output of operation 1 into an input for operation 2. That layer could be performed by a lightweight javascript interpreter. Instead of running in parallel, operations would run in sequence, following the ordering of the operations array, with javascript transformation performed in between each operation.The javascript interpreter must run into a sandbox. It must not be allowed to perform IOs. The agent must control the stdin and stdout of a javascript sandbox.
If valid JSON is returned to the scheduler that is missing some fields, the scheduler will not load it (CmdFromFile() fails). The data ends up sitting in the returned directory though and is reparsed every few seconds. If a number of invalid events were returned this could potentially get inefficient and may cause problems.
We could somehow mark a failed returned file, so the first time it fails maybe it's renamed, and subsequent walks of the spool directory ignore these failed files.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1201659
Assigned to: Julien Vehent [:ulfr]
On 2015-09-03 12:27:06 -0700, Julien Vehent [:ulfr] wrote:
Recent changes in the MIG backend infrastructure required us to break backward compatibility. When we release the latest version of MIG scheduler, old agents will not be reachable any longer. For this reason, we want to synchronize a release of the backend with a deployment of new agents across IT, releng and services and opsec.
An important note: while agents running the old version will not be reachable any more, the agents themselves will keep functioning correctly and not notice they are talking to the void. The impact to their host will be null and won't break anything. It won't even show in the logs until we delete the old rabbitmq exchanges, days after the release. By that time, hopefully, all agents will have been upgraded.
Below is a list of tasks that need to be performed before, during and after the release. Most actions will be taken by Aaron and myself. There is one action for Dustin to merge the patch in build-puppet on release day.
Preparation
- Perform sanity check on scribe, pkg and memory modules {ulfr+alm}
- Update GPG keys {alm}
- ops -> access to file, pkg and netstat
- sec -> access to *
- Create an automated investigator key, give acl to scribe, pkg, file and netstat {alm}
- Update build script to copy available module to new location {alm}
- Enable scribe, pkg, memory modules everywhere {alm}
- Build mig-agent packages for all organizations {alm}
- Prepare patch for releng deployment {ulfr}
- publish new mig-agent packages to releng repositories
- create deploy bug for releng, wait for r+
- Prepare patch for opsec puppet {ulfr}
- create new exchanges with new ACLs in rabbitmq
- deploy new scheduler/api/workers {ulfr}
Release Day (starts 1100EDT, 0800PDT)
- Deploy new servers in moz-opsec {ulfr}
- Merge patch into releng build-puppet {dustin}
- Publish new packages in mrepo {ulfr}
- Publish new packages in SVC mrepo {ulfr}
- Publish new packages in opsec puppet & ansible {alm}
Cleanup
- Write script that pull heartbeats from old exchange to detect non updated agents {ulfr}
- Delete old exchange and queues in rabbitmq admin UI {ulfr}
On 2015-09-03 13:25:32 -0700, Dustin J. Mitchell [:dustin] wrote:
Sounds good. Keep in mind that because of the way we bake AWS instance nightly, there's a 24-hour lag on changes to all of our AWS buildslaves. It doesn't sound like that would be a problem. We can accelerate that to "a few hours" by manually kicking off a baking process, but it's still not instantaneous.
On 2015-09-04 08:36:48 -0700, Julien Vehent [:ulfr] wrote:
That should be alright, we can live without MIG for half a day.
On 2015-09-09 11:21:25 -0700, Julien Vehent [:ulfr] wrote:
QA took longer than expected, so we're going to release tomorrow instead.
Example:
"check for yum compliance on RHEL, not Ubuntu"
"check for apt compliance on Ubuntu, not RHEL"
It could be a system target linux-ubuntu/linux-rhel, etc.
If all actions are standard-based, this should be possible to do at the scheduler level, too. This might be better because sometimes we'll want targets to be more like "linux-ubuntu-high", and that could be a lot of target and complex to maintain checks for these actions.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1149496
Assigned to: Julien Vehent [:ulfr]
On 2015-03-31 05:45:42 -0700, Julien Vehent [:ulfr] wrote:
In some instances, endpoints cannot establish NTP connections with pools from the internet. Timedrift should attempt to connect to locally configured NTP servers first, by reading /etc/ntp.conf and parsing the ^server entries, and then use internet servers if no local ones are available.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1117147
Assigned to: Aaron Meihm [:alm]
On 2015-01-02 09:54:23 -0800, Julien Vehent [:ulfr] wrote:
MIG needs to allow releng spot instances to register. They are ephemeral, such that manual whitelisting requires constant verification work. Instead, I�d like the scheduler to query the aws api to verify that an agent runs on an instance that exists and can be allowed to register.
Proposed algorithm:
- mig-agent retrieves local instance meta-data by querying http://169.254.169.254/latest/meta-data/ and sends that information to the scheduler [1].
- the scheduler queries AWS to look for an instance that matches the meta-data sent by the agent. It should, at a minimum, compare the following:
- placement (region + AZ)
- instance-id
- instance-type
- mac address
- if a match is found, the agent is allowed to register. If not, a warning is logged.
The implementation must allow the scheduler to query multiple AWS accounts. We could use hostnames mapping to tie a hostname with one (or more) AWS access key.
For example:
{
"domains": [
{
"match": ".+.dmz.use1.mozilla.com$",
"lookup": {
"aws": {
"accesskey": "12345",
"secretkey": "abcdef"
}
}
},
{
"match": ".+.dmz.use1.mozilla.com$",
"lookup": {
"aws": {
"accesskey": "98271",
"secretkey": "ffffff"
}
}
}
]
}The scheduler would go through the list of domain regexes, and attempt to verify the instance against each aws account that matches the regex until a match is found.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1116677
Assigned to: Nobody; OK to take it and work on it
On 2014-12-30 19:36:03 -0800, Julien Vehent [:ulfr] wrote:
Agents collect some information about their environment. This information is currently only available in the database, and should be exposed in the API to be rendered in the console.
This should also allows users to search agents based on environment criteria, like the family of operating system.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1037966
Assigned to: Julien Vehent [:ulfr]
On 2014-07-13 06:49:51 -0700, Julien Vehent [:ulfr] wrote:
The console needs support for advanced search queries, through the API.
Some example:select agents where command.id='12345' and commands.results.0.foundanything=true
select action where name='some action name' and starttime>='2014-07-01'
select command where agent.name='someserver.example.net' and action.threat.family='compliance'
etc...
As a step 2, it would be useful to have multiple search targets. The default target would be the API, but the result a query could be stored in a local buffer, and further queries ran against that buffer.
select commands where action.id='123456'
==> results stored in BUF1select agents from BUF1 where command.status != 'done'
select agents from BUF1 where command.results.*.foundanything=true
On 2014-07-23 05:14:24 -0700, Julien Vehent [:ulfr] wrote:
Instead of re-implementing complex search support in the console, another idea would be to give investigators unbounded sql access, aka "SQL Injection".
The API would require a strong authentication mechanism and session management. Authentication should use GPG to avoid introducing a second key/password. The API would also need support for ACLs, and a read-only database user.Some investigators would gain search access by tying their identity to a search role in the API ACLs. That would also let us open us the API without giving full access to its data.
In the console, we would pass whole SQL queries as POST bodies, and return the results in json.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1037965
Assigned to: Julien Vehent [:ulfr]
On 2014-07-13 06:41:50 -0700, Julien Vehent [:ulfr] wrote:
Provide a standard module that can verify the presence of a given account or group on a system. It should also be able to create and disable accounts, or compare a given hash string with the current password (to check when a password has not been properly changed).
On 2014-12-31 10:40:51 -0800, Julien Vehent [:ulfr] wrote:
One interesting feature would be to parse /var/log/lastlog to check where a user has recently logged in.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1014740
Assigned to: Julien Vehent [:ulfr]
On 2014-05-22 11:06:22 -0700, Julien Vehent [:ulfr] wrote:
There is currently a 1:1 mapping between the scheduler and the rabbitmq relay. We want to change that to a 1:n mapping, such that relays can be deployed in network zones that don't have limited connectivity to the internet.
The scheduler would then be able to route messages to the relay where a given agent is connected.
On 2014-06-16 10:54:54 -0700, Gene Wood [:gene] wrote:
An additional/alternative feature would be to enable mig to talk to a proxy server (http/https) since the Network Security Policy requires one anyhow.
On 2014-07-20 10:07:42 -0700, Julien Vehent [:ulfr] wrote:
Proxying isn't straighforward. Talking to a proxy server means supporting the CONNECT HTTP method.
$ nc -v some.proxy.example.net 3128 << EOF > CONNECT relay.mig.example.com:443 HTTP/1.1 > Host: relay.mig.example.com:443 > > > EOF Connection to some.proxy.example.net 3128 port [tcp/squid] succeeded! HTTP/1.0 200 Connection established
Maybe the function that connects to the relay could be rewritten to first establish a CONNECT to the proxy, and then pass the resulting connection back to the agent. Maybe...
https://github.com/mozilla/mig/blob/master/src/mig/agent/context.go#L350-L352
On 2014-07-20 11:11:59 -0700, Julien Vehent [:ulfr] wrote:
Well, turns out if was easier than I thought. Proxy support has been added to the agent in SHA: cc0658b
I'm leaving this bug open for now, in case that some agents are still unreachable without local relays.
On 2014-07-20 14:55:26 -0700, Julien Vehent [:ulfr] wrote:
Well, turns out if was almost as easy as I thought. Took me a couple hours to iron out the bugs, but it's now stable in SHA: bf080b1
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1135542
Assigned to: Julien Vehent [:ulfr]
On 2015-02-22 16:48:59 -0800, Julien Vehent [:ulfr] wrote:
The service package in mig agent currently uses a fork that solves a systemd issue. The issue has now been fixed upstream, so there is no need for the fork anymore. The repository has also moved from bitbucket to github and provides a new configuration API that should be integrated in mig-agent.
new location is https://github.com/kardianos/service
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1014714
Assigned to: Nobody; OK to take it and work on it
Original attachment names and IDs:
On 2014-05-22 10:45:42 -0700, Julien Vehent [:ulfr] wrote:
This module parses netstat on a host and looks for connected IPs. Add support for windows and macos.
On 2014-09-17 08:21:43 -0700, Julien Vehent [:ulfr] wrote:
The
connected
module has been renamed tonetstat
. It currently supports linux 100%, and macos/windows are in the work.The commits are:
SHA: 0926c73
SHA: 49498bf
SHA: 01a7a9e
SHA: c4d6f50
On 2014-11-28 08:58:13 -0800, Julien Vehent [:ulfr] wrote:
For windows, see if https://godoc.org/golang.org/x/sys/windows can help
On 2015-03-13 15:10:48 -0700, Dustin J. Mitchell [:dustin] wrote:
I tried running netstat in 'dtruss' to see if there was a nicer API to get connection data. Nope, it accesses /AppleInternal\0. But running netstat isn't so bad. Except:
r5-6:mig root# netstat -n -a -W -f inet6
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp6 0 0 .88 *. LISTEN
tcp6 0 0 .22 *. LISTEN
tcp6 0 0 ::1.631 . LISTEN
tcp6 0 0 .5900 *. LISTEN
tcp6 0 0 .625 *. LISTEN
udp6 0 0 fe80::3e07:54ff:.123 .
udp6 0 0 ::1.123 .
udp6 0 0 fe80::1%lo0.123 .
udp6 0 0 .123 *.
udp6 0 0 .88 *.
udp6 0 0 .5353 *.
udp46 0 0 . .
icm6 0 0 . .Note that long local address -- it's truncated, despite using the "-W" flag. I don't know of any workaround.
On 2015-03-13 15:19:30 -0700, Dustin J. Mitchell [:dustin] wrote:
Created attachment 8577488
#42
On 2015-03-13 15:22:42 -0700, Dustin J. Mitchell [:dustin] wrote:
Specifically, that's a truncated version of the link-local address for
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=2b<RXCSUM,TXCSUM,VLAN_HWTAGGING,TSO4>
ether 3c:07:54:72:4c:34
inet6 fe80::3e07:54ff:fe72:4c34%en0 prefixlen 64 scopeid 0x4
inet 10.26.78.47 netmask 0xffffff00 broadcast 10.26.78.255
media: autoselect (1000baseT <full-duplex,flow-control>)
status: active
On 2015-03-17 08:35:45 -0700, Julien Vehent [:ulfr] wrote:
Dustin added support for
HasConnectedIP
to darwin, so what's left is:
- on darwin: support for
HasListeningPort
- on windows: support for
HasListeningPort
andHasConnectedIP
On 2015-03-20 06:06:48 -0700, Dustin J. Mitchell [:dustin] wrote:
I'm going to work on some testing before returning to this.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1116683
Assigned to: Julien Vehent [:ulfr]
On 2014-12-30 19:54:06 -0800, Julien Vehent [:ulfr] wrote:
Actions and commands are persisted on disk while processed, in a spool directory on the scheduler node. In order to run multiple schedulers in parallel, this spool needs to be moved to a distributed location.
Action and commands should be committed to the database while being processed, and events should be published into rabbitmq to notify schedulers that work is pending.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1014744
Assigned to: Julien Vehent [:ulfr]
On 2014-05-22 11:09:19 -0700, Julien Vehent [:ulfr] wrote:
If a command as the wrong signature, the errors that is returned to the scheduler and displayed in the logs isn't specific enough. We need a better, more detailled, way to log and report failures.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1116681
Assigned to: Julien Vehent [:ulfr]
On 2014-12-30 19:50:12 -0800, Julien Vehent [:ulfr] wrote:
The File module is able to inspect the content of files, but does not support automated decompression. This feature would be useful for scanning log files that are often compressed when rotated. It could also be used to search and inspect database dumps, or various suspicious files.
The performance impact of inspecting compressed file is an important aspect of this bug. MIG Agent must remain lightweight and not extravagantly consume CPU when scanning the file system.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1163598
Assigned to: Julien Vehent [:ulfr]
On 2015-05-11 06:57:45 -0700, Julien Vehent [:ulfr] wrote:
The file module should allow searching for files that contain a specific hexadecimal encoded bytes string.
This feature is implemented in the memory module under the
-bytes
options, we should reuse the name in the file module.The implementation should use a sliding window to match bytes string at the edge of search buffers, and not using line by line search as is currently the case for regexp searches.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1116687
Assigned to: Julien Vehent [:ulfr]
On 2014-12-30 20:20:12 -0800, Julien Vehent [:ulfr] wrote:
MIG needs a web interface to provide a similar level of functionality than the console, but in a web browser.
It should provide a simple way to run actions, visualize results, review the activity of investigators and agents, manage investigators and ACLs, and implement an easy way to perform multi-signature on an action.
The interface should use OpenPGPJS to implement the X-PGPAUTHORIZATION token required to communicate with the API.
It should be lightweight and fast. The goal of MIG is to launch investigations rapidly, and the user experience on the webui should reflect this goal.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1149502
Assigned to: Julien Vehent [:ulfr]
On 2015-03-31 05:59:32 -0700, Julien Vehent [:ulfr] wrote:
Investigators are currently managed manually, created in the MIG Database and deactivated when needed. Their keys are uploaded manually too.
A better approach would be to populate the investigator database using LDAP Groups, and fetch investigator's keys using key ID stored in LDAP.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1151660
Assigned to: Julien Vehent [:ulfr]
On 2015-04-06 15:56:34 -0700, Julien Vehent [:ulfr] wrote:
Instead of pushing a binary, the upgrade module should use a rpm/deb/dmg/wmi package to install new versions of the agent.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1104992
Assigned to: Julien Vehent [:ulfr]
On 2014-11-25 13:45:23 -0800, Julien Vehent [:ulfr] wrote:
https://github.com/bradfitz/http2/ provides support for HTTP2 in Go. Considering the large number of API requests between a client and the MIG API, HTTP/2 could provide a significant speedup. It would also provide a cleaner alternative to websocket, to allow the API to push action status updates without requiring requests from the client.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1167354
Assigned to: Julien Vehent [:ulfr]
On 2015-05-21 13:06:41 -0700, Julien Vehent [:ulfr] wrote:
I often happens during vulnerability patching that libraries are upgraded but processes are not restarted. When that happens, the process is marked as linked to a
deleted
library.
MIG should have a facility to detect this, probably in the memory module.-deleted
could be a boolean added to the-lib
parameter to return processes that link to a given library that has been deleted on disk.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1104708
Assigned to: Julien Vehent [:ulfr]
On 2014-11-25 05:52:21 -0800, Julien Vehent [:ulfr] wrote:
Row level security is a postgres 9.5 feature that allows setting permissions at the row level of a table. The goal is to give investigators visibility of their own perimeter, but not beyond. For example, an investigator "A" that operates agents tagged "team123" should be able to view all agents, actions and commands ran on its agents, as well as schedule new actions on these agents. But not beyond.
With row level security, when investigator "A" creates an action X that targets all agents, the policy will limit the list of targeted agents to ones that are operated by "team123" only.
This functionality will allow mig to manage agents from multiple teams in the same platform, while limiting the perimeters of investigators at the most granular level.
source: http://michael.otacoo.com/postgresql-2/postgres-9-5-feature-highlight-row-level-security/
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1198304
Assigned to: Nobody; OK to take it and work on it
On 2015-08-25 08:31:43 -0700, Guillaume Destuynder [:kang] wrote:
Ex:
mig file -path /etc/ssl/certs -name localhost.crt -md5 2b547043e4c84892cbebcf47a3e9d2b9does not return a match even thus /etc/ssl/certs/localhost.crt matches the md5.
that's because /etc/ssl/certs directory is a symlink to /etc/pki/tls/certs (which is not in the real directory scope of /etc/ssl/certs/*)This causes confusion for users since paths would have to be manually checked. Note that doing symlink resolution may also cause infinite loops (for ex /etc/ssl/certs could be linked to itself, or to a directory linking back again to /etc/ssl/certs!)
Here's a more thorough real-life example for reproducibility:
--- snip ---
pbx1 /home/gdestuynder � mig-agent -m file -d <<< '{"class":"parameters","parameters":{"searches":{"s1":{"paths":["/etc/ssl/certs"],"md5":["2b547043e4c84892cbebcf47a3e9d2b9"]}}}}';echo
[info] using builtin conf
{"foundanything":false,"success":true,"elements":{"s1":[{"file":"","search":{"paths":["/etc/ssl/certs"],"md5":["2B547043E4C84892CBEBCF47A3E9D2B9"],"options":{"maxdepth":0,"matchall":false,"matchlimit":0}},"fileinfo":{"size":0,"mode":"","lastmodified":""}}]},"statistics":{"filescount":1,"openfailed":0,"totalhits":0,"exectime":"3.754553ms"},"errors":["checkHash() -\u003e getHash() -\u003e read /etc/ssl/certs: is a directory"]}
pbx1 /home/gdestuynder � mig-agent -m file -d <<< '{"class":"parameters","parameters":{"searches":{"s1":{"paths":["/etc/pki/tls/certs"],"md5":["2b547043e4c84892cbebcf47a3e9d2b9"]}}}}';echo
[info] using builtin conf
{"foundanything":true,"success":true,"elements":{"s1":[{"file":"/etc/pki/tls/certs/localhost.crt","search":{"paths":["/etc/pki/tls/certs"],"md5":["2B547043E4C84892CBEBCF47A3E9D2B9"],"options":{"maxdepth":0,"matchall":false,"matchlimit":0}},"fileinfo":{"size":1241,"mode":"-rw-------","lastmodified":"2011-08-13 17:10:15.753058153 +0000 UTC"}}]},"statistics":{"filescount":9,"openfailed":0,"totalhits":1,"exectime":"13.336304ms"},"errors":null}
pbx1 /home/gdestuynder � ls -l /etc/ssl/certs
lrwxrwxrwx 1 root root 16 Jan 28 2015 /etc/ssl/certs -> ../pki/tls/certs
pbx1 /home/gdestuynder � md5sum /etc/pki/tls/certs/localhost.crt 2b547043e4c84892cbebcf47a3e9d2b9 /etc/pki/tls/certs/localhost.crt
pbx1 /home/gdestuynder � md5sum /etc/ssl/certs/localhost.crt
2b547043e4c84892cbebcf47a3e9d2b9 /etc/ssl/certs/localhost.crt
--- snap ---
On 2015-08-25 08:53:51 -0700, Julien Vehent [:ulfr] wrote:
MIG's
file
module will follow symlinks on files only, not directories, using Go's filepath.EvalSymlinks() function (which has a hardcoded limit of 255 levels). We could make it follow symlinks on directories, but it would be difficult to constrain a given search to a root path, as we do today. Any extra limitation we impose, such as limited the number of links that can be followed, or limiting the depth search of a symlinked directory, are likely to cause confusion.I would prefer to not follow symlink of directories at all and add a soft error in the results of the search that indicates a directory symlink was encountered and not followed. It think it would help reduce confusion and keep the behavior consistent. Note that
find
has a similar behavior and will not follow symlinks by default (-L is required).That said, if we decide to follow directory symlinks, it should be implemented as an additional case of the symlink handling in the pathWalk() function, right after https://github.com/mozilla/mig/blob/SHA: f308713/src/mig/modules/file/file.go#L885-L887
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1116684
Assigned to: Julien Vehent [:ulfr]
On 2014-12-30 19:56:32 -0800, Julien Vehent [:ulfr] wrote:
This bug is a reference to an upstream bug in golang openpgp implementation: golang/go#9312
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1116003
Assigned to: Julien Vehent [:ulfr]
On 2014-12-28 10:43:22 -0800, Julien Vehent [:ulfr] wrote:
Running mig-agent on windows currently works in foreground, with options "-f" or "-d". From today's tests, it seems that the mig-agent service is not properly installed and configured, which prevents it from running in background. This may be an issue with the service-go package.
MSI packaging mostly works, but two things need to be added to the WXS file:
- shortcut creation to link C:\Program Files\mig\mig-agent-.exe to C:\Program Files\mig\mig-agent.exe
- command execution at the end of the MSI install to run mig-agent
Today's work is committed in SHA: e6ed8a4 . To be continued...
Create an restful API for investigators to submit signed actions and receive real time results via a websocket. The API must provide authentication (possibly 2FA). The first iteration will have a limited number of endpoints:
POST /action/create
will take a signed action file that follow SyntaxVersion 1
and return a href with an action idGET /action/<action_id>
will take an action ID to return the status of the action with all MetaAction
attributesGET /action/<action_id>/command/<command_id>
will take an action ID and a command ID to return the results of a command ran on a target agent.The action signature scheme MIG uses protects the most important fields of the Action JSON file. Agents verify those fields to make sure that a trusted investigator has issued the action.
In particular, the signature covers the action.target fields that contains the target string used to sent the action to selected agents. The target fields is expanded into a list of agents by the scheduler, then each agent receives a Command JSON file which includes the signed action and some other fields.
The signature does not cover the command JSON file because commands are created later in the process and agent-specific.
Moreover, agents themselves do not verify that action.target and cannot verify that an action was truly targeted at them or not. That decision is made by the scheduler.
Put together, these two issue allow for an attacker to take a signed action and craft a command file targeted at an agent that was not originally meant to be.
This is currently mitigated using RabbitMQ's access controls which:
But rabbitmq is fragile and directly exposed to the internet. We should provide an additional layer of security to prevent a rabbitmq compromise from putting agents at risk.
The proposed solution is to add a second signature, this time on the Command envelope, that contains:
Upon reception of a command, an agent:
This protocol would prevent a command reuse and add an extra layer of security behind RabbitMQ's ACLs.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1014726
Assigned to: Julien Vehent [:ulfr]
Original attachment names and IDs:
On 2014-05-22 10:57:16 -0700, Julien Vehent [:ulfr] wrote:
Camlistore has a nice package to handle the signing of JSON.
https://camlistore.googlesource.com/camlistore/+/master/pkg/jsonsign/
It would most likely be cleaner than the current "stringified json" approach.
On 2014-06-14 12:19:13 -0700, Julien Vehent [:ulfr] wrote:
Created attachment 8440318
Getting one step closer: no more cgo with gpgme. Using pinentry and gpgagent wrappers from camlistore.
On 2014-11-16 10:27:13 -0800, Julien Vehent [:ulfr] wrote:
I've done some analysis and it doesn't seem like replacing the current signing method with camlistore's brings much value. Signing and verifying still relies on the ordering of the json parameters applied by the json package. For now, since we use the same json encoder/decoder provided by the Go standard lib everywhere, ordering is not an issue. It may be if we start using clients that use various json libs.
Leaving open for future ref.
On 2014-12-23 10:10:42 -0800, Julien Vehent [:ulfr] wrote:
Need to evaluate https://github.com/square/go-jose for this.
On 2014-12-23 10:12:47 -0800, Julien Vehent [:ulfr] wrote:
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1207709
Assigned to: Nobody; OK to take it and work on it
On 2015-09-23 10:51:39 -0700, Hal Wine [:hwine](use NI) wrote:
May only be an issue when retrieved via "go get", but after update:
$ ./mig -V$ ./mig-console -V
$
On 2015-09-23 10:58:29 -0700, Julien Vehent [:ulfr] wrote:
This is because -V reads the content of
var version string
which is set toGOLDFLAGS := -ldflags "-X main.version=$(BUILDREV)"
in the Makefile.
But when retrieve the agent using go get, the makefile isn't called, and the flag isn't set. I'm not yet sure how to go about this.
On 2015-09-24 05:53:06 -0700, Julien Vehent [:ulfr] wrote:
Proposed fix in #88
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1141910
Assigned to: Julien Vehent [:ulfr]
On 2015-03-10 20:56:03 -0700, Julien Vehent [:ulfr] wrote:
Add options that automatically implements http://mig.mozilla.org/doc/cheatsheet.rst.html#target-agents-that-found-results-in-a-previous-action
mig file -t "id IN (
SELECT agentid FROM commands, json_array_elements(commands.results) AS r
WHERE commands.actionid = 12345 AND r#>>'{foundanything}' = 'true')"
-path /etc/passwd -content "^spongebob"For example:
- --target-foundanything would limit the targets to agents that returned positive results in
- --target-foundnothing would do the opposite and target agents that did not return positive results in
This option should concatenate with the regular -t option such that an investigator could set both and filter the results even further.
On 2015-03-10 20:56:41 -0700, Julien Vehent [:ulfr] wrote:
Implementation should take the form of two new options in the command line client at https://github.com/mozilla/mig/blob/master/src/mig/client/cmd/main.go#L75
Example:
"check all repositories in /etc/yum/yum.d/* have gpgcheck=1"
The filenames are not known, thus, need a wildcard.
The NeighborIP parameter in the netstat module does not appear to be used; the command line provides the ability to search using -ni but it doesn't look like this incoming parameter is processed anywhere currently.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1198653
Assigned to: Nobody; OK to take it and work on it
On 2015-08-26 03:06:48 -0700, Guillaume Destuynder [:kang] wrote:
For ex:
mig file -t "environment->>'os'='linux' AND tags->>'operator'='IT'" -name '.crt$' -path /etc/pki/tls/certs -maxdepth 1 -mtime '>1460d'that's 4 years, not super convenient. moreover, for more precision, that date interval needs to be converted to hours (35040h for ex) otherwise you're limited to a specific day. And of course, that's a rough simplification since I just multiplicated by 365 days for a year, but in our calendar-based reality this won't always give the expected results.
ideally, so that the user doesn't have to calculate the "time before" or "time after" manually for long periods, maybe we should parse a timestamp such as:
-mtime ">2015-01-01 10:52:00 UTC"
the module would then convert it
On 2015-08-26 04:59:21 -0700, Julien Vehent [:ulfr] wrote:
Not a bad idea. Here again we simply reproduce the behavior of find's -mtime parameter, without the rounding aspect.
We could parse a timestamp, but I'd want to make it a separate parameter. '-msince' for example. Parsing dates can be hard, so we need to enforce a specific data format. MIG uses RFC3339 is most places.
Calculating months or years accurately seems like a lot of work for little benefit. I'm not sold on that.
On 2015-08-26 05:01:02 -0700, Julien Vehent [:ulfr] wrote:
Also note that the results from the file module contain the exact 'lastmodified' date, so you can always grep on the command line.
server.example.net /etc/somefile [lastmodified:2013-04-07 15:42:10 +0000 UTC, mode:-rw-r--r--, size:651075] in search 's1'
On 2015-08-26 11:09:52 -0700, Guillaume Destuynder [:kang] wrote:
yeah im doing as per comment 2 for now as a work-around (while overshooting the date a bit), works ok, but its not the most convenient. generally only an issue for searches on really old stuff.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1014742
Assigned to: Julien Vehent [:ulfr]
On 2014-05-22 11:07:41 -0700, Julien Vehent [:ulfr] wrote:
The client
mig-action-generator
must be able to load an action that has already been signed once, and append another signature to it. This would allow investigators to multi-sign actions that require a higher level of privileges.
On 2014-12-30 19:40:56 -0800, Julien Vehent [:ulfr] wrote:
This work needs a different strategy: an investigator must be able to sign an action and submit it to the api for more signatures, then launched.
The API should have a staging area, maybe under /api/v1/action/pending, where action can be registered but not sent. The dashboard resource at /api/v1/dashboard should list action pending signatures.
The console should list actions pending signatures, allow an investigator to retrieve a pending action, and repost it to the staging area for more signatures, or launch it.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1014720
Assigned to: Julien Vehent [:ulfr]
On 2014-05-22 10:51:03 -0700, Julien Vehent [:ulfr] wrote:
See winter of security project: https://wiki.mozilla.org/Security/Automation/WinterOfSecurity2014#Cross-platform_firewall_driver_in_Go
We currently store DB records forever, and after two years of running MIG on thousands of production systems, we still use a minimal amount of space (~70GB). Still, most of these records are not useful anymore and could be purge to speed up search queries.
I'm proposing the following strategy:
agents
table after 30 days if the agent has not received any commandactions
, signatures
and investigator
tablesThis should be implemented in the scheduler using a new periodic task.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1134390
Assigned to: Julien Vehent [:ulfr]
On 2015-02-18 13:53:27 -0800, Julien Vehent [:ulfr] wrote:
MIG provides a search API endpoint documented at http://mig.mozilla.org/doc/api.rst.html#get-root-search
It is currently very limited in functionality. It can search for action, commands, agents and investigators, but it only supports searching on a subset of fields, and each search query results in a very inefficient JOIN of all database tables that is slow to process.The search API needs a revamp. Here's a few requirements:
be fast. JOINs should only be perform when needed. Unlike now: https://github.com/mozilla/mig/blob/master/src/mig/database/searches.go#L90-L93
be flexible. The API should support searching inside of the json fields stores in postgres. I would like to avoid statically listing all supported JSON fields, but instead have the API try and fail with a meaningful error message when a given search field is not found.
example: searching for an agent using its IP address, which is stored in a JSON array inside of the agent.environment column
allow for complex queries. For example, list agents that ran an action of threat family "malware" launched by investigator named "julien vehent" over the last 20 days. If possible, I would like to do this without accepting raw SQL in API parameters, without statically defining all possible search parameters in the code and with decent performances.
The current code, that statically lists search parameters is here https://github.com/mozilla/mig/blob/master/src/mig/api/search.go#L46-L98
control the data returned. Right now, a lot of unnecessary data is returned by the search API, because it has no way to define which fields the requester wants.
So, in fact, we really want SQL flexibility, but in API queries, and without the risk of taking raw sql as input.
In parallel to this work, the client library should be updated with a flexible search syntax for the command line. https://github.com/mozilla/mig/blob/master/src/mig/client/client.go
Right now, the scribe module duplicates some of the functionality mig already provided in terms of results filtering. The module has an -onlytrue option that means only true evaluations will be shown. We want to make this more in line with mig and use the -show found/notfound to provide filtering on true/false evaluations in the module.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1037967
Assigned to: Julien Vehent [:ulfr]
On 2014-07-13 07:11:16 -0700, Julien Vehent [:ulfr] wrote:
Compliance checks need to run daily. However, signing actions currently requires access to GPG private keys that only investigators have. Thus, compliance checks need to be launched daily by an investigator.
MIG needs a way to launch recurring actions without requiring human interventions, and without compromising the security model. Recurring actions need to be derived from master actions that are signed by an investigator. Designed a secure and practical way to do so is the purpose of this bug.
On 2015-09-23 09:41:20 -0700, Julien Vehent [:ulfr] wrote:
:alm wrote a mig-runner client that automates the scheduling of recurring actions. We'll use that moving forward.
#86
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1069988
Assigned to: Julien Vehent [:ulfr]
On 2014-09-19 08:58:21 -0700, Julien Vehent [:ulfr] wrote:
This bug is a notepad for general usability issues in the MIG Console. Dump stuff here and I'll triage into separate bugs.
On 2014-09-19 09:05:32 -0700, Julien Vehent [:ulfr] wrote:
Need to clarify the launcher workflow. It's not clear that, when an action is launched, the user can exit the console or launch another action without affecting the currently running one.
The interface must clarify that a running action is completely unaffected by what a user does in the console after launching it.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1014735
Assigned to: Julien Vehent [:ulfr]
On 2014-05-22 11:03:10 -0700, Julien Vehent [:ulfr] wrote:
ACLs are currently hardcoded in the agent. Instead, ACLs should be signed by a master key and stored in the postgres database. The Scheduler could then distribute ACLs to agents via AMQP. Agents would verify the master signature and update their ACLs accordingly.
On 2015-05-18 12:55:42 -0700, Julien Vehent [:ulfr] wrote:
The end goal is to remove the AGENTACL and PUBLICPGPKEYS configuration variables from the agent's static conf, and instead distribute them via AMQP or API calls.
The tricky part is distributing the AGENTACL in a secure manner. We don't want to just trust whatever is in the database because a server compromise could rewrite these values and grant access to a malicious investigator. The AGENTACL list provided to an agent MUST be signed by an entity the agent trusts, and the signing key MUST be kept offline.
The proposal is to add two new configuration variables to the agent:
- MEMBER_OF: an array of strings containing groups the agent is a member of. If empty, or does not contain any group known to the database, a membership to a group named "default" is assumed.
- MASTERKEYS: an array of public pgp keys used to verify ACLs
- REQUIRED_ACL_SIGNATURES: an integer that indicates the number of valid signature an ACL must provide to be considered valid
On startup, the agent requests one ACL for each group listed in MEMBER_OF, and receives a JSON document containing a list of permissions to modules, and a list of signatures. The agent verifies each signature, and if enough are valid to satisfy the value of REQUIRED_ACL_SIGNATURES, loads the ACL.
The verification process MUST require that at least two signatures from two different MASTERKEYS are provided for an ACL to be valid, thus preventing a single MASTERKEY compromise from putting the infrastructure at risk. In production, the MASTERKEYS array will contain 5 to 10 keys from trusted individuals, and we will probably require three valid signatures to update an ACL. But to facilitate development and adoption in small environments, REQUIRED_ACL_SIGNATURES is made configurable and set to 1 by default.
The format of the JSON document is described below:
{
"modules": {
"default": {
"minimumweight": 2,
"investigators": {
"Bob Kelso": {
"fingerprint": "E60892BB....",
"weight": 2
},
"Morpheus": {
"fingerprint": "AD595634....",
"weight": 3
}
}
},
"mymodule1": {
"minimumweight": 1,
"investigators": {
"Jean-Kevin": {
"fingerprint": "E60892BB...",
"weight": 1
}
}
}
},
"pgpsignatures": [
"wsBcBAABCAAQBQJVWjgMCRC0wd1JzRH...",
"SQNRn2dG7abwbID7bV7+F+TdtIzOhfq..."
]
}In environment where builtin configuration is disabled, the variables MEMBER_OF and MASTERKEY will be read from the configuration file in <config_dir>/mig-agent.cfg. In the future, removing these variables from the built-in configuration will also allow us to distribute compiled binaries of the agent publicly.
RISKS
Building ACLs into the agent binary currently provides a good level of security, which we are removing. Providing a mechanism to override ACLs opens the door to privilege escalation attacks. Requiring multiple signatures on ACLs mitigates that risks if the keys listed in the MASTERKEYS are securely kept, and if enough signatures are required.
I would like to hear what others think of this plan. Implementation will probably wait until Q3 anyway, so we have time to discuss it.
On 2015-07-07 15:31:12 -0700, Guillaume Destuynder [:kang] wrote:
Doc
doc nit: According to this, the MASTERKEYS should also be configurable to a minimum of 2 so that it works with the defaults
doc nit: Should specify that by ACLs we mean the investigators, their weight, fingerprint, and minimum required weight per MIG module.on topic
I think the signature vs builtin trade off is good.
on format
It doesn't explain how the pgp signatures are generated (which data can you exactly verify, json doc without the pgpsignatures entry?)
on logic
How does it work exactly for the MEMBER_OF selection?
If my understanding is correct:With the scenario of the attacker having control of the server queue - I could see an attack where he would change the MEMBER_OF to a group of keys he has control of (even if those are signed by masterkeys), weakening the system.
this means MEMBER_OF needs to also be strongly verified, otherwise we have no other security than checking the keys have been signed by a MASTERKEY.
On 2015-07-10 06:57:27 -0700, Aaron Meihm [:alm] wrote:
Some questions I have on this:
- How will the new ACLs be published to the database? Is this anticipated to happen through the API, or would we have some out-of-band tool to sign an ACL and install it in the database? The latter is probably the best way to ensure compromise of the API/scheduler can't result in escalation of privilege in the agent. In the past the scheduler/API has always been considered "less trusted" then the agent, so that's one thing to consider for this is we are now relying more on the security of the API/scheduler for the agent itself. Granted, the ACL still needs a valid signature, but that wouldn't stop somebody from owning the API and pushing a intentionally bad ACL into the system to stop the agents from operating correctly.
- Should we include a "lifetime" for the ACL, so if new ACLs are not received after a given period of time the agent reverts to some default ACL configuration?
- Should we include a global group, that always passes for a given agent to avoid needing to duplicate users into multiple groups in the ACL?
On 2015-08-02 07:30:29 -0700, Julien Vehent [:ulfr] wrote:
With the scenario of the attacker having control of the server queue - I could see an attack where he would change the MEMBER_OF to a group of keys he has control of (even if those are signed by masterkeys), weakening the system.
I don't see this as a concern, because for that attack to happen, the attacker would need permission to change the configuration of the agent. That means either commit access to the configuration management system (puppet) or root on the target host. In both cases, that level of access give the attacker full root, and compromising MIG is probably the least of our worries.
How will the new ACLs be published to the database?
We will use code in the MIG console to build, sign and post ACLs to the API. The API endpoint will simply store the ACL in the database. If someone owns the API or database, they won't be able to sign fraudulent ACLs.
Should we include a "lifetime" for the ACL?
I can see pros and cons with this. It's definitely a good way to prevent an ACL from going stale and grant access to users who have left the company. It might also add some amount of work on our side to resign the ACL on a regular basis. I think we can decide on this later, and for now make sure we store the necessary timestamps.
Should we include a global group?
Yes. It's the group named
default
in comment 1.
On 2015-08-02 09:29:17 -0700, Julien Vehent [:ulfr] wrote:
I started implementation. We have two options for managing multiple ACLs: merging them, or treating them independently.
=== Merging ACLs ===
Given an agent that is member of two groups: base and mygroup. The agent retrieves two signed ACLs from the API, one for each group.Each ACL contains a list of modules, each containing a list of investigator's keyid and weight.
The agent loads the first ACL
base
(or whatever is first in the MEMBEROF array). Then loads the second ACLmygroup
, and for each module that is present in both ACLs, merges the content.Meaning that base->file->investigators and mygroup->file->investigators will be merged together. Any investigator present in both will be kept only ones.
It is possible that both ACLs will have different values for
minimumweight
, in which case only the highest value should be kept.
Similarly, it is possible that a given investigator present in both lists would have a differentweight
, in which case only the highest value is kept.=== Treating ACLs independently ===
We could also treat ACLs independently, and thus avoid the complexity of merging their content. It would make the code slightly simpler, and probably remove confusion when defining new ACLs that do not behave as expected because another ACL is being merged on top.
The downside is dealing with
minimumweight
across ACLs. A given action could have two signatures from investigators in two different ACLs.
- The first signature could belong to investigator X, have a weight of 2 and a minimum weight of 3.
- The second signature could belong to investigator Y, have a weight of 2 and minimum weight of 5.
In this situation, the total weight would be 4, which satisfies the minimumweight of the first ACL, but not the second one. What should MIG do in this case is an open question...
I haven't decided which approach to take yet. Any thought?
On 2015-08-03 04:30:39 -0700, Aaron Meihm [:alm] wrote:
It seems the simplest approach would be to treat it like an access list. Merging the ACLs sounds useful but would the behavior outlined always hold as being desired? For example, would we always merge in the higher weight ACL, or would there be a scenario where if somebody has assigned a lower weight to the ACL, that would win in the merge logic as we'd default to the lowest agreed upon privilege?
In the second example, would it make things simpler if we just have a minimumweight value for modules that is the same across ACLs to avoid the confusion?
On 2015-08-03 07:45:28 -0700, Julien Vehent [:ulfr] wrote:
The problem with detecting conflicts server-side is that the server doesn't know which ACLs an agent will request, so it cannot detect in advance that ACLs A and B have conflicting values.
I like the idea of merging the ACLs better than treating them separately, and I agree with you that we should default to better security, with the following rules:
- When merging ACLs that have different
minimumweight
values for a given module, the highestminimumweight
value wins.- When merging investigators that have different weight on a given module, the lowest weight (least privilege) value wins.
Sounds good?
On 2015-08-03 09:34:20 -0700, Julien Vehent [:ulfr] wrote:
Here's an example of ACL:
{ "name": "testacl", "validfrom": "2015-08-01T12:05:23.008403417-04:00", "expireafter": "2015-08-31T12:05:23.008403445-04:00", "permissions": [ { "module": "testmodule", "minimumweight": 1, "investigators": [ { "id": 1, "name": "Test Investigator", "pgpfingerprint": "abcdef0123456789", "weight": 1 } ] } ], "pgpsignatures": [ "wpwEAAEIABAFAlW/k2QJEF4AASq0LBwdAADY3gQACOKNzFlw1mQNB92vyeUUnCeD6CeGI+qlACrIgOIr09FJQOip7X4rRqrr4oi3e0oj9upqH0TiWacJBVK3VubD5J7NcGNlSjgcwe6oVtt2ieMci8z2ZiXt4+eJ/bUfRsAKw+c9qV0oi0Sxm4cdxsEBwratQrMFUv3qIcoPjSkLx4==Sw35" ] }
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=896579
Assigned to: Julien Vehent [:ulfr]
On 2013-07-22 10:59:30 -0700, Julien Vehent [:ulfr] wrote:
mounting points such as NFS and CIFS shouldn't be inspected. Or at least, there should be an option to prevent their inspection.
On 2014-05-25 10:31:51 -0700, Marco Vanotti wrote:
This can be achieved using the statfs syscall1(syscall.Statfs in Go), and then checking the Type attribute from the result.
However, should this be checked only against NFS and CIFS? (what about fuse? something mounted via sshfs should be avoided too?).Is there a better way to do this?
On 2014-05-30 23:56:16 -0700, Marco Vanotti wrote:
An update for this issue:
In bsd, the statfs syscall gives you more info1: there's a f_flags field that will tell you if the filesystem is local (MNT_LOCAL). I think that will work for NFS/CIFS but reading freebsd's source code, it looks like FUSE filesystems are considered as local2(even if it's a sshfs filesystem).
In linux there's also a f_flag field in statfs, but it doesn't have the MNT_LOCAL flag.
FUSE filesystems don't report properly their f_type, so it will be hard to check.
I'm not sure if the syscall is actually waiting for some response from the filesystem. At least with sshfs it is laggy.
Maybe the easiest way will be to parse /proc/mounts and check against a list of mounted fs.
On 2014-05-31 09:27:24 -0700, Julien Vehent [:ulfr] wrote:
Thanks for the research Marco.
I played with the statfs syscall in Go a little bit: https://gist.github.com/jvehent/8d86426a0b0918131410
The above code is for Linux. It looks like building a blacklist of FS Types is the way to go.About Fuse, I'm getting mixed results on Fedora 20. One path returns 0x27E0EB, another one 0x65735543. I'll need to look into it further to figure out why.
On 2014-05-31 12:23:07 -0700, Marco Vanotti wrote:
Cool!
About FUSE: Both results look weird, in Ubuntu 13.10 (kernel 3.11) I'm getting: 0x65735546 which is also defined in 1. Reading previous mailing list discussions2 it looks like the only way to identify a fuse filesystem is by parsing /proc/mounts.
The problem with using statfs is that we are accessing the remote filesystem somehow. If there's an error with sshfs or nfs (for example, remote server disconnected), the program will wait forever. I think that's a problem that we should try to avoid.
If we parse /proc/mounts before starting we can get an idea of which filesystems are mounted and have a map[mountpoint]type to ask after accessing each folder. The problem with this is that it may happen that a remote fs can be mounted after parsing /proc/mounts.
Parsing /proc/mounts in Go: https://gist.github.com/mvanotti/66f23c32f1aa4cc2f042
Currently the agent package targets produce filenames that are slightly different depending on the type of package. For example if it is .deb or .rpm we end up with _'s or -'s in different places which adds complexity to automating fetch of the package using the filename.
This may be required for a given package type, but if not it would be nice to standardize on a common name to simplify.
I'm not sure the method mig uses to find the gpg key but it might be extended to read the gpg conf file and use the key location indicated in the conf.
It also might be good to improve the error message to clarify that "no suitable key found" means something like
"Unable to find GPG key after searching in these places. To manually set your gpg key do this"
mig file --path /home -path /root -name "^authorized_keys" -content "foo"
no configuration file found at /home/jdoe/.migrc
would you like to generate a new configuration file at /home/jdoe/.migrc? Y/n> y
ReadConfiguration() -> MakeConfiguration() -> no suitable key found
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1071279
Assigned to: Julien Vehent [:ulfr]
On 2014-09-22 15:08:29 -0700, Guillaume Destuynder [:kang] wrote:
This could be either done in the netstat module either by chaining the filechecker module with the netstat module.
Feel free to edit the summary if you want to chain the modules.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1134394
Assigned to: Julien Vehent [:ulfr]
On 2015-02-18 13:59:00 -0800, Brian Hourigan [:digi] wrote:
Our greenfield AWS environment nubis does not support remote access via ssh. We don't want to configure a break-glass account with credentials that any employee could of seen, otherwise we would be forced to redeploy everything during an exit and also have the burden of managing these credentials.
After talking with :ulfr we think the best way to facilitate emergency access is to create a mig plugin that would add a user, and configure some public ssh key.
On 2015-02-18 14:27:30 -0800, Julien Vehent [:ulfr] wrote:
This should be built into its own plugin, and not reuse the account plugin planned in bug 1037965. Having it in a separate module will allow us to limit the individuals having access to it, such that only opsec and a few selected ops can invoke it.
The module should support two operations:
- open emergency access: start sshd and add a public ssh key to a given user (either create a new user, or add it to /root/.ssh/authorized_keys)
- close emergency access: stop sshd and remove keys previously added (need to figure out how, maybe by restoring a backup file?)
On 2015-02-18 14:54:11 -0800, Brian Hourigan [:digi] wrote:
- close emergency access: stop sshd and remove keys previously added (need
to figure out how, maybe by restoring a backup file?)I'd vote to not have this feature. Once an AMI has been accessed it is 'dirty' and should be terminated and re-deployed from a known AMI.
The scheduler in AWS regularly loses the connection to rabbitmq, possibly because the ELBs in front of the rabbitmq cluster are rotating, or because long lived connections are killed. When that happens, the scheduler continues to live in a state of limbo, not receiving any message anymore.
The scheduler should either exit entirely, like mig-agent does, which would force systemd/upstart to spawn a new scheduler with a fresh connection, or attempt to reconnect.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1109877
Assigned to: Julien Vehent [:ulfr]
On 2014-12-10 13:54:06 -0800, Julien Vehent [:ulfr] wrote:
The MIG scheduler should be able to output logs into MozDef's AMQP endpoint in a proper JSON format.
Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1014719
Assigned to: Julien Vehent [:ulfr]
On 2014-05-22 10:50:01 -0700, Julien Vehent [:ulfr] wrote:
The PCAP go package would be a good candidate for this: http://godoc.org/code.google.com/p/gopacket/pcap
The module would not return data to the scheduler, but only answer questions such as "is this host sending DNS requests for that domain".
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.