• Install from scratch
• Run any zeustracker feed
• Create an export feed for tag zeus
• Manually run export
• Export is empty

Add a censys.io one time analytic in the investigation view

Description

When having multiple investigations popping up from different analysts, it would be nice to know who created (started/working on/owns) which.

Description

Some times Im clumsy and ad a link between two nodes that should not be there. Then I get sad when I cant delete it. Solution for has been to delete on of the nodes, reload the webpage, add the node and re-add new links. If I dont reload the webpage, it will add the old links if you try to add the node again.

Environment

Steps to Reproduce

1. In investigation, add two nodes.
2. Add a link between the nodes
3. No way to delete the link, help :)

Expected behavior

Some button or way to delete just the link, without doing the long hack of deleting the node and all its link.

Actual behavior

No way to delete links :(

Can anyone share the configuration to link Yeti to MISP platform?
Thanks in advance!

Description

Support the Virustotal public and private API, similar to the enrichment from PassiveTotal.

Public API documentation can be found here:
https://www.virustotal.com/en/documentation/public-api/

Private API documentation can be found here:
https://www.virustotal.com/en/documentation/private-api/

Yeti generates a 502 error when I try to add an IDN in the database.

eg. Searching for a URL also queries its domain, etc.

Add the ability to tag email observables

Be able to search for http://github.com when entering hxxp://github[.].com

Description

I am embarrassed to ask, but I cannot find the default creds. It's certainly possible I missed a config file, but can't seem to find it.

I looked in yeti/core/auth/local/user_management.py and found:

def get_default_user():
    try:
        # Assume authentication is anonymous if only 1 user
        if User.objects.count() == 1:
            return User.objects.get(username="yeti")
        return AnonymousUserMixin()
    except DoesNotExist:
        return create_user("yeti", "yeti")

...but yeti:yeti didn't seem to work.

Environment

Steps to Reproduce

Browse to http://server:5000
I attempted admin:admin, admin:password, yeti:yeti, and every combination in between.

Expected behavior

Log into the web app

Actual behavior

Invalid credentials

Send files to Yeti so that it is stored in the database as an Observable (of type File), derive hashes automatically.

Description

After creating a template, it does not show up in Exports tab in the drop down menu for templates in New export. Reloading the page populates the drop down menu.

Environment

Steps to Reproduce

1. Add new template (/dataflows#export-templates)
2. Click on the Export tab
3. Make a new export. The new template is missing.

Expected behavior

New template should be there.

Actual behavior

New template is not there.

Add the ability to perform Whois analytics on hostnames not just domain names.

Question

Can you recommend on how to update the Yeti instance with the latest version without losing your data such as observables investigations, etc.?

Query search view is confusing. Redesign it to put important information first and ease navigation

Description

I get an "Internal Server Error" when I try to connect to http://localhost:5000

Environment

Steps to Reproduce

1. $ python yeti.py
2. http://localhost:5000
3. Internal Server Error

Expected behavior

Expect the web UI to start up.

Actual behavior

Browser hangs for a few seconds and then throws an error.

"Internal Server Error

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application."

I've also installed threat_note which is similar platform to Yeti. When I have it running I also have to connect to http://localhost:5000 in the browser. When I run Yeti and try to open in the browser the error message above is displayed and the favicon on the browser tab is the threat_note favicon; however, I didn't have threat_note running at the time.

I'm wondering if these two platforms are colliding in some way even though I'm only running one at a time. I've attached the error message from the command line.

Thanks!
yeti_error.txt

It appears that the Palevo Tracker has been discontinued and may need to be removed from the public feeds.

https://palevotracker.abuse.ch/?rssfeed

Description

Yeti is throwing an exception when we attempt to upload observables. We are having difficulties identifying the commonalities between the observables that fail and the observables that work. We believe it might be associated with uploading large numbers of duplicate observables.

Has anyone seen this exception before?

Apr  4 14:56:27 <hostname> uwsgi[13723]: [2017-04-04 14:56:27,095] ERROR in app: Exception on /observable/ [POST]
Apr  4 14:56:27 <hostname> uwsgi[13723]: Traceback (most recent call last):
Apr  4 14:56:27 <hostname> uwsgi[13723]:   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1982, in wsgi_app
Apr  4 14:56:27 <hostname> uwsgi[13723]:     response = self.full_dispatch_request()
Apr  4 14:56:27 <hostname> uwsgi[13723]:   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1614, in full_dispatch_request
Apr  4 14:56:27 <hostname> uwsgi[13723]:     rv = self.handle_user_exception(e)
Apr  4 14:56:27 <hostname> uwsgi[13723]:   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1517, in handle_user_exception
Apr  4 14:56:27 <hostname> uwsgi[13723]:     reraise(exc_type, exc_value, tb)
Apr  4 14:56:27 <hostname> uwsgi[13723]:   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1612, in full_dispatch_request
Apr  4 14:56:27 <hostname> uwsgi[13723]:     rv = self.dispatch_request()
Apr  4 14:56:27 <hostname> uwsgi[13723]:   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1598, in dispatch_request
Apr  4 14:56:27 <hostname> uwsgi[13723]:     return self.view_functions[rule.endpoint](**req.view_args)
Apr  4 14:56:27 <hostname> uwsgi[13723]:   File "/usr/local/lib/python2.7/dist-packages/flask_classy.py", line 200, in proxy
Apr  4 14:56:27 <hostname> uwsgi[13723]:     response = view(**request.view_args)
Apr  4 14:56:27 <hostname> uwsgi[13723]:   File "./core/web/helpers.py", line 32, in inner
Apr  4 14:56:27 <hostname> uwsgi[13723]:     return f(*args, **kwargs)
Apr  4 14:56:27 <hostname> uwsgi[13723]:   File "./core/web/frontend/observables.py", line 87, in index
Apr  4 14:56:27 <hostname> uwsgi[13723]:     o = Observable.add_text(txt)
Apr  4 14:56:27 <hostname> uwsgi[13723]:   File "./core/observables/observable.py", line 105, in add_text
Apr  4 14:56:27 <hostname> uwsgi[13723]:     o = Observable.guess_type(text).get_or_create(value=text)
Apr  4 14:56:27 <hostname> uwsgi[13723]:   File "./core/database.py", line 297, in get_or_create
Apr  4 14:56:27 <hostname> uwsgi[13723]:     return cls.objects.get(value=obj.value)
Apr  4 14:56:27 <hostname> uwsgi[13723]:   File "/usr/local/lib/python2.7/dist-packages/mongoengine/queryset/base.py", line 271, in get
Apr  4 14:56:27 <hostname> uwsgi[13723]:     raise queryset._document.DoesNotExist(msg)
Apr  4 14:56:27 <hostname> uwsgi[13723]: DoesNotExist: Observable.Ip matching query does not exist.

Environment

Steps to Reproduce

1. Upload more than 10 observables
2. Upload them again

Expected behavior

Observables should be inserted into Yeti

Actual behavior

We are receiving an "500 Internal Server Error" instead

Description

Just need to delete a saved investigation as I was just testing.

Environment

Steps to Reproduce

1. Add new investigation via new malware
2. Attempt to delete the test investigation

Expected behavior

To be able to remove the investication

Actual behavior

There's no button or link or anything that allows removing an investigation.

Description

All step was fine until i start python2 yeti.py (tried also ./yeti.py)
[+] Yeti started. Point browser to http://localhost:5000/
Any clue?
When i try to open page all the noise come up:

ERROR:core.web.webapp:Exception on / [GET]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1473, in full_dispatch_request
rv = self.preprocess_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1666, in preprocess_request
rv = func()
File "/home/docplague/Scrivania/arsenale/yeti/core/web/webapp.py", line 53, in frontend_login_required
if not current_user.is_active and (request.endpoint and request.endpoint != 'frontend.static'):
File "/usr/lib/python2.7/dist-packages/werkzeug/local.py", line 343, in getattr
return getattr(self._get_current_object(), name)
File "/usr/lib/python2.7/dist-packages/werkzeug/local.py", line 302, in _get_current_object
return self.__local()
File "/usr/local/lib/python2.7/dist-packages/flask_login/utils.py", line 26, in
current_user = LocalProxy(lambda: _get_user())
File "/usr/local/lib/python2.7/dist-packages/flask_login/utils.py", line 302, in _get_user
current_app.login_manager._load_user()
File "/usr/local/lib/python2.7/dist-packages/flask_login/login_manager.py", line 313, in _load_user
return self._load_from_request(request)
File "/usr/local/lib/python2.7/dist-packages/flask_login/login_manager.py", line 370, in _load_from_request
user = self.request_callback(request)
File "/home/docplague/Scrivania/arsenale/yeti/core/web/webapp.py", line 44, in api_auth
return User.objects.get(api_key=request.headers.get('X-Api-Key'))
File "/usr/local/lib/python2.7/dist-packages/mongoengine/queryset/manager.py", line 37, in get
queryset = queryset_class(owner, owner._get_collection())
File "/usr/local/lib/python2.7/dist-packages/mongoengine/document.py", line 206, in _get_collection
cls.ensure_indexes()
File "/usr/local/lib/python2.7/dist-packages/mongoengine/document.py", line 836, in ensure_indexes
collection.create_index(fields, background=background, **opts)
File "/usr/lib/python2.7/dist-packages/pymongo/collection.py", line 1529, in create_index
self.__create_index(keys, kwargs)
File "/usr/lib/python2.7/dist-packages/pymongo/collection.py", line 1417, in __create_index
with self._socket_for_writes() as sock_info:
File "/usr/lib/python2.7/contextlib.py", line 17, in enter
return self.gen.next()
File "/usr/lib/python2.7/dist-packages/pymongo/mongo_client.py", line 823, in _get_socket
server = self._get_topology().select_server(selector)
File "/usr/lib/python2.7/dist-packages/pymongo/topology.py", line 214, in select_server
address))
File "/usr/lib/python2.7/dist-packages/pymongo/topology.py", line 189, in select_servers
self._error_message(selector))
ServerSelectionTimeoutError: localhost:27017: [Errno 111] Connection refused

Environment

Debian 8.5

Steps to Reproduce

1. git clone the repo
2. cd yeti
3. pip install -r
4. observe stacktrace

Expected behavior

[How are you expecting the application to behave?]

Actual behavior

[How is the application behaving? (include any stacktraces, logs, screenshots, etc.)]

Description

Running "curl https://raw.githubusercontent.com/yeti-platform/yeti/master/extras/bootstrap.sh | sudo /bin/bash" in terminal clone and downloads as expected until collecting "pythonwhois" from requirements.txt entry "git+git://github.com/joepie91/python-whois#egg=pythonwhois" (line 15)

Occurs when repo is cloned as well.

Environment

Steps to Reproduce

1. git clone the repo
   

2. cd yeti

3. pip install -r requirements.txt

Expected behavior

Expect complete installation.

Actual behavior

Incomplete installation, no IP address is provided. Not clear if this is an error in requirements.txt or version.

Description

Saved investigation graphs should retain their name

Environment

Steps to Reproduce

1. Make a new investigation
2. Make a graph and name it
3. Leave and come back

Expected behavior

Graph should retain name.

Actual behavior

Graph is unnamed

Add a shodan.io one time analytic in the investigation view

• multi tag
• single tag

First, GREAT app...just what I've been looking for. Second, in reading the API docs, is there a way to submit a file via API? Thank you.

Build equivalence dictionaries for tags, eg. if an observable is tagged c&c automatically change that to c2.

Description

I run the quick install for Yeti and after all is back to command prompt, I then try to move to the next step.
I have tried the pip install -r requirements.txt (no such file or directory)

Also, I cannot find yeti.py anywhere on my system. I have done the grep and looked in every file and folder i can find. I rebuilt my Ubuntu using 16.04 LTS in a virtual environment.

I ran the command
"sudo apt-get install build-essential git python-dev mongodb redis-server libxml2-dev libxslt-dev zlib1g-dev python-virtualenv"

Then "sudo pip install -r requirements.txt" it tells me no such file or directory

I skip that and try to run the "./yeti.py" doesn't find anything. I need some help.

Want to get this up and going, however, doesn't seem as easy as I was hoping.
I'm a Linux novice, but though that the instructions were fairly simple.
I suppose they are when they work.

Just a quick fix in the install docs. The systemd scripts for "Feeds" and "Exports" have unbalanced quotes.

https://github.com/daniel-gallagher/yeti/commit/b6597582b8ed43d3ea344e0a7ec5afb2f7054e8d

It would be helpful to have some graphs representing metrics on the Observable/tags to give you an over of the data set over the last 7/30 days, I was thinking about the following:

1. Stacked bargraph showing observable counts by sources over time
2. Piechart showing data type distribution
3. Piechart showing tag distribution
4. Piechart showing feed source distribution
5. table with sparklines showing feed activity by feed name over time (quick way to eyeball failures or defunct feeds)

I would assume it would entail some aggregations of mongo data and maybe something like pygal. Don't mind taking a crack at it if it seems like a good idea, but looking for some suggestions on what to do the charts in,

Another option would be to put together a Splunk app, but that would limit it to folks who had access to Splunk. I do, but not everyone does,

When tags are renamed from admin, changes are not applied to observables.

Description

Additional feed plugins for openphish, phishtank, and vxvault. I'll try to post more as time allows and figured this would be the best way since I do not have push access to the repo.

feeds.zip

Description

1. Export
   Be able to export investigations in a structural way (json etc).
   The export should contain at least the data displayed in the graph, probably with options to include descriptions, tags (include/exclude tags?), observables, context, TLP (Traffic Light Protocol (if the concept is ever implemented), etc.

2. Import
   Be able to import an investigation (previously exported from yeti). Maybe with options to add a tag or source (like a feed) etc. (and maybe import it with a TLP constraint to the user importing it? if the TLP concept is ever implemented).

Be able to import x509 certificates and run analytics on them (censys.io...)

RUN systemctl daemon-reload
---> Running in 7a7a7661c963
Failed to connect to bus: No such file or directory
os:ubuntu 16.04

Description

Add support for Metadefeder Cloud public APIs and Threat Intel Feeds.

• Metadefender Cloud Public APIs
• Metadefender Cloud Threat Intelligence Feeds

• List loaded feeds
• Control (enable / disable / run) particular feeds
• List loaded analyzers
• Control (enable / disable / run) particular analyzers

There are several links to https://raw.githubusercontent.com/yeti-platform/yeti/master/extras/bootstrap.sh

These should be updated to centos_bootstrap.sh and ubuntu_bootstrap.sh, respectively.

(also at http://yeti-platform.readthedocs.io/en/latest/installation.html)

Description

From what I can tell, importing domain names when those names have an underscore character does not work. Here are some samples (replaced the periods to prevent people from clicking on potentially malicious domain links...):

ERROR:root:Invalid hostname: gb_validateapplecareassitancesecurelog_in(dot)verfiction1(dot)link
ERROR:root:Invalid hostname: gb_validateapplecareassitancesecurelog_in(dot)vrfied1(dot)link
ERROR:root:Invalid hostname: gb_validateapplecareassitancesecurelog_in(dot)vrify1(dot)link
ERROR:root:Invalid hostname: hack_king10(dot)net23(dot)net

My initial thought is this may be related to the tldexport package used to split up the hostname for validation but I haven't had time to fully run through it.

Environment

Steps to Reproduce

Import hostname observable with an underscore via a feed script.

Expected behavior

No, don't die.

Actual behavior

Death to domains with underscores.

I am unable to do export as export format drop down box is empty and hence unable to save the export.

OS: Ubuntu 16.04
Browser: firefox, chrome

Hello,

After adding some users on the system, it is not possible to logout of yeti account. So it is not possible to disable it from the panel.

Best regards,

Write documentation for updating Yeti and making backups.

Create a timeline from an investigation.

When launching ./yeti.py, I get:

ImportError: No module named flask_restful

The python librairy flask-restful should be added to requirements.txt.

Mixed Content: The page at 'https://hostname.com/investigation/graph/observable/577f910a4e563b05955cac9b' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://hostname.com/observable/577f9be24e563b1b5bf05876'. This request has been blocked; the content must be served over HTTPS.

Description

[Can't start yeti service]

Environment

Linux kali 4.9.0-kali3-amd64 #1 SMP Debian 4.9.16-1kali1 (2017-03-24) x86_64 GNU/Linux

Steps to Reproduce

1. git clone the repo
2. cd yeti
3. pip install -r
4. observe stacktrace

Expected behavior

[How are you expecting the application to behave?]
Start on local port 5000

Actual behavior

File "yeti.py", line 11
print "[+] Yeti started. Point browser to http://localhost:5000/"

• Malware
• Actors
• Indicators

Add the ability to add BTC addresses as observable