google / vulncode-db Goto Github PK
View Code? Open in Web Editor NEWVulncode-DB project
Home Page: https://www.vulncode-db.com
License: Apache License 2.0
Vulncode-DB project
Home Page: https://www.vulncode-db.com
License: Apache License 2.0
Currently, the application uses different identifiers such as:
id
suggested_id
vuln_id
vcdb_id
cve_id
We should write a wrapper class and add type annotations to avoid any confusion with this.
Increase the number of entries with patches assigned from 3.7k to >10k.
Allow to modify the following fields:
As we're not using TypeScript we're prone to unexpected JS issues such as with silent third-party dependency updates which break code on our side.
We should generally investigate ways to improve the JS code quality and maintainability for code such as
We should make sure people can delete their accounts if there have been no contributions. Otherwise, I'm not sure how to approach this best as accounts might have data dependencies. For this reason, it's not possible to delete accounts from the admin UI either if an account has an existing/approved proposal.
Adding this to this milestone for now but this can likely be moved out, too.
We need metrics regarding user activity to see how well the project is doing.
One option might be to implement Google Analytics server side.
It would be a neat quality-of-life feature to have the vuln ID appear in the title, for easier navigation between browser tabs.
A quick look at the app routes shows that the vuln_view.html
could be changed to update the document title, possibly inside the <script> tag.
Maybe something along the lines of:
<script>
...
document.title = "{{ vulnerability_details.id }} - Vulncode-DB";
</script>
I have not tested this edit, but I would love to see such a feature at vulncode-db.com
Fix a redirect bug for loading file content when the entry uses a custom repository.
We should quickly check and test the OAuth changes provided by commit: 1b738f9
Project KB contains manually curated commit links which fix particular CVE. It contains commits which actually rectified "vulnerable code" unlike NVD which many times contains commits which tagged release.
Vulncode-db can leverage project KB's data and provide more examples of real world vulnerable code.
Data is at :
https://github.com/SAP/project-kb/blob/master/MSR2019/dataset/vulas_db_msr2019_release.csv
https://github.com/SAP/project-kb/tree/vulnerability-data/statements
FYI project KB is used by https://github.com/eclipse/steady .
Create a first release entry for the Github repository.
We could likely use:
curl -H "Accept: application/vnd.github.v3+json" 'https://api.github.com/repos/google/vulncode-db/releases'
to then fetch and generate a .html
release file with markdown support that can be included in prod.
We're currently serving some content like specific JS files over Cloudflare and other providers like Bootstrap.
Instead, we should directly provide these files ourselves to avoid extensions which block such providers from breaking the service for some users.
The review system only keeps track of one single reviewer feedback per proposal.
This means new feedback will overwrite old feedback.
We'll likely have to introduce a new table like feedback_history
to keep a full history of what feedback was given so far for a new reviewer to better understand the full context.
Vulncode-db uses https://github.com/kotakanbe/go-cve-dictionary which is present as a git submodule. It uses https://github.com/kotakanbe/go-cve-dictionary/tree/5fe52611f0b8dff9f95374d9cd7bdb23cc5fc67a to be more specific which is 1+ year old version.
https://github.com/kotakanbe/go-cve-dictionary/tree/5fe52611f0b8dff9f95374d9cd7bdb23cc5fc67a no longer works due to some changes. Here's the log :
Sending build context to Docker daemon 608.8kB
Step 1/16 : FROM golang:alpine as builder
---> 1a87ceb1ace5
Step 2/16 : RUN apk add --no-cache git make gcc musl-dev
---> Using cache
---> 90867a3f9751
Step 3/16 : ENV REPOSITORY github.com/kotakanbe/go-cve-dictionary
---> Using cache
---> 2fc568001ee8
Step 4/16 : COPY . $GOPATH/src/$REPOSITORY
---> 28d1be732cc6
Step 5/16 : RUN cd $GOPATH/src/$REPOSITORY && make install
---> Running in 24a0108db67c
go get -u github.com/golang/dep/...
dep ensure -v
# Gopkg.lock is out of sync with Gopkg.toml and project imports:
github.com/pkg/errors: imported or required, but missing from Gopkg.lock's input-imports
Due to this ./docker-admin.sh init
does not work too.
./docker-admin.sh init
works alright after updating the go-cve-dictionary
to their current master branch.
Please update go-cve-dictionary
Currently, it's possible to delete entries that have been marked as 'archived'.
This will lead to unexpected gaps in the change history.
We should fix this if this is not WAI.
For historic reasons when running the application outside of Docker all Python dependencies were meant to be installed to the third_party
directory as AppEngine required them to be shipped.
With the newest AppEngine version this is not required anymore. We should move any third_party
dependencies to be installed inside a virtual environment instead and remove third_party
directory mentions from the code.
The navigation bar for production is deprecated.
Replace with a version that automatically considers the newest version in the repostitory.
The latest changes have broken the code preview in widgets such as can be seen with the Heartbleed example in:
We should make sure that this is fixed before the new release.
Things like code annotations are meant to be working in the next milestone.
Until then we should simplify the UI and remove any data or functionality which is not working and might cause confusion.
One of the main problem of the NVD / CVE raw data is that they are not enough qualified so that SAST vendors can easily find vulnerabilities corresponding to a given language or type of vuln.
Having such information will really help and speed up the development of enhancements in SAST products. For example, I would like to get a full list of XSS vulnerabilities involving Python (*.py) files.
Currently VulnCode-DB assumes every link which passes the regex
Line 121 in e6c5008
vulncode-db/data/models/nvd.py
Line 301 in e6c5008
https://www.vulncode-db.com/CVE-2018-7749 the patch link is identified as ronf/asyncssh@c161e26 . But this link does not contain any example of "vulnerable code".
Such type of error(s) can be avoided if we add one more condition for a link to be considered as a patch, this condition would be to check whether NVD tags the reference/link as a Patch
. For CVE-2018-7749
NVD has rightly tagged it as a Third Party Advisory
NOT Patch
at https://nvd.nist.gov/vuln/detail/CVE-2018-7749 .
If such errors don't occur at considerable frequency close this ticket.
The OAuth Client ID / consent screen is currently configured under google.com:ruslanh-vulncodedb
.
Instead, it should be running fully under vulncode-db
.
We currently do handle sqlakeyset errors poorly. We should explicitly handle them and show nicer error messages.
Makes communication easier.
Changing the CVE-ID in the proposal section seems to break things:
The CDN changes introduced with #31 have broken the JSTree view for files. Particularly, the folder and file icons are missing. We should fix this before releasing.
=========================================================== warnings summary ===========================================================
tests/app_tests/api/test_routes.py::test_save_editor_data[query0-data0-400-expected_response0]
/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:1917: SAWarning: Setting backref / back_populates on relationship RepositoryFileComments.commit to refer to viewonly relationship VulnerabilityGitCommits.comments will be deprecated in SQLAlchemy 1.4, and will be disallowed in a future release. viewonly relationships should not be mutated (this warning may be suppressed after 10 occurrences)
(self, other),
tests/app_tests/api/test_routes.py::test_save_editor_data[query0-data0-400-expected_response0]
/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:1917: SAWarning: Setting backref / back_populates on relationship VulnerabilityGitCommits.comments to refer to viewonly relationship RepositoryFileComments.commit will be deprecated in SQLAlchemy 1.4, and will be disallowed in a future release. viewonly relationships should not be mutated (this warning may be suppressed after 10 occurrences)
(self, other),
tests/app_tests/api/test_routes.py::test_save_editor_data[query0-data0-400-expected_response0]
/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:1917: SAWarning: Setting backref / back_populates on relationship RepositoryFileMarkers.commit to refer to viewonly relationship VulnerabilityGitCommits.markers will be deprecated in SQLAlchemy 1.4, and will be disallowed in a future release. viewonly relationships should not be mutated (this warning may be suppressed after 10 occurrences)
(self, other),
tests/app_tests/api/test_routes.py::test_save_editor_data[query0-data0-400-expected_response0]
/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:1917: SAWarning: Setting backref / back_populates on relationship VulnerabilityGitCommits.markers to refer to viewonly relationship RepositoryFileMarkers.commit will be deprecated in SQLAlchemy 1.4, and will be disallowed in a future release. viewonly relationships should not be mutated (this warning may be suppressed after 10 occurrences)
(self, other),
-- Docs: https://docs.pytest.org/en/latest/warnings.html
Some actions like clicking "propose changes" in the vulnerability view currently show a restrictive "forbidden" when you're not logged in.
Instead, we should:
a) If the user is not logged in forward to the login page after which one gets forwarded to the original page.
b) Otherwise, check if the authN/Z conditions are met and if not redirect to the previous page and flash an error which displays that this action can't be performed due to a lack of permissions.
Otherwise, the application flow breaks unexpectedly which will likely deter people from using it.
We should make sure we avoid regressions regarding the project setup process.
Currently, we advice to execute steps like ./setup.sh
and ./docker/docker-admin.sh init
to get things started but we don't regularly verify if this works correctly.
Suggestion: Add a daily job which builds the project and verifies that the base version works as intended.
We should display an icon (at the top right?) that links to this issue tracker to allow convenient bug or feature request filing.
Allow users to annotate vulnerabilities and to propose vulnerable code for them. This requires to implement support for:
Describe the bug
When logging in on vulncode-db.com you get an internal error.
Details seem to be:
ModuleNotFoundError: No module named '_cffi_backend'
at <module> (third_party/cryptography/hazmat/bindings/openssl/binding.py:16)
at <module> (third_party/cryptography/hazmat/backends/openssl/backend.py:117)
at <module> (third_party/cryptography/hazmat/backends/openssl/__init__.py:7)
at default_backend (third_party/cryptography/hazmat/backends/__init__.py:15)
at loads_public_key (third_party/authlib/jose/rfc7518/_backends/_jwk_cryptography.py:91)
at loads (third_party/authlib/jose/rfc7518/_backends/_jwk_cryptography.py:97)
at _load_obj (third_party/authlib/jose/rfc7517/jwk.py:50)
at _load_jwk_set (third_party/authlib/jose/rfc7517/jwk.py:62)
at loads (third_party/authlib/jose/rfc7517/jwk.py:76)
at load_key (third_party/authlib/integrations/base_client/remote_app.py:168)
at load_key (third_party/authlib/jose/jwk.py:21)
at key_func (third_party/authlib/jose/jwk.py:27)
at prepare_algorithm_key (third_party/authlib/jose/util.py:8)
at deserialize_compact (third_party/authlib/jose/rfc7515/jws.py:115)
at decode (third_party/authlib/jose/rfc7519/jwt.py:121)
at _parse_id_token (third_party/authlib/integrations/base_client/remote_app.py:198)
at parse_id_token (third_party/authlib/integrations/flask_client/remote_app.py:81)
at authorized (vulncode-db/app/auth/routes.py:128)
at decorated_function (third_party/flask_bouncer.py:66)
at dispatch_request (third_party/flask/app.py:1936)
at full_dispatch_request (third_party/flask/app.py:1950)
at reraise (third_party/flask/_compat.py:39)
at handle_user_exception (third_party/flask/app.py:1821)
at full_dispatch_request (third_party/flask/app.py:1952)
at wsgi_app (third_party/flask/app.py:2447)
I've tried adding cffi>=1.14.2
as a dependency which didn't seem to help here.
The repository file tree currently only displays a subset of all available files for repositories.
Example
https://www.vulncode-db.com/CVE-2016-8675
WithFilter Settings
set toAll Files
Goal: Add support to display all files within a given repository.
In data/models/vulnerability.py we use joinedload
which in some cases seems to lead to inconsistent unit tests SQL behavior where tests sometimes pass and sometimes fail. We need to investigate this further and address this as it's impacting all automatically run tests for push and pull requests to the repository.
The go-cve-dictionary
third-party dependency maintains its own tables. If there is an update it will automatically migrate/update the table structure.
Currently, Vulncode-DB explicitly ensures over alembic migrations that the data models are in sync. However, this breaks if go-cve-dictionary
is run before Vulncode-DB was upgraded to the newest version as some columns or database modifications might have already happened through go-cve-dictionary
.
go-cve-dictionary
from all migrations after that.This way it is expected that go-cve-dictionary
will be run at least once if there have been any updates.
Vulnerability entry shows up
Vulnerability entry does not show up
or add "request reassignment" option on author side?
The checkbox for sharing the profile picture and the full name is:
We might want to change this to be on by default (opt-out) or at least make it way better visible on first sign-up/login.
VCDB: 0.1.1559103842
PYPI: 0.1.1579837191
When starting the docker containers, frontend crashes beacause of an old marshmallow-sqlalchemy
version. For reference marshmallow-code/marshmallow-sqlalchemy#208
git clone https://github.com/google/vulncode-db
cd vulncode-db/docker
sudo docker-compose up --build
frontend_1 | Usage: flask db upgrade [OPTIONS] [REVISION]
frontend_1 |
frontend_1 | Error: While importing "main", an ImportError was raised:
frontend_1 |
frontend_1 | Traceback (most recent call last):
frontend_1 | File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 235, in locate_app
frontend_1 | __import__(module_name)
frontend_1 | File "/app/main.py", line 30, in <module>
frontend_1 | from app.vulnerability.views.vulncode_db import VulncodeDB
frontend_1 | File "/app/app/vulnerability/views/vulncode_db.py", line 18, in <module>
frontend_1 | from app.vulnerability.views.vulnerability import VulnerabilityView
frontend_1 | File "/app/app/vulnerability/views/vulnerability.py", line 19, in <module>
frontend_1 | from data.database import DEFAULT_DATABASE
frontend_1 | File "/app/data/database.py", line 22, in <module>
frontend_1 | from data.models import *
frontend_1 | File "/app/data/models/__init__.py", line 15, in <module>
frontend_1 | from data.utils import populate_models
frontend_1 | File "/app/data/utils.py", line 18, in <module>
frontend_1 | from data.models.base import db, ma
frontend_1 | File "/app/data/models/base.py", line 16, in <module>
frontend_1 | from flask_marshmallow import Marshmallow
frontend_1 | File "/usr/local/lib/python3.7/site-packages/flask_marshmallow/__init__.py", line 19, in <module>
frontend_1 | from . import fields
frontend_1 | File "/usr/local/lib/python3.7/site-packages/flask_marshmallow/fields.py", line 15, in <module>
frontend_1 | from marshmallow.compat import iteritems
frontend_1 | ModuleNotFoundError: No module named 'marshmallow.compat'
frontend_1 |
frontend_1 | Traceback (most recent call last):
frontend_1 | File "<string>", line 1, in <module>
frontend_1 | File "/app/main.py", line 30, in <module>
frontend_1 | from app.vulnerability.views.vulncode_db import VulncodeDB
frontend_1 | File "/app/app/vulnerability/views/vulncode_db.py", line 18, in <module>
frontend_1 | from app.vulnerability.views.vulnerability import VulnerabilityView
frontend_1 | File "/app/app/vulnerability/views/vulnerability.py", line 19, in <module>
frontend_1 | from data.database import DEFAULT_DATABASE
frontend_1 | File "/app/data/database.py", line 22, in <module>
frontend_1 | from data.models import *
frontend_1 | File "/app/data/models/__init__.py", line 15, in <module>
frontend_1 | from data.utils import populate_models
frontend_1 | File "/app/data/utils.py", line 18, in <module>
frontend_1 | from data.models.base import db, ma
frontend_1 | File "/app/data/models/base.py", line 16, in <module>
frontend_1 | from flask_marshmallow import Marshmallow
frontend_1 | File "/usr/local/lib/python3.7/site-packages/flask_marshmallow/__init__.py", line 19, in <module>
frontend_1 | from . import fields
frontend_1 | File "/usr/local/lib/python3.7/site-packages/flask_marshmallow/fields.py", line 15, in <module>
frontend_1 | from marshmallow.compat import iteritems
frontend_1 | ModuleNotFoundError: No module named 'marshmallow.compat'
frontend_1 | [~] Executed outside AppEngine context. Manually loading config.
docker_frontend_1 exited with code 1
Hi what would be the license for the data at https://www.vulncode-db.com/ ? is there a way to get that through some API too?
I am asking as I would want to include your data in this project most likely https://nlnet.nl/project/vulnerabilitydatabase/
Thank you!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.