Git Product home page Git Product logo

datastage's People

Contributors

alexdutton avatar bhavanaananda avatar gklyne avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar

Forkers

no-reply mkozuharov mishin mdeguzis luckyfriends pdonwad sonijain htc-abdelkrim leylane

datastage's Issues

sudoers file is overwritten

The sheel script admiral-base/firstboot.sh overwrites the sudoers file, along with the default apache config file. This is done on firstboot, but still seems wrong to me.

Also a default ssl certificate is installed, a bunch of apache modules are enabled and a proxy config file is copied to apache

CGI (and other structural issues)

Maintaining a web app as CGI with no templating is going to be unsustainable. Problems include:

  • Changing style or JS library versions requires going through every one of the HTML files
  • Lots of sys.path('..') calls make the code highly dependent on the current working directory; arranging the code into a Python package (in this sense would make this easier, particularly if ever we need to refactor.
  • It's all very procedural. Using classes, modules and packages would help to modularise the code, encouraging re-use and extensibility.

Shell injection attack on AdminUIHandler

The code in question is https://github.com/dataflow/DataStage/blob/master/src/AdminUIHandler/services/AdminUIHandler.py#L155, lines 155 through 166. The variable remoteUser is pulled out of the HTTP Authorization header (L159), and then transcluded verbatim into commandString (L163). This is then passed to subprocess.Popen(.., shell=True) (L166) which executes commandString in a shell.

It would be trivial to craft an Authorization header (e.g. b64encoding "'; rm -rf / #:") to execute arbitrary commands on the server. It's also worth noting that remotPasswd is not used for any kind of authorization, so this attack is also available to untrusted users.

External Python libraries in repository

There are various external Python libraries in the repository which need to be removed. Having local copies is a security risk as they won't receive updates.

Things found so far:

  • test/RDFDatabank/rdflib
  • test/RDFDatabank/rdfextras
  • test/FileShare/tests/urllib2.py

sys.path.append('..')

Lots of Python files in the repo use sys.path.append(...) to place other directories on the Python path. They really should be properly namespaced packages. Here's the list:

[alex@karpathos DataStage]$ find -name "*.py" | xargs grep "sys.path.append"
./test/MiscLib/tests/TestDomHelpers.py:sys.path.append("../..")
./test/MiscLib/tests/TestTestUtils.py:sys.path.append("../..")
./test/MiscLib/tests/TestFunctions.py:sys.path.append("../..")
./test/MiscLib/tests/TestSuperGlobal.py:sys.path.append("../..")
./test/MiscLib/tests/TestScanFiles.py:sys.path.append("../..")
./test/MiscLib/tests/TestNetUtils.py:sys.path.append("../..")
./test/MiscLib/tests/TestCombinators.py:sys.path.append("../..")
./test/MiscLib/tests/TestAll.py:sys.path.append("../..")
./test/RDFDatabank/TestSubmission.py:    sys.path.append("..")
./test/TestLib/SparqlQueryTestCase.py:    sys.path.append("..")
./test/ZipTest/tests/TestRegEx.py:sys.path.append("../..")
./test/ZipTest/tests/ZipDirectory.py:sys.path.append("../..")
./test/FileShare/tests/TestFilePrivateArea.py:sys.path.append("../..")
./test/FileShare/tests/TestFileCollabAreaOld.py:sys.path.append("../..")
./test/FileShare/tests/TestFileSharedArea.py:sys.path.append("../..")
./test/FileShare/tests/TestWebDAVAccess.py:sys.path.append("../..")
./test/FileShare/tests/TestWebDAVbyHTTP.py:sys.path.append("../..")
./test/FileShare/tests/TestFileCommonArea.py:sys.path.append("../..")
./test/FileShare/tests/TestFileAccess.py:sys.path.append("../..")
./test/FileShare/tests/TestDeletedUserCheckFileAccess.py:sys.path.append("../..")
./test/FileShare/tests/TestFileDefaultArea.py:sys.path.append("../..")
./test/FileShare/tests/TestFileHTTPwriteCIFSread.py:sys.path.append("../..")
./test/FileShare/tests/TestFileCIFSwriteHTTPread.py:sys.path.append("../..")
./test/FileShare/tests/TestFileSilkGroup.py:sys.path.append("../..")
./test/FileShare/tests/TestFileCollabArea.py:sys.path.append("../..")
./test/FileShare/tests/TestFileUserAPublic.py:sys.path.append("../..")
./test/FileShare/tests/TestAll.py:sys.path.append("../..")
./test/FileShare/tests/TestFileUserASharedPublic.py:sys.path.append("../..")
./src/SubmitDatasetHandler/tests/TestMetadataMerging.py:sys.path.append("..")
./src/SubmitDatasetHandler/tests/TestMetadataMerging.py:sys.path.append("../cgi-bin")
./src/SubmitDatasetHandler/tests/TestSubmitDatasetHandler.py:sys.path.append("..")
./src/SubmitDatasetHandler/tests/TestSubmitDatasetHandler.py:sys.path.append("../cgi-bin")
./src/SubmitDatasetHandler/tests/TestDirectoryListingHandler.py:sys.path.append("..")
./src/SubmitDatasetHandler/tests/TestDirectoryListingHandler.py:sys.path.append("../cgi-bin")
./src/SubmitDatasetHandler/tests/TestGetDatasetMetadataHandler.py:sys.path.append("..")
./src/SubmitDatasetHandler/tests/TestGetDatasetMetadataHandler.py:sys.path.append("../cgi-bin")
./src/SubmitDatasetHandler/tests/TestRegEx.py:sys.path.append("../..")
./src/SubmitDatasetHandler/tests/TestSubmitDataset.py:###sys.path.append("../../../test")
./src/SubmitDatasetHandler/tests/TestSubmitDataset.py:sys.path.append("..")
./src/SubmitDatasetHandler/tests/TestAll.py:sys.path.append("../..")
./src/SubmitDatasetHandler/tests/TestAll.py:sys.path.append("..")
./src/SubmitDatasetHandler/tests/TestHttpSession.py:###sys.path.append("../../../test")
./src/SubmitDatasetHandler/tests/TestHttpSession.py:sys.path.append("..")
./src/SubmitDatasetHandler/MiscLib/tests/TestDomHelpers.py:sys.path.append("../..")
./src/SubmitDatasetHandler/MiscLib/tests/TestTestUtils.py:sys.path.append("../..")
./src/SubmitDatasetHandler/MiscLib/tests/TestScanDirectories.py:sys.path.append("../..")
./src/SubmitDatasetHandler/MiscLib/tests/TestFunctions.py:sys.path.append("../..")
./src/SubmitDatasetHandler/MiscLib/tests/TestSuperGlobal.py:sys.path.append("../..")
./src/SubmitDatasetHandler/MiscLib/tests/TestScanFiles.py:sys.path.append("../..")
./src/SubmitDatasetHandler/MiscLib/tests/TestNetUtils.py:sys.path.append("../..")
./src/SubmitDatasetHandler/MiscLib/tests/TestCombinators.py:sys.path.append("../..")
./src/SubmitDatasetHandler/MiscLib/tests/TestAll.py:sys.path.append("../..")
./src/SubmitDatasetHandler/cgi-bin/DirectoryContentsListingHandler.py:sys.path.append("..")
./src/SubmitDatasetHandler/cgi-bin/DirectoryContentsListingHandler.py:sys.path.append("../..")
./src/SubmitDatasetHandler/cgi-bin/SubmitDatasetConfirmationHandler.py:sys.path.append("..")
./src/SubmitDatasetHandler/cgi-bin/SubmitDatasetConfirmationHandler.py:sys.path.append("../..")
./src/SubmitDatasetHandler/cgi-bin/DirectoryListingHandler.py:sys.path.append("..")
./src/SubmitDatasetHandler/cgi-bin/DirectoryListingHandler.py:sys.path.append("../..")
./src/SubmitDatasetHandler/cgi-bin/GetDatasetMetadataHandler.py:sys.path.append("..")
./src/SubmitDatasetHandler/cgi-bin/GetDatasetMetadataHandler.py:sys.path.append("../..")
./src/SubmitDatasetHandler/cgi-bin/SubmitDatasetSummaryHandler.py:sys.path.append("..")
./src/SubmitDatasetHandler/cgi-bin/SubmitDatasetSummaryHandler.py:sys.path.append("../..")
./src/SubmitDatasetHandler/cgi-bin/DatasetDetailsErrorHandler.py:sys.path.append("..")
./src/SubmitDatasetHandler/cgi-bin/DatasetDetailsErrorHandler.py:sys.path.append("../..")
./src/SubmitDatasetHandler/cgi-bin/SubmitDatasetDetailsHandler.py:sys.path.append("..")
./src/SubmitDatasetHandler/cgi-bin/SubmitDatasetDetailsHandler.py:sys.path.append("../..")
./src/AdminUI/MiscLib/tests/TestDomHelpers.py:sys.path.append("../..")
./src/AdminUI/MiscLib/tests/TestTestUtils.py:sys.path.append("../..")
./src/AdminUI/MiscLib/tests/TestScanDirectories.py:sys.path.append("../..")
./src/AdminUI/MiscLib/tests/TestFunctions.py:sys.path.append("../..")
./src/AdminUI/MiscLib/tests/TestSuperGlobal.py:sys.path.append("../..")
./src/AdminUI/MiscLib/tests/TestScanFiles.py:sys.path.append("../..")
./src/AdminUI/MiscLib/tests/TestNetUtils.py:sys.path.append("../..")
./src/AdminUI/MiscLib/tests/TestCombinators.py:sys.path.append("../..")
./src/AdminUI/MiscLib/tests/TestAll.py:sys.path.append("../..")
./src/AdminUIHandler/tests/TestAdminInterfaceRestfulAPIs.py:sys.path.append(".")
./src/AdminUIHandler/tests/TestAdminInterfaceRestfulAPIs.py:sys.path.append("..")
./src/AdminUIHandler/services/AdminUIHandler.py:sys.path.append("..")
./src/AdminUIHandler/services/AdminUIHandler.py:sys.path.append("../..")
./src/AdminUIHandler/MiscLib/tests/TestDomHelpers.py:sys.path.append("../..")
./src/AdminUIHandler/MiscLib/tests/TestTestUtils.py:sys.path.append("../..")
./src/AdminUIHandler/MiscLib/tests/TestScanDirectories.py:sys.path.append("../..")
./src/AdminUIHandler/MiscLib/tests/TestFunctions.py:sys.path.append("../..")
./src/AdminUIHandler/MiscLib/tests/TestSuperGlobal.py:sys.path.append("../..")
./src/AdminUIHandler/MiscLib/tests/TestScanFiles.py:sys.path.append("../..")
./src/AdminUIHandler/MiscLib/tests/TestNetUtils.py:sys.path.append("../..")
./src/AdminUIHandler/MiscLib/tests/TestCombinators.py:sys.path.append("../..")
./src/AdminUIHandler/MiscLib/tests/TestAll.py:sys.path.append("../..")
./src/AllTests/tests/AllTests.py:sys.path.append("../../SubmitDatasetHandler")
./src/AllTests/tests/AllTests.py:sys.path.append("../../SubmitDatasetHandler/cgi-bin")
./src/AllTests/tests/AllTests.py:sys.path.append("../../SubmitDatasetHandler/tests")
./spike/applications/Admiral/Unused/Logout/LogoutHandler.py:sys.path.append("..")
./spike/applications/Admiral/Unused/Logout/LogoutHandler.py:sys.path.append("../..")
./spike/applications/Admiral/Unused/Logout/LogoutResponseHandler.py:sys.path.append("..")
./spike/applications/Admiral/Unused/Logout/LogoutResponseHandler.py:sys.path.append("../..")

Debian package should have additional dependencies

The debian/control file should be updated to include these additional dependencies under Depends::

    net-tools,
    sudo

Without sudo, the debian/postinst script fails and exits.

Without net-tools, the datastage-config interactive config submenu crashes (due to external calls to the ifconfig tool, provided by net-tools.

These packages are usually present by default, but in my case I was trying to set up DataStage on a more barebones Ubuntu environment (Docker).

Licensing issues of bundled JS libraries

DataStage currently bundles (and hence distributes) jQuery, jQuery-treeview, mochikit, rdfquery and maybe a couple of other JavaScript libraries. Details:

  • jQuery: Dual-licensed under MIT and GPL in code headers. Not immediately obvious; missing license text.
  • jQuery-treeview: Dual-licensed under MIT and GPL. In code header with hrefs to license documents. Raised issue asking for a LICENSE file..
  • mochikit: Dual-licensed under MIT and AFLv2.1, with included BSD code. Our copy only contains a pointer to the mochikit website for license and documentation, which technically doesn't meet the requirements of the license (which requires the license and copyright notice to be distributed with the source). This may be implicitly okay for people serving it alongside websites, but it's not a brilliant state of affairs for source distributions.
  • rdfquery: Looks mostly okay; files contain copyright notice, name of license (MIT) and pointer to LICENSE.txt. However, LICENSE.txt is missing.
  • qunit: Dual-licensed under MIT and GPL. Pointer to license in source.

SubmitDatasetConfirmationHandler doesn't check permissions when submitting a dataset

src/SubmitDatasetHandler/cgi-bin/SubmitDatasetConfirmationHandler.py doesn't check permissions on the directory to be packaged and sent to DataBank. It is fairly trivial to use something like Firebug to edit the path to be packaged.

This would lead to a user being able to access other users' private files by submitting them to a repository and then retrieving them from there.

Apache WebDAV doesn't respect ACLs

Apache WebDAV doesn't respect ACLs of authenticated user, so these are implemented as Apache config files on a per-user basis, requiring an apache restart whenever a user is added or removed.

The current functionality could be implemented by wrapping PyWebDAV. It would also remove a dependency on Apache for anyone who wants to deploy the DataStage web interface in another environment.

Shell injection attack on ScanDirectories

The code in question is https://github.com/dataflow/DataStage/blob/master/src/AdminUI/MiscLib/ScanDirectories.py. IsDirectoryWritable() contains a call to os.system() using input containing dirPath. Following things up a potential call stack, dirPath can be the name of a directory on the file system. As users are able to create directories with whatever names they like, a malicious user could invoke any command as www-data by visiting </tool/SubmitDatasetHandler/cgi-bin/DirectoryListingHandler.py>. This would then traverse the user's home, and in so doing, execute the malicious command.

This one requires an authenticated user, as the above CGI script is behind basic auth.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.