Git Product home page Git Product logo

phar-stream-wrapper's Introduction

Scrutinizer Code Quality GitHub Build Status Ubuntu GitHub Build Status Windows Downloads

PHP Phar Stream Wrapper

Abstract & History

Based on Sam Thomas' findings concerning insecure deserialization in combination with obfuscation strategies allowing to hide Phar files inside valid image resources, the TYPO3 project decided back then to introduce a PharStreamWrapper to intercept invocations of the phar:// stream in PHP and only allow usage for defined locations in the file system.

Since the TYPO3 mission statement is inspiring people to share, we thought it would be helpful for others to release our PharStreamWrapper as standalone package to the PHP community.

The mentioned security issue was reported to TYPO3 on 10th June 2018 by Sam Thomas and has been addressed concerning the specific attack vector and for this generic PharStreamWrapper in TYPO3 versions 7.6.30 LTS, 8.7.17 LTS and 9.3.1 on 12th July 2018.

License

In general the TYPO3 core is released under the GNU General Public License version 2 or any later version (GPL-2.0-or-later). In order to avoid licensing issues and incompatibilities this PharStreamWrapper is licenced under the MIT License. In case you duplicate or modify source code, credits are not required but really appreciated.

Credits

Thanks to Alex Pott, Drupal for creating back-ports of all sources in order to provide compatibility with PHP v5.3.

Installation

The PharStreamWrapper is provided as composer package typo3/phar-stream-wrapper and has minimum requirements of PHP v5.3 (v2 branch) and PHP v7.0 (master branch).

Installation for PHP v7.0

composer require typo3/phar-stream-wrapper ^3.0

Installation for PHP v5.3

composer require typo3/phar-stream-wrapper ^2.0

Example

The following example is bundled within this package, the shown PharExtensionInterceptor denies all stream wrapper invocations files not having the .phar suffix. Interceptor logic has to be individual and adjusted to according requirements.

\TYPO3\PharStreamWrapper\Manager::initialize(
    (new \TYPO3\PharStreamWrapper\Behavior())
        ->withAssertion(new \TYPO3\PharStreamWrapper\Interceptor\PharExtensionInterceptor())
);

if (in_array('phar', stream_get_wrappers())) {
    stream_wrapper_unregister('phar');
    stream_wrapper_register('phar', \TYPO3\PharStreamWrapper\PharStreamWrapper::class);
}
  • PharStreamWrapper defined as class reference will be instantiated each time phar:// streams shall be processed.
  • Manager as singleton pattern being called by PharStreamWrapper instances in order to retrieve individual behavior and settings.
  • Behavior holds reference to interceptor(s) that shall assert correct/allowed invocation of a given $path for a given $command. Interceptors implement the interface Assertable. Interceptors can act individually on following commands or handle all of them in case not defined specifically:
    • COMMAND_DIR_OPENDIR
    • COMMAND_MKDIR
    • COMMAND_RENAME
    • COMMAND_RMDIR
    • COMMAND_STEAM_METADATA
    • COMMAND_STREAM_OPEN
    • COMMAND_UNLINK
    • COMMAND_URL_STAT

Interceptors

The following interceptor is shipped with the package and ready to use in order to block any Phar invocation of files not having a .phar suffix. Besides that individual interceptors are possible of course.

class PharExtensionInterceptor implements Assertable
{
    /**
     * Determines whether the base file name has a ".phar" suffix.
     *
     * @param string $path
     * @param string $command
     * @return bool
     * @throws Exception
     */
    public function assert(string $path, string $command): bool
    {
        if ($this->baseFileContainsPharExtension($path)) {
            return true;
        }
        throw new Exception(
            sprintf(
                'Unexpected file extension in "%s"',
                $path
            ),
            1535198703
        );
    }

    /**
     * @param string $path
     * @return bool
     */
    private function baseFileContainsPharExtension(string $path): bool
    {
        $baseFile = Helper::determineBaseFile($path);
        if ($baseFile === null) {
            return false;
        }
        $fileExtension = pathinfo($baseFile, PATHINFO_EXTENSION);
        return strtolower($fileExtension) === 'phar';
    }
}

ConjunctionInterceptor

This interceptor combines multiple interceptors implementing Assertable. It succeeds when all nested interceptors succeed as well (logical AND).

\TYPO3\PharStreamWrapper\Manager::initialize(
    (new \TYPO3\PharStreamWrapper\Behavior())
        ->withAssertion(new ConjunctionInterceptor([
            new PharExtensionInterceptor(),
            new PharMetaDataInterceptor(),
        ]))
);

PharExtensionInterceptor

This (basic) interceptor just checks whether the invoked Phar archive has an according .phar file extension. Resolving symbolic links as well as Phar internal alias resolving are considered as well.

\TYPO3\PharStreamWrapper\Manager::initialize(
    (new \TYPO3\PharStreamWrapper\Behavior())
        ->withAssertion(new PharExtensionInterceptor())
);

PharMetaDataInterceptor

This interceptor is actually checking serialized Phar meta-data against PHP objects and would consider a Phar archive malicious in case not only scalar values are found. A custom low-level Phar\Reader is used in order to avoid using PHP's Phar object which would trigger the initial vulnerability.

\TYPO3\PharStreamWrapper\Manager::initialize(
    (new \TYPO3\PharStreamWrapper\Behavior())
        ->withAssertion(new PharMetaDataInterceptor())
);

Reader

  • Phar\Reader::__construct(string $fileName): Creates low-level reader for Phar archive
  • Phar\Reader::resolveContainer(): Phar\Container: Resolves model representing Phar archive
  • Phar\Container::getStub(): Phar\Stub: Resolves (plain PHP) stub section of Phar archive
  • Phar\Container::getManifest(): Phar\Manifest: Resolves parsed Phar archive manifest as documented at http://php.net/manual/en/phar.fileformat.manifestfile.php
  • Phar\Stub::getMappedAlias(): string: Resolves internal Phar archive alias defined in stub using Phar::mapPhar('alias.phar') - actually the plain PHP source is analyzed here
  • Phar\Manifest::getAlias(): string - Resolves internal Phar archive alias defined in manifest using Phar::setAlias('alias.phar')
  • Phar\Manifest::getMetaData(): string: Resolves serialized Phar archive meta-data
  • Phar\Manifest::deserializeMetaData(): mixed: Resolves deserialized Phar archive meta-data containing only scalar values - in case an object is determined, an according Phar\DeserializationException will be thrown
$reader = new Phar\Reader('example.phar');
var_dump($reader->resolveContainer()->getManifest()->deserializeMetaData());

Helper

  • Helper::determineBaseFile(string $path): string: Determines base file that can be accessed using the regular file system. For instance the following path phar:///home/user/bundle.phar/content.txt would be resolved to /home/user/bundle.phar.
  • Helper::resetOpCache(): Resets PHP's OPcache if enabled as work-around for issues in include() or require() calls and OPcache delivering wrong results. More details can be found in PHP's bug tracker, for instance like https://bugs.php.net/bug.php?id=66569

Security Contact

In case of finding additional security issues in the TYPO3 project or in this PharStreamWrapper package in particular, please get in touch with the TYPO3 Security Team.

phar-stream-wrapper's People

Contributors

alexpott avatar ausi avatar ktomk avatar lolli42 avatar longwave avatar ohader avatar ryanaslett avatar xknown avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

phar-stream-wrapper's Issues

Including dependencies packed with clue/phar-composer results in exception

In a non-composer TYPO3 project I follow @helhumโ€˜s suggestions here https://insight.helhum.io/post/148112375750/how-to-use-php-libraries-in-legacy-extensions to require third party dependencies.
After an update to a TYPO3 version including phar-stream-wrapper Iโ€˜m facing this exception:

#1530103999: Method stream_select() cannot be used

thrown in the line that does actually require the autoload.php file inside the phar, equally to

@include 'phar://' . ExtensionManagementUtility::extPath('ext-key') . 'Libraries/symfony-process.phar/vendor/autoload.php';

Expected behaviour:
Register classes inside phar and load classes from there as needed.

PHP is 5.6.24 and target TYPO3 version is 6.2.40

Check meta-data deserialization capabilities in PHP 8

Also, change the signature from getMetadata()
to getMetadata(array $unserialize_options = []).
Start throwing earlier if setMetadata() is called and serialization threw.

Scope for this package, craft a bunch of exploits for PHP 8 and see whether it works.
In case it does, this package probably could "hand over" Phar handling to native PHP 8 then...

Performance down between 0469d9f and b7a21f0

When we upgraded to Drupal 8.6.13 due to the recent security issue we also had several dependencies update, one of them being this library due to Core's version requirement being set to "^2.0.1". So instead of using 2.0.1 as bundled with Core we ended up with the pre-release 2.1.0 and the time it took to run simple Drush commands like status skyrocketed.

With v2.1.0 (b7a21f0) installed we get these numbers:

time drupal status
...
real    0m29.862s
user    0m19.492s
sys     0m10.350s

With v2.0.1 (0469d9f) we get these numers:

time drupal status
...
real    0m2.422s
user    0m1.695s
sys     0m0.705s

The diff looks fairly small, but does seem to introduce brumann/polyfill-unserialize, could that be what's making it take a lot longer?

Does not handle fgets returning an error gracefully

If fgets returns false due to an error, this package will crash as it passes a bool to strpos:

TypeError: strpos() expects parameter 1 to be string, bool given in strpos() (line 109 of /var/www/html/jwtest-com/vendor/typo3/phar-stream-wrapper/src/Phar/Reader.php).

I'm still not certain how we got an error from fgets. We were trying to launch the phpstan.phar:

TYPO3\PharStreamWrapper\Phar\Reader->extractData('/var/www/html/jwtest-com/vendor/phpstan/phpstan/phpstan.phar') (Line: 53)

Relevant issue on Drupal.org. This gets triggered with:

    if (!class_exists('PHPStan\ExtensionInstaller\GeneratedConfig')) {

That causes the PHPStan\PharAutoloader to execute:

PHPStan\PharAutoloader::loadClass('PHPStan\ExtensionInstaller\GeneratedConfig')
spl_autoload_call('PHPStan\ExtensionInstaller\GeneratedConfig')
class_exists('PHPStan\ExtensionInstaller\GeneratedConfig') (Line: 589)

And we end up broken. I don't know how many lines it takes to get an error, but it's before feof returns true.

PHP Notice: stream_wrapper_restore(): phar:// was never changed

PHPUnit\Framework\Exception: PHP Notice:  stream_wrapper_restore(): phar:// was never changed, nothing to restore in /home/travis/build/TYPO3/phar-stream-wrapper/src/PharStreamWrapper.php on line 498
PHP Stack trace:
PHP   1. TYPO3\PharStreamWrapper\PharStreamWrapper->dir_closedir() /home/travis/build/TYPO3/phar-stream-wrapper/src/PharStreamWrapper.php:0
PHP   2. TYPO3\PharStreamWrapper\PharStreamWrapper->invokeInternalStreamWrapper() /home/travis/build/TYPO3/phar-stream-wrapper/src/PharStreamWrapper.php:51
PHP   3. TYPO3\PharStreamWrapper\PharStreamWrapper->restoreInternalSteamWrapper() /home/travis/build/TYPO3/phar-stream-wrapper/src/PharStreamWrapper.php:481
PHP   4. stream_wrapper_restore() /home/travis/build/TYPO3/phar-stream-wrapper/src/PharStreamWrapper.php:498

This probably changed when fix for PHP bug 76943 was applied - and was back-ported to PHP 7.3.24 and 7.4.12.

Version 2.2.2 not backwards compatible because of strict_types=1

Verison 2.2.2 seems to be published with the wrong sources.

Version 2.2.1 did not have strict_types=1 and no type annotations, version 2.2.2 now includes those which breaks compatibility for 2.2.x. In some projects we still have to use Typo3 7 which exits with an fatal error after update to 2.2.2.

Please publish a release with the correct sources for 2.2.x

Manifest / End of Stub Detection

Due to differences of how PHP Phar detects the end of the Phar stub and how this Phar parser does, it is possible to circumvent the PharMetaDataInterceptor.

Breaks typo3 8.7.25 on symlinked webroot

On a typo3 8.7.25 installation that has a symlinked webroot, the following errors occur:

Warning: Uncaught TYPO3\PharStreamWrapper\Exception: Executing phar:///data/sites/web/depontnl/www/typo3conf/ext/emogrifier/Resources/Private/Php/Emogrifier.phar/vendor/autoload.php is denied in /data/sites/web/depontnl/subsites/typo3_src-8.7.25/typo3/sysext/core/Classes/IO/PharStreamWrapperInterceptor.php:39 Stack trace: #0 /data/sites/web/depontnl/subsites/typo3_src-8.7.25/vendor/typo3/phar-stream-wrapper/src/Behavior.php(72): TYPO3\CMS\Core\IO\PharStreamWrapperInterceptor->assert('phar:///data/si...', 'stream_open') #1 /data/sites/web/depontnl/subsites/typo3_src-8.7.25/vendor/typo3/phar-stream-wrapper/src/Manager.php(110): TYPO3\PharStreamWrapper\Behavior->assert('phar:///data/si...', 'stream_open') #2 /data/sites/web/depontnl/subsites/typo3_src-8.7.25/vendor/typo3/phar-stream-wrapper/src/PharStreamWrapper.php(421): TYPO3\PharStreamWrapper\Manager->assert('phar:///data/si...', 'stream_open') #3 /data/sites/web/depontnl/subsites/typo3_src-8.7.25/vendor/typo3/phar-stream-wrapper/src/PharStreamWrapper.php(256): TYPO3\ in /data/sites/web/depontnl/subsites/typo3_src-8.7.25/typo3/sysext/core/Classes/IO/PharStreamWrapperInterceptor.php on line 39

Fatal error: TYPO3\CMS\Core\Utility\GeneralUtility::requireOnce(): Failed opening required 'phar:///data/sites/web/depontnl/www/typo3conf/ext/emogrifier/Resources/Private/Php/Emogrifier.phar/vendor/autoload.php' (include_path='.') in /data/sites/web/depontnl/subsites/typo3_src-8.7.25/typo3/sysext/core/Classes/Utility/GeneralUtility.php on line 4249

Reverting to 8.7.24 resolves it.
The same setup, but then a non-symlinked webroot on staging, generates no errors on 8.7.25, so it seems related to the symlink.

Interceptors fail using Phar archives using internal aliases

Based on

Description

Helper::determineBaseFile($path) is only capable of resolving files that are actually available in the system's file system. Since Phar allows to set internal alias names - in order to be used inside of Phar archives only - the real system path cannot be resolved any more.

Alias names either could be defined using

  • Phar::setAlias($alias) - persisted to Phar Manifest
  • Phar::mapPhar($alias) - only in-memory during runtime and executed in Phar's Stub section

Inside Phar archives Phar::running could be used, but this won't work "outside", e.g. in PharStreamWrapper or interceptors (http://php.net/manual/en/phar.running.php) - thus, not an option here.

Breaking included AWS Phar

Hi. Joomla 3.9.4 updated/included this package.
Currently I'm using aws.phar to use for S3, but since last update, I'm getting this error:

Uncaught TYPO3\PharStreamWrapper\Exception: Unexpected file extension in "phar://aws-3.67.5.phar/aws-autoloader.php"

This error is related to this package, not in Joomla or my own code. Maybe a bug?

Add possibility to retrieve low-level Phar internals

Since using new \Phar($path) on a compromised Phar archive would already trigger meta-data extraction, low-level internals like Phar meta-data as well as stub related information shall be extracted.

PHP documentation gives some pointers on Phar internals:

Phar signatures are out-of-scope for this change.

Enhance variety of test fixtures

  • use compromised archives that really triggering action
  • generalize invocation and loading behavior
  • extend examples for alias Phar archives

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.