consilience / flysystem-azure-file-storage Goto Github PK
View Code? Open in Web Editor NEWFlysystem adapter for Windows Azure File Storage
License: MIT License
Flysystem adapter for Windows Azure File Storage
License: MIT License
I'm able to list files and read file contents from Azure File Storage, so I'm pretty sure my connection string is fine.
When I try to write/update a file, I get the following error:
403: Fail: Code: 403 Value: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. details (if any): <?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. RequestId:33698f74-a01a-005c-7ab3-6f6a14000000 Time:2020-08-11T07:48:02.1835762Z</Message><AuthenticationErrorDetail>The MAC signature found in the HTTP request 'l+Lz9lBDaKq9VZ3k9zoBozQ5j9prREu0VkO9i/LxzxU=' is not the same as any computed signature. Server used following string to sign: 'GET application/x-www-form-urlencoded Tue, 11 Aug 2020 07:48:02 GMT x-ms-request-id:5f324d322c9eb x-ms-version:2016-05-31 /sjlmsib/s3ib/smarthub_sync/new/ restype:directory'.</AuthenticationErrorDetail></Error>.
I'm able to write to the file using the same connection string, but Microsoft API's createFileFromContent function, so I'm sure that my access rights allow me to create and write the destination file.
For reference, this works (it creates the file in the Azure File Share)
$microsoftFileClient->createFileFromContent('share', 'test.csv', $testdata);
And this doesn't work (it produces the error above)
$azureFileAdapterClient->write('test.csv', $testdata);
Any way to fix?
When creating a file using a prefix, the file uploads, but then the following exception is thrown:
Code: 404
Value: The specified parent path does not exist.
details (if any): <?xml version="1.0" encoding="utf-8"?><Error><Code>ParentNotFound</Code><Message>The specified parent path does not exist.
Was trying to copy a file between two separate disks on Laravel:
$destinationFileSystem->writeStream(
$filename,
$sourceFileSystem->readStream($filename)
);
Error from Azure is:
Code: 400
Value: An HTTP header that's mandatory for this request is not specified.
details (if any): <?xml version="1.0" encoding="utf-8"?><Error><Code>MissingRequiredHeader</Code><Message>An HTTP header that's mandatory for this request is not specified.
Workaround in my application for now:
$destinationFileSystem->put(
$filename,
stream_get_contents($sourceFileSystem->readStream($filename))
);
My files are limited in size, so this is not such a problem. It would still be nice to get to the bottom of this issue though, albeit not with much urgency.
Given that workaround, the stream reading is working, but the stream writing is not.
This would be a simpler line than the stream reading, but I've left it in as a reminder:
$sourceFileSystem->read($filename)
Problem
In version 1.x of this package does a separate metadata call for each file and directory found in a directory content listing.
Additional information
Since the update to version 1.x I started to receive rate limit warnings from Azure. This turned out to be from the listContents
method call used on Flysystem using the Azure Adapter. This methods uses the getContents
call https://github.com/consilience/flysystem-azure-file-storage/blob/1.0.0/src/AzureFileAdapter.php#L407 in the AzureFileAdapter. In the getContents
a request is done to retrieve the listDirectoriesAndFiles
method from the client to retrieve a list from the Azure File Storage. This returns a list of all the directories and files and their sizes. On version 0.1.6 and before of the package this was the only information used. https://github.com/consilience/flysystem-azure-file-storage/blob/0.1.6/src/AzureFileAdapter.php#L456
In 1.0 this information is expanded with additional meta information retrieved by calling getFileProperties
and getDirectoryProperties
for each file or directory found https://github.com/consilience/flysystem-azure-file-storage/blob/1.0.0/src/AzureFileAdapter.php#L483. This generates a lot of calls quickly. Their is the first call (listDirectoriesAndFiles
). If their are for example 3 directories and 60 files, then the total would be 1 + 3 + 60 = 64 calls just to retrieve the list of contents of a directory.
Possible solution
Not retrieving the additional information for files (and directories). This forces the user to retrieve the additional information if necessary, but I would estimate that the additional information besides the filename is not always used and therefore removing the need to have the information available by default.
In a project it was for now mitigated by patching the file with the following code:
diff --git a/vendor/consilience/flysystem-azure-file-storage/src/AzureFileAdapter.php b/vendor/consilience/flysystem-azure-file-storage/src/AzureFileAdapter.php
--- a/vendor/consilience/flysystem-azure-file-storage/src/AzureFileAdapter.php
+++ b/vendor/consilience/flysystem-azure-file-storage/src/AzureFileAdapter.php
@@ -477,14 +481,11 @@
foreach ($listResults->getFiles() as $fileObject) {
$pathName = trim(sprintf('%s/%s', $path, $fileObject->getName()), '/');
-
- $contents[] = $this->normalizeFileProperties(
- $pathName,
- $this->azureClient->getFileProperties(
- $this->container,
- $this->prefixer->prefixPath($pathName),
- ),
- );
+ $contents[] = FileAttributes::fromArray([
+ 'type' => 'file',
+ 'path' => $pathName,
+ 'dirname' => dirname($pathName),
+ ]);
}
return $contents;
Issue from PR #1 (@maetthu sorry, I had issues disabled for some reason)
Copying/Renaming files containing spaces, unicode characters or other non-url-safe characters results in following error:
Fail: Code: 403 Value: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. details (if any): AuthenticationFailedServer failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. [...] The MAC signature found in the HTTP request 'XXX' is not the same as any computed signature.
Uploading files to root folder w/o path prefix results in a similar error.
The vendor is consilience so it makes sense to move the repo. There were technical reasons why this could not be done initially.
Moving the vendor on packagist is going to be an important part of this move.
Laravel 9.x uses PHP 8 and Flysystem 3. A two-stage upgrade should take us to PHP 8.1 first, then Flysystem 3.
League\Flysystem\PathPrefixer
object).getContents()
is returning files, but no directories.
I'm sure this used to work, but now throws "missing header" exceptions at the HTTP level communicating with Azure. That threw me quite a lot.
It appears that the read stream the API provides no longer can be stated, i.e. details about its length is not available. When writing a stream to the API, it expects to know the content length from the start. The MS library was assuming the length was available just because a resource was provided, when in fact the length was unknown - null. This caused the request to the stream writing to the API to be corrupted - it had a null length. And that was throwing up all sorts of obscure errors.
Issue raised here:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.