On Windows, AzCopy Cannot Handle Long File Paths about azure-storage-azcopy HOT 23 CLOSED

I'll have to walk back on that. I realized that the previous times, I did not have verbose logging. So, I ran again with verbose logging on and it is now working and transferring at a decent pace compared to ASE. Strange thing is, otherwise, the command was identical - it was a batch file and I just appended the logging. Not sure what the issue was the first time but I can't imagine turning on logging would fix it.

from azure-storage-azcopy.

Hiya, we currently have a refactor in progress and will be fixing this up shortly after we finish refactoring upload and download.

from azure-storage-azcopy.

Hi @MRayermannMSFT, thanks for reporting this issue!

It looks really strange, but I was able to reproduce it. In my testing, it seems that an absolute path works fine, but a relative path fails with such a long file name. Could you please try with an absolute path as well?

from azure-storage-azcopy.

Do you consider a path starting with \\ to be absolute?

from azure-storage-azcopy.

Hi @MRayermannMSFT, "\\" is for network drives, it's a single "\" for local paths:

cd \Users\zezha

After some testing, here are my findings:

1. For absolute paths that include the volume, like "C:\.....", AzCopy works regardless of the length of path, the longest path I tested was 569 characters long.
2. For unix-style absolute paths, like "\Users\...", there is indeed a problem when the length of path gets longer.
3. For relative paths, like ".\....", the result is the same as unix-style absolute paths.
4. For network paths, like "\\server\share", there is also a problem with longer paths.

To summarize, the results are mostly consistent with what was described here. However, the exception is that in my testing, even though I did not enable the new behavior in Win10, specifying a volume worked for me.

I tried adding "\\?\UNC\" for the network paths, but it didn't work with AzCopy. I need to look into this further and get back to you.

from azure-storage-azcopy.

I've logged this item for further investigation. I saw no easy solution for the long network paths unfortunately.

from azure-storage-azcopy.

I'll second this as a pretty major issue. I don't think the path even has to be that long - mine was failing with paths of around 30 characters. If I changed to a local drive the exact same copy otherwise worked fine.

from azure-storage-azcopy.

Hi @jim-jezioranski, which version are you using? Are you also uploading from a network drive? Could you please send me the log file? Please rest assured that we scrub away sensitive account info (SAS signature) before logging the requests, but the file names are still exposed.

from azure-storage-azcopy.

@jim-jezioranski thanks for letting us know! Please don't hesitate if you see any other problem.

from azure-storage-azcopy.

Actually here's a solid question. Does this issue get reproduced on UNIX based systems? I know Windows imposes an arbitrary file path length of 260 characters.

As a result, my thought is that this could be a purely windows-specific issue that can't really be fixed.

from azure-storage-azcopy.

Hi @Virepri, yes it is a Windows specific issue. There is a workaround that I found that works on the command line, but it didn't work with the Go APIs. I didn't have enough time to dig deeper, there may be ways to mitigate the issue.

from azure-storage-azcopy.

Does enabling long file names in windows 10 help? https://www.howtogeek.com/266621/how-to-make-windows-10-accept-file-paths-over-260-characters/

from azure-storage-azcopy.

@yoginm I'm not sure. There are server different ways to sort it out. We hope to get one that doesn't require users to make changes to registry settings.
The issue is on our backlog for further investigation.

from azure-storage-azcopy.

Wanted to check in to see if there has been any progress on this issue? We're unfortunately experiencing this with the latest version of AzCopy(10.2.1).

from azure-storage-azcopy.

Thank you for the update!

from azure-storage-azcopy.

Hi @adreed-msft, I was wondering if you had a timeline on when the fix might be deployed. We have a client that would like to move a bunch of data to Azure and we need to decide if we should wait for the new code or come up with an alternate solution to move the data to the cloud.

Thanks!

from azure-storage-azcopy.

This should be coming in the next release, but we don't have an exact date of that release yet.

from azure-storage-azcopy.

Ok, thank you!

from azure-storage-azcopy.

Any workarounds for this or should I have to use other tools as I have a very urgent requirement for this.

from azure-storage-azcopy.

We just merged the final fix for this into our official dev branch about 1 hour ago. If you can wait a week (or a bit more) then waiting for the official release of that would make sense.

If you can't wait that long, can I ask you one question please: are you accessing a UNC path, or a path that starts with a drive letter? (Asking because in the current release the former is more broken than the latter).

from azure-storage-azcopy.

@JohnRusk Thanks for prompt response.
The files that I am copying are at NFS so It's UNC Path (\xyz).
But I have a requirement to finish the copy within this week so can I use the dev branch version?

from azure-storage-azcopy.

Good question, but sorry, please don't use the dev branch. It would only be considered at "preview" quality at this time, and Microsoft generally does not allow use of preview software for real use on real (production) data.

Please email me (address is in my profile) if you'd like to discuss your options in more detail.

from azure-storage-azcopy.

The new release (10.3.0) is out and resolves this issue. Enjoy!

from azure-storage-azcopy.