[Question] Error: Combination 'Localfile' not supported for sync command (File Storage) about azure-storage-azcopy HOT 42 CLOSED

Do we have ETA for adding support of syncing between Local and Azure Files?

from azure-storage-azcopy.

We will do this for out 10.9 release. @adreed-msft to make it happen.

from azure-storage-azcopy.

Hi @oscarwest, I apologize for the inconvenience.

We had to do some major refactoring, so this feature is not implemented yet. But it will much easier to implement once we get to it.

from azure-storage-azcopy.

@zezha-msft nicest guy ever... I hope they pay you well

from azure-storage-azcopy.

We run some stuff in AKS which mounts from Fileshares, in our CD pipe we'd love to have this sync capability so we never get diffs. It's not necessarily time sensitive but would make our life a lot easier :)

from azure-storage-azcopy.

Yes that's right, sure i'll create an issue over there so you guys can sync

from azure-storage-azcopy.

That sounds fine.

from azure-storage-azcopy.

Parity between robocopy and azcopy would make cloud migrations much simpler. +1 for local to file sync.

from azure-storage-azcopy.

@Kapanther Currently we don't support sync between Local and Azure File. We support sync only between Local and Azure Blob. But we plan to implement sync between Local and Azure File eventually.

from azure-storage-azcopy.

No Support for sync between Local and Azure Files.

from azure-storage-azcopy.

@prjain-msft thanks for confirming. Tested against blob storage and it works fine. (a bit slow, but possibly my awful australian internet)

from azure-storage-azcopy.

I've been waiting on this for a couple of months now, looking forward to it being implemented.

from azure-storage-azcopy.

@zezha-msft Sounds good :)

from azure-storage-azcopy.

Is this still planned? @zezha-msft

from azure-storage-azcopy.

It's on our approved backlog of work, but we don't have any planned date to share yet. Is your need for it time sensitive, with a particular deadline? If so please let us know here or via email. Can't promise anything of course, but all information is helpful in planning.

from azure-storage-azcopy.

If we could integrate this with Azure DevOps it would be even greater, right now we're deploying from Octopus Deploy with the binary

from azure-storage-azcopy.

Any suggestions on what such integration, with DevOps, would look like?

from azure-storage-azcopy.

Maybe expanding on something like this?
https://github.com/microsoft/azure-pipelines-tasks/blob/master/Tasks/AzureFileCopyV1/README.md

from azure-storage-azcopy.

So that one is using AzCopy v8, right? And you'd be looking for:

1. The same thing, but with v10... AND
2. With "sync" supported, from Local to Azure Files?

Is that correct? If so, would you mind popping over to the azure-pipelines-tasks GitHub project, and logging an issue there to request v10 support? You can include a link back to this issue, with a comment that its Local to Files sync that you most want.

from azure-storage-azcopy.

:-)

from azure-storage-azcopy.

We also need this feature. Is there a workaround to have the same behavior with the azcopy copy command?

from azure-storage-azcopy.

@Symbianx The copy command already works from local disk to Azure Files. Is that what you were asking about, or are you looking for something else?

from azure-storage-azcopy.

We also need this feature. Is there a workaround to have the same behavior with the azcopy copy command?

A temporary solution I used is mount Azure File to a local machine, and use robotcopy to sync(To be exact, it's mirroring).
Robocopy S:\FolderPath\ T:\FolderPath\ /MIR /FFT /Z /XA:H /W:5 /MT:8

from azure-storage-azcopy.

@JohnRusk From what I understood, the azcopy copy doesn't remove the files that no longer exist (like rsync --delete).

Our use case is pushing application backups to the Azure File Share from a container running in kubernetes, we also need to remove the files that don't exist locally anymore (hence the need to sync).

In one of our environments we are mounting the file share inside the container and we rsync the directories, however this forces us to run the container in privileged mode which is not ideal but works.
On the other environment the only way to access the internet is through an http_proxy, but unfortunately we can't mount the file system through the proxy.

from azure-storage-azcopy.

@Symbianx. Yes, it does sound like local->Files sync would be ideal for you there. Sorry its not ready yet!

Maybe you could to a workaround where the copying is done by AzCopy but the deletion is done differently. How are you authenticating? With a SAS url/token? If so, it's probably possible (with a little work) to call the rest API directly from script, to list the contents of the file share. Then, still in script, you could compare the local file listing with the one retrieved via the rest API, and then delete the ones that need to be deleted one by one (probably again with the REST API).

In theory you could do the same with other authentication types, but they are not as easy to script as SAS.

That's the best workaround I can think of right now. (And I'm not sure its terribly good... but its the best I can think of at the moment).

Note that you can't compare last modified dates with copy. You either re-copy everything, or else sett overwrite=false and don't copy anything that already exists (last modified dates are not checked, only existence is checked).

from azure-storage-azcopy.

@JohnRusk Thanks for the suggestion, we are using SAS.
For now, we ended up cleaning the entire directory with azcopy remove --recursive and uploading all of the files over again.
This is not ideal but since the files are relatively small I think this is a nice compromise until sync supports Local->Files.

Do you think there will be a downside that we may not be aware of?

from azure-storage-azcopy.

Just tossing in a vote of support for this functionality!

from azure-storage-azcopy.

Since Azure Files Fileshare with On premises AD ACL support has gone GA, I'm guessing a lot more will start missing this feature :) Big fileshares need the incremental sync otherwise we need a few days of down time for the share, which is never going to happen.

from azure-storage-azcopy.

Any progress on prioritisation for this?

from azure-storage-azcopy.

This is a must have functionality. We have to sync GBs of small files of which only a few change every week. Now we have to copy everything every time.

from azure-storage-azcopy.

@rpohane

from azure-storage-azcopy.

Hey Guys, its can help me alot.Is there some update about azcopy sync with Azure FileShare ?

from azure-storage-azcopy.

cc @amishra-dev

from azure-storage-azcopy.

Hi Guys, do we have any new status here?

from azure-storage-azcopy.

Amishra-dev;

Any chance of getting a hight priority assigned to this? This feature has been in request since 2018. Getting bi-direction sync is a featuer much-needed in this feature set.

from azure-storage-azcopy.

Hi @wsandmanw, we've already committed to a scope for 10.8, so 10.9 is where we are adding the high priority items such as this feature.

from azure-storage-azcopy.

Thanks for the update. Any update on when 10.9 will be released ?

from azure-storage-azcopy.

+1

from azure-storage-azcopy.

@zezha-msft any word?

from azure-storage-azcopy.

Thanks for following up, we shuffled the numbers a bit and the original 10.9 is now 10.10. We are targeting to land it in early Q2.

from azure-storage-azcopy.

ok...thanks for the update...

from azure-storage-azcopy.

Hey @Kapanther,

We've addressed this issue in the recent release of AzCopy 10.10.0. Please use the same and do reach out to us for more information.

from azure-storage-azcopy.