Will this support the /mov function that was deprecated? about azure-storage-azcopy HOT 17 CLOSED

Hello @Mark7even - thanks for the feedback. For now let's keep this request open and hear what others have to say. We as the Storage team are worried about wrongfully deletion of source data, and possible data loss issues...

For now you can accomplish this yourself using an additional 'del' command for each file that is uploaded successfully.

Once AzCopy upload completes, you can get a list of succeeded file transfers using azcopy jobs show command. Here is how you can view each file that succeeded:

.\azcopy jobs show 8f925120-9bb6-b540-707e-4be86899b4ff --with-status=Success

You can for instance parse out the source files, and prepend 'DEL' and delete files one by one.. E.g.

 .\azcopy jobs show 8f925120-9bb6-b540-707e-4be86899b4ff --with-status=Success | ForEach-Object {
    Write-Host "DEL " $_.Split(" ")[2]
 }

If you think this is a good enough workaround, we can place this in the documentation so others can make use of it as well.

from azure-storage-azcopy.

@seguler I appreciate the response, but the issue comes when writing multiple small files to blob storage in an archive scenario.

Manually deleting these is not a great workaround even when pulling the azcopy jobs show. I understand the concern about wrongfully deleting source data, but this already worked great in AzCopy 2.4 with the slash /mov function.

I don't understand why it cannot be brought back in some capacity or re-engineered for V10? The only reason it was removed was not because of wrongful source data deletion, but because of potential destination overwrites.

from azure-storage-azcopy.

Thanks @srakins for adding to this thread.

We will discuss it within the AzCopy team, but for now the best option remains the one suggested by @seguler above.

from azure-storage-azcopy.

The MOVE functionality is very basic and AzCopy really need to have this option.
Manually deleting the data using azcopy job show is not a viable workaround.
In our case, we need to continuously transfer files from one Azure server to an on-prem server. Except once during midnight, we cannot purge the source files, so a MOVE option would be optimal.
Have voted to request this feature back.

from azure-storage-azcopy.

Hi @Mark7even, thanks for reaching out!

Could you please elaborate on what the /mov function does? I couldn't find it in the documentation.

from azure-storage-azcopy.

In AzCopy 2.5 /mov to move files instead of copy was removed. Will this be returning in this version?

https://blogs.msdn.microsoft.com/windowsazurestorage/2014/08/07/azcopy-2-5-release/

from azure-storage-azcopy.

@zezha-msft

from azure-storage-azcopy.

Hi @Mark7even, thanks for providing the context!

I see that this feature was discontinued due to data corruption concerns. If two users both move data into a container simultaneously, but some of the blobs share the same name, then one of the users will lose the original data due to it being deleted after the copy operation is complete.

Is there a particular reason why you would like to have this feature? It is currently not on our radar.

from azure-storage-azcopy.

@zezha-msft Well since blob storage is endless, the goal would be to make weekly transfers (moves) from On-premise to low-cost blob storage. I guess a copy and subsequent delete could work, but I am unsure how to do that.

For instance a scenario is data being written to a 10TB share which fills up weekly, once a week AzCopy runs and moves this data solely to blob freeing up the 10TB on-premise share. For Archival purposes.

from azure-storage-azcopy.

Hi @Mark7even, thanks for providing more context!

To be clear, is more data coming in as AzCopy is running? Is that why it's hard for you to delete the files that were transferred?

@seguler what's your opinion?

from azure-storage-azcopy.

@zezha-msft Not necessarily more data coming in as AzCopy is running, but the point would be to have AzCopy do a move, or a copy-delete so that only the files that were successfully transferred would be deleted. As opposed to copying the directory to blob, deleting it from on-premise manually, and then realizing the transfer failed on a couple items and they are now deleted from on-premise.

from azure-storage-azcopy.

My client has the same requirement, and manually deleting the data using azcopy job show is not a good workaround.
the orignal request was that if multiple users do the move then there might be chances of data being deleted as one operation is completed and it deleted the data.... but this happens only when a lame user did that from multiple machine.
this is a awesome feature which is removed now and because of this I might loss one client for bringing him to azure.

from azure-storage-azcopy.

We have the exact same scenario and can't upgrade our azcopy version and have to remain on the old version that has the /mov.

from azure-storage-azcopy.

We have the exact same scenario and can't upgrade our azcopy version and have to remain on the old version that has the /mov.

Request you to vote for this feature

https://feedback.azure.com/forums/217298-storage/suggestions/37164085-azcopy-with-move-feature-from-on-premise-to-azure

from azure-storage-azcopy.

@rishabpoh

from azure-storage-azcopy.

hey John, Just wanted to know is this added in roadmap? As feature request vote is above 190+ in Azcopy section of user voice.

from azure-storage-azcopy.

It's not currently. We'll discuss it.

from azure-storage-azcopy.