Comments (17)
@zezha-msft Thank you guys so much for this!
from azure-storage-azcopy.
Hi @123dev, thanks for reaching out!
We are still in the process of getting the download links ready, this should happen very soon.
In the mean time, please install the dep tool and run dep ensure
before doing go build -o azcopy
.
from azure-storage-azcopy.
Thanks zezha for a very quick reply.
followed your suggestion, and at the step of go build -o azcopy
I'm getting
# github.com/Azure/azure-storage-azcopy/vendor/github.com/jiacfan/keychain
vendor/github.com/jiacfan/keychain/corefoundation_1.10.go:55:33: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFDataCreate
vendor/github.com/jiacfan/keychain/corefoundation_1.10.go:81: cannot use nil as type _Ctype_CFAllocatorRef in argument to func literal
vendor/github.com/jiacfan/keychain/corefoundation_1.10.go:118:41: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFStringCreateWithBytes
vendor/github.com/jiacfan/keychain/corefoundation_1.10.go:153:232: cannot use nil as type _Ctype_CFAllocatorRef in argument to func literal
Thanks
from azure-storage-azcopy.
Hi @123dev, please verify that you have the latest code from our master branch. This should work. 😄
from azure-storage-azcopy.
Our CI machine seems to be having the same errors as you, but I couldn't reproduce it locally yet. I'm investigating now.
from azure-storage-azcopy.
Hi, @123dev
Thanks for reporting the issue!
Are you using go 1.11? There is change in go which could lead the build failure mentioned in one of azcopy's dependency, we have fixed that issue and updated dependency in azcopy(dev branch). Now, dev branch can pass build successfully in my local machine with go 1.11, would you like to have a try?
Please note to checkout dev branch's code, run dep ensure
again to get latest dependencies, and then run go build
.
Thanks,
Jiachen
from azure-storage-azcopy.
Hi jiacfan
Thanks for the follow up and the fix.
Indeed I'm using go 1.11
dev build worked, thank you.
Quick question, this azcopy is not backward compatible with the original azcopy?
We already have code that issues azcopy (multi-platform) with parameters such as
/Source
/DestKey
/Dest
And were thinking that this would just be a drop in replacement that supports copying to $web
Thanks
from azure-storage-azcopy.
Hi @123dev, the new AzCopy(Version 10) is indeed NOT backward compatible with the current AzCopy(Version 8). However, the new tool is much simpler to use, to see examples, please use ./azcopy copy -h
:
Examples:
Upload a single file:
- azcopy cp "/path/to/file.txt" "https://[account].blob.core.windows.net/[existing-filesystem]/[path/to/destination/directory/or/file]"
Upload an entire directory:
- azcopy cp "/path/to/dir" "https://[account].blob.core.windows.net/[existing-filesystem]/[path/to/destination/directory]" --recursive=true
Upload files using wildcards:
- azcopy cp "/path/*foo/*bar/*.pdf" "https://[account].blob.core.windows.net/[existing-filesystem]/[path/to/destination/directory]"
Upload files and/or directories using wildcards:
- azcopy cp "/path/*foo/*bar*" "https://[account].blob.core.windows.net/[existing-filesystem]/[path/to/destination/directory]" --recursive=true
Download a single file:
- azcopy cp "https://[account].blob.core.windows.net/[existing-filesystem]/[path/to/source/file]" "/path/to/file.txt"
Download an entire directory:
- azcopy cp "https://[account].blob.core.windows.net/[existing-filesystem]/[path/to/source/dir]" "/path/to/file.txt" --recursive=true
The new version also works with SAS tokens only, so that it's much more secure and you don't have to expose your Storage Key. To easily generate an account or container token, please use Storage Explorer, and include the token in the URL that you provide to the copy command.
Please let me know if you have any question. To see what commands are available, you can always view the help pages by ./azcopy -h
.
from azure-storage-azcopy.
Thank you zszha for the follow up and confirmation of not being backward compatible.
And thanks for the links.
I am able to create SAS and use azcopy to copy files including to $web
Many thanks.
This issue can be closed.
from azure-storage-azcopy.
@123dev Awesome!
Please let us know if you encounter any other issue using the new tool.
from azure-storage-azcopy.
Hi zezha,
Considering that this new version 10 is not backward compatible with the previous version 8,
I tried to build version 10 on windows as I want to make all our platforms run the same tool the same way.
and encountered the following error.
# runtime
C:\U\Go\src\runtime\lfstack_amd64.go:16:6: lfstackPack redeclared in this block
previous declaration at C:\U\Go\src\runtime\lfstack_64bit.go:33:45
C:\U\Go\src\runtime\lfstack_amd64.go:20:6: lfstackUnpack redeclared in this block
previous declaration at C:\U\Go\src\runtime\lfstack_64bit.go:37:33
C:\U\Go\src\runtime\map.go:64:2: bucketCntBits redeclared in this block
previous declaration at C:\U\Go\src\runtime\hashmap.go:62:18
C:\U\Go\src\runtime\map.go:65:2: bucketCnt redeclared in this block
previous declaration at C:\U\Go\src\runtime\hashmap.go:63:23
C:\U\Go\src\runtime\map.go:76:2: maxKeySize redeclared in this block
previous declaration at C:\U\Go\src\runtime\hashmap.go:72:17
C:\U\Go\src\runtime\map.go:77:2: maxValueSize redeclared in this block
previous declaration at C:\U\Go\src\runtime\hashmap.go:73:17
C:\U\Go\src\runtime\map.go:82:2: dataOffset redeclared in this block
previous declaration at C:\U\Go\src\runtime\hashmap.go:81:4
C:\U\Go\src\runtime\map.go:91:2: empty redeclared in this block
previous declaration at C:\U\Go\src\runtime\hashmap.go:87:19
C:\U\Go\src\runtime\map.go:92:2: evacuatedEmpty redeclared in this block
previous declaration at C:\U\Go\src\runtime\hashmap.go:88:19
C:\U\Go\src\runtime\map.go:93:2: evacuatedX redeclared in this block
previous declaration at C:\U\Go\src\runtime\hashmap.go:89:19
C:\U\Go\src\runtime\map.go:93:2: too many errors
Steps I took.
Installed Go from here
Installed dep from here
Setup environment
Ran dep ensure
and then go build -o azcopy.exe
Also I would like to know is if azcopy binary is self contained or it has go dependencies?
Can I just copy the binary to another computer and execute it without having to install dependencies?
if no? what do I need to install?
Thanks
from azure-storage-azcopy.
Hi @123dev, it looks like your Go environment has some problem, please follow the instructions here to verify that Go was installed properly.
To build AzCopy, you could CD into the directory of azure-storage-azcopy
, and either:
- Run
dep ensure
followed bygo build -o azcopy.exe
. - Or install Docker, and run
make all
, which will produce 3 executables, one for Mac, one for Linux, and one for Windows.
And yes, the AzCopy binary is self-contained: you can copy the binary to another computer and execute it just like that.
I apologize for the inconvenience. We should have the download links ready very soon. Thanks!
from azure-storage-azcopy.
Thanks zezha for all the assistance,
I re-ran the Go installer and chose repair, and it repaired it, I am no able to build it on windows.
Sorry for the false alarm, my inexperience with Go is the culprit :)
Thanks
from azure-storage-azcopy.
Hi @123dev, it's great to hear that you got it working!
Please let us know if you have any feedback on the tool. We are actively working on it to improve the user experience!
from azure-storage-azcopy.
Sure zezha,
One thing I came across is that wherever I issue the command azcopy there are two log files in that directory.
azcopy.log which is always 0 bytes long
and .log
Any way to suppress the logging? or at least control where they are output and what name?
So far it's working great on windows and Mac
Great work.
Thanks
from azure-storage-azcopy.
Hi @123dev, thanks for the feedbacks!
We have fixed the log file location issue on the dev branch, we'll publish it soon!
In the future, the log files will be placed under a folder called .azcopy
in the user's home directory, so that it's still visible, in case the user wants to find them.
from azure-storage-azcopy.
Awesome, thanks
from azure-storage-azcopy.
Related Issues (20)
- AZCopy Sync with Exclude-path does not work with DFS as source
- 0.0% while uploading larger files HOT 3
- azcopy copy --list-of-files ignores failures when listed files do not exist
- High number of calls to GetBlobProperties when running azcopy copy HOT 3
- Permanent download link for specific versions
- Azcopy command taking lot of time while copying large number of files having small size. HOT 3
- list option + Azure files cannot work with MSI "azure files requires a SAS token for authentication" HOT 1
- Data missing without indications of it HOT 6
- build new azcopy binary with go 1.22.5 HOT 4
- Azcopy on Mac: too many open files HOT 1
- AzCopy not working on Azure Cloud Shell HOT 2
- azcopy don't work in alpine since 10.25.0 HOT 1
- VHD file was not deleted when Azcopy ongoing process was terminated, however the VHD file in Azure Blob show the same size as AWS S3 HOT 4
- azcopy url-decodes slashes from directory names HOT 2
- storage-preview for az storage azcopy blob download is missing --overwrite option HOT 1
- azcopy jobs resume fails with "panic" or "sp is mandatory. Cannot be empty" HOT 2
- azcopy jobs list command is broken HOT 4
- AzCopy sync with hash-meta-dir hides original files HOT 4
- azcopy download shuffles lines on windows but works on linux HOT 5
- AzCopy fails to write xattr HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from azure-storage-azcopy.