Git Product home page Git Product logo

bbcp's People

Contributors

abh3 avatar klabattermann avatar rtobar avatar

Stargazers

avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

yabostone niki-on-github

bbcp's Issues

"free(): invalid pointer" on Ubuntu 20.04 and Debian Buster

I was hunting for a more recent fork of bbcp - this one seems maintained and with more recent commits. Unfortunately - I had some issues using it. The copy seems to happen with no issues, but there's an issue with cleanup at the end and the return status of the program is not 0. I tried -s 1 and I still had the same problem.

Went with https://github.com/scaleengine/bbcp and no such issues. I can help debug if you guys are interested in getting to the bottom of this.

BTW I had it built in a container too if you want to easily reproduce it and you don't have access to debian or ubuntu.

FROM ruby:2.7-slim as BBCP_BUILDER
RUN apt-get update && \
  apt-get install -y build-essential libssl-dev git
RUN git clone https://github.com/ICRAR/bbcp.git
# uname -i is not available in docker. Hardcode arch.
RUN sed -i 's/`\/bin\/uname -i`/x86_64/' bbcp/src/Makefile
RUN cd bbcp/src && make clean && make

Some examples:

free(): invalid pointer, killed by signal 6

bbcp -s 1 bbcp.C [email protected]:/home/costi/                                                                                                               
bbcp: Copy process 815813 was killed via signal 6
free(): invalid pointer
bbcp: Copy process 815807 was killed via signal 6

Copy process was killed via signal 11

bbcp -s 1 foo [email protected]:/home/costi/                                                                                                                  
bbcp: Copy process 815345 was killed via signal 11
bbcp: Copy process 815341 was killed via signal 11

Full debug:

costi@bf3a566c00b4:/app$ bbcp --debug -i /ssh/id_rsa blah [email protected]:/home/costi/
bbcp_CTL 946: Optimum MT level is 8
bbcp_CTL: Running as pid 947: ssh -x -a -oFallBackToRsh=no -i /ssh/id_rsa -l costi ryzen7.home bbcp SNK 
bbcp_CTL: Sending to ryzen7.home:  -D -m 644 -s 4 -W 131072 -Y 80dd4960 -H none:0

bbcp_bbcp_CTLCTL: Sending to  ryzen7.home: 947/home/costi/: 
PATH=
bbcp_CTL/usr/local/bundle/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin: Sending to 
ryzen7.home: 
bbcp_SNK 17887: MaxWin=425984 MaxSeg=536 AT=1
bbcp_SNK 17887: Window set to 131072 AT=1
bbcp_SNK 17887: rwbsz=163840 BNum=12
bbcp_SNK 17887: Optimum MT level is 9
bbcp_SNK 17888: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
bbcp_SNK 17887: Thread 140132234241792 monitoring process 17835
bbcp_SNK 17887: bind recv window set to -1 (actual=131072 segsz=536)
bbcp_CTL 946: Received from ryzen7.home: 200 Port: 5031 Host: ryzen7

bbcp_CTL: Running as pid 948: bbcp SRC 
bbcp_CTL: Sending to bf3a566c00b4:  -D -m 644 -s 4 -W 131072 -Y 80dd4960 -H ryzen7:5031

bbcp_CTL: Sending to bf3a566c00b4: blah

bbcp_CTL: Sending to bf3a566c00b4: 
bbcp_CTL 948: PATH=/usr/local/bundle/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
bbcp_SRC 948: Max Window=425984 MaxSeg=536 AT=1
bbcp_SRC 948: Window set to 131072 AT=1
bbcp_SRC 948: rwbsz=163840 BNum=12
bbcp_SRC 948: Optimum MT level is 8
bbcp_SRC 949: PATH=/usr/local/bundle/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
bbcp_SRC 948: Thread 139768807147264 monitoring process 877
bbcp_SRC 948: getFS: blah running in /tmp
bbcp_SRC 948: getFS norm blah
bbcp_SRC 948: FS_Unix::Stat for blah
bbcp_SRC.Stat: No such file or directory processing blah
bbcp_SRC 948: connect send window set to -1 (actual=16384 segsz=536)
bbcp_SRC 948: Connected to: ryzen7.home:5031
bbcp_SRC: Sending to ryzen7.home: login ctlr 80dd4960 wsz: +131072 ver: 17.12.00.00.0 dsz: 163840

bbcp_SRC 948: Received from ryzen7.home: 204 loginok wsz: 131072 163840

bbcp_SNK 17887: accept recv window set to -1 (actual=131072 segsz=1448)
bbcp_SNK 17887: Received from costi-linux-zuper.home: login ctlr 80dd4960 wsz: +131072 ver: 17.12.00.00.0 dsz: 163840

bbcp_SNK: Sending to costi-linux-zuper.home: 204 loginok wsz: 131072 163840

bbcp_SRC 948: connect send window set to -1 (actual=16384 segsz=536)
bbcp_SRC 948: Connected to: ryzen7.home:5031
bbcp_SRC: Sending to ryzen7.home: login data 80dd4960

bbcp_SRC 948: connect send window set to -1 (actual=16384 segsz=536)
bbcp_SRC 948: Connected to: ryzen7.home:5031
bbcp_SRC: Sending to ryzen7.home: login data 80dd4960

bbcp_SNK 17887: accept recv window set to -1 (actual=131072 segsz=1448)
bbcp_SNK 17887: Received from costi-linux-zuper.home: login data 80dd4960

bbcp_SNK 17887: accept recv window set to -1 (actual=131072 segsz=1448)
bbcp_SNK 17887: Received from costi-linux-zuper.home: login data 80dd4960

bbcp_SRC 948: connect send window set to -1 (actual=16384 segsz=536)
bbcp_SRC 948: Connected to: ryzen7.home:5031
bbcp_SRC: Sending to ryzen7.home: login data 80dd4960

bbcp_SRC 948: connect send window set to -1 (actual=16384 segsz=536)
bbcp_SRC 948: Connected to: ryzen7.home:5031
bbcp_SRC: Sending to ryzen7.home: login data 80dd4960

bbcp_SNK 17887: accept recv window set to -1 (actual=131072 segsz=1448)
bbcp_SNK 17887: Received from costi-linux-zuper.home: login data 80dd4960

bbcp_SRC 948: Allocated 12 163840 (159744) byte buffs in the net pool.
bbcp_SRC: Sending to ryzen7.home: eol

bbcp_SNK 17887: accept recv window set to -1 (actual=131072 segsz=1448)
bbcp_SNK 17887: Received from costi-linux-zuper.home: login data 80dd4960

bbcp_SNK 17887: Allocated 12 167936 (163840) byte buffs in the net pool.
bbcp_SNK 17887: getFS: /home/costi/ running in /tmp
bbcp_SNK 17887: getFS norm /home/costi/
bbcp_SNK 17887: FS_Unix::Stat for /home/costi/
bbcp_SNK: Sending to costi-linux-zuper.home: flist

bbcp_SNK 17887: Received from costi-linux-zuper.home: eol

bbcp_bbcp_CTL 946: Received from ryzen7.home: 200 End: 0 0

bbcp_CTL 946: At end files=0 bytes=0
SNK: Sending to costi-linux-zuper.home: exit 0

bbcp_CTL 946: Process 947 ended; rc=0
bbcp_CTL.Waitpid: Copy process 948 was killed via signal 11
bbcp_CTL 946: Process 948 ended; rc=255
bbcp_CTL 946: Ending; rc=255 files=0 bytes=0
costi@bf3a566c00b4:/app$ bbcp --debug -i /ssh/id_rsa blah [email protected]:/home/costi/
bbcp_CTL 955: Optimum MT level is 8
bbcp_CTL: Running as pid 956: ssh -x -a -oFallBackToRsh=no -i /ssh/id_rsa -l costi ryzen7.home bbcp SNK 
bbcp_bbcp_CTLCTL: Sending to  ryzen7.home:  -D -m 644 -s 4 -W 131072 -Y 9736e633 -H none:0
956
: PATH=bbcp_CTL: Sending to ryzen7.home/usr/local/bundle/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin: 
/home/costi/

bbcp_CTL: Sending to ryzen7.home: 
bbcp_SNK 17948: MaxWin=425984 MaxSeg=536 AT=1
bbcp_SNK 17948: Window set to 131072 AT=1
bbcp_SNK 17948: rwbsz=163840 BNum=12
bbcp_SNK 17948: Optimum MT level is 9
bbcp_SNK 17949: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
bbcp_SNK 17948: Thread 140651933218560 monitoring process 17896
bbcp_SNK 17948: bind recv window set to -1 (actual=131072 segsz=536)
bbcp_CTL 955: Received from ryzen7.home: 200 Port: 5031 Host: ryzen7

bbcp_CTL: Running as pid 957: bbcp SRC 
bbcp_CTL: Sending to bf3a566c00b4:  -D -m 644 -s 4 -W 131072 -Y 9736e633 -H ryzen7:5031

bbcp_CTL: Sending to bf3a566c00b4: blah

bbcp_CTL: Sending to bf3a566c00b4: 
bbcp_CTL 957: PATH=/usr/local/bundle/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
bbcp_SRC 957: Max Window=425984 MaxSeg=536 AT=1
bbcp_SRC 957: Window set to 131072 AT=1
bbcp_SRC 957: rwbsz=163840 BNum=12
bbcp_SRC 957: Optimum MT level is 8
bbcp_SRC 958: PATH=/usr/local/bundle/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
bbcp_SRC 957: Thread 140299876443904 monitoring process 877
bbcp_SRC 957: getFS: blah running in /tmp
bbcp_SRC 957: getFS norm blah
bbcp_SRC 957: FS_Unix::Stat for blah
bbcp_SRC.Stat: No such file or directory processing blah
bbcp_SRC 957: connect send window set to -1 (actual=16384 segsz=536)
bbcp_SRC 957: Connected to: ryzen7.home:5031
bbcp_SRC: Sending to ryzen7.home: login ctlr 9736e633 wsz: +131072 ver: 17.12.00.00.0 dsz: 163840

bbcp_SRC 957: Received from ryzen7.home: 204 loginok wsz: 131072 163840

bbcp_SNK 17948: accept recv window set to -1 (actual=131072 segsz=1448)
bbcp_SNK 17948: Received from costi-linux-zuper.home: login ctlr 9736e633 wsz: +131072 ver: 17.12.00.00.0 dsz: 163840

bbcp_SNK: Sending to costi-linux-zuper.home: 204 loginok wsz: 131072 163840

bbcp_SRC 957: connect send window set to -1 (actual=16384 segsz=536)
bbcp_SRC 957: Connected to: ryzen7.home:5031
bbcp_SRC: Sending to ryzen7.home: login data 9736e633

bbcp_SRC 957: connect send window set to -1 (actual=16384 segsz=536)
bbcp_SRC 957: Connected to: ryzen7.home:5031
bbcp_SRC: Sending to ryzen7.home: login data 9736e633

bbcp_SRC 957: connect send window set to -1 (actual=16384 segsz=536)
bbcp_SRC 957: Connected to: ryzen7.home:5031
bbcp_SRC: Sending to ryzen7.home: login data 9736e633

bbcp_bbcp_SNK 17948: accept recv window set to -1 (actual=131072 segsz=1448)
bbcp_SNK 17948: Received from costi-linux-zuper.home: login data 9736e633

SRC 957: connect send window set to -1 (actual=16384 segsz=536)
bbcp_SRC 957: Connected to: ryzen7.home:5031
bbcp_SRC: Sending to ryzen7.home: login data 9736e633

bbcp_SNK 17948: accept recv window set to -1 (actual=131072 segsz=1448)
bbcp_SNK 17948: Received from costi-linux-zuper.home: login data 9736e633

bbcp_SRC 957: Allocated 12 163840 (159744) byte buffs in the net pool.
bbcp_SRC: Sending to ryzen7.home: eol

free(): invalid pointer
bbcp_SNK 17948: accept recv window set to -1 (actual=131072 segsz=1448)
bbcp_SNK 17948: Received from costi-linux-zuper.home: login data 9736e633

bbcp_SNK 17948: accept recv window set to -1 (actual=131072 segsz=1448)
bbcp_SNK 17948: Received from costi-linux-zuper.home: login data 9736e633

bbcp_SNK 17948: Allocated 12 167936 (163840) byte buffs in the net pool.
bbcp_SNK 17948: getFS: /home/costi/ running in /tmp
bbcp_SNK 17948: getFS norm /home/costi/
bbcp_SNK 17948: FS_Unix::Stat for /home/costi/
bbcp_SNK: Sending to costi-linux-zuper.home: flist

bbcp_SNK 17948: Received from costi-linux-zuper.home: eol

bbcp_SNK: Sending to costi-linux-zuper.home: exit 0

bbcp_CTL 955: Received from ryzen7.home: 200 End: 0 0

bbcp_CTL 955: At end files=0 bytes=0
bbcp_CTL 955: Process 956 ended; rc=0
bbcp_CTL.Waitpid: Copy process 957 was killed via signal 6
bbcp_CTL 955: Process 957 ended; rc=255
bbcp_CTL 955: Ending; rc=255 files=0 bytes=0

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.