使用root权限运行go run main.go,"main.go"如下所示：

//main.go
package main

import (
	"log"
	"os"
	"os/exec"
	"syscall"
)

func main() {
	cmd := exec.Command("sh")
	cmd.SysProcAttr = &syscall.SysProcAttr{
		Cloneflags: syscall.CLONE_NEWUTS | syscall.CLONE_NEWIPC | syscall.CLONE_NEWPID |
			syscall.CLONE_NEWNS | syscall.CLONE_NEWUSER,
	}
	cmd.SysProcAttr.Credential = &syscall.Credential{
		Uid: uint32(1), Gid: uint32(1)}
	cmd.Stdin = os.Stdin
	cmd.Stdout = os.Stdout
	cmd.Stderr = os.Stderr

	if err := cmd.Run(); err != nil {
		log.Fatal(err)
	}
	os.Exit(-1)
}

会出现下列报错

2017/07/30 01:27:36 fork/exec /bin/sh: operation not permitted
exit status 1

谁能解答一下？

发行版: centos 7

uname -a : Linux localhost.localdomain 3.10.0-514.6.2.el7.toa.2.x86_64 #1 SMP Tue Oct 31 14:54:31 CST 2017 x86_64 x86_64 x86_64 GNU/Linux

有 #3 这个提示内核问题，里边有座 Uid Gid map 的 ，有 添加这个 Credential 属性的 , 等等，
挨个试了一遍，实在是试不出来结果。求解决。

./mydocker run -ti busybox sh
{"level":"info","msg":"createTty true","time":"2017-08-23T14:10:44+08:00"}
{"level":"error","msg":"get init process error readlink /proc/self/exe: no such file or directory","time":"2017-08-23T14:10:44+08:00"}
{"level":"error","msg":"New parent process error","time":"2017-08-23T14:10:44+08:00"}

after mount -t proc proc /proc

ok

针对不同版本的系统或者kernel我们没做过兼容性的测试，可以提交PR一起来提高mydocker的兼容性。

known issue:

1. 4.2+的内核中userns的使用方式变化导致operation not permitted： #3
2. Centos 7.x中的内核不支持userns
3. 4.4内核运行后会导致/proc挂载不释放，导致后续命令报错 #8
4. Ubuntu 14.04.4之后的版本使用systemd去管理cgroup，会导致cgroup的限制失效

fork/exec /usr/bin/sh: no such file or directory

环境

Linux vm_16_14_centos 3.10.0-862.3.2.el7.x86_64 #1 SMP Mon May 21 23:36:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

为搭建本书中的运行环境花了好多功夫，可不可以为运行环境构建一个docker镜像，这样就能很方便的在所有操作系统上进行学习

code-3.2

# ./mydocker run -ti -m 100m stress --vm-bytes 200m --vm-keep -m 1

{"level":"info","msg":"command all is stress --vm-bytes 200m --vm-keep -m 1","time":"2017-08-23T20:04:08+08:00"}
{"level":"info","msg":"init come on","time":"2017-08-23T20:04:08+08:00"}
{"level":"info","msg":"Current location is /root/go/src/github.com/xianlubird/mydocker","time":"2017-08-23T20:04:08+08:00"}
{"level":"error","msg":"Exec loop path error exec: \"stress\": executable file not found in $PATH","time":"2017-08-23T20:04:08+08:00"}
exec: "stress": executable file not found in $PATH
{"level":"warning","msg":"remove cgroup fail cgroup path error stat mydocker-cgroup: no such file or directory","time":"2017-08-23T20:04:08+08:00"}
{"level":"warning","msg":"remove cgroup fail cgroup path error stat mydocker-cgroup: no such file or directory","time":"2017-08-23T20:04:08+08:00"}

# lsb_release -a; uname -r
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 14.04.5 LTS
Release:	14.04
Codename:	trusty
4.4.0-31-generic

书不在身边，凭记忆描述下问题：
第五章 第一节说docker daemon退出会导致容器关闭，原因是容器都是docker deamon fork出来的。
然后说1.12? 之后做了改进（runc？），docker daemon退出不会导致容器退出了，然后借着mydocker的代码说了下原理，大体就是docker daemon退出后容器进程会被init接管，所以不会退出。

那么问题来了，我的理解子进程在父进程退出后被init接管，这个是linux本身就自带的机制，那么为什么1.12？之前的docker daemon反而能忽略这个机制，自己退出了还顺带把子进程也搞退出呢？

请指教，谢谢！

目前看到第三章

有一些问题

1、比如mount -t proc proc /proc命令执行后，书中没有恢复挂载的命令，会导致3-1的程序执行得到fork/exec /proc/self/exe: no such file or directory的错误，因为/proc的内容变了，/proc/self/exe文件没有了，看书的人一般应该都不熟这些环境变化的命令，希望每个实验做完有说明恢复环境的方法，毕竟拿自己的机器实验不知道怎么恢复比较惨，此外其他的挂载之类的实验也没有说明

2、2.2里限制内存后stress命令无法执行，比如显示100M则命令无法使用200M做参数无法启动，只能使用100M以下的参数，不知道是stress命令版本高了还是Ubuntu版本高了（我用16.04），希望能提供含有无问题完成所有实验的虚拟机环境（有所有软件和包）下载？

目前看书实验做下来感觉还好，环境问题还能自己鼓捣鼓捣解决，书写得很好，感觉做应用不懂一些依赖系统接口原理的我增长不少知识，当然最好比如aufs这些系统实现的算法原理书里也有提就更好了，看完再过来提建议

network/bridge.go:172: unknown netlink.Addr field 'Broadcast' in struct literal

Linux ubuntu 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
go version go1.7.1 linux/amd64

I am a new user, and when i print
echo $(docker logs wordpress-test | grep password)

can't find mysql account

just have this

Complete! WordPress has been successfully copied to /var/www/html
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
[Thu Jan 04 06:30:23.474586 2018] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.25 (Debian) PHP/7.2.0 configured -- resuming normal operations
[Thu Jan 04 06:30:23.474760 2018] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'

code-3.1运行./mydocker run -ti ./bin/sh
之前

运行之后

然后要重启才能恢复正常。

求解答。感谢！

2.1.6 User Namespace 第17页以root运行时报错：

fork/exec /bin/sh: operation not permitted.

但是将cmd.SysProcAttr.Credential = &syscall.Credential{Did: uint32(1), Gid: uint32(1) }删掉后可以正常运行。

版本： Ubuntu 14.04 kernel: 4.4.0-31-generic

root@vagrant-ubuntu-trusty-64:/vagrant/mydocker# go version
go version go1.9.2 linux/amd64
root@vagrant-ubuntu-trusty-64:/vagrant/mydocker# godep version
godep v80 (linux/amd64/go1.9.2)

root@vagrant-ubuntu-trusty-64:/vagrant/mydocker# go env
GOARCH="amd64"
GOBIN="/usr/local/go/bin"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/vagrant/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build537744011=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

root@vagrant-ubuntu-trusty-64:/vagrant/mydocker# git branch -v

• code-3.1-branch d7511e7 Merge commit '4b834f5f611c5a40f25dacdcc73b078fddfadbeb'
  master 9ea8dbc Merge pull request #31 from s3341458/simplified-delete-volume
  root@vagrant-ubuntu-trusty-64:/vagrant/mydocker# godep go build
  godep: [WARNING]: godep should only be used inside a valid go package directory and
  godep: [WARNING]: may not function correctly. You are probably outside of your $GOPATH.
  godep: [WARNING]: Current Directory: /vagrant/mydocker
  godep: [WARNING]: $GOPATH: /vagrant/go
  main_command.go:7:2: cannot find package "github.com/xianlubird/mydocker/container" in any of:
  /usr/local/go/src/github.com/xianlubird/mydocker/container (from $GOROOT)
  /vagrant/mydocker/Godeps/_workspace/src/github.com/xianlubird/mydocker/container (from $GOPATH)
  /vagrant/go/src/github.com/xianlubird/mydocker/container
  godep: go exit status 1

in setns.go
should have #include <unistd.h> for close function.

I will make pr later if it is indeed a issue

I just record issue I found, after I got through all of this book, I will submit pr for it.

seems the struct initializer in code-6.5 are wrong.
in branch code-6.5 network/bridge.go
br := &netlink.Bridge{la}
addr := &netlink.Addr{ipNet}
will cause "too few values in struct initializer"

I got error:
{"level":"error","msg":"fork/exec /proc/self/exe: no such file or directory","time":"2018-04-25T23:19:48-04:00"}
when I try ./mydocker run -ti /bin/ls
after run command ./mydocker run -ti /bin/ls success.

fix by run "mount -t proc proc /proc"

Could anybody tell me what happen, many thanks

这不算是mydocker代码中的bug，只是我的一点疑惑。但是我在尝试自己实现nsenter.go后，想在C代码中执行setns系统调用，然后在Go代码中执行用户提交的Cmd。
然后在成功进入namespaces后，执行Go代码前，会报错：

runtime/cgo: pthread_create failed: Invalid argument
SIGABRT: abort
PC=0x7f3eab6bc207 m=0 sigcode=18446744073709551610

我谷歌了一下没有找到相关资料，唯一一个资料是在moby的issue里，但moby的member说可能是内核版本问题（我的环境是centos7，内核版本是4.20.8-1，猜测应该不是内核问题）。
我在C代码中去掉setns系统调用后，是不会报错的。
然后我去看mydocker的实现，它是在C中system(mydocker_cmd)，然后exit(0)，没有进入Go Runtime；
runC的实现比较复杂， 它是有runc init-1、2、3，在init-2中执行了setns，然后clone了init-3，然后exit。init-3中会进入Go Runtime。
我看到这两种实现都避开了setns后在当前进程中进入Go Runtime的操作，想问一下作者是否也遇到过这个问题，或者能帮忙看一下这个问题可能出现的原因。
谢谢。

I have fixed missing of resource command & cgroup manager in code-4.1.But Github does not support pulling request the tag to another tag.
Could you create a new branch based in tag code-4.1 so that I can pull request ?
@xianlubird

uname -a
Linux i-5x80cleb 3.13.0-24-generic #47-Ubuntu SMP Fri May 2 23:30:00 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

sudo ./mydocker run -ti busybox sh
{"level":"info","msg":"createTty true","time":"2017-08-24T14:34:11+08:00"}
{"level":"error","msg":"Run command for creating mount point failed exit status 32","time":"2017-08-24T14:34:11+08:00"}
{"level":"info","msg":"command all is sh","time":"2017-08-24T14:34:11+08:00"}
{"level":"info","msg":"init come on","time":"2017-08-24T14:34:11+08:00"}
{"level":"info","msg":"Current location is /root/mnt/4672873324","time":"2017-08-24T14:34:11+08:00"}
{"level":"error","msg":"Exec loop path error exec: "sh": executable file not found in $PATH","time":"2017-08-24T14:34:11+08:00"}
exec: "sh": executable file not found in $PATH
{"level":"error","msg":"Unmount /root/mnt/4672873324 error exit status 1","time":"2017-08-24T14:34:11+08:00"}

作者好，我在阅读本书3.2小节，动手做关于本节的实验时，遇到了一些问题，望解答

在提issue之前，我也翻了相关的issue，比如#35，#6，也想过通过自己解决，但是我go语言基本没怎么学习过，尝试了很久后问题仍然没解决，所以才提的这个issue，望作者能进行详细解答

我的操作系统是使用vagrant起的ubuntu14.04，vagrant box镜像是bento/ubuntu-14.04，具体实验环境如下

uname -r
3.13.0-135-generic
go version
go1.7.1 linux/amd64

我的操作及遇到的问题

git clone https://github.com/xianlubird/mydocker.git
cd mydocker
git checkout code-3.2 #HEAD is now at b500ded... fix bugs 看commit信息是fix bug之后的
go build .

抛出的错误是

./run.go:14: not enough arguments in call to container.NewParentProcess

这个错误相关issue可见#35，但是我的go版本本来就是1.7.1了，后来我又升级到1.9.2之后，还是有这个问题

./run.go:14:49: not enough arguments in call to container.NewParentProcess
have (bool)
want (bool, string, string, string, []string)

然后我阅读了tag-3.2的代码，看到引用的包都是

"github.com/xianlubird/mydocker/xxx"

之类的路径。

我的做法是将报错的那个包路径改成本地包引用

"github.com/xianlubird/mydocker/container" -> "./container"

然后go build .就没问题了，但是执行

./mydocker run -ti -m 100m stress --vm-bytes 200m --vm-keep -m 1

报错了

Incorrect Usage: flag provided but not defined: -vm-bytes

NAME:
   mydocker run - Create a container with namespace and cgroups limit
      mydocker run -ti [command]

USAGE:
   mydocker run [command options] [arguments...]

OPTIONS:
   --ti              enable tty
   -m value          memory limit
   --cpushare value  cpushare limit
   --cpuset value    cpuset limit

{"level":"fatal","msg":"flag provided but not defined: -vm-bytes","time":"2018-08-24T02:02:17Z"}

希望作者能复现一下这个问题，帮忙解决一下，感谢

您好 我的环境是centos7 go1.6.3 在运行code5-1时报了如下的错 请问怎么解决这个错误
[root@localhost mydocker]# ./mydocker run -d top
{"level":"info","msg":"createTty false","time":"2019-04-17T19:13:21-07:00"}
{"level":"error","msg":"fork/exec /proc/self/exe: no such file or directory","time":"2019-04-17T19:13:21-07:00"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x548036]

goroutine 1 [running]:
main.Run(0x5a4b00, 0xc42004a630, 0x1, 0x1, 0xc4200584e0)
/root/gocode/mydocker/run.go:26 +0x126
main.glob..func1(0xc42006c420, 0xc420088000, 0xc42006c420)
/root/gocode/mydocker/main_command.go:57 +0x3f1
github.com/urfave/cli.HandleAction(0x56d880, 0x5ae0f0, 0xc42006c420, 0x0, 0xc42004c180)
/home/mygo/src/github.com/urfave/cli/app.go:502 +0xbe
github.com/urfave/cli.Command.Run(0x5a35e1, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x5ad3d7, 0x51, 0x0, ...)
/home/mygo/src/github.com/urfave/cli/command.go:165 +0x4bb
github.com/urfave/cli.(*App).Run(0xc420080000, 0xc42000e080, 0x4, 0x4, 0x0, 0x0)
/home/mygo/src/github.com/urfave/cli/app.go:259 +0x740
main.main()
/root/gocode/mydocker/main.go:31 +0x17a

$ git checkout code-3.2
$ go build .
出现错误：
./main_command.go:58: not enough arguments in call to container.RunContainerInitProcess
./run.go:14: not enough arguments in call to container.NewParentProcess
./run.go:14: assignment count mismatch: 2 = 1

运行环境：ubuntu16.04
go version：1.6.2

请教~

in code-5.5 function stopContainer

a function "GetContainerPidByName" is used, but the actual function name declared in exec.go is "getContainerPidByName" notice the first character g is in lowercase.

https://github.com/xianlubird/mydocker/blob/code-3.2/container/init.go#L14-L32

line 16: if cmdArray == nil || len(cmdArray) == 0 既然cmdArray是切片类型，nil值的len也是0前一半没必要吧
line 20: setUpMount() 为什么不是 setupMount() setUp？
line 24: log.Errorf("Exec loop path error %v", err) loop? look吧？
line 27: log.Infof("Find path %s", path) Find? Found吧？
line 28: cmdArray[0:]? 切片类型从0开始取新切片的目的是啥？ 和cmdArray有啥区别？

I think I am very likely be wrong.
in code-4.3
But I thought docker volume was originally implemented by "bind mount" instead using union file system for data consistency and performance reason.

容器不能ping通，而且域名服务器那么配也有问题，不起作用，书中的测试都有问题，并且我使用的是个人热点开的Wi-Fi，然后在Mac Ubuntu14.04虚拟机上会在配置完网桥后出现网络连接不上

操作系统

root@ubuntu:/tools/go# uname -a
Linux ubuntu 4.10.0-35-generic #3916.04.1-Ubuntu SMP Wed Sep 13 09:02:42 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
root@ubuntu:~/tools/go# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial

root@ubuntu:~/project/docker-step# go version
go version go1.9.1 linux/amd64

测试步骤

package main
import (
"os/exec"
"syscall"
"os"
"log"
)

func main() {
cmd := exec.Command("sh")
cmd.SysProcAttr = &syscall.SysProcAttr{
Cloneflags: syscall.CLONE_NEWUTS | syscall.CLONE_NEWIPC | syscall.CLONE_NEWPID |syscall.CLONE_NEWNS | syscall.CLONE_NEWUSER,
}
cmd.SysProcAttr.Credential = &syscall.Credential{Uid: uint32(1),Gid: uint32(1)}
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err := cmd.Run(); err !=nil{
log.Fatal(err)
}
os.Exit(-1)
}

实际结果

1.1. root@ubuntu:~/project/docker-step# go run main.go
2017/10/11 16:17:52 fork/exec /bin/sh: operation not permitted
exit status 1。

code-2.1

OS: centos 7.4

Linux Core: 3.10.0-693.el7.x86_64

Golang Version: go1.8.3 linux/amd64

command: go run main.go.

ERROR:

command-line-arguments

./main.go:22: undefined: initCommand
./main.go:23: undefined: runCommand

切换到tag code-3.2,按照《自己动手写docker》上3.2小节root权限运行如下命令：

# git checkout code-3.2
# go build .
# ./mydocker  run -ti -m 100m stress  --vm-bytes 200m  --vm-keep  -m 1

产生如下报错

{"level":"error","msg":"fork/exec /proc/self/exe: no such file or directory","time":"2017-08-10T16:20:27+08:00"}
{"level":"warning","msg":"remove cgroup fail cgroup path error stat mydocker-cgroup: no such file or directory","time":"2017-08-10T16:20:27+08:00"}
{"level":"warning","msg":"remove cgroup fail cgroup path error stat mydocker-cgroup: no such file or directory","time":"2017-08-10T16:20:27+08:00"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x401342]

goroutine 1 [running]:
panic(0x557520, 0xc42000e0c0)
	/usr/local/go/src/runtime/panic.go:500 +0x1a1
main.Run(0x564201, 0xc420074100, 0x6, 0x8, 0xc42001a390)
	/home/aimingzhen/gopath/src/github.com/xianlubird/mydocker/run.go:26 +0x102
main.glob..func1(0xc420078140, 0x100, 0xc420078140)
	/home/aimingzhen/gopath/src/github.com/xianlubird/mydocker/main_command.go:48 +0x31d
github.com/xianlubird/mydocker/vendor/github.com/urfave/cli.HandleAction(0x5507c0, 0x5939c0, 0xc420078140, 0xc42004e200, 0x0)
	/home/aimingzhen/gopath/src/github.com/xianlubird/mydocker/vendor/github.com/urfave/cli/app.go:485 +0xd4
github.com/xianlubird/mydocker/vendor/github.com/urfave/cli.Command.Run(0x57ee39, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x588a83, 0x51, 0x0, ...)
	/home/aimingzhen/gopath/src/github.com/xianlubird/mydocker/vendor/github.com/urfave/cli/command.go:193 +0xb96
github.com/xianlubird/mydocker/vendor/github.com/urfave/cli.(*App).Run(0xc420001860, 0xc42000a0b0, 0xb, 0xb, 0x0, 0x0)
	/home/aimingzhen/gopath/src/github.com/xianlubird/mydocker/vendor/github.com/urfave/cli/app.go:250 +0x812
main.main()
	/home/aimingzhen/gopath/src/github.com/xianlubird/mydocker/main.go:31 +0x153

能帮忙解决一下吗？
运行环境如下：
os: ubuntu14.04 ,linux 3.13.0.83
go:1.7.1

很奇怪，一本讲 docker 的书，居然没有实验环境的 Dockerfile……

在runc项目中的实现是使用了unix.PivotRoot(".", ".")来切换rootfs的，而这个系统调用我在man page上看到的说明是两个目录是不能相同的，请问一下这句代码要怎么理解呢？

go build
go build github.com/xianlubird/mydocker/vendor/github.com/urfave/cli: /usr/local/go/pkg/tool/linux_amd64/compile: open /dev/null: no such file or directory
go build github.com/xianlubird/mydocker/vendor/github.com/vishvananda/netns: /usr/local/go/pkg/tool/linux_amd64/compile: open /dev/null: no such file or directory
go build github.com/xianlubird/mydocker/vendor/github.com/Sirupsen/logrus: /usr/local/go/pkg/tool/linux_amd64/compile: open /dev/null: no such file or directory
go build github.com/xianlubird/mydocker/cgroups/subsystems: /usr/local/go/pkg/tool/linux_amd64/compile: open /dev/null: no such file or directory
go build github.com/xianlubird/mydocker/nsenter: /usr/local/go/pkg/tool/linux_amd64/cgo: open /dev/null: no such file or directory

请问3.1节 ，也就是code-3.1 main_command.go中的initCommand的Action函数在哪里调用的，我gdb跟踪代码一直没跟踪到这个函数里面去，就直接运行了，所有我不知道这个方法是从哪里进入并运行的，谢谢！

2.1.2 UTS Namespace
in elementary os 0.41(ubuntu 16.04, Linux version 4.8.0-58-generic)

solitarius@Lain:~/GoglandProjects/DockerTry$ go run UTS_NAMESPACE.go
2017/07/11 10:10:47 fork/exec /bin/sh: operation not permitted

请问，为啥通过/proc/self/exe 而不是直接调用init方法，有啥区别？

NewPipe should introduced at code-3.3
setUpMount should introduced at code-4.1
but they all included in 6a066fe

os: Linux chen-pc 4.15.0-30deepin-generic 2018 x86_64 GNU/Linux
go : go version go1.12.5 linux/amd64

在书的2.1.5例子上碰到问题, 我也知道和书上要求的kernel version不一致, 但还是希望有人知道的话帮忙答复一下解决方法

...
cmd.SysProcAttr = &syscall.SysProcAttr{
    Cloneflags: syscall.CLONE_NEWNS,
}
...

运行:

$ go build -o mount
$ sudo ./mount
[sudo] chen 的密码：
# ls /proc
1      11173  13946  1770   20 ...
# mount -t proc proc /proc
mount: proc is already mounted or /proc busy
       proc is already mounted on /proc

According to the code in the book, git checkout branch failed. Only two branches now. Having you executed git checkout code-3.1 ????

i just test "user namespaces", and i found after run my code, it will exit status 1

Finally got to the last step (the network), experiencing a weird issue after trying to test code-6.5 when tried to ping another container. The ping command hangs there without any progressing (no package loss or something).

Really appreciate it If you guys can give me some hints.

However arp commend can get the correct hardware mac address.
container can ping bridge and host eth, however can not ping another container or externel website.

details:
container 1:
/bin/ifconfig
cif-27020 Link encap:Ethernet HWaddr 3A:2C:B8:E3:EA:20
inet addr:192.169.0.11 Bcast:192.169.0.255 Mask:255.255.255.0
inet6 addr: fe80::382c:b8ff:fee3:ea20/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:54 errors:0 dropped:0 overruns:0 frame:0
TX packets:74 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4100 (4.0 KiB) TX bytes:6228 (6.0 KiB)
/bin/arp -a
? (192.169.0.12) at 16:dd:0d:87:05:d5 [ether] on cif-27020
? (192.169.0.10) at on cif-27020
? (192.169.0.1) at 4a:a3:1c:01:f3:7b [ether] on cif-27020

.....

container 2:
ifconfig
cif-27020 Link encap:Ethernet HWaddr 3A:2C:B8:E3:EA:20
inet addr:192.169.0.11 Bcast:192.169.0.255 Mask:255.255.255.0
inet6 addr: fe80::382c:b8ff:fee3:ea20/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:54 errors:0 dropped:0 overruns:0 frame:0
TX packets:74 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4100 (4.0 KiB) TX bytes:6228 (6.0 KiB)
.....
/ # /bin/arp -a
(192.169.0.11) at 3a:2c:b8:e3:ea:20 [ether] on cif-33306
(192.169.0.10) at on cif-33306
(192.169.0.1) at 4a:a3:1c:01:f3:7b [ether] on cif-33306

host:
ifconfig
...
testbridge: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.169.0.1 netmask 255.255.255.0 broadcast 192.169.0.255
inet6 fe80::4ccf:70ff:fe3f:d0aa prefixlen 64 scopeid 0x20
ether 4a:a3:1c:01:f3:7b txqueuelen 1000 (Ethernet)
RX packets 422 bytes 29504 (28.8 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 198 bytes 14380 (14.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

...

brctl show
...
testbridge 8000.4aa31c01f37b no 27020
33306

//Delete the AUFS filesystem while container exit
func DeleteWorkSpace(rootURL string, mntURL string, volume string){
/f(volume != ""){
volumeURLs := volumeUrlExtract(volume)
length := len(volumeURLs)
if(length == 2 && volumeURLs[0] != "" && volumeURLs[1] !=""){
DeleteMountPointWithVolume(rootURL, mntURL, volumeURLs)
}else{
DeleteMountPoint(rootURL, mntURL)
}
}else {
DeleteMountPoint(rootURL, mntURL)
}
DeleteWriteLayer(rootURL)
}

seems can be done by
if (volume) {
.. umount volume
}
.. umount container mnt
delete write layer

basically the if else clause have room be simplified