本文主要解析harbor打离线包的流程，并在此基础上给出如果需要替换registry的操作方案。
- harbor源码基于1.5.1版本
为什么要分析harbor打离线包的流程
在我们的项目过程会涉及到修改harbor源码的需求，然后修改源码之后自然需要出包部署；
通过文章从Habor源码进行安装及打包可以得知修改harbor源码之后可以用如下命令出离线包（需要将下面命令里的版本替换自己实际的版本）：1
make package_offline GOBUILDIMAGE=golang:1.9.2 COMPILETAG=compile_golangimage CLARITYIMAGE=vmware/harbor-clarity-ui-builder:1.3.0
除此之外还可能会涉及到修改docker registry源码，通过前面的文章Habor初印象可以知道harbor是依赖于docker registry的，所以要搞清楚修改docker registry之后如何将修改打包到harbor的离线包中。本文最后会基于流程分析，给出两个将registry修改打包到harbor的离线包中的方案。
harbor打离线包的流程
harbor打离线包流程源码
在项目下github.com/docker/distribution有Makefile1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19 ll
total 100
-rw-r--r-- 1 root root 2016 May 22 2018 AUTHORS
-rw-r--r-- 1 root root 3223 May 22 2018 CHANGELOG.md
drwxr-xr-x 7 root root 4096 May 22 2018 contrib
-rw-r--r-- 1 root root 4123 May 22 2018 CONTRIBUTING.md
drwxr-xr-x 3 root root 4096 May 22 2018 docs
-rw-r--r-- 1 root root 10771 May 22 2018 LICENSE
drwxr-xr-x 7 root root 4096 Apr 17 16:01 make
-rw-r--r-- 1 root root 19284 Mar 26 17:08 Makefile
-rw-r--r-- 1 root root 482 May 22 2018 NOTICE
-rw-r--r-- 1 root root 976 May 22 2018 partners.md
-rw-r--r-- 1 root root 5206 May 22 2018 README.md
-rw-r--r-- 1 root root 2214 May 22 2018 ROADMAP.md
drwxr-xr-x 9 root root 4096 May 22 2018 src
drwxr-xr-x 9 root root 4096 May 22 2018 tests
drwxr-xr-x 6 root root 4096 May 22 2018 tools
-rw-r--r-- 1 root root 3 Apr 15 17:50 UIVERSION
-rw-r--r-- 1 root root 6 May 22 2018 VERSION
出离线包的源码为：1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20package_offline: compile version build modify_sourcefiles modify_composefile
@echo "packing offline package ..."
@cp -r make $(HARBORPKG)
@cp LICENSE $(HARBORPKG)/LICENSE
@cp NOTICE $(HARBORPKG)/NOTICE
@cp $(HARBORPKG)/photon/db/registry.sql $(HARBORPKG)/ha/
@if [ "$(MIGRATORFLAG)" = "true" ] ; then \
echo "pulling Harbor migrator..."; \
$(DOCKERPULL) vmware/harbor-migrator:$(MIGRATORVERSION); \
fi
@echo "saving harbor docker image"
@$(DOCKERSAVE) $(DOCKERSAVE_PARA) > $(HARBORPKG)/$(DOCKERIMGFILE).$(VERSIONTAG).tar
@gzip $(HARBORPKG)/$(DOCKERIMGFILE).$(VERSIONTAG).tar
@$(TARCMD) $(PACKAGE_OFFLINE_PARA)
@rm -rf $(HARBORPKG)
@echo "Done."
出离线包的命令如下（请将版本号修改为自己环境实际的版本号）：1
make package_offline GOBUILDIMAGE=golang:1.9.2 COMPILETAG=compile_golangimage CLARITYIMAGE=vmware/harbor-clarity-ui-builder:1.3.0
代码分析
通过代码可以看到出离线包大致需要如下步骤：
- compile
- version
- build
- modify_sourcefiles
- modify_composefile
- 保存harbor docker镜像
- 使用tar -zcvf命令进行打包
下面详细分析每一步：
compile && version
1 | compile:check_environment compile_golangimage |
可以看到compile就做了检查环境check_environment和编译go镜像compile_golangimage两件事；
check_environment:1
2check_environment:
@$(MAKEPATH)/$(CHECKENVCMD)
其中1
2
3BUILDPATH=$(CURDIR)
MAKEPATH=$(BUILDPATH)/make
CHECKENVCMD=checkenv.sh
所以检查环境就是执行checkenv.sh脚本
脚本内容如下：1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148#/bin/bash
#docker version: 1.11.2
#docker-compose version: 1.7.1
#Harbor version: 0.4.5+
set +e
set -o noglob
#
# Set Colors
#
bold=$(tput bold)
underline=$(tput sgr 0 1)
reset=$(tput sgr0)
red=$(tput setaf 1)
green=$(tput setaf 76)
white=$(tput setaf 7)
tan=$(tput setaf 202)
blue=$(tput setaf 25)
#
# Headers and Logging
#
underline() { printf "${underline}${bold}%s${reset}\n" "$@"
}
h1() { printf "\n${underline}${bold}${blue}%s${reset}\n" "$@"
}
h2() { printf "\n${underline}${bold}${white}%s${reset}\n" "$@"
}
debug() { printf "${white}%s${reset}\n" "$@"
}
info() { printf "${white}➜ %s${reset}\n" "$@"
}
success() { printf "${green}✔ %s${reset}\n" "$@"
}
error() { printf "${red}✖ %s${reset}\n" "$@"
}
warn() { printf "${tan}➜ %s${reset}\n" "$@"
}
bold() { printf "${bold}%s${reset}\n" "$@"
note() { printf "\n${underline}${bold}${blue}Note:${reset} ${blue}%s${reset}\n" "$@"
}
set -e
usage=$'Checking environment for harbor build and install. Include golang, docker and docker-compose.'
while [ $# -gt 0 ]; do
case $1 in
--help)
note "$usage"
exit 0;;
*)
note "$usage"
exit 1;;
esac
shift || true
done
function check_golang {
if ! go version &> /dev/null
then
warn "No golang package in your enviroment. You should use golang docker image build binary."
return
fi
# docker has been installed and check its version
if [[ $(go version) =~ (([0-9]+)\.([0-9]+)([\.0-9]*)) ]]
then
golang_version=${BASH_REMATCH[1]}
golang_version_part1=${BASH_REMATCH[2]}
golang_version_part2=${BASH_REMATCH[3]}
# the version of golang does not meet the requirement
if [ "$golang_version_part1" -lt 1 ] || ([ "$golang_version_part1" -eq 1 ] && [ "$golang_version_part2" -lt 6 ])
then
warn "Better to upgrade golang package to 1.6.0+ or use golang docker image build binary."
return
else
note "golang version: $golang_version"
fi
else
warn "Failed to parse golang version."
return
fi
}
function check_docker {
if ! docker --version &> /dev/null
then
error "Need to install docker(1.10.0+) first and run this script again."
exit 1
fi
# docker has been installed and check its version
if [[ $(docker --version) =~ (([0-9]+)\.([0-9]+)([\.0-9]*)) ]]
then
docker_version=${BASH_REMATCH[1]}
docker_version_part1=${BASH_REMATCH[2]}
docker_version_part2=${BASH_REMATCH[3]}
# the version of docker does not meet the requirement
if [ "$docker_version_part1" -lt 1 ] || ([ "$docker_version_part1" -eq 1 ] && [ "$docker_version_part2" -lt 10 ])
then
error "Need to upgrade docker package to 1.10.0+."
exit 1
else
note "docker version: $docker_version"
fi
else
error "Failed to parse docker version."
exit 1
fi
}
function check_dockercompose {
if ! docker-compose --version &> /dev/null
then
error "Need to install docker-compose(1.7.1+) by yourself first and run this script again."
exit 1
fi
# docker-compose has been installed, check its version
if [[ $(docker-compose --version) =~ (([0-9]+)\.([0-9]+)([\.0-9]*)) ]]
then
docker_compose_version=${BASH_REMATCH[1]}
docker_compose_version_part1=${BASH_REMATCH[2]}
docker_compose_version_part2=${BASH_REMATCH[3]}
# the version of docker-compose does not meet the requirement
if [ "$docker_compose_version_part1" -lt 1 ] || ([ "$docker_compose_version_part1" -eq 1 ] && [ "$docker_compose_version_part2" -lt 6 ])
then
error "Need to upgrade docker-compose package to 1.7.1+."
exit 1
else
note "docker-compose version: $docker_compose_version"
fi
else
error "Failed to parse docker-compose version."
exit 1
fi
}
check_golang
check_docker
check_dockercompose
可以看到检查环境主要是检查go环境、docker环境和docker-compose。
在笔者实验环境执行该脚本的结果如下：
compile_golangimage：1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25compile_clarity:
@echo "compiling binary for clarity ui..."
@if [ "$(HTTPPROXY)" != "" ] ; then \
$(DOCKERCMD) run --rm -v $(BUILDPATH)/src:$(CLARITYSEEDPATH) $(CLARITYIMAGE) $(SHELL) $(CLARITYBUILDSCRIPT) -p $(HTTPPROXY); \
else \
$(DOCKERCMD) run --rm -v $(BUILDPATH)/src:$(CLARITYSEEDPATH) $(CLARITYIMAGE) $(SHELL) $(CLARITYBUILDSCRIPT); \
fi
@echo "Done."
compile_golangimage: compile_clarity
@echo "compiling binary for adminserver (golang image)..."
@echo $(GOBASEPATH)
@echo $(GOBUILDPATH)
@$(DOCKERCMD) run --rm -v $(BUILDPATH):$(GOBUILDPATH) -w $(GOBUILDPATH_ADMINSERVER) $(GOBUILDIMAGE) $(GOIMAGEBUILD) -o $(GOBUILDMAKEPATH_ADMINSERVER)/$(ADMINSERVERBINARYNAME)
@echo "Done."
@echo "compiling binary for ui (golang image)..."
@echo $(GOBASEPATH)
@echo $(GOBUILDPATH)
@$(DOCKERCMD) run --rm -v $(BUILDPATH):$(GOBUILDPATH) -w $(GOBUILDPATH_UI) $(GOBUILDIMAGE) $(GOIMAGEBUILD) -o $(GOBUILDMAKEPATH_UI)/$(UIBINARYNAME)
@echo "Done."
@echo "compiling binary for jobservice (golang image)..."
@$(DOCKERCMD) run --rm -v $(BUILDPATH):$(GOBUILDPATH) -w $(GOBUILDPATH_JOBSERVICE) $(GOBUILDIMAGE) $(GOIMAGEBUILD) -o $(GOBUILDMAKEPATH_JOBSERVICE)/$(JOBSERVICEBINARYNAME)
@echo "Done."
version:1
2version:
@printf $(UIVERSIONTAG) > $(VERSIONFILEPATH)/$(VERSIONFILENAME);
1 | UIVERSIONTAG=dev |
version就是将版本（这里就是将“dev”，也就是默认是dev，可以自行修改定义）重定向输入到了当前路径的UIVERSION文件中
build
build对应源码：1
2
3
4
5build:
make -f $(MAKEFILEPATH_PHOTON)/Makefile build -e DEVFLAG=$(DEVFLAG) -e MARIADBVERSION=$(MARIADBVERSION) \
-e REGISTRYVERSION=$(REGISTRYVERSION) -e NGINXVERSION=$(NGINXVERSION) -e NOTARYVERSION=$(NOTARYVERSION) \
-e CLAIRVERSION=$(CLAIRVERSION) -e CLAIRDBVERSION=$(CLAIRDBVERSION) -e VERSIONTAG=$(VERSIONTAG) \
-e BUILDBIN=$(BUILDBIN) -e REDISVERSION=$(REDISVERSION)
可以看到build就是调用$(MAKEPATH)/photon目录中的Makefile
参数分别为：1
2
3
4
5
6
7
8
9
10MAKEFILEPATH_PHOTON=$(MAKEPATH)/photon
DEVFLAG=true
MARIADBVERSION=$(VERSIONTAG) #这个前面就已经看到了也是dev
REGISTRYVERSION=v2.6.2
NGINXVERSION=$(VERSIONTAG) #---dev
NOTARYVERSION=v0.5.1
CLAIRVERSION=v2.0.1
CLAIRDBVERSION=$(VERSIONTAG) #---dev
BUILDBIN=false
REDISVERSION=$(VERSIONTAG) #---dev
查看$(MAKEPATH)/photon中Makefile:1
build: _build_postgresql _build_db _build_adminiserver _build_ui _build_jobservice _build_log _build_nginx _build_registry _build_notary _build_clair _build_redis
可以看到build阶段，各种依赖的组件都在这build，如果关注那个组件的build就看对应的标记，这里笔者关注 _build_registr，便以 _build_registr为例。
_build_registry：
1
2
3
4
5
6
7
8
9
10
11
12_build_registry:
@if [ "$(BUILDBIN)" != "true" ] ; then \
rm -rf $(DOCKERFILEPATH_REG)/binary && mkdir -p $(DOCKERFILEPATH_REG)/binary && \
$(call _get_binary, https://storage.googleapis.com/harbor-builds/bin/registry, $(DOCKERFILEPATH_REG)/binary/registry); \
else \
cd $(DOCKERFILEPATH_REG) && $(DOCKERFILEPATH_REG)/builder $(REGISTRYVERSION); \
fi
@echo "building registry container for photon..."
@cd $(DOCKERFILEPATH_REG) && chmod 655 $(DOCKERFILEPATH_REG)/binary/registry && $(DOCKERBUILD) -f $(DOCKERFILEPATH_REG)/$(DOCKERFILENAME_REG) -t $(DOCKERIMAGENAME_REG):$(REGISTRYVERSION)-$(VERSIONTAG) .
@rm -rf $(DOCKERFILEPATH_REG)/binary
@echo "Done."
其中BUILDBIN字段在最外层的Makefile中定义
1
BUILDBIN=false
也就是是否需要编译二进制，如果不编译，那么就会去https://storage.googleapis.com/harbor-builds/bin/registry下载，如果需要编译二进制就git clone源码进行编译。
其他build流程诸如_build_postgresql _build_db _build_adminiserver _build_ui，如果有需要可以进一步细看，本文就不一一列出。
modify_sourcefiles && modify_composefile
1 | modify_sourcefiles: |
可以看到modify_sourcefiles主要就是修改notary-signer.key、notary-signer.crt等文件的权限。
1 | modify_composefile: modify_composefile_notary modify_composefile_clair |
可以看到modify_composefile主要是将各文件中的版本__xxx_version替换为设置的版本。
保存harbor docker镜像 && 打包
保存harbor docker镜像 && 打包其实很简单，直接看代码即可：1
2
3
4
5
6
7@echo "saving harbor docker image"
@$(DOCKERSAVE) $(DOCKERSAVE_PARA) > $(HARBORPKG)/$(DOCKERIMGFILE).$(VERSIONTAG).tar
@gzip $(HARBORPKG)/$(DOCKERIMGFILE).$(VERSIONTAG).tar
@$(TARCMD) $(PACKAGE_OFFLINE_PARA)
@rm -rf $(HARBORPKG)
@echo "Done.1
2
3
4
5
6
7
8
9
10
11
12
13
14
15DOCKERSAVE_PARA=$(DOCKERIMAGENAME_ADMINSERVER):$(VERSIONTAG) \
$(DOCKERIMAGENAME_UI):$(VERSIONTAG) \
$(DOCKERIMAGENAME_LOG):$(VERSIONTAG) \
$(DOCKERIMAGENAME_DB):$(VERSIONTAG) \
$(DOCKERIMAGENAME_JOBSERVICE):$(VERSIONTAG) \
vmware/redis-photon:$(REDISVERSION) \
vmware/nginx-photon:$(NGINXVERSION) vmware/registry-photon:$(REGISTRYVERSION)-$(VERSIONTAG) \
vmware/photon:$(PHOTONVERSION)
PACKAGE_OFFLINE_PARA=-zcvf harbor-offline-installer-$(PKGVERSIONTAG).tgz \
$(HARBORPKG)/common/templates $(HARBORPKG)/$(DOCKERIMGFILE).$(VERSIONTAG).tar.gz \
$(HARBORPKG)/prepare $(HARBORPKG)/NOTICE \
$(HARBORPKG)/LICENSE $(HARBORPKG)/install.sh \
$(HARBORPKG)/harbor.cfg $(HARBORPKG)/$(DOCKERCOMPOSEFILENAME) \
$(HARBORPKG)/ha
打包就是使用tar -zcvf命令进行打包,当然需要把“$(HARBORPKG)/prepare $(HARBORPKG)/NOTICE \
$(HARBORPKG)/LICENSE $(HARBORPKG)/install.sh \
$(HARBORPKG)/harbor.cfg $(HARBORPKG)/$(DOCKERCOMPOSEFILENAME) \
$(HARBORPKG)/ha
”等包含进去。
修改docker registry后如何打离线包
看这段代码（这段代码分析可参考上面build小节）1
2
3
4
5
6_build_registry:
@if [ "$(BUILDBIN)" != "true" ] ; then \
rm -rf $(DOCKERFILEPATH_REG)/binary && mkdir -p $(DOCKERFILEPATH_REG)/binary && \
$(call _get_binary, https://storage.googleapis.com/harbor-builds/bin/registry, $(DOCKERFILEPATH_REG)/binary/registry); \
else \
cd $(DOCKERFILEPATH_REG) && $(DOCKERFILEPATH_REG)/builder $(REGISTRYVERSION); \
很容易想到两种方案：
方案一
将修改后docker registry编译出对应的二进制文件registry，编译方法如下(可参考文章Docker Registry-源码安装)：
到项目路径下执行：1
make PREFIX=/go clean binaries
然后把二进制放到固定位置registry_bin_path,然后修改脚本；
可以把这1
2rm -rf $(DOCKERFILEPATH_REG)/binary && mkdir -p $(DOCKERFILEPATH_REG)/binary && \
$(call _get_binary, https://storage.googleapis.com/harbor-builds/bin/registry, $(DOCKERFILEPATH_REG)/binary/registry);
修改为1
2rm -rf $(DOCKERFILEPATH_REG)/binary && mkdir -p $(DOCKERFILEPATH_REG)/binary && \
cp $(registry_bin_path) $(DOCKERFILEPATH_REG)/binary/registry
只要达到使用本地编译出来的二进制文件即可
方案二
自己到github建立一个自己的存放修改后的docker registry代码的项目，然后把脚本中
harbor-1.5.1/make/photon/registry/builder中项目地址改为自己的docker registry代码的项目地址即可。
即把1
2
3
4# the temp folder to store distribution source code...
TEMP=`mktemp -d /$TMPDIR/distribution.XXXXXX`
git clone -b $VERSION https://github.com/docker/distribution.git $TEMP
改为1
2
3
4# the temp folder to store distribution source code...
TEMP=`mktemp -d /$TMPDIR/distribution.XXXXXX`
git clone -b $VERSION $(你自己的docker registry github项目地址) $TEMP
修改完之后重新出包部署即可 : )
构建带安全扫描工具的harbor离线包
1 | make package_offline GOBUILDIMAGE=golang:1.9.2 COMPILETAG=compile_golangimage CLARITYIMAGE=vmware/harbor-clarity-ui-builder:1.3.0 CLAIRFLAG=true |
此时安装时需要带参数了：1
./install.sh --with-clair
