通过harbor搭建私有docker registry

  • 流程:
    • 安装docker
    • 安装docker-compose
    • 安装harbor
    • 修改docker启动选项,使默认docker login为http
    • 简单使用示例
  • 系统环境:
    • CentOS 7.4.1708
    • docker-ce 18.06.0-ce (client/server)
    • docker-compose 1.16.1
      • 安装路径:/usr/local/bin/
  • harbor v1.6.0
    • 安装路径:/usr/local/harbor/
  • 一. 安装docker

    • 配置yum源
      • 在/etc/yum.repos.d/目录下创建docker.repo文档,并添加以下内容
    [docker]
    name=docker
    enabled=1
    baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/7/x86_64/stable/
    gpgcheck=0
    enabled=1
    • 执行以下命令安装docker-ce,等待安装完成即可
    [root@node ~]# yum install -y docker-ce

    二. 安装docker-compose

    • 下载二进制文档至指定路径下、给予执行权限(不×××死慢)
    curl -L https://github.com/docker/compose/releases/download/1.16.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
    chmod + x /usr/local/bin/docker-compose
    • 也可到https://github.com/docker/compose/releases/下载最新版
    • 执行docker-compose需在包含docker-compose.yml(harbor自带该文档)的目录
    • 验证docker-compose是否安装好
    [root@node ~]# docker-compose version
    docker-compose version 1.16.1, build 6d1ac21
    docker-py version: 2.5.1
    CPython version: 2.7.13
    OpenSSL version: OpenSSL 1.0.1t  3 May 2016
    • 卸载docker-compose
    rm -rf /usr/local/bin/docker-compose

    三. 安装 Harbor

    • 系统需求:
      • docker:1.10.0+
      • docker-compose:1.6.0+
      • Python:2.7或更高
      • Openssl:若使用https方式,需安装最新版

    1. 下载harbor安装包

    - Online installer:
        - 下载链接:https://storage.googleapis.com/harbor-releases/harbor-offline-installer-v1.5.2.tgz
        - md5: 49f5ce1cab8125e59d45af305b8f46fe
    - Offline installer:
        - 下载链接:https://storage.googleapis.com/harbor-releases/harbor-online-installer-v1.5.2.tgz
        - md5: abd7a80c052cc10b3346062f65f96b96

    2. 解压安装包至/usr/local/harbor/目录下

    3. 编辑harbor.cfg文档:

    • 必须项:
    hostname = reg.lxk.com     #IP地址或FQDN
    ui_url_protocol = http      
    #通讯协议。默认docker pull/push通讯协议为https。若为https,需配置证书。
    max_job_workers = 50        #最大工作进程(默认值)
    db_password = root123       #harbor数据密码(默认值)
    customize_crt = on          #默认on:准备脚本为registry的令牌的生成/验证创建私钥和根证书。
                                #off :密钥和根证书由外部存储提供
    ssl_cert = /data/cert/server.crt            #SSL证书的路径,仅在ui_url_protocol为https时有效
    ssl_cert_key = /data/cert/server.key        #SSL密钥的路径,仅在ui_url_protocol为https时有效
    secretkey_path = /data      #The path of secretkey storage
    log_rotate_count = 50       #日志轮转次数(保留多少次轮转日志,使用默认值)
    log_rotate_size = 200M      #日志达到多大时执行轮转操作(使用默认值)
    • 可选项:
    self_registration = off                     #禁止用户注册
    project_creation_restriction = adminonly    #设置只有管理员可以创建项目
    harbor_admin_password = centos              #网页登录管理帐号的密码,默认账号密码为:admin/Harbor12345
    • 邮箱设置:
      • 只有此处设置了邮箱设置,才允许用户发送“密码重置”电子邮件
    email_server = smtp.mydomain.com
    email_server_port = 25
    email_identity =
    email_username = sample_admin@mydomain.com
    email_password = abc
    email_from = admin sample_admin@mydomain.com
    email_ssl = false
    email_insecure = false

    4. 执行安装脚本

    • 这是已经安装过又执行一次的结果,可以下载离线安装包,也可以用镜像加速。
    [root@node ~]# cd /usr/local/harbor/
    [root@node harbor]# ./install.sh 
    
    [Step 0]: checking installation environment ...
    
    Note: docker version: 18.06.0
    
    Note: docker-compose version: 1.16.1
    
    [Step 1]: preparing environment ...
    Clearing the configuration file: ./common/config/adminserver/env
    Clearing the configuration file: ./common/config/ui/env
    Clearing the configuration file: ./common/config/ui/app.conf
    Clearing the configuration file: ./common/config/ui/private_key.pem
    Clearing the configuration file: ./common/config/db/env
    Clearing the configuration file: ./common/config/jobservice/env
    Clearing the configuration file: ./common/config/jobservice/config.yml
    Clearing the configuration file: ./common/config/registry/config.yml
    Clearing the configuration file: ./common/config/registry/root.crt
    Clearing the configuration file: ./common/config/nginx/nginx.conf
    Clearing the configuration file: ./common/config/log/logrotate.conf
    loaded secret from file: /data/secretkey
    Generated configuration file: ./common/config/nginx/nginx.conf
    Generated configuration file: ./common/config/adminserver/env
    Generated configuration file: ./common/config/ui/env
    Generated configuration file: ./common/config/registry/config.yml
    Generated configuration file: ./common/config/db/env
    Generated configuration file: ./common/config/jobservice/env
    Generated configuration file: ./common/config/jobservice/config.yml
    Generated configuration file: ./common/config/log/logrotate.conf
    Generated configuration file: ./common/config/jobservice/config.yml
    Generated configuration file: ./common/config/ui/app.conf
    Generated certificate, key file: ./common/config/ui/private_key.pem, cert file: ./common/config/registry/root.crt
    The configuration files are ready, please use docker-compose to start the service.
    
    [Step 2]: checking existing instance of Harbor ...
    
    Note: stopping existing Harbor instance ...
    Removing nginx              ... done
    Removing harbor-jobservice  ... done
    Removing harbor-ui          ... done
    Removing redis              ... done
    Removing harbor-adminserver ... done
    Removing registry           ... done
    Removing harbor-db          ... done
    Removing harbor-log         ... done
    Removing network harbor_harbor
    
    [Step 3]: starting Harbor ...
    Creating network "harbor_harbor" with the default driver
    Creating harbor-log ... 
    Creating harbor-log ... done
    Creating harbor-db ... 
    Creating redis ... 
    Creating harbor-adminserver ... 
    Creating registry ... 
    Creating harbor-db
    Creating redis
    Creating registry
    Creating harbor-db ... done
    Creating harbor-ui ... 
    Creating harbor-ui ... done
    Creating harbor-jobservice ... 
    Creating nginx ... 
    Creating nginx
    Creating nginx ... done
    
    ✔ ----Harbor has been installed and started successfully.----
    
    Now you should be able to visit the admin portal at http://reg.lxk.com. 
    For more details, please visit https://github.com/vmware/harbor .

    5. 查看安装好的harbor

    [root@node harbor]# docker ps
    CONTAINER ID        IMAGE                                  COMMAND                  CREATED             STATUS                   PORTS                                                              NAMES
    45c849240289        vmware/harbor-jobservice:v1.5.2        "/harbor/start.sh"       2 minutes ago       Up 2 minutes                                                                                harbor-jobservice
    24df8c8d740e        vmware/nginx-photon:v1.5.2             "nginx -g 'daemon of…"   2 minutes ago       Up 2 minutes (healthy)   0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp   nginx
    49a2e63d33eb        vmware/harbor-ui:v1.5.2                "/harbor/start.sh"       2 minutes ago       Up 2 minutes (healthy)                                                                      harbor-ui
    44edfc92d5c2        vmware/harbor-adminserver:v1.5.2       "/harbor/start.sh"       2 minutes ago       Up 2 minutes (healthy)                                                                      harbor-adminserver
    a2d2f2a08e77        vmware/registry-photon:v2.6.2-v1.5.2   "/entrypoint.sh serv…"   2 minutes ago       Up 2 minutes (healthy)   5000/tcp                                                           registry
    229dddfc0e34        vmware/redis-photon:v1.5.2             "docker-entrypoint.s…"   2 minutes ago       Up 2 minutes             6379/tcp                                                           redis
    97ac1f88d6a7        vmware/harbor-db:v1.5.2                "/usr/local/bin/dock…"   2 minutes ago       Up 2 minutes (healthy)   3306/tcp                                                           harbor-db
    d96f1ce61867        vmware/harbor-log:v1.5.2               "/bin/sh -c 'crond &…"   2 minutes ago       Up 2 minutes             514/tcp, 127.0.0.1:1514->10514/tcp                                 harbor-log

    四. 用docker-compose管理harbor

    1. docker-compose命令注释
    docker-compose
        Define and run multi-container applications with Docker.    
        #定义并运行多个docker容器
    Usage:
      docker-compose [-f <arg>...] [options] [COMMAND] [ARGS...]
      docker-compose -h|--help
    Options:
      -f, --file FILE             Specify an alternate compose file (default: docker-compose.yml)
                                  #指定配置文档,默认当前目录下docker-compose.yml
      --verbose                   Show more output
      -v, --version               Print version and exit
    Commands:
      down               Stop and remove containers, networks, images, and volumes
                         #停止并删除容器、网络、docker镜像和卷组
      kill               Kill containers        #关闭容器
      logs               View output from containers        #显示容器的日志
      pause              Pause services                     #暂停服务
      ps                 List containers                    #显示容器列表
      pull               Pull service images                #下载一个服务镜像   
      push               Push service images                #推送一个服务镜像至服务器
      restart            Restart services                   #重启镜像
      rm                 Remove stopped containers          #删除停止的窗口
      run                Run a one-off command              #运行一个停止的命令
      start              Start services                     #启动服务
      stop               Stop services                      #停止服务
      top                Display the running processes      #显示运行中的进程
      unpause            Unpause services                   #恢复暂停中的服务
      up                 Create and start containers        #创建并运行一个容器
      version            Show the Docker-Compose version information        #显示docker-compose的版本
    1. docker-compose命令运行时需要配置文档docker-compose.yml,该文档在harbor目录下,故运行docker-compose命令需要在/usr/local/harbor目录。也可用-f选项指定compose文档。

    例:使用-f参数指定docker-compose.yml文档

    [root@node ~]# docker-compose -f /usr/local/harbor/docker-compose.yml ps
           Name                     Command               State                                Ports                              
    ------------------------------------------------------------------------------------------------------------------------------
    harbor-adminserver   /harbor/start.sh                 Up                                                                      
    harbor-db            /usr/local/bin/docker-entr ...   Up      3306/tcp                                                        
    harbor-jobservice    /harbor/start.sh                 Up                                                                      
    harbor-log           /bin/sh -c crond && rsyslo ...   Up      127.0.0.1:1514->10514/tcp, 514/tcp                              
    harbor-ui            /harbor/start.sh                 Up                                                                      
    nginx                nginx -g daemon off;             Up      0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp, 0.0.0.0:80->80/tcp
    redis                docker-entrypoint.sh redis ...   Up      6379/tcp                                                        
    registry             /entrypoint.sh serve /etc/ ...   Up      5000/tcp

    例:不指定docker-compose.yml文档时会报错

    [root@node ~]# docker-compose ps
    ERROR: 
            Can't find a suitable configuration file in this directory or any
            parent. Are you in the right directory?
    
            Supported filenames: docker-compose.yml, docker-compose.yaml
    1. 使用docker-compose管理容器:
    [root@node harbor]# docker-compose ps       #查看运行中的容器
           Name                     Command               State                                Ports                              
    ------------------------------------------------------------------------------------------------------------------------------
    harbor-adminserver   /harbor/start.sh                 Up                                                                      
    harbor-db            /usr/local/bin/docker-entr ...   Up      3306/tcp                                                        
    harbor-jobservice    /harbor/start.sh                 Up                                                                      
    harbor-log           /bin/sh -c crond && rsyslo ...   Up      127.0.0.1:1514->10514/tcp, 514/tcp                              
    harbor-ui            /harbor/start.sh                 Up                                                                      
    nginx                nginx -g daemon off;             Up      0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp, 0.0.0.0:80->80/tcp
    redis                docker-entrypoint.sh redis ...   Up      6379/tcp                                                        
    registry             /entrypoint.sh serve /etc/ ...   Up      5000/tcp     
    
    [root@node harbor]# docker-compose stop         #关闭harbor各容器
    Stopping harbor-jobservice  ... done
    Stopping nginx              ... done
    Stopping harbor-ui          ... done
    Stopping harbor-adminserver ... done
    Stopping registry           ... done
    Stopping redis              ... done
    Stopping harbor-db          ... done
    Stopping harbor-log         ... done
    [root@node harbor]# docker-compose ps           #查看当前各harbor container状态
           Name                     Command                State     Ports
    ----------------------------------------------------------------------
    harbor-adminserver   /harbor/start.sh                 Exit 137        
    harbor-db            /usr/local/bin/docker-entr ...   Exit 0          
    harbor-jobservice    /harbor/start.sh                 Exit 137        
    harbor-log           /bin/sh -c crond && rsyslo ...   Exit 137        
    harbor-ui            /harbor/start.sh                 Exit 137        
    nginx                nginx -g daemon off;             Exit 0          
    redis                docker-entrypoint.sh redis ...   Exit 0          
    registry             /entrypoint.sh serve /etc/ ...   Exit 137 
    
    [root@node harbor]# docker-compose start    #该命令运行时会报错,用restart就可以。
    Starting log         ... done
    Starting redis       ... error
    Starting adminserver ... error
    Starting registry    ... error
    Starting ui          ... error
    Starting mysql       ... error
    Starting jobservice  ... error
    Starting proxy       ... error
    
    ERROR: for mysql  Cannot start service mysql: failed to initialize logging driver: dial tcp 127.0.0.1:1514: connect: connection refused
    
    ERROR: for redis  Cannot start service redis: failed to initialize logging driver: dial tcp 127.0.0.1:1514: connect: connection refused
    
    ERROR: for registry  Cannot start service registry: failed to initialize logging driver: dial tcp 127.0.0.1:1514: connect: connection refused
    
    ERROR: for adminserver  Cannot start service adminserver: failed to initialize logging driver: dial tcp 127.0.0.1:1514: connect: connection refused
    • 关于服务启动报错的原因:

      guy-hub该项目issue上有提到这个问题的,原因是日志服务未先启动

      而其它服务需要先到日志服务器注册,所以会造成端口访问拒绝。

      解决方法没有,答主只说后续会关注。

    [root@node harbor]# docker-compose restart      #使用restart可以正常启动,但是有时也会报错,多来两次就好了。
    Restarting harbor-jobservice  ... done 
    Restarting nginx              ... done 
    Restarting harbor-ui          ... done 
    Restarting harbor-adminserver ... done
    Restarting registry           ... done
    Restarting redis              ... done
    Restarting harbor-db          ... done
    Restarting harbor-log         ... done

    五. 测试访问harbor

    • 在浏览器输入 reg.lxk.com,请大家根据自己的配置情况输入访问的域名;
    • 默认账号密码: admin / Harbor12345 登录后修改密码
      通过harbor搭建私有docker registry
      通过harbor搭建私有docker registry

    六. 测试上传和下载镜像

    1. docker registry通讯协议默认为https,需要配置证书。若未配置证书,需要做以下修改:
    [root@node ~]# vim /usr/lib/systemd/system/docker.service 
    ExecStart=/usr/bin/dockerd --insecure-registry reg.lxk.com
    #在ExecStart后面加上--insecure-registry reg.lxk.com
    #后面的reg.lxk.com为harbor.cfg中hostname配置的值
    1. 重载systemd并重启docker
    [root@node test]# systemctl daemon-reload
    [root@node test]# systemctl  restart docker
    1. 创建一个Dockerfile文档
    [root@node ~]# mkdir test
    [root@node ~]# cd test/
    [root@node test]# vim Dockerfile 
    # vim Dockerfile 
    FROM centos:centos7.1.1503
    ENV TZ "Asia/Shanghai"
    1. 创建镜像
    [root@node test]# docker build -t reg.lxk.com/library/centos7.1:v0.1 ./
    Sending build context to Docker daemon  2.048kB
    Step 1/2 : FROM centos:centos7.1.1503
     ---> fbe8925ecf55
    Step 2/2 : ENV TZ "Asia/Shanghai"
     ---> Using cache
     ---> 930eec2ed889
    Successfully built 930eec2ed889
    Successfully tagged reg.lxk.com/library/centos7.1:v0.1
    1. 登录到reg.lxk.com并push镜像
    [root@node harbor]# docker login reg.lxk.com
    Authenticating with existing credentials...
    #以现有证书认证
    WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
    #警告!你的密码会以明文保存在/root/.docker/config.json
    Configure a credential helper to remove this warning. See
    #配置证书就会不再显示该警告
    https://docs.docker.com/engine/reference/commandline/login/#credentials-store
    #详情请看这网址
    
    Login Succeeded
    #登录成功
    [root@node harbor]# docker tag 9432976b676f reg.lxk.com/library/swaggerapi/swagger-ui:latest
    #给已存在的镜像打标签
    [root@node harbor]# docker push reg.lxk.com/library/swaggerapi/swagger-ui:latest
    #把打好标签的镜像push至私有registry
    The push refers to repository [reg.lxk.com/library/swaggerapi/swagger-ui]
    47c77f5f4ee4: Pushed 
    ab4588773347: Pushed 
    5382149040dc: Pushed 
    a8d7d0b05699: Pushed 
    a9031380f2d7: Pushed 
    7105cc56962c: Pushed 
    latest: digest: sha256:0b5457c35fa0b21c08780dd84afe3f27525bee462261dff9b8e08a1e70414109 size: 1571
    1. 验证镜像文档是否已push至私有registry
      • 打好标签的镜像文档已保存至reg.lxk.com的library下
    通过harbor搭建私有docker registry
    通过harbor搭建私有docker registry
    1. 用局域网中另外一台机器下载镜像
      • 安装docker
      • 修改/usr/lib/systemd/system/docker.service
      • 点击图中图标即可复制docker pull命令,至shell下粘贴即可。
        通过harbor搭建私有docker registry
        通过harbor搭建私有docker registry
      • 下载镜像文档:
    [root@node ~]# docker pull reg.lxk.com/library/swaggerapi/swagger-ui:latest
    #命令中的内容是由上图中直接复制而来,不必进行任何修改。
    latest: Pulling from library/swaggerapi/swagger-ui
    f4900964ff56: Pull complete 
    6f8087d9ed5d: Pull complete 
    31023fcfba5a: Pull complete 
    8c462391de19: Pull complete 
    ba9c0a3c3f9a: Pull complete 
    6a4540734666: Pull complete 
    Digest: sha256:0b5457c35fa0b21c08780dd84afe3f27525bee462261dff9b8e08a1e70414109
    Status: Downloaded newer image for reg.lxk.com/library/swaggerapi/swagger-ui:latest
    [root@node ~]# docker image ls
    REPOSITORY                                   TAG                 IMAGE ID            CREATED             SIZE
    reg.lxk.com/library/centos7.1               0.1                 930eec2ed889        23 hours ago        212MB
    reg.lxk.com/library/swaggerapi/swagger-ui   latest              9432976b676f        6 days ago          15.4MB