Git Product home page Git Product logo

installer's Introduction

项目由来

设想一下: 你公司某个节点要建设,进货了200台服务器(不同规格),都是安装同样的系统,区别在于hostname和业务相关的IP以及IPMI(带外的)网络信息。接好所有的线后机柜和网络规划为

                                            +----------------+
                                            |  Corenet-Spine |
                                            |                |
                                            +-------+--------+
                                                    ^
                                                    |
         +---------------------+-------------------------------------------+---------------------+
         |                     ^                    |                      ^                     |
         |                     |                    |                      |                     |
         |                     |                    |                      |                     |
+--------+-------+    +--------+-------+    +-------+--------+    +--------+-------+    +--------+-------+
|    M/C/S-ACC   |    |    M/C/S-ACC   |    |    M/C/S-ACC   |    |    M/C/S-ACC   |    |    M/C/S-ACC   |
|                |    |                |    |                |    |                |    |                |
+----------------+    +----------------+    +----------------+    +----------------+    +----------------+
|     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |
+----------------+    +----------------+    +----------------+    +----------------+    +----------------+
|     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |
+----------------+    +----------------+    +----------------+    +----------------+    +----------------+
|     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |
+----------------+    +----------------+    +----------------+    +----------------+    +----------------+
|     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |
+----------------+    +----------------+    +----------------+    +----------------+    +----------------+
|     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |
+----------------+    +----------------+    +----------------+    +----------------+    +----------------+
|     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |
+----------------+    +----------------+    +----------------+    +----------------+    +----------------+
|     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |
+-----------------    ------------------    ------------------    ------------------    -----------------+

每个机柜都有各个网络接入的交换机,考虑到最坏最省钱的情况,由于每台服务器都没有做raid(出场做raid得加服务费)。你得到现场去用笔记本一台台直连服务器的带外口网线设置好带外(IPMI)的ip后(出场配置好指定ip或者配置成DHCP得加服务费),才能在办公室通过带外远程继续操作。重启服务器进bios或者阵列卡里去配置阵列(raid)(服务器厂商没有cli tool来远程配置raid),最后在pxe安装完后所有系统后人为一台台上去改hostname和相关的网络设置。 但是如果所有服务器只需要开机通电,就能达到最终需求呢

目前做到的

上架布好线后,统计服务器的序列号和标签名,整理成一份excel表格(现场的人员统计前两列,后面可以让办公室的人填写),例如

设备名 序列号 hostname IPMI IP MASK GW MG IP MASK GW
HN1-xxx-R4900G3-001 210200A00QH19700xxx1 HN1-XX-ComCVK-001 10.101.0.101 255.255.255.0 10.101.0.254 10.102.0.1 255.255.255.0 10.102.0.254
HN1-xxx-R4900G3-002 210200A00QH19700xxx2 HN1-XX-ComCVK-002 10.101.0.102 255.255.255.0 10.101.0.254 10.102.0.2 255.255.255.0 10.102.0.254
HN1-xxx-R4900G3-003 210200A00QH19700xxx3 HN1-XX-ComCVK-003 10.101.0.103 255.255.255.0 10.101.0.254 10.102.0.3 255.255.255.0 10.102.0.254
HN1-xxx-R4900G3-004 210200A00QH19700xxx4 HN1-XX-ComCVK-004 10.101.0.104 255.255.255.0 10.101.0.254 10.102.0.4 255.255.255.0 10.102.0.254

然后(通过curl命令或者后期开发个前端页面)上传excel到后端,autoInstaller相关进程启动后,交换机配置好pxe需要的DHCP的boot-filename,服务器开机后最终就会按照表格里的设置好,全程不需要人为干预

技术前提需知

前提

总结

  • 修改grub.cfg适应pxe场景以及ks依赖的俩boot option
  • 修改iso的squashfs.img增加阵列卡的cli tool命令
  • 编写后端,支持上传excel和根据来源序列号返回渲染是ks文件

实际部署步骤

因为pxe和请求ks的时候都用到了dhcp,所以定义了一个段10.1.0.0/16供安装阶段使用。最终的实际ip和安装段不重合,也就是下面的组网,核心上设置路由都能通,我自己笔记本接核心上设置静态ip:


                              10.1.0.2      +----------------+
                                +---+       |  Corenet-Spine |
                                |pc +------>+                |
                                +---+       +-------+--------+
                                                    ^
                                                    |
         +---------------------+-------------------------------------------+---------------------+
         |                     ^                    |                      ^                     |
         |                     |                    |                      |                     |
         |                     |                    |                      |                     |
+--------+-------+    +--------+-------+    +-------+--------+    +--------+-------+    +--------+-------+
|    M/C/S-ACC   |    |    M/C/S-ACC   |    |    M/C/S-ACC   |    |    M/C/S-ACC   |    |    M/C/S-ACC   |
|                |    |                |    |                |    |                |    |                |
+----------------+    +----------------+    +----------------+    +----------------+    +----------------+
|     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |
+----------------+    +----------------+    +----------------+    +----------------+    +----------------+
|     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |
+----------------+    +----------------+    +----------------+    +----------------+    +----------------+
|     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |
+----------------+    +----------------+    +----------------+    +----------------+    +----------------+
|     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |
+----------------+    +----------------+    +----------------+    +----------------+    +----------------+
|     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |
+----------------+    +----------------+    +----------------+    +----------------+    +----------------+
|     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |
+----------------+    +----------------+    +----------------+    +----------------+    +----------------+
|     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |    |     Machine    |
+-----------------    ------------------    ------------------    ------------------    -----------------+
    10.1.1.0/24           10.1.1.0/24           10.1.1.0/24           10.1.1.0/24           10.1.1.0/24

一个机柜是一个最小单位,所以一个机柜一个段

实操部署步骤:

$ env | grep GO
GO111MODULE=on
GOPROXY=https://goproxy.io
GOPATH=xxxxx
  • 下载编译
git clone https://github.com/zhangguanzhang/Installer.git $GOPATH/src/Installer #下载文件
go build -o docker/main main.go # 编译可执行文件到docker目录下的main
  • cd docker确认相关目录mkdir http/centos tftp mysql创建好后docker-compose up -d启动tftp,nginx,mysql
  • cp -a复制centos iso的EFI/BOOT/下文件到tftp的根目录,更改grub.cfg(tftp里的grub.cfg文件可参考),把iso解压到http/centos下,复制iso里/images/pxeboot/vmlinuzinitrd.img到tftp目录里
  • 我们这6张网卡是ovs,所以ks里没用到用于ssh的管理网ip,但是代码预留了它,自行根据自己的场景写好templates/ks.tmpl(语法是go template),ks的密码可以用scripts里运行python2.7 grub-crypt.py --sha-512生成,./main启动服务,并dos2unix转换下ks.tmpl防止windows的换行出现
  • scripts目录里有脚本,参照machine.xlsx的列要求写好这个excel(列的作用是在代码里的常量固定的),然后用脚本upload去curl模拟http上传excel到我后端(url的path为localhost:8080/api/v1/ks POST请求),后端会把excel信息导入到mysql里,也有ks的header的部分,自己带url模拟物理机请求ks(url的path为localhost:8080/api/v1/ks, GET请求)
  • 现场每个机柜交换机都配置好后dhcp和pxe部分
[H3C-S5130-dhcp-pool-1] bootfile-name BOOTX64.EFI
[H3C-S5130-dhcp-pool-1] next-server 10.1.0.2
  • Installer机器配置好ip为10.1.0.2, 并配置10.1.0.0/16的路由route add -net 10.1.0.0/16 gw 10.1.0.254 dev ens37,我pc是虚拟机桥接,根据自己情况去配置
  • 所有物理机开机即可

有条件先一台物理机测试下,可行了后续基本就都固化了

可扩展的设想

  • 目前的dhcp是在交换机上,可以开发dhcp server,因为ipmi出场是固定ip的,有些服务器厂商的mac地址贴在机器表面了,可以根据mac地址返回不同的boot-filename
  • 同样的,可以用tftp server根据请求grub.cfg时候第一次请求的是自身mac地址,可以提前导入mac地址去渲染返回不同的grub.cfg从而安装不同的操作系统

installer's People

Contributors

zhangguanzhang avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.