0x1 备注

当centos系统发生迁移时,往往会导致 系统无法启动,这时候需要手动修复

0x2 故障复现

当操作系统/服务器 出现以下情况时会导致服务器服务器无法进去系统:

  1. 更换底层处理器
  2. 大量硬件发生改变
  3. 虚拟机平台发生改变
  4. 内核升级异常

具体表现为加载时间超长,并且最后加载失败,加载界面如下图所示:

image-20200714161144158.png

下面的进度条虽然一直会走但是始终走不到头,最终会停在这个位置

image-20200714161336325.png

当进度条走完后并没有进入操作系统,直接打开了一个临时的错误窗口

image-20200714161513960.png

0x3 修复内核

按照如下步骤进行修复就可以解决这个问题了:

  1. 首先使用 linux的dvd进入修复模式(这里直接使用centos7的 dvd,与例子中的系统匹配)
  2. 接着挂载原始系统的磁盘分区到恢复模式的临时系统中
  3. 刷新内核

0x3.1 进入修复模式

  • 首先在系统关机时 挂载 centos7的安装盘
  • 这里以xen为例

挂载安装盘后直接启动

image-20200714165628695.png

选择 修复模式

image-20200714165719587.png

选择修复操作系统

image-20200714165808927.png

0x3.2 挂载原始系统

  • 这一步主要是为了将原始的操作系统完整的挂载出来,为之后的修复做准备
  • 承接上一步

恢复系统 查找到原始操作系统时会询问我们 修复方式,这里选择方式1

image-20200714170407123.png

按回车键确认就可以获得修复系统的shell 了

image-20200714170559103.png

注意:这里获取到的shell是当前 恢复模式中临时系统的shell,我们的目标是获取 原始系统的shell

仔细观察上一个提示信息,可以发现:临时系统已经寻找到了 原始系统文件,并且已经自动挂载到目录中了

image-20200714171533215.png

这时候直接使用如下命令就可以获取到原始系统的shell了

# 这是centos7额外需要运行的指令
systemctl start multipathd.service
# 获取shell
chroot /mnt/sysimage

如果临时系统没有找到原始系统,那么就需要我们手动挂载了,这里给一些参考指令

mount --bind /proc /mnt/sysimage/proc
mount --bind /dev /mnt/sysimage/dev
mount --bind /sys /mnt/sysimage/sys
# 这两条是centos7额外需要运行的指令
mount --bind /run /mnt/sysimage/run
systemctl start multipathd.service

chroot /mnt/sysimage

挂载成功后会话会发生改变

image-20200714172123272.png

0x3.3 修复内核

  • 通过以上两个步骤,已经将原始系统 完整的挂载出来
  • 此时我们已经获取到了 原始系统的shell
  • 接下来的操作就是针对 内核进行修复了

通过以下命令可以查询出当前操作系统已经安装的内核

ls /boot/|grep img

image-20200714172540144.png

中间这个内核就是我们需要修复的目标内核,先对他进行备份

cp -p /boot/initramfs-3.10.0-957.el7.x86_64.img /boot/initramfs-3.10.0-957.el7.x86_64.img.bac

自动修复内核(不建议用这个,经常会失败。建议使用下面那个直接指定内核的方式来修复)

dracut -f

如果安装了多个内核,或者自动刷新失败,那么就需要 指定目标内核

# 目标内核为 3.10.0-957.el7.x86_64
dracut -f /boot/initramfs-3.10.0-957.el7.x86_64.img 3.10.0-957.el7.x86_64

执行成功后重启就可以进入原始的系统了

sync
exit
exit

0x1 Remarks

When a centos migration occurs, the system often fails to boot and needs to be repaired manually.

0x2 Fault Replication

The server server will not go in when the operating system/server

  1. the underlying processor is replaced
  2. a large number of hardware changes
  3. a change in the virtual machine platform
  4. kernel upgrade exceptions

The loading time is very long and the loading interface fails as shown below:

image-20200714161144158.png

The following progress bar keeps going but never goes to the end, and eventually stops at this position

image-20200714161336325.png

When the progress bar finishes it doesn't go to the OS, it opens a temporary error window

image-20200714161513960.png

0x3 Fixing the kernel

The problem can be solved by following these steps.

  1. first use linux dvd to enter repair mode (here use centos7 dvd directly, matching the system in the example)
  2. then mount the original system disk partition to the recovery mode temporary system
  3. refresh the kernel

0x3.1 Enter recovery mode

  • first mount the centos7 installation disk while the system is shut down
  • Here is an example of xen

After mounting the installation disk, boot directly

image-20200714165628695.png

Select repair mode

image-20200714165719587.png

Select Repair OS

image-20200714165808927.png

0x3.2 Mount the original system

  • This step is to mount the original OS in order to prepare it for the repair
  • Following on from the previous step

When Recovery System finds the original OS it will ask us how to repair it, here choose Mode 1.

Here choose Mode 1image-20200714170407123.png

Press enter to confirm and you will get the shell to repair the system

image-20200714170559103.png

Note: The shell we get here is the temporary system shell in the current recovery mode, our goal is to get the original system shell.

Look carefully at the previous prompt and you can see that the temporary system has already found the original system file and has automounted it in the directory

image-20200714171533215.png

At this point, the raw system shell can be retrieved directly with the following command

# This is an additional command that needs to be run for centos7
systemctl start multipathd.service
# Get the shell
chroot /mnt/sysimage

If the temporary system does not find the original system, then we need to mount it manually, here are some reference commands

mount --bind /proc /mnt/sysimage/proc
mount --bind /dev /mnt/sysimage/dev
mount --bind /sys /mnt/sysimage/sys
# These are the two additional commands that centos7 needs to run
mount --bind /run /mnt/sysimage/run
systemctl start multipathd.service

chroot /mnt/sysimage

The session will change after a successful mount

image-20200714172123272.png

0x3.3 Repairing the kernel

  • With the above two steps, the original system has been completely mounted
  • At this point we have the shell of the original system.
  • The next step is to repair the kernel

You can find out which kernel is installed in the current operating system by using the following command

ls /boot/|grep img

image-20200714172540144.png

The kernel in the middle is the target kernel we need to fix, so first make a backup of it

cp -p /boot/initramfs-3.10.0-957.el7.x86_64.img /boot/initramfs-3.10.0-957.el7.x86_64.img.bac

Automatically repair the kernel (this is not recommended, it often fails. It is recommended to use the following one to repair the kernel directly by specifying it)

dracut -f

If multiple kernels are installed, or if the automatic refresh fails, then you need to specify the target kernel

# Target kernel is 3.10.0-957.el7.x86_64
dracut -f /boot/initramfs-3.10.0-957.el7.x86_64.img 3.10.0-957.el7.x86_64

After successful execution, reboot and you will be able to enter the original system

sync
exit
exit

本文章禁止任何形式的转载

Last modification:February 5, 2021
如果觉得我的文章对你有用,请随意赞赏