云原生是基于分布部署和统一运管的分布式云，以容器、微服务、DevOps等技术为基础建立的一套云技术产品体系。
1. 什么是自愿中断？在 Kubernetes 中，中断分为两种：自愿中断（Voluntary Disruptions）：由用户或管理员主动触发的操作，例如：升级或维护节点（Node Drain）。手动删除 Pod。更新 Deployment 或 StatefulSet 的 Pod 模板（触发滚动更新）。非自愿中断（Involuntary Disruptions）：由系统故障或意外事件引起的，例如：节点硬件故障。网络分区。内核崩溃。PDB 主要用于管理自愿中断，确保在中断过程中应用的高可用性。2. PDB 的作用PDB 的核心作用是：限制同时中断的 Pod 数量，确保在自愿中断期间，应用仍然能够保持一定数量的可用 Pod。防止因中断操作导致的服务不可用或性能下降。例如：如果你有一个包含 10 个 Pod 的 Deployment，设置 PDB 可以确保在节点升级时，最多只有 2 个 Pod 被终止，剩下的 8 个 Pod 仍然可以提供服务。3. PDB 的关键参数PDB 有两个关键参数，用于定义中断的容忍度：minAvailable：指定在中断期间，必须保持的最小可用 Pod 数量。例如：minAvailable: 3 表示在任何时候，至少有 3 个 Pod 必须是可用的。maxUnavailable：指定在中断期间，最多允许多少个 Pod 不可用。例如：maxUnavailable: 1 表示在任何时候，最多只能有 1 个 Pod 不可用。注意：minAvailable 和 maxUnavailable 是互斥的，只能设置其中一个。4. PDB 的工作原理当 Kubernetes 进行自愿中断操作时（例如节点 Drain 或滚动更新），它会检查相关的 PDB 规则：如果中断操作会导致 Pod 数量低于 minAvailable 或超过 maxUnavailable，则中断操作会被阻止。只有当中断操作符合 PDB 规则时，才会继续执行。例如：假设你有一个 Deployment，包含 5 个 Pod，并且设置了 maxUnavailable: 1。当你触发滚动更新时，Kubernetes 会确保在任何时候，最多只有 1 个 Pod 被终止，剩下的 4 个 Pod 仍然可以提供服务。5. PDB 的适用场景PDB 特别适用于以下场景：有状态服务：例如数据库（如 MySQL、PostgreSQL）或消息队列（如 Kafka、RabbitMQ），需要确保在中断期间仍然有足够的副本提供服务。高可用服务：例如 Web 服务或 API 服务，需要确保在中断期间不会导致服务不可用。滚动更新：在更新应用时，确保服务的可用性不受影响。6. PDB 的示例以下是一个 PDB 的 YAML 示例：apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: my-app-pdb namespace: default spec: minAvailable: 2 # 确保至少有 2 个 Pod 可用 selector: matchLabels: app: my-app # 选择需要保护的 Pod 或者使用 maxUnavailable：apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: my-app-pdb namespace: default spec: maxUnavailable: 1 # 最多允许 1 个 Pod 不可用 selector: matchLabels: app: my-app # 选择需要保护的 Pod 7. PDB 的注意事项仅对自愿中断有效：PDB 只能管理自愿中断（如节点 Drain、滚动更新），无法防止非自愿中断（如节点故障）。需要合理设置参数：如果 minAvailable 或 maxUnavailable 设置过于严格，可能会导致中断操作无法执行。与 HPA 结合使用：如果应用使用了 Horizontal Pod Autoscaler（HPA），需要确保 PDB 的设置与 HPA 的伸缩策略兼容。8. PDB 的实际应用假设你有一个包含 3 个 Pod 的 Deployment，并且希望确保在节点升级时，至少有两个 Pod 可用。你可以创建一个 PDB：apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: my-app-pdb namespace: default spec: minAvailable: 2 selector: matchLabels: app: my-app 当管理员执行节点 Drain 操作时，Kubernetes 会确保最多只有一个 Pod 被终止，剩下的两个 Pod 仍然可以提供服务。总结PodDisruptionBudget（PDB） 是 Kubernetes 中用于管理自愿中断的重要机制，通过设置 minAvailable 或 maxUnavailable，可以确保在中断操作期间，应用的高可用性不受影响。理解并合理使用 PDB，可以帮助你在 Kubernetes 集群中更好地管理应用的稳定性和可用性。
前言随着docker的爆火，万物皆可容器化，当然Minecraft游戏服务端也不例外，早在docker刚出现的时候，itzg大佬在github的便推出了Minecraft running on docker的项目，支持各类服务端的运行。但是随着笔者接触到云原生之后，便热衷于用k8s去部署各种支持容器化的项目，幸运的是itzg大佬也写了可以使用k8s的helm部署的chart包，这样我们就可以一键梭哈部署我们的Minecraft服务器啦！准备工作环境需求8G以上的内存,100G以上硬盘正常运行的Kubernetes 集群，版本1.4以上提前预定义好存储卷PV provisioner 或者使用动态存储卷Helm版本v3.x以上可选环境部署Nginx ingress (Kubernetes ingress流量解决方案)使用OpenEBS动态制备本地存储卷 (Kubernetes存储解决方案)使用Cert-manager实现服务端域名https自动证书申请准备部署当以上环境条件满足后，我们就可以开始愉快部署了：在终端输入以下命令后，于是我们的部署就完成啦 ，happy helming :)helm install minecraft \ --set minecraftServer.eula=true itzg/minecraft很显然我们不可能稀里糊涂的就这么部署了，我们既没有指定使用哪种服务端，也没有指定对应java版本等等，这不是我们所期望的结果。所以在开始前，我们先得获取作者提供的Helm chart包，在ArtifactHub这个网站就可以获取到我们所需的Minecraft chart包。然后我们将下载好的包解压之后，会得到如下文件：minecraft ├── Chart.yaml ├── README.md ├── ci │ └── test-values.yaml ├── templates │ ├── NOTES.txt │ ├── _helpers.tpl │ ├── backupdir-pvc.yaml │ ├── datadir-pvc.yaml │ ├── deployment.yaml │ ├── extra-list.yaml │ ├── extraports-ing.yaml │ ├── extraports-svc.yaml │ ├── minecraft-svc.yaml │ ├── rclone-secret.yaml │ ├── rcon-svc.yaml │ └── secrets.yaml ├── values.schema.json └── values.yaml首先我们新建一份yaml格式的空白文件，命名为customized-values.yaml,后面我们会用这份文件自定义配置，达到我们期望部署的Minecraft服务端。然后我们打开values.yaml，我们会看到许多配置参数，令人眼花缭乱，但是我们只看对我们部署关键的参数，然后把这些参数按照yaml语法格式复制到我们新建的values文件里，可以参考如下我在新的values文件里定义的参数。images tag这里可以参考作者给的文档详细展示了容器镜像tag和java版本的对应关系，这样可以找到适合运行Minecraft服务端的java版本。#指定容器的镜像地址，以及镜像tag使用哪个版本的java # ref: https://hub.docker.com/r/itzg/minecraft-server/ image: repository: itzg/minecraft-server tag: java11-jdk pullPolicy: IfNotPresent pullSecret: ""CPU/Memory资源限制个人建议可以把resource设置上，以免容器内存或者CPU过高把节点搞崩。resources: requests: memory: 4096Mi cpu: 1000m limits: cpu: 5500m memory: 8192Mi工作节点选择(可跳过不设置)如果你是多个工作节点的话，可以忽略scheduler的调度算法，指定pod到哪个节点上去部署nodeSelector: kubernetes.io/hostname: home-local-worker07探活设置#这里我把initialDelaySeconds调成了300s,视个人网络情况而定，以便给服务端足够的启动时间，防止服务端没能在探活周期正常运行，导致被强制kill掉。 livenessProbe: command: - mc-health initialDelaySeconds: 300 periodSeconds: 5 failureThreshold: 20 successThreshold: 1 timeoutSeconds: 1 readinessProbe: command: - mc-health initialDelaySeconds: 300 periodSeconds: 5 failureThreshold: 20 successThreshold: 1 timeoutSeconds: 1好了，接下来是重头戏了~Minecraft服务端参数设置这一长串参数就是我们指定使用哪种服务端和版本，以及预定义游戏参数的地方，咋一看挺多，没关系，特别需要注意的地方，我已经在下面加上了中文注释，其他部分，英文注释已经解释得很清楚了，接下来让我们一个一个地看吧。minecraftServer: #首当其冲，我们必须先同意EULA协议，至于这个是什么协议，我相信只要开过mc服务器的同学应该都知道 eula: "TRUE" #版本我就不细说了，个人选择需要的版本 # One of: LATEST, SNAPSHOT, or a specific version (ie: "1.7.9"). version: "1.16.5" #下面的注释是作者提供的可以使用的服务端类型，这里笔者选的是Forge服务端,但经过本人测试实际上支持像猫端这样的服务端。 #具体细节可以参考这个网址: https://docker-minecraft-server.readthedocs.io/en/latest/types-and-platforms/ # This can be one of "VANILLA", "FORGE", "SPIGOT", "BUKKIT", "PAPER", "FTBA", "SPONGEVANILLA", "CURSEFORGE" type: "FORGE" #forge的版本号，不必多说，如果你开mod服需要特定forge版本号的话，可以在这里指定 # If type is set to FORGE, this sets the version; this is ignored if forgeInstallerUrl is set forgeVersion: # If type is set to SPONGEVANILLA, this sets the version spongeVersion: #下面4个url，可以自定义下载直链运行 # If type is set to FORGE, this sets the URL to download the Forge installer forgeInstallerUrl: # If type is set to BUKKIT, this sets the URL to download the Bukkit package bukkitDownloadUrl: # If type is set to SPIGOT, this sets the URL to download the Spigot package spigotDownloadUrl: # If type is set to PAPER, this sets the URL to download the PaperSpigot package paperDownloadUrl: # If type is set to FTBA, this sets the modpack to run ftbModpackId: # If type is set to FTBA and a modpack is set, this sets the version to run ftbModpackVersionId: # If type is set to CURSEFORGE, this sets the server mod to run. Can also provide url to curseforge package. cfServerMod: # Set to true if running Feed The Beast and get an error like "unable to launch forgemodloader" ftbLegacyJavaFixer: default # One of: peaceful, easy, normal, and hard difficulty: easy # A comma-separated list of player names to whitelist. whitelist: #指定玩家名字为op # A comma-separated list of player names who should be admins. ops: #设置服务器图标的地方 # A server icon URL for server listings. Auto-scaled and transcoded. icon: "https://your-icon.example.com/i/2023/10/07/652172629ec2e.png" # Max connected players. maxPlayers: 20 # This sets the maximum possible size in blocks, expressed as a radius, that the world border can obtain. maxWorldSize: 20000 # Allows players to travel to the Nether. allowNether: default # Allows server to announce when a player gets an achievement. announcePlayerAchievements: default # Enables command blocks. enableCommandBlock: default # If true, players will always join in the default gameMode even if they were previously set to something else. forcegameMode: default # Defines whether structures (such as villages) will be generated. generateStructures: default # If set to true, players will be set to spectator mode if they die. hardcore: default # The maximum height in which building is allowed. maxBuildHeight: 256 # The maximum number of milliseconds a single tick may take before the server watchdog stops the server with the message. -1 disables this entirely. maxTickTime: 60000 # Determines if animals will be able to spawn. spawnAnimals: default # Determines if monsters will be spawned. spawnMonsters: default # Determines if villagers will be spawned. spawnNPCs: default # Sets the area that non-ops can not edit (0 to disable) spawnProtection: 16 # Max view distance (in chunks). viewDistance: 10 # Define this if you want a specific map generation seed. levelSeed: # One of: creative, survival, adventure, spectator gameMode: survival # Message of the Day motd: "Welcome to §nMinecraft on §l§cKubernetes!§r" # If true, enable player-vs-player damage. pvp: default #如果你使用了地形mod，例如低版本的多生物群系需要更改 levelType的名字，就可以在这里设置 # One of: DEFAULT, FLAT, LARGEBIOMES, AMPLIFIED, CUSTOMIZED levelType: DEFAULT # When levelType == FLAT or CUSTOMIZED, this can be used to further customize map generation. # ref: https://hub.docker.com/r/itzg/minecraft-server/ generatorSettings: worldSaveName: world # If set, this URL will be downloaded at startup and used as a starting point downloadWorldUrl: # force re-download of server file forceReDownload: false # If set, the modpack at this URL will be downloaded at startup downloadModpackUrl: # If true, old versions of downloaded mods will be replaced with new ones from downloadModpackUrl removeOldMods: false # A list of VanillaTweaks Share Codes to download. (https://vanillatweaks.net/share#wUq1iz => "wUq1iz") vanillaTweaksShareCodes: [] # Optional URI to a resource pack. The player may choose to use it. resourcePackUrl: # Optional SHA-1 digest of the resource pack, in lowercase hexadecimal. # It is recommended to specify this, because it is used to verify the integrity of the resource pack. resourcePackSha: # When true, players will be prompted for a response and will be disconnected if they decline the required pack resourcePackEnforce: false #在线验证模式，请支持正版 # Check accounts against Minecraft account service. onlineMode: false # Require public key to be signed by Mojang to join enforceSecureProfile: default #这个地方请特别注意，内存一定要和之前在resource下的limit的memory的值要对应上，否则会出现OOM Killed的情况 # If you adjust this, you may need to adjust resources.requests above to match. memory: 8192M #自定义jvm的启动参数的地方，如果服务端启动需要特别优化，可以在这里设置 # General JVM options to be passed to the Minecraft server invocation jvmOpts: "" # Options like -X that need to proceed general JVM options jvmXXOpts: "" #默认情况下，服务器配置将根据以下环境变量创建和设置，但仅在服务器第一次启动时创建和设置,如果想在每次容器启动时重写服务器配置，可以将此设置为true # By default, the server configuration will be created and set based on the following environment variables, but only the first time the server is started # If you would like to override the server configuration each time the container starts up, you can set this to true # see https://github.com/itzg/docker-minecraft-server#server-configuration overrideServerProperties: false # DEPRECATED: use top-level rconServiceAnnotations instead serviceAnnotations: {} #这里涉及到kubernetes的service对外访问的设置，后文会详细解释这部分。 serviceType: ClusterIP ## Set the port used if the serviceType is NodePort nodePort: # Set the external port of the service, usefull when using the LoadBalancer service type servicePort: 25565 clusterIP: loadBalancerIP: # loadBalancerSourceRanges: [] ## Set the externalTrafficPolicy in the Service to either Cluster or Local # externalTrafficPolicy: Cluster externalIPs:RCON这里我们需要开启RCON，以便可以进入服务端后台执行op命令，记得更改密码 rcon: # If you enable this, make SURE to change your password below. enabled: true # By default, the container will generate a random password at startup # to ensure internal RCON tooling, including a backup container, port: 25575 password: "CHANGEME!"游戏数据持久化由于我这边设置的动态制备存储卷，所以在 storageClass这里设置了提供动态制备存储卷的服务名字如果你不是动态制备，手动创建好pvc和pv通过设置existingClaim参数认领存储卷。persistence: labels: {} annotations: {} ## minecraft data Persistent Volume Storage Class ## If defined, storageClassName: <storageClass> ## If set to "-", storageClassName: "", which disables dynamic provisioning ## If undefined (the default) or set to null, no storageClassName spec is ## set, choosing the default provisioner. (gp2 on AWS, standard on ## GKE, AWS & OpenStack) ## storageClass: "openebs-hostpath" dataDir: # Set this to false if you don't care to persist state between restarts. enabled: true Size: 50Gi # existingClaim: nil ## specify a subpath in the volume where the data is. Useful when sharing volumes with other apps. # subPath: /path/to/dataDirMinecraft存档备份(可跳过不设置)如果你的存储空间足够可观的话，可以开启作者提供的存档自动增量备份边车容器，详细参数可以看原始values文件的英文注释。请特别注意开启自动备份的话，RCON控制台必须先启用。# PLEASE NOTE! rcon must be enabled above! It does NOT require a nodePort or loadBalancerIP mcbackup: enabled: true #默认resource参数设置，根据自身需求更改 resources: requests: memory: 256Mi cpu: 100m limits: cpu: 200m memory: 256Mi #备份数据持久化，如果不是动态制备，请手动创建好pvc和pv通过设置existingClaim参数认领存储卷。 persistence: annotations: {} ## minecraft data Persistent Volume Storage Class ## If defined, storageClassName: <storageClass> ## If set to "-", storageClassName: "", which disables dynamic provisioning ## If undefined (the default) or set to null, no storageClassName spec is ## set, choosing the default provisioner. (gp2 on AWS, standard on ## GKE, AWS & OpenStack) ## # storageClass: "-" storageClass: "openebs-hostpath" backupDir: # Set this to false if you don't care to persist state between restarts. enabled: true # existingClaim: nil Size: 20Gi开始部署将chart包传到对应的地方，然后执行如下命令:helm install prod -f customized-values.yaml minecraft-4.15.0.tgz --namespace game --create-namespace --debug --timeout 600s然后等待一段时间，你会看到如下输出，说明部署已经完成NAME: prod LAST DEPLOYED: Sat Apr 20 23:48:00 2024 NAMESPACE: game STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: Get the IP address of your Minecraft server by running these commands in the same shell: export POD_NAME=$(kubectl get pods \ --namespace game \ -l "component=prod-minecraft" \ -o jsonpath="{.items[0].metadata.name}") kubectl port-forward $POD_NAME 25565:25565 echo "Point your Minecraft client at 127.0.0.1:25565"接下来我们使用helm命令查看刚刚部署的releaseroot@tencent-beijing-master:~# helm list -n game NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION prod game 1 2024-04-20 23:48:00.875150927 +0800 CST deployed minecraft-4.15.0 SeeValues 我们再查看一下pod的运行情况吧root@tencent-beijing-master:~# kubectl get pod -n game NAME READY STATUS RESTARTS AGE prod-minecraft-bf95f5499-c299p 2/2 Running 0 30m再logs看一下pod的后台日志是否正常kubectl logs -n game prod-minecraft-bf95f5499-c299p prod-minecraft看起来一切正常，请注意这里，如果你在前文开启了自动备份的话，pod里应该有2个容器，我们需要查看的是服务端的容器，另外一个容器是我们的备份容器。root@tencent-beijing-master:~# kubectl logs -n game prod-minecraft-bf95f5499-c299p prod-minecraft [init] Running as uid=1000 gid=3000 with /data as 'drwxrwsrwx 2 0 2000 4096 Apr 20 15:48 /data' [init] Resolving type given FORGE [mc-image-helper] 15:49:33.893 INFO : Downloading Forge installer 36.2.34 for Minecraft 1.16.5 [mc-image-helper] 15:49:38.782 INFO : Running Forge 36.2.34 installer for Minecraft 1.16.5. This might take a while... [init] Creating server properties in /data/server.properties [init] Disabling whitelist functionality [init] Setting mode [mc-image-helper] 15:51:36.567 INFO : Created/updated 16 properties in /data/server.properties [init] Using server icon from https://pic.lazytoki.cn/i/2023/10/07/652172629ec2e.png... [init] Converting image to 64x64 PNG... [init] Setting initial memory to 8192M and max to 8192M [init] Starting the Minecraft server... [Log4jPatcher] [INFO] Transforming org/apache/logging/log4j/core/lookup/JndiLookup [Log4jPatcher] [INFO] Transforming org/apache/logging/log4j/core/pattern/MessagePatternConverter [Log4jPatcher] [WARN] Unable to find noLookups:Z field in org/apache/logging/log4j/core/pattern/MessagePatternConverter [15:51:45] [main/INFO] [cp.mo.mo.Launcher/MODLAUNCHER]: ModLauncher running: args [--gameDir, ., --launchTarget, fmlserver, --fml.forgeVersion, 36.2.34, --fml.mcpVersion, 20210115.111550, --fml.mcVersion, 1.16.5, --fml.forgeGroup, net.minecraftforge] [15:51:45] [main/INFO] [cp.mo.mo.Launcher/MODLAUNCHER]: ModLauncher 8.1.3+8.1.3+main-8.1.x.c94d18ec starting: java version 11.0.11 by AdoptOpenJDK WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by cpw.mods.modlauncher.SecureJarHandler (file:/data/libraries/cpw/mods/modlauncher/8.1.3/modlauncher-8.1.3.jar) to field java.util.jar.Manifest.jv WARNING: Please consider reporting this to the maintainers of cpw.mods.modlauncher.SecureJarHandler WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release [15:51:45] [main/INFO] [ne.mi.fm.lo.FixSSL/CORE]: Added Lets Encrypt root certificates as additional trust [15:51:45] [main/INFO] [mixin/]: SpongePowered MIXIN Subsystem Version=0.8.4 Source=file:/data/libraries/org/spongepowered/mixin/0.8.4/mixin-0.8.4.jar Service=ModLauncher Env=SERVER [15:51:45] [main/WARN] [ne.mi.fm.lo.FMLConfig/CORE]: Configuration file /data/config/fml.toml is not correct. Correcting [15:51:45] [main/WARN] [ne.mi.fm.lo.FMLConfig/CORE]: Incorrect key [defaultConfigPath] was corrected from null to defaultconfigs [15:51:46] [main/INFO] [STDERR/]: [jdk.nashorn.api.scripting.NashornScriptEngine:<init>:143]: Warning: Nashorn engine is planned to be removed from a future JDK release [15:51:46] [main/INFO] [STDERR/]: [jdk.nashorn.api.scripting.NashornScriptEngine:<init>:143]: Warning: Nashorn engine is planned to be removed from a future JDK release [15:51:46] [main/INFO] [STDERR/]: [jdk.nashorn.api.scripting.NashornScriptEngine:<init>:143]: Warning: Nashorn engine is planned to be removed from a future JDK release [15:51:46] [main/INFO] [cp.mo.mo.LaunchServiceHandler/MODLAUNCHER]: Launching target 'fmlserver' with arguments [--gameDir, .] [15:51:52] [modloading-worker-3/INFO] [ne.mi.co.ForgeMod/FORGEMOD]: Forge mod loading, version 36.2.34, for MC 1.16.5 with MCP 20210115.111550 [15:51:52] [modloading-worker-3/INFO] [ne.mi.co.MinecraftForge/FORGE]: MinecraftForge v36.2.34 Initialized [15:51:53] [main/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Configuration file /data/config/forge-common.toml is not correct. Correcting [15:51:53] [main/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Incorrect key general was corrected from null to its default, SimpleCommentedConfig:{}. [15:51:53] [main/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Incorrect key general.defaultWorldType was corrected from null to its default, default. [15:51:53] [Forge Version Check/INFO] [ne.mi.fm.VersionChecker/]: [forge] Starting version check at https://files.minecraftforge.net/net/minecraftforge/forge/promotions_slim.json [15:51:55] [Forge Version Check/INFO] [ne.mi.fm.VersionChecker/]: [forge] Found status: UP_TO_DATE Current: 36.2.34 Target: null [15:51:56] [main/INFO] [mojang/YggdrasilAuthenticationService]: Environment: authHost='https://authserver.mojang.com', accountsHost='https://api.mojang.com', sessionHost='https://sessionserver.mojang.com', servicesHost='https://api.minecraftservices.com', name='PROD' [15:51:57] [main/WARN] [minecraft/Commands]: Ambiguity between arguments [teleport, destination] and [teleport, targets] with inputs: [Player, 0123, @e, dd12be42-52a9-4a91-a8a1-11c01849e498] [15:51:57] [main/WARN] [minecraft/Commands]: Ambiguity between arguments [teleport, location] and [teleport, destination] with inputs: [0.1 -0.5 .9, 0 0 0] [15:51:57] [main/WARN] [minecraft/Commands]: Ambiguity between arguments [teleport, location] and [teleport, targets] with inputs: [0.1 -0.5 .9, 0 0 0] [15:51:57] [main/WARN] [minecraft/Commands]: Ambiguity between arguments [teleport, targets] and [teleport, destination] with inputs: [Player, 0123, dd12be42-52a9-4a91-a8a1-11c01849e498] [15:51:57] [main/WARN] [minecraft/Commands]: Ambiguity between arguments [teleport, targets, location] and [teleport, targets, destination] with inputs: [0.1 -0.5 .9, 0 0 0] [15:51:57] [main/INFO] [minecraft/SimpleReloadableResourceManager]: Reloading ResourceManager: Default, forge-1.16.5-36.2.34-universal.jar [15:51:58] [Worker-Main-9/INFO] [minecraft/RecipeManager]: Loaded 7 recipes [15:51:59] [Worker-Main-9/INFO] [minecraft/AdvancementList]: Loaded 927 advancements [15:52:00] [Server thread/INFO] [minecraft/DedicatedServer]: Starting minecraft server version 1.16.5 [15:52:00] [Server thread/INFO] [minecraft/DedicatedServer]: Loading properties [15:52:00] [Server thread/INFO] [minecraft/DedicatedServer]: Default game type: SURVIVAL [15:52:00] [Server thread/INFO] [minecraft/MinecraftServer]: Generating keypair [15:52:00] [Server thread/INFO] [minecraft/DedicatedServer]: Starting Minecraft server on *:25565 [15:52:00] [Server thread/INFO] [minecraft/NetworkSystem]: Using epoll channel type [15:52:00] [Server thread/WARN] [minecraft/DedicatedServer]: **** SERVER IS RUNNING IN OFFLINE/INSECURE MODE! [15:52:00] [Server thread/WARN] [minecraft/DedicatedServer]: The server will make no attempt to authenticate usernames. Beware. [15:52:00] [Server thread/WARN] [minecraft/DedicatedServer]: While this makes the game possible to play without internet access, it also opens up the ability for hackers to connect with any username they choose. [15:52:00] [Server thread/WARN] [minecraft/DedicatedServer]: To change this, set "online-mode" to "true" in the server.properties file. [15:52:00] [Server thread/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Configuration file ./world/serverconfig/forge-server.toml is not correct. Correcting [15:52:00] [Server thread/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Incorrect key server was corrected from null to its default, SimpleCommentedConfig:{}. [15:52:00] [Server thread/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Incorrect key server.removeErroringEntities was corrected from null to its default, false. [15:52:00] [Server thread/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Incorrect key server.removeErroringTileEntities was corrected from null to its default, false. [15:52:00] [Server thread/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Incorrect key server.fullBoundingBoxLadders was corrected from null to its default, false. [15:52:00] [Server thread/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Incorrect key server.zombieBaseSummonChance was corrected from null to its default, 0.1. [15:52:00] [Server thread/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Incorrect key server.zombieBabyChance was corrected from null to its default, 0.05. [15:52:00] [Server thread/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Incorrect key server.logCascadingWorldGeneration was corrected from null to its default, true. [15:52:00] [Server thread/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Incorrect key server.fixVanillaCascading was corrected from null to its default, false. [15:52:00] [Server thread/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Incorrect key server.dimensionUnloadQueueDelay was corrected from null to its default, 0. [15:52:00] [Server thread/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Incorrect key server.treatEmptyTagsAsAir was corrected from null to its default, false. [15:52:00] [Server thread/WARN] [ne.mi.co.ForgeConfigSpec/CORE]: Incorrect key server.fixAdvancementLoading was corrected from null to its default, true. [15:52:00] [Server thread/INFO] [minecraft/DedicatedServer]: Preparing level "world" [15:52:05] [Server thread/INFO] [minecraft/MinecraftServer]: Preparing start region for dimension minecraft:overworld [15:52:05] [Worker-Main-9/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 0% [15:52:05] [Worker-Main-9/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 0% [15:52:06] [Worker-Main-6/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 3% [15:52:06] [Worker-Main-8/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 6% [15:52:07] [Worker-Main-9/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 11% [15:52:07] [Worker-Main-8/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 16% [15:52:08] [Worker-Main-7/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 21% [15:52:08] [Worker-Main-7/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 32% [15:52:09] [Worker-Main-7/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 32% [15:52:09] [Worker-Main-6/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 42% [15:52:10] [Worker-Main-9/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 42% [15:52:10] [Worker-Main-8/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 60% [15:52:11] [Worker-Main-9/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 68% [15:52:12] [Worker-Main-9/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 78% [15:52:12] [Worker-Main-10/INFO] [minecraft/LoggingChunkStatusListener]: Preparing spawn area: 78% [15:52:12] [Server thread/INFO] [minecraft/LoggingChunkStatusListener]: Time elapsed: 7497 ms [15:52:12] [Server thread/INFO] [minecraft/DedicatedServer]: Done (11.918s)! For help, type "help" [15:52:12] [Server thread/INFO] [minecraft/DedicatedServer]: Starting remote control listener [15:52:12] [Server thread/INFO] [minecraft/RConThread]: Thread RCON Listener started [15:52:12] [Server thread/INFO] [minecraft/MainThread]: RCON running on 0.0.0.0:25575设置管理员权限最简单方法是通过kubectl exec的方式进入容器内部的rcon控制台设置管理员权限，由于笔者这里使用了存档备份的容器，所以在进入之前需要指定进入哪个容器。有备份容器命令kubectl exec -it -n game prod-minecraft-bf95f5499-c299p -c prod-minecraft rcon-cli无备份容器命令kubectl exec -it -n game prod-minecraft-bf95f5499-c299p rcon-cli成功设置管理员权限root@tencent-beijing-master:~# kubectl exec -it -n game prod-minecraft-bf95f5499-c299p -c prod-minecraft rcon-cli kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. > op Wllo_YU Made Wllo_YU a server operator > help /advancement (grant|revoke) /attribute <target> <attribute> (base|get|modifier) /ban <targets> [<reason>] /ban-ip <target> [<reason>] /banlist [ips|players] /bossbar (add|get|list|remove|set) /clear [<targets>] /clone <begin> <end> <destination> [filtered|masked|replace] /config showfile <mod> <type> /data (get|merge|modify|remove) /datapack (disable|enable|list) /debug (report|start|stop) /defaultgamemode (adventure|creative|spectator|survival) /deop <targets> /difficulty [easy|hard|normal|peaceful] /effect (clear|give) /enchant <targets> <enchantment> [<level>] /execute (align|anchored|as|at|facing|if|in|positioned|rotated|run|store|unless) /experience (add|query|set) /fill <from> <to> <block> [destroy|hollow|keep|outline|replace] /forceload (add|query|remove) /forge (dimensions|entity|generate|mods|setdimension|tps|track) /function <name> /gamemode (adventure|creative|spectator|survival) /gamerule (announceAdvancements|commandBlockOutput|disableElytraMovementCheck|disableRaids|doDaylightCycle|doEntityDrops|doFireTick|doImmediateRespawn|doInsomnia|doLimitedCrafting|doMobLoot|doMobSpawning|doPatrolSpawning|doTileDrops|doTraderSpawning|doWeatherCycle|drowningDamage|fallDamage|fireDamage|forgiveDeadPlayers|keepInventory|logAdminCommands|maxCommandChainLength|maxEntityCramming|mobGriefing|naturalRegeneration|randomTickSpeed|reducedDebugInfo|sendCommandFeedback|showDeathMessages|spawnRadius|spectatorsGenerateChunks|universalAnger) /give <targets> <item> [<count>] /help [<command>] /kick <targets> [<reason>] /kill [<targets>] /list [uuids] /locate (bastion_remnant|buried_treasure|desert_pyramid|endcity|fortress|igloo|jungle_pyramid|mansion|mineshaft|monument|nether_fossil|ocean_ruin|pillager_outpost|ruined_portal|shipwreck|stronghold|swamp_hut|village) /locatebiome <biome> /loot (give|insert|replace|spawn) /me <action> /msg <targets> <message> /op <targets> /pardon <targets> /pardon-ip <target> /particle <name> [<pos>] /playsound <sound> (ambient|block|hostile|master|music|neutral|player|record|voice|weather) /recipe (give|take) /reload /replaceitem (block|entity) /save-all [flush] /save-off /save-on /say <message> /schedule (clear|function) /scoreboard (objectives|players) /seed /setblock <pos> <block> [destroy|keep|replace] /setidletimeout <minutes> /setworldspawn [<pos>] /spawnpoint [<targets>] /spectate [<target>] /spreadplayers <spreadDistance> <maxRange> (under|<respectTeams>) /stop /stopsound <targets> [*|ambient|block|hostile|master|music|neutral|player|record|voice|weather] /summon <entity> [<pos>] /tag <targets> (add|list|remove) /team (add|empty|join|leave|list|modify|remove) /teammsg <message> /teleport (<destination>|<location>|<targets>) /tell -> msg /tellraw <targets> <message> /time (add|query|set) /title <targets> (actionbar|clear|reset|subtitle|times|title) /tm -> teammsg /tp -> teleport /trigger <objective> [add|set] /w -> msg /weather (clear|rain|thunder) /whitelist (add|list|off|on|reload|remove) /worldborder (add|center|damage|get|set|warning) /xp -> experience > 设置服务端对外访问方式既然我们的服务端部署完成了，那么如何暴露我们服务器的地址给小伙伴们愉快地玩耍喃？我们现在get svc应该可以看到类型为ClusterIP的service，ClusterIP是用于集群内部通信的地址，很显然这是没法让外部客户端访问我们的服务端的。root@tencent-beijing-master:~# kubectl get svc -n game NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE prod-minecraft ClusterIP 10.96.3.43 <none> 25565/TCP 14d prod-minecraft-rcon ClusterIP 10.96.2.218 <none> 25575/TCP 14dKubernetes 提供了多种方式让pod内的服务可以被外部访问，这些方式包括：NodePort：NodePort 是 Kubernetes 中最简单的一种对外暴露服务的方式之一。通过 NodePort，Kubernetes 会在每个节点上开放一个固定的端口，然后将该端口映射到 Service 中指定的端口上。外部用户可以通过访问节点的 IP 地址和这个端口来访问服务。NodePort 方式适合于测试和快速原型开发，但不适合在生产环境中大规模使用，因为它暴露了节点上的所有服务，可能会导致安全风险。LoadBalancer：LoadBalancer 是一种更安全和灵活的对外暴露服务的方式。通过 LoadBalancer，Kubernetes 可以自动创建一个云服务提供商（如 AWS、GCP、Azure 等）所支持的负载均衡器，并将其配置为服务的前端。外部用户可以通过访问负载均衡器的 IP 地址或域名来访问服务。这种方式适合于生产环境中对高可用性和弹性要求较高的服务。Ingress：Ingress 是一种在集群外部提供 HTTP 和 HTTPS 路由的方法。它通过将入站请求路由到集群内部的服务来实现对外暴露服务。Ingress 控制器可以根据请求的主机名或路径将请求路由到不同的服务。这种方式适合于需要在单个 IP 地址上托管多个服务的情况，同时提供灵活的路由配置和 TLS 终止功能。ExternalName：ExternalName 是一种将服务映射到集群外部的服务的方式。它将一个 Kubernetes 服务映射到一个 DNS 名称，这个 DNS 名称指向集群外部的某个服务或者域名。这种方式适合于需要与集群外部的服务进行集成的情况，例如将集群内部的服务访问外部的数据库或者第三方服务。这是笔者只提供1和3的访问方式，2和4需要有外部服务的支持，所以我们暂不使用。NodePort NAT访问方式两种方式设置NodePorta.通过kubectl edit直接修改svckubectl edit svc -n game prod-minecraft我们需要把type这里改成NodePort的类型，然后wq保存退出。type: NodePortb.通过helm更新设置为NodePort类型还记得我们在Minecraft服务端参数设置时候，在serviceType这一行设置的值么，现在我们需要将其改成NodePort的类型。serviceType: NodePort更新完这行配置后，执行helm upgrade命令helm upgrade -n game --atomic --debug --timeout 600s prod -f customized-values.yaml minecraft-4.15.0.tgz查看service状态这时我们可以看到在PORT这里已经有了我们容器内部端口25565映射为端口32048root@tencent-beijing-master:~# kubectl get svc -n game NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE prod-minecraft NodePort 10.96.3.43 <none> 25565:32048/TCP 14d prod-minecraft-rcon ClusterIP 10.96.2.218 <none> 25575/TCP 14d访问服务端我们只需要将任意工作节点ip地址和映射出去的端口填入客户端，即可访问到我们的服务端。Ingress TCP代理访问方式(需要部署Nginx ingress)众所周知，Minecraft服务端是通过TCP进行连接的，所以我们可以通过ingress的四层代理方式为服务端提供访问。由于Kubernetes 有多种ingress控制器，所以这里我们选择Nginx ingress。虽然nginx ingress是专门针对七层的控制器，但是官方提供了文档 设置四层代理，需要我们在nginx ingress的控制器里加一行参数，让nginx ingress能够正确代理到我们的服务器端口。创建configmap我们需要在data这里设置我们需要代理端口的键值对，从左到右按照 需要代理出去的端口->命名空间->服务端service的名字->service端口号设置。apiVersion: v1 kind: ConfigMap metadata: name: tcp-proxy-service namespace: ingress-nginx data: "25565": game/gateway-mc-router:25565编辑ingress的controller的副本kubectl edit ds -n ingress-nginx ingress-nginx-controller在args层级里添加如下配置ingress-nginx(命名空间)/tcp-proxy-service(configmap名字)和上面创建的cm名字对应,添加完毕后wq保存退出。 spec: containers: - args: - --tcp-services-configmap=ingress-nginx/tcp-proxy-servicenginx默认更新方式是RollingUpdate，等待pod一个一个重启完毕，并查看状态。root@tencent-beijing-master:~/ingress# kubectl get pod -n ingress-nginx NAME READY STATUS RESTARTS AGE ingress-nginx-controller-5szfv 1/1 Running 0 5m56s ingress-nginx-controller-5t2jd 1/1 Running 0 4m16s ingress-nginx-controller-9fjqp 1/1 Running 0 4m50s ingress-nginx-controller-fkjrk 1/1 Running 0 6m29s ingress-nginx-controller-vhzs6 1/1 Running 0 3m44s ingress-nginx-controller-vw7s9 1/1 Running 0 2m38s ingress-nginx-controller-w9f7l 1/1 Running 0 5m23s ingress-nginx-controller-xqmlg 1/1 Running 0 3m11s ingress-nginx-defaultbackend-78d476db97-wr99h 1/1 Running 0 51m我们get任意ingress controller pod，查看其配置，可以看到我们添加的args参数和pod在主机节点开启了25565这个端口。因为笔者的ingress环境是采用了daemonset的方式部署的controller pod和使用了host network也就是和主机共享网络栈的模式暴露对外的端口，所以理论上可以通过多个节点负载均衡流量以及不通过controller service的NodePort的方式访问服务端kubectl get pod -n ingress-nginx ingress-nginx-controller-vhzs6 -o yaml特别注意：如果加完参数，查看任意pod配置发现端口并没有开放，请按照yaml格式手动添加port的配置,并等待pod重启完毕。至于为什么不生效，我也没有想明白，欢迎大佬解答。 - args: - --tcp-services-configmap=ingress-nginx/tcp-proxy-service . . . . . ports: - containerPort: 25565 hostPort: 25565 name: 25565-tcp protocol: TCP访问服务端但在我们访问服务端之前，我们需要做一件事，那就是设置域名解析到我们的工作节点的公网地址上，这样我们就可以通过域名访问我们的服务端啦。ps:如果你的节点都处于公网的环境下且采用daemonset的方式部署，理论上你可以将域名解析到多个工作节点上，实现4层流量负载均衡，但这取决于你的dns服务提供商，最多允许你添加几个相同域名的负载均衡设置。接下来，大功告成!游戏截图总结这次算是Minecraft服务端在云原生方向部署的一次小小探索，欢迎各位大佬多多交流，批评指正。
Basic Authentication¶This example shows how to add authentication in a Ingress rule using a secret that contains a file generated with htpasswd. It's important the file generated is named auth (actually - that the secret has a key data.auth), otherwise the ingress-controller returns a 503.Create htpasswd file¶$ htpasswd -c auth foo New password: <bar> New password: Re-type new password: Adding password for user fooConvert htpasswd into a secret¶$ kubectl create secret -n generic basic-auth --from-file=auth secret "basic-auth" createdExamine secret¶$ kubectl get secret basic-auth -o yaml apiVersion: v1 data: auth: Zm9vOiRhcHIxJE9GRzNYeWJwJGNrTDBGSERBa29YWUlsSDkuY3lzVDAK kind: Secret metadata: name: basic-auth namespace: default type: Opaque
Automated Certificate Management with cert-managercert-manager automatically requests missing or expired certificates from a range of supported issuers (including Let's Encrypt) by monitoring ingress resources.To set up cert-manager you should take a look at this full example.To enable it for an ingress resource you have to deploy cert-manager, configure a certificate issuer update the manifest:apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-demo annotations: cert-manager.io/issuer: "letsencrypt-staging" # Replace this with a production issuer once you've tested it [..] spec: tls: - hosts: - ingress-demo.example.com secretName: ingress-demo-tls [...]
podSecurityContext 和 securityContext 是两个相关但用途不同的配置选项，用于 Kubernetes 中的容器和 Pod 安全性。podSecurityContext：podSecurityContext 是用于配置整个 Pod 的安全上下文的选项。它通常包括 Pod 层面的配置，例如指定运行 Pod 的用户和组、Seccomp 配置、AppArmor 配置等。podSecurityContext 中的设置将应用于 Pod 内的所有容器，除非在容器级别明确覆盖。示例：apiVersion: v1 kind: Pod metadata: name: my-pod spec: securityContext: runAsUser: 1000 containers: - name: my-container image: my-imagesecurityContext：securityContext 是用于配置单个容器的安全上下文的选项。它通常包括容器级别的配置，例如指定容器运行时的用户和组、特权模式、Linux 内核功能、Seccomp 配置、AppArmor 配置等。securityContext 中的设置将应用于特定容器，允许在同一 Pod 内的不同容器中使用不同的安全配置。示例：apiVersion: v1 kind: Pod metadata: name: my-pod spec: containers: - name: container-1 image: image-1 securityContext: runAsUser: 1000 - name: container-2 image: image-2 securityContext: runAsUser: 2000总之，podSecurityContext 用于配置整个 Pod 的安全性，而 securityContext 用于配置单个容器的安全性。你可以根据需要在 Pod 和容器级别分别配置它们，以满足特定的安全需求。podSecurityContext 是 Kubernetes 中用于定义 Pod 安全上下文的配置选项。这允许你为特定的 Pod 配置安全属性，以确保其行为与安全性符合你的需求。Pod 安全上下文通常用于控制以下方面的安全性：Linux 用户和组的权限：你可以通过 runAsUser 和 runAsGroup 字段设置 Pod 中容器的运行用户和组。这有助于限制容器的权限，防止恶意进程访问主机上的资源。Linux 特权模式：你可以使用 privileged 字段来指定是否容器应该以特权模式运行，这将影响容器的访问权限，包括主机上的文件系统。Seccomp 和 AppArmor 配置：通过 seccompProfile 和 apparmorProfile 字段，你可以配置容器的 Seccomp 和 AppArmor 配置，以限制容器中进程的系统调用和访问权限。sysctl 参数：使用 sysctls 字段，你可以配置容器的内核参数，以增强容器的安全性。Linux 安全上下文标签：你可以使用 securityContext 字段配置 Linux 安全上下文标签，以限制容器的访问权限。资源限制：虽然资源限制通常与 Pod 的 resources 字段相关，但它们也与安全性有关。通过限制 CPU 和内存资源，你可以避免容器使用过多的资源，从而确保其他 Pod 仍然能够正常工作。示例：apiVersion: v1 kind: Pod metadata: name: my-pod spec: securityContext: runAsUser: 1000 runAsGroup: 3000 seccompProfile: type: RuntimeDefault containers: - name: my-container image: my-image在这个示例中，securityContext 设置了运行容器的用户和组，以及 Seccomp 配置。这有助于确保容器以指定的用户和组运行，并且受到 Seccomp 配置的限制。这是一个简化的示例，实际配置可能会更复杂，根据安全需求来定制。securityContext 在 Kubernetes 中是用于定义 Pod 或容器的安全上下文的配置选项。这些配置允许你控制容器运行时的安全性特性。以下是一些常见的 securityContext 字段及其作用：runAsUser 和 runAsGroup： 用于指定容器运行时的用户和用户组。这有助于限制容器内进程的权限，确保它们以指定的用户和用户组身份运行。securityContext: runAsUser: 1000 runAsGroup: 3000privileged： 一个布尔值，用于指定容器是否应该以特权模式运行。特权容器具有更多的访问权限，可以绕过一些安全限制。设置为 true 表示容器以特权模式运行。securityContext: privileged: truecapabilities： 用于指定容器的 Linux 内核功能集。可以根据需要添加或删除特定的功能，以减少容器的权限。securityContext: capabilities: add: ["NET_ADMIN"] drop: ["ALL"]seccompProfile： 用于指定容器的 Seccomp 配置，以限制容器内进程的系统调用。可以指定自定义的 Seccomp 配置文件或使用预定义的模板。securityContext: seccompProfile: type: RuntimeDefaultapparmorProfile： 用于指定容器的 AppArmor 配置，以限制容器的访问权限。可以指定自定义的 AppArmor 配置文件或使用预定义的模板。securityContext: apparmorProfile: "my-custom-profile"readOnlyRootFilesystem： 一个布尔值，指定容器的根文件系统是否为只读。设置为 true 表示根文件系统为只读。securityContext: readOnlyRootFilesystem: true这些配置项可以根据你的安全性需求进行定制。通过适当地配置 securityContext，可以提高容器的安全性，限制对主机和其他容器的潜在威胁。
Nginx Ingress Controller 在 Kubernetes 中充当了一个代理和负载均衡器的角色，它可以根据 Ingress 资源的规则将外部的 HTTP/HTTPS 流量转发到集群内部的服务。然而，对于非 HTTP/HTTPS 的流量，例如 TCP/UDP，Nginx Ingress Controller 也提供了代理的功能，但它并不是通过 Ingress 资源来配置，而是需要通过 ConfigMap 进行配置。以下是 Nginx Ingress 的 TCP 代理的工作原理：集群外部的 TCP 连接请求：当有一个来自集群外部的 TCP 连接请求到达 Nginx Ingress Controller 时，Nginx 会根据这个请求的目标端口来决定如何处理这个请求。ConfigMap 配置：Nginx Ingress Controller 会读取一个特定的 ConfigMap（通常名为 tcp-services 或 udp-services），在这个 ConfigMap 中，你可以定义一个端口到服务的映射。这个映射告诉 Nginx Ingress Controller，对于到达特定端口的 TCP 请求，应该将其转发到哪个服务。TCP 请求转发：根据 ConfigMap 的配置，Nginx Ingress Controller 会将 TCP 请求转发到对应的服务。这个服务可以是集群内的任何服务，不论它是运行在哪个 Namespace 中，或者是使用哪种协议。服务响应：当服务处理完请求并返回响应时，Nginx Ingress Controller 会将这个响应返回给原始的 TCP 客户端。值得注意的是，Nginx Ingress 的 TCP/UDP 代理比 HTTP/HTTPS 代理少了很多功能，例如基于路径的路由、SSL 终止、主机名路由等。因为 TCP 和 UDP 是传输层协议，它们并不像 HTTP/HTTPS 那样提供丰富的应用层信息，所以 Nginx Ingress Controller 只能根据 TCP/UDP 请求的目标端口来决定如何转发请求。Nginx Ingress Controller 提供了 TCP 服务的代理功能，但是它并不支持基于路径或主机名的路由，这是因为 TCP 是一种传输层协议，它并不包含这些应用层信息。所以，如果你想要代理 TCP 服务到集群内部的多个服务，你需要为每个服务指定一个唯一的端口。你可以在一个名为 tcp-services 或 udp-services 的 ConfigMap 中定义端口到服务的映射。例如：apiVersion: v1 kind: ConfigMap metadata: name: tcp-services namespace: ingress-nginx data: 9000: "default/my-service-1:9000" 9001: "default/my-service-2:9001"在这个例子中，所有发送到端口 9000 的 TCP 请求将被转发到名为 my-service-1 的服务的端口 9000，所有发送到端口 9001 的 TCP 请求将被转发到名为 my-service-2 的服务的端口 9001。请注意，你需要在 Nginx Ingress Controller 的部署配置中添加一个参数，指定这个 ConfigMap 的名称。例如：args: - /nginx-ingress-controller - --configmap=$(POD_NAMESPACE)/tcp-services这样，Nginx Ingress Controller 就会加载这个 ConfigMap，并根据它的配置来代理 TCP 请求。最后，你需要在 Nginx Ingress Controller 的 Service 中公开这些端口，以便外部的 TCP 客户端可以连接到它们。例如：apiVersion: v1 kind: Service metadata: name: ingress-nginx namespace: ingress-nginx spec: ports: - name: http port: 80 targetPort: http - name: https port: 443 targetPort: https - name: tcp-9000 port: 9000 targetPort: 9000 - name: tcp-9001 port: 9001 targetPort: 9001 selector: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx这样，你就可以把集群外部的 TCP 请求转发到集群内部的多个服务了。
1.准备工作需要以下基础Kubernetes基础概念Kubernetes集群(当前集群版本 version 1.23.4)#当前环境版本 Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.4", GitCommit:"e6c093d87ea4cbb530a7b2ae91e54c0842d8308a", GitTreeState:"clean", BuildDate:"2022-02-16T12:38:05Z", GoVersion:"go1.17.7", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.4", GitCommit:"e6c093d87ea4cbb530a7b2ae91e54c0842d8308a", GitTreeState:"clean", BuildDate:"2022-02-16T12:32:02Z", GoVersion:"go1.17.7", Compiler:"gc", Platform:"linux/amd64"}helm服务#当前环境版本 version.BuildInfo{Version:"v3.8.1", GitCommit:"5cb9af4b1b271d11d7a97a71df3ac337dd94ad37", GitTreeState:"clean", GoVersion:"go1.17.5"}Minecraft helm chart包在Artifact hub下载并解压对应的Minecraft chart包到当前目录 (chart version 4.3.0)2.配置基础Values文件先把服务端跑起来,体验一把由于values文件内容较多，我们只需要修改如下地方就可以把服务端跑起来.修改如下地方,在values文件中搜索即可找到: 服务端版本类型以及服务端属性文件的地方 minecraftServer: #同意EULA，搭建过mc服务器的童鞋应该都知道这个地方 This must be overridden, since we can't accept this for the user. eula: "TRUE" # 服务端的版本,例如填入版本1.12.2或者1.16.5... One of: LATEST, SNAPSHOT, or a specific version (ie: "1.7.9"). version: "1.12.2" # 服务端类型,默认是原版VANILLA，也就是官方服务端 This can be one of "VANILLA", "FORGE", "SPIGOT", "BUKKIT", "PAPER", "FTBA", "SPONGEVANILLA", "CURSEFORGE" type: "VANILLA" # 服务端内存限制 If you adjust this, you may need to adjust resources.requests above to match. memory: 1024M # 如果需要离线登录，请关闭online mode Check accounts against Minecraft account service. onlineMode: false 这个地方是选择k8s集群的存储驱动，我这里使用的是nfs的动态provisioner，大家可以自寻选择自己合适的CSI插件作为自己的Minecraft后端存储 注意层级关系，value中有两处persistence的配置地方，选择最外层的persistence进行配置 persistence: annotations: {} ## minecraft data Persistent Volume Storage Class ## If defined, storageClassName: <storageClass> ## If set to "-", storageClassName: "", which disables dynamic provisioning ## If undefined (the default) or set to null, no storageClassName spec is ## set, choosing the default provisioner. (gp2 on AWS, standard on ## GKE, AWS & OpenStack) ## storageClass: "填入你的storage class的名称" dataDir: # Set this to false if you don't care to persist state between restarts. enabled: true Size: 1Gi # existingClaim: nil ## specify a subpath in the volume where the data is. Useful when sharing volumes with other apps. # subPath: /path/to/dataDir 设置你服务端需要k8s集群为你分配多少CPU和Memory，个人建议加上limit限制，否则有可能服务端request的资源过大，影响集群内其他服务 注意如果加了limits限制，必须要和前面你的定义的memory的值一样 ## Configure resource requests and limits ## ref: http://kubernetes.io/docs/user-guide/compute-resources/ ## resources: requests: memory: 512Mi cpu: 500m #limits: #cpu: 500m #memory: 1024Mi 存活探针设置，个人建议将 initialDelaySeconds 的时间设置长一些，个人是改为300+秒,留给服务端足够启动时间. livenessProbe: initialDelaySeconds: 300 periodSeconds: 5 failureThreshold: 20 successThreshold: 1 timeoutSeconds: 1完整Values文件：# ref: https://hub.docker.com/r/itzg/minecraft-server/ image: repository: itzg/minecraft-server tag: latest pullPolicy: IfNotPresent pullSecret: "" # ### WARNING ### # Minecraft is not horizontally scalable, adjusting this will most likely break your setup. # ### WARNING ### replicaCount: 1 ## Configure resource requests and limits ## ref: http://kubernetes.io/docs/user-guide/compute-resources/ ## resources: requests: memory: 512Mi cpu: 500m # upgrade strategy type (e.g. Recreate or RollingUpdate) strategyType: Recreate nodeSelector: {} tolerations: [] affinity: {} securityContext: # Security context settings runAsUser: 1000 fsGroup: 2000 # Most of these map to environment variables. See Minecraft for details: # https://hub.docker.com/r/itzg/minecraft-server/ livenessProbe: initialDelaySeconds: 30 periodSeconds: 5 failureThreshold: 20 successThreshold: 1 timeoutSeconds: 1 readinessProbe: initialDelaySeconds: 30 periodSeconds: 5 failureThreshold: 20 successThreshold: 1 timeoutSeconds: 1 startupProbe: enabled: false failureThreshold: 30 periodSeconds: 10 # initContainers: # - name: do-something # image: busybox # command: ['do', 'something'] # volumesMounts: # - name: nfs # mountPath: /mnt/volume # readOnly: true initContainers: [] # sidecarContainers: # - name: do-something # image: busybox # command: ['do', 'something'] # volumesMounts: # - name: nfs # mountPath: /mnt/volume # readOnly: true sidecarContainers: [] # extraVolumes: # - volumeMounts: # - name: nfs # mountPath: /mnt/volume # readOnly: true # volumes: # - name: nfs # server: some.nfs.server.com # path: / # mountOptions: # - port=2049 # - hard # - vers=4 extraVolumes: [] minecraftServer: # This must be overridden, since we can't accept this for the user. eula: "FALSE" # One of: LATEST, SNAPSHOT, or a specific version (ie: "1.7.9"). version: "LATEST" # This can be one of "VANILLA", "FORGE", "SPIGOT", "BUKKIT", "PAPER", "FTBA", "SPONGEVANILLA", "CURSEFORGE" type: "VANILLA" # If type is set to FORGE, this sets the version; this is ignored if forgeInstallerUrl is set forgeVersion: # If type is set to SPONGEVANILLA, this sets the version spongeVersion: # If type is set to FORGE, this sets the URL to download the Forge installer forgeInstallerUrl: # If type is set to BUKKIT, this sets the URL to download the Bukkit package bukkitDownloadUrl: # If type is set to SPIGOT, this sets the URL to download the Spigot package spigotDownloadUrl: # If type is set to PAPER, this sets the URL to download the PaperSpigot package paperDownloadUrl: # If type is set to FTBA, this sets the modpack to run ftbModpackId: # If type is set to FTBA and a modpack is set, this sets the version to run ftbModpackVersionId: # If type is set to CURSEFORGE, this sets the server mod to run. Can also provide url to curseforge package. cfServerMod: # Set to true if running Feed The Beast and get an error like "unable to launch forgemodloader" ftbLegacyJavaFixer: false # One of: peaceful, easy, normal, and hard difficulty: easy # A comma-separated list of player names to whitelist. whitelist: # A comma-separated list of player names who should be admins. ops: # A server icon URL for server listings. Auto-scaled and transcoded. icon: # Max connected players. maxPlayers: 20 # This sets the maximum possible size in blocks, expressed as a radius, that the world border can obtain. maxWorldSize: 10000 # Allows players to travel to the Nether. allowNether: true # Allows server to announce when a player gets an achievement. announcePlayerAchievements: true # Enables command blocks. enableCommandBlock: true # If true, players will always join in the default gameMode even if they were previously set to something else. forcegameMode: false # Defines whether structures (such as villages) will be generated. generateStructures: true # If set to true, players will be set to spectator mode if they die. hardcore: false # The maximum height in which building is allowed. maxBuildHeight: 256 # The maximum number of milliseconds a single tick may take before the server watchdog stops the server with the message. -1 disables this entirely. maxTickTime: 60000 # Determines if animals will be able to spawn. spawnAnimals: true # Determines if monsters will be spawned. spawnMonsters: true # Determines if villagers will be spawned. spawnNPCs: true # Sets the area that non-ops can not edit (0 to disable) spawnProtection: 16 # Max view distance (in chunks). viewDistance: 10 # Define this if you want a specific map generation seed. levelSeed: # One of: creative, survival, adventure, spectator gameMode: survival # Message of the Day motd: "Welcome to Minecraft on Kubernetes!" # If true, enable player-vs-player damage. pvp: false # One of: DEFAULT, FLAT, LARGEBIOMES, AMPLIFIED, CUSTOMIZED levelType: DEFAULT # When levelType == FLAT or CUSTOMIZED, this can be used to further customize map generation. # ref: https://hub.docker.com/r/itzg/minecraft-server/ generatorSettings: worldSaveName: world # If set, this URL will be downloaded at startup and used as a starting point downloadWorldUrl: # force re-download of server file forceReDownload: false # If set, the modpack at this URL will be downloaded at startup downloadModpackUrl: # If true, old versions of downloaded mods will be replaced with new ones from downloadModpackUrl removeOldMods: false # A list of VanillaTweaks Share Codes to download. (https://vanillatweaks.net/share#wUq1iz => "wUq1iz") vanillaTweaksShareCodes: [] # Optional URI to a resource pack. The player may choose to use it. resourcePackUrl: # Optional SHA-1 digest of the resource pack, in lowercase hexadecimal. # It is recommended to specify this, because it is used to verify the integrity of the resource pack. resourcePackSha: # When true, players will be prompted for a response and will be disconnected if they decline the required pack resourcePackEnforce: false # Check accounts against Minecraft account service. onlineMode: true # If you adjust this, you may need to adjust resources.requests above to match. memory: 1024M # General JVM options to be passed to the Minecraft server invocation jvmOpts: "" # Options like -X that need to proceed general JVM options jvmXXOpts: "" # By default, the server configuration will be created and set based on the following environment variables, but only the first time the server is started # If you would like to override the server configuration each time the container starts up, you can set this to true # see https://github.com/itzg/docker-minecraft-server#server-configuration overrideServerProperties: false # DEPRECATED: use top-level rconServiceAnnotations instead serviceAnnotations: {} serviceType: ClusterIP ## Set the port used if the serviceType is NodePort nodePort: # Set the external port of the service, usefull when using the LoadBalancer service type servicePort: 25565 clusterIP: loadBalancerIP: # loadBalancerSourceRanges: [] ## Set the externalTrafficPolicy in the Service to either Cluster or Local # externalTrafficPolicy: Cluster externalIPs: # A list of Spigot resources/plugins IDs to download. spigetResources: [] rcon: # If you enable this, make SURE to change your password below. enabled: false port: 25575 password: "CHANGEME!" existingSecret: secretKey: rcon-password serviceType: ClusterIP ## Set the external port if the rcon serviceType is NodePort nodePort: clusterIP: loadBalancerIP: # loadBalancerSourceRanges: [] ## Set the externalTrafficPolicy in the Service to either Cluster or Local # externalTrafficPolicy: Cluster extraPorts: [] # These options allow you to expose another port from the Minecraft server, plugins such # as dynmap (8123) and bluemap (8100) will require this for access to their web interfaces # # - name: map # containerPort: 8123 # protocol: TCP # service: # enabled: false # embedded: false # annotations: {} # type: ClusterIP # ## Set the external port if the rcon serviceType is NodePort ## nodePort: # loadBalancerIP: "" # loadBalancerSourceRanges: [] # externalTrafficPolicy: Cluster # port: 8123 # ingress: # ingressClassName: nginx # enabled: false # annotations: ## Deprecated way for specifying the ingressClass. Kube.version < 1.18 ## kubernetes.io/ingress.class: nginx # kubernetes.io/tls-acme: "true" # hosts: # - name: map.local # path: / # tls: # - secretName: map-tls # hosts: # - map.local query: # If you enable this, your server will be "published" to Gamespy enabled: false port: 25565 ## Additional minecraft container environment variables ## Values can be either variable values or `valueFrom` yaml ## extraEnv: {} # some_variable: some value # another_variable: # valueFrom: # fieldRef: # fieldPath: status.hostIP ## Additional environment variables to add to the minecraft container from ## ConfigMaps and Secrets ## envFrom: [] persistence: annotations: {} ## minecraft data Persistent Volume Storage Class ## If defined, storageClassName: <storageClass> ## If set to "-", storageClassName: "", which disables dynamic provisioning ## If undefined (the default) or set to null, no storageClassName spec is ## set, choosing the default provisioner. (gp2 on AWS, standard on ## GKE, AWS & OpenStack) ## # storageClass: "-" dataDir: # Set this to false if you don't care to persist state between restarts. enabled: false Size: 1Gi # existingClaim: nil ## specify a subpath in the volume where the data is. Useful when sharing volumes with other apps. # subPath: /path/to/dataDir podAnnotations: {} deploymentAnnotations: {} serviceAnnotations: {} rconServiceAnnotations: {} # PLEASE NOTE! rcon must be enabled above! It does NOT require a nodePort or loadBalancerIP mcbackup: enabled: false image: repository: itzg/mc-backup tag: latest pullPolicy: IfNotPresent # wait 2 minutes before starting initialDelay: 2m # ***set to 0 or smaller, script will run once and exit. DO NOT SET TO 0 or smaller, this will cause K8s to kill your pod!*** # backupInterval="1.5d" -> backup every one and a half days (36 hours) # backupInterval="2h 30m" -> backup every two and a half hours backupInterval: 24h # option lets you pause backups if no players are online. pauseIfNoPlayers: "false" # is set to a positive number, it'll delete old .tgz backup files from DEST_DIR. By default deletes backups older than a week. pruneBackupsDays: 7 # Set to a negative value to retry indefinitely rconRetries: 5 rconRetryInterval: 10s # is a comma-separated list of glob(3) patterns to exclude from backups. By default excludes all jar files (plugins, server files), # logs folder and cache (used by i.e. PaperMC server). excludes: "*.jar,cache,logs" # backup methods, see https://github.com/itzg/docker-mc-backup e.g. tar, rclone, restic backupMethod: tar # tar and rclone methods destDir: /backups # is a true/false flag that creates a symbolic link to the latest backup linkLatest: "false" # is the compression method used by tar. Valid value: gzip bzip2 zstd compressMethod: "gzip" # sets the parameters for zstd compression. The --long parameter affects RAM requirements for both compression and decompression # (the default of 25 means 2^25 bytes = 32 MB). zstdParameters: "-3 --long=25 --single-thread" # the name of the remote you've configured in your rclone.conf rcloneRemote: rcloneDestDir: rcloneCompressMethod: gzip # see https://rclone.org/ for details # this value is evaluated as a template rcloneConfig: # [remote] # type = google cloud storage # client_id = # client_secret = # token = {"AccessToken":"super","RefreshToken":"secret","Expiry":"date","Extra":null} # project_number = 12345678 # object_acl = private # bucket_acl = private # if you prefer to create a secret from file (e.g. kubectl create secret generic my-rclone-config --from-file=~/.config/rclone/rclone.conf) # rcloneConfigExistingSecret: my-rclone-config resticRepository: # variable to define a space separated list of additional restic tags. see https://hub.docker.com/r/itzg/mc-backup resticAdditionalTags: "mc_backups" # see https://restic.readthedocs.io/en/latest/060_forget.html pruneResticRetention: "--keep-daily 7 --keep-weekly 5 --keep-monthly 12 --keep-yearly 75" # At least one of RESTIC_PASSWORD* env variables need to be defined, see https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html resticEnvs: [] # RESTIC_PASSWORD: restic-password extraEnvs: [] # Can be set to the timezone to use for logging # tz: resources: requests: memory: 512Mi cpu: 500m persistence: annotations: {} ## minecraft data Persistent Volume Storage Class ## If defined, storageClassName: <storageClass> ## If set to "-", storageClassName: "", which disables dynamic provisioning ## If undefined (the default) or set to null, no storageClassName spec is ## set, choosing the default provisioner. (gp2 on AWS, standard on ## GKE, AWS & OpenStack) ## # storageClass: "-" backupDir: # Set this to false if you don't care to persist state between restarts. enabled: false # existingClaim: nil Size: 1Gi 3.开始部署3.1 使用helm部署改好的value文件:helm install -n <namespace> minecraft-test -f minecraft-values.yaml minecraft-4.3.0.tgz预期输出结果：NAME: minecraft-test LAST DEPLOYED: Mon Sep 26 21:03:41 2022 NAMESPACE: lab STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: Get the IP address of your Minecraft server by running these commands in the same shell: export POD_NAME=$(kubectl get pods \ --namespace lab \ -l "component=minecraft-test-minecraft" \ -o jsonpath="{.items[0].metadata.name}") kubectl port-forward $POD_NAME 25565:25565 echo "Point your Minecraft client at 127.0.0.1:25565" [root@director minecraft]#3.2 查看部署状态：[root@director minecraft]# helm list -n lab NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION minecraft-test lab 1 2022-09-26 21:03:41.687369562 +0800 CST deployed minecraft-4.3.0 SeeValues kubectl get pod -n <namespace> NAME READY STATUS RESTARTS AGE minecraft-test-minecraft-7cb4f8df9b-jnvwb 2/2 Running 0 10m [root@director minecraft]# [root@director minecraft]# k logs -n <namespace> -f minecraft-test-minecraft-67579d447-xcssb minecraft-test-minecraft chmod: changing permissions of '/data': Operation not permitted [init] Running as uid=1000 gid=1000 with /data as 'drwxrwxrwx 5 0 0 4096 Sep 26 13:31 /data' [init] Resolved version given 1.12.2 into 1.12.2 and major version 1.12 [init] Resolving type given VANILLA [init] server.properties already created, skipping [init] Checking for JSON files. [init] Setting initial memory to 1024M and max to 1024M [init] Starting the Minecraft server... 2022-09-26 13:31:54,476 main ERROR Error processing element Queue ([Appenders: null]): CLASS_NOT_FOUND 2022-09-26 13:31:54,674 main ERROR Unable to locate appender "ServerGuiConsole" for logger config "root" [13:31:59] [Server thread/INFO]: Starting minecraft server version 1.12.2 [13:31:59] [Server thread/INFO]: Loading properties [13:31:59] [Server thread/INFO]: Default game type: SURVIVAL [13:31:59] [Server thread/INFO]: Generating keypair [13:32:00] [Server thread/INFO]: Starting Minecraft server on *:25565 [13:32:01] [Server thread/INFO]: Using epoll channel type [13:32:01] [Server thread/INFO]: Preparing level "world" [13:32:02] [Server thread/INFO]: Loaded 488 advancements [13:32:02] [Server thread/INFO]: Preparing start region for level 0 [13:32:04] [Server thread/INFO]: Preparing spawn area: 0% [13:32:05] [Server thread/INFO]: Preparing spawn area: 12% [13:32:06] [Server thread/INFO]: Preparing spawn area: 32% [13:32:07] [Server thread/INFO]: Preparing spawn area: 61% [13:32:08] [Server thread/INFO]: Preparing spawn area: 96% [13:32:09] [Server thread/INFO]: Done (7.591s)! For help, type "help" or "?" [13:32:09] [Server thread/INFO]: Starting remote control listener [13:32:09] [RCON Listener #1/INFO]: RCON running on 0.0.0.0:25575 [13:32:11] [RCON Listener #1/INFO]: Rcon connection from: /127.0.0.14.暴露服务给外部客户端4.1 port-forward快速访问使用kubectl port-forward将 minecraft的api server监听在使用这条命令的主机的25565端口将流量送到pod的25565端口kubectl -n lab port-forward minecraft-test-minecraft-7cb4f8df9b-jnvwb --address 0.0.0.0 25565:25565[root@director minecraft]# kubectl -n lab port-forward minecraft-test-minecraft-67579d447-xcssb --address 0.0.0.0 25565:25565 Forwarding from 0.0.0.0:25565 -> 25565打开Minecraft客户端，输入当前主机Ip,即可访问 PS：如果不是默认25565端口号，请在Ip后面跟上你自己指定的端口号,即 :这时候，我们会发现服务器成功被发现，接下来就双击进入服务器吧！成功以survivor模式登入服务器,happy gaming:):4.2 补充说明:获取服务器op权限kubectl exec -it minecraft-test-minecraft-546d85dbff-mgtdj -n lab -- rcon-cli[root@director minecraft]# k exec -it minecraft-test-minecraft-5667f8bffd-j9p8g -n lab -- rcon-cli Defaulted container "minecraft-test-minecraft-mc-backup" out of: minecraft-test-minecraft-mc-backup, minecraft-test-minecraft > help --- Showing help page 1 of 10 (/help <page>) ---/advancement <grant|revoke|test> <player>/ban <name> [reason ...]/ban-ip <address|name> [reason ...]/banlist [ips|players]/blockdata <x> <y> <z> <dataTag>/clear [player] [item] [data] [maxCount] [dataTag]/clone <x1> <y1> <z1> <x2> <y2> <z2> <x> <y> <z> [maskMode] [cloneMode]Tip: Use the <tab> key while typing a command to auto-complete the command or its arguments > list There are 1/20 players online:wllo > op wllo Opped wllo > 上帝模式，起飞!关闭online mode如果是离线模式的话，记得把on-line mode设置为false，否则就会如下图登录失败:可以使用helm upgrade将更新好的values进行进行in-service重新部署helm upgrade -n lab --atomic minecraft-test -f minecraft-values.yaml minecraft-4.3.0.tgz[root@director minecraft]# helm upgrade -n lab --atomic minecraft-test -f minecraft-values.yaml minecraft-4.3.0.tgz Release "minecraft-test" has been upgraded. Happy Helming! NAME: minecraft-test LAST DEPLOYED: Mon Sep 26 22:30:05 2022 NAMESPACE: lab STATUS: deployed REVISION: 2 TEST SUITE: None NOTES: Get the IP address of your Minecraft server by running these commands in the same shell: export POD_NAME=$(kubectl get pods \ --namespace lab \ -l "component=minecraft-test-minecraft" \ -o jsonpath="{.items[0].metadata.name}") kubectl port-forward $POD_NAME 25565:25565 echo "Point your Minecraft client at 127.0.0.1:25565" [root@director minecraft]# k get pod -n lab NAME READY STATUS RESTARTS AGE minecraft-test-minecraft-5667f8bffd-4qq5n 2/2 Running 0 3m36s [root@director minecraft]# 4.2 通过service服务的Node port方式访问宿主机+高端口号进行访问，不再赘述5.结语上述部署方式只是一个快速的Minecraft部署的Demo演示，具体详细进阶，大家可以参考value文件进行深入部署研究，欢迎交流分享经验!这份chart包还有很多功能值得探索，比如，部署forge，spigot，paperspigot等服务端需要设置的参数，还有全栈大佬(作者)提供的mc-backup自动备份的功能(默认关闭)。也可以使用作者开发的其他项目进行搭配使用，比如docker版的bungeecord，mc-router，mc-monitor，以及k8s的helm chart版的minecraft-bedrock，rcon-web-adminminecraft-proxy.
Kubeadm部署Kubernetes 1.23.4完整过程-Centos7.6
1.预先主机环境设置1.设置主机名hostname，管理节点设置主机名为 master 。hostnamectl set-hostname master hostnamectl set-hostname node1 hostnamectl set-hostname node2需要设置其他主机名称时，可将 master 替换为正确的主机名node1、node2即可。2.编辑 /etc/hosts 文件，添加域名解析。#这里一般是填内网地址映射 cat << EOF >>/etc/hosts < YOUR IP > master < YOUR IP > node1 < YOUR IP > node2 EOF3.关闭防火墙、selinux和swap。systemctl stop firewalld systemctl disable firewalld setenforce 0 sed -i "s/^SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config swapoff -a sed -i 's/.*swap.*/#&/' /etc/fstab4.配置内核参数，将桥接的IPv4流量传递到iptables的链# 创建 .conf 文件以在启动时加载模块 cat <<EOF | sudo tee /etc/modules-load.d/crio.conf overlay br_netfilter EOF sudo modprobe overlay sudo modprobe br_netfilter # 配置 sysctl 参数，这些配置在重启之后仍然起作用 cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-ip6tables = 1 EOF sudo sysctl --system5.依赖项要求(可选)==>部署KubeSphere需要KubeKey 可以一同安装 Kubernetes 和 KubeSphere。根据要安装的 Kubernetes 版本，需要安装的依赖项可能会不同。您可以参考下表，查看是否需要提前在节点上安装相关依赖项。依赖项Kubernetes 版本 ≥ 1.18Kubernetes 版本 < 1.18socat必须可选，但建议安装conntrack必须可选，但建议安装ebtables可选，但建议安装可选，但建议安装ipset可选，但建议安装可选，但建议安装yum -y install socat yum -y install conntrack yum -y install ebtables yum -y install ipset2.安装容器运行时（Container Runtime）容器运行环境是负责运行容器的软件。Kubernetes 支持多个容器运行环境: Docker、 containerd、CRI-O 以及任何实现 Kubernetes CRI (容器运行环境接口)。为了在 Pod 中运行容器，Kubernetes 使用 容器运行时（Container Runtime）。Linux 节点其它操作系统默认情况下，Kubernetes 使用 容器运行时接口（Container Runtime Interface，CRI） 来与你所选择的容器运行时交互。如果你不指定运行时，则 kubeadm 会自动尝试检测到系统上已经安装的运行时， 方法是扫描一组众所周知的 Unix 域套接字。 下面的表格列举了一些容器运行时及其对应的套接字路径：运行时域套接字Docker/var/run/dockershim.sockcontainerd/run/containerd/containerd.sockCRI-O/var/run/crio/crio.sock如果同时检测到 Docker 和 containerd，则优先选择 Docker。 这是必然的，因为 Docker 18.09 附带了 containerd 并且两者都是可以检测到的， 即使你仅安装了 Docker。 如果检测到其他两个或多个运行时，kubeadm 输出错误信息并退出。kubelet 通过内置的 dockershim CRI 实现与 Docker 集成。参阅容器运行时 以了解更多信息。选择1: CentOS Docker 安装Docker 支持以下的 64 位 CentOS 版本：CentOS 7CentOS 8更高版本...使用官方安装脚本自动安装1.安装命令如下：curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun也可以使用国内 daocloud 一键安装命令：curl -sSL https://get.daocloud.io/docker | sh配置镜像加速器和cgroup驱动配置 Docker 守护程序，尤其是使用 systemd 来管理容器的 cgroup。配置阿里云镜像加速地址2.您可以通过修改daemon配置文件/etc/docker/daemon.json来使用加速器sudo mkdir /etc/docker cat <<EOF | sudo tee /etc/docker/daemon.json { "registry-mirrors": ["https://<XXXX>.mirror.aliyuncs.com"], #填入个人阿里云加速地址 "exec-opts": ["native.cgroupdriver=systemd"], "log-driver": "json-file", "log-opts": { "max-size": "100m" }, "storage-driver": "overlay2" } EOF sudo systemctl enable docker sudo systemctl daemon-reload sudo systemctl restart docker选择2: CentOS CRI-O 安装本节包含安装 CRI-O 作为容器运行时的必要步骤。使用以下命令在系统中安装 CRI-O：说明：CRI-O 的主要以及次要版本必须与 Kubernetes 的主要和次要版本相匹配。 更多信息请查阅 CRI-O 兼容性列表。DebianUbuntuCentOSopenSUSE TumbleweedFedora在下列操作系统上安装 CRI-O, 使用下表中合适的值设置环境变量 OS:操作系统$OSCentos 8CentOS_8Centos 8 StreamCentOS_8_StreamCentos 7CentOS_7然后，将 $VERSION 设置为与你的 Kubernetes 相匹配的 CRI-O 版本。 例如，如果你要安装 CRI-O 1.20, 请设置 VERSION=1.20. 你也可以安装一个特定的发行版本。 例如要安装 1.20.0 版本，设置 VERSION=1.20:1.20.0.1.执行sudo curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable.repo https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/devel:kubic:libcontainers:stable.repo sudo curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo sudo yum install cri-o2.启动 CRI-O：sudo systemctl daemon-reload sudo systemctl enable crio --now参阅CRI-O 安装指南 了解进一步的详细信息。*cgroup 驱动(不推荐切换驱动)默认情况下，CRI-O 使用 systemd cgroup 驱动程序。要切换到 cgroupfs 驱动程序，或者编辑 / etc / crio / crio.conf 或放置一个插件 在 /etc/crio/crio.conf.d/02-cgroup-manager.conf 中的配置，例如：[crio.runtime] conmon_cgroup = "pod" cgroup_manager = "cgroupfs"另请注意更改后的 conmon_cgroup，将 CRI-O 与 cgroupfs 一起使用时， 必须将其设置为 pod。通常有必要保持 kubelet 的 cgroup 驱动程序配置 （通常透过 kubeadm 完成）和 CRI-O 一致。选择3: containerd本节包含使用 containerd 作为 CRI 运行时的必要步骤。使用以下命令在系统上安装 Containerd：1.安装 containerd:LinuxWindows (PowerShell)从官方Docker仓库安装 containerd.io 软件包。可以在 安装 Docker 引擎 中找到有关为各自的 Linux 发行版设置 Docker 存储库和安装 containerd.io 软件包的说明。2.配置 containerd：sudo mkdir -p /etc/containerd containerd config default | sudo tee /etc/containerd/config.toml3.重新启动 containerd:sudo systemctl restart containerd*使用 systemd cgroup 驱动程序结合 runc 使用 systemd cgroup 驱动，在 /etc/containerd/config.toml 中设置[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc] ... [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] SystemdCgroup = true如果您应用此更改，请确保再次重新启动 containerd：sudo systemctl restart containerd当使用 kubeadm 时，请手动配置 kubelet 的 cgroup 驱动3.安装 kubeadm、kubelet 和 kubectl1.你需要在每台机器上安装以下的软件包：kubeadm：用来初始化集群的指令。kubelet：在集群中的每个节点上用来启动 Pod 和容器等。kubectl：用来与集群通信的命令行工具。kubeadm 不能 帮你安装或者管理 kubelet 或 kubectl，所以你需要 确保它们与通过 kubeadm 安装的控制平面的版本相匹配。 如果不这样做，则存在发生版本偏差的风险，可能会导致一些预料之外的错误和问题。 然而，控制平面与 kubelet 间的相差一个次要版本不一致是支持的，但 kubelet 的版本不可以超过 API 服务器的版本。 例如，1.7.0 版本的 kubelet 可以完全兼容 1.8.0 版本的 API 服务器，反之则不可以。有关安装 kubectl 的信息，请参阅安装和设置 kubectl文档。警告：这些指南不包括系统升级时使用的所有 Kubernetes 程序包。这是因为 kubeadm 和 Kubernetes 有特殊的升级注意事项。关于版本偏差的更多信息，请参阅以下文档：Kubernetes 版本与版本间的偏差策略Kubeadm 特定的版本偏差策略基于 Debian 的发行版基于 Red Hat 的发行版无包管理器的情况#以下两个镜像源2选一 #国内镜像源(推荐) cat > /etc/yum.repos.d/kubernetes.repo << EOF [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 repo_gpgcheck=0 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF #官方镜像源 cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg exclude=kubelet kubeadm kubectl EOF # 安装 yum install -y kubelet kubeadm kubectl systemctl enable --now kubelet请注意：通过运行命令 setenforce 0 和 sed ... 将 SELinux 设置为 permissive 模式 可以有效地将其禁用。 这是允许容器访问主机文件系统所必需的，而这些操作时为了例如 Pod 网络工作正常。你必须这么做，直到 kubelet 做出对 SELinux 的支持进行升级为止。如果你知道如何配置 SELinux 则可以将其保持启用状态，但可能需要设定 kubeadm 不支持的部分配置kubelet 现在每隔几秒就会重启，因为它陷入了一个等待 kubeadm 指令的死循环。4.使用 kubeadm 创建集群Kubelet负责与其他节点集群通信，并进行本节点Pod和容器生命周期的管理。Kubeadm是Kubernetes的自动化部署工具，降低了部署难度，提高效率。Kubectl是Kubernetes集群管理工具。注：在master节点上进行如下操作1.在master进行Kubernetes集群初始化。kubeadm init \ --apiserver-advertise-address=< YOUR Master IP > \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version v1.23.4 \ --service-cidr=<Service ip range> \ --pod-network-cidr=<Pod i定义POD的网段为: 10.233.0.0/16， api server地址就是master本机IP地址。这一步很关键，由于kubeadm 默认从官网k8s.grc.io下载所需镜像，国内无法访问，因此需要通过–image-repository指定阿里云镜像仓库地址，很多新手初次部署都卡在此环节无法进行后续配置。集群初始化成功后返回如下信息：记录生成的最后部分内容，此内容需要在其它节点加入Kubernetes集群时执行。有关 kubeadm init 参数的更多信息，请参见 kubeadm 参考指南。要使用配置文件配置 kubeadm init 命令，请参见带配置文件使用 kubeadm init。要自定义控制平面组件，包括可选的对控制平面组件和 etcd 服务器的活动探针提供 IPv6 支持，请参阅自定义参数。要再次运行 kubeadm init，你必须首先卸载集群。如果将具有不同架构的节点加入集群， 请确保已部署的 DaemonSet 对这种体系结构具有容器镜像支持。kubeadm init 首先运行一系列预检查以确保机器 准备运行 Kubernetes。这些预检查会显示警告并在错误时退出。然后 kubeadm init 下载并安装集群控制平面组件。这可能会需要几分钟。 完成之后你应该看到：Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a Pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: /docs/concepts/cluster-administration/addons/ You can now join any number of machines by running the following on each node as root: kubeadm join <control-plane-host>:<control-plane-port> --token <token> --discovery-token-ca-cert-hash sha256:<hash>2.要使非 root 用户可以运行 kubectl，请运行以下命令， 它们也是 kubeadm init 输出的一部分：mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config或者，如果你是 root 用户，则可以运行：export KUBECONFIG=/etc/kubernetes/admin.conf警告：kubeadm 对 admin.conf 中的证书进行签名时，将其配置为 Subject: O = system:masters, CN = kubernetes-admin。 system:masters 是一个例外的、超级用户组，可以绕过鉴权层（例如 RBAC）。 不要将 admin.conf 文件与任何人共享，应该使用 kubeadm kubeconfig user 命令为其他用户生成 kubeconfig 文件，完成对他们的定制授权。记录 kubeadm init 输出的 kubeadm join 命令。 你需要此命令将节点加入集群。令牌用于控制平面节点和加入节点之间的相互身份验证。 这里包含的令牌是密钥。确保它的安全， 因为拥有此令牌的任何人都可以将经过身份验证的节点添加到你的集群中。 可以使用 kubeadm token 命令列出，创建和删除这些令牌。 请参阅 kubeadm 参考指南。5.安装 Pod 网络附加组件Install Calico on nodesBased on your datastore and number of nodes, select a link below to install Calico.Note: The option, Kubernetes API datastore, more than 50 nodes provides scaling using Typha daemon. Typha is not included for etcd because etcd already handles many clients so using Typha is redundant and not recommended.Install Calico with Kubernetes API datastore, 50 nodes or lessInstall Calico with Kubernetes API datastore, more than 50 nodesInstall Calico with etcd datastoreInstall Calico with Kubernetes API datastore, 50 nodes or lessDownload the Calico networking manifest for the Kubernetes API datastore.$ curl https://projectcalico.docs.tigera.io/manifests/calico.yaml -OIf you are using pod CIDR 192.168.0.0/16, skip to the next step. If you are using a different pod CIDR with kubeadm, no changes are required - Calico will automatically detect the CIDR based on the running configuration. For other platforms, make sure you uncomment the CALICO_IPV4POOL_CIDR variable in the manifest and set it to the same value as your chosen pod CIDR.Customize the manifest as necessary.Apply the manifest using the following command.$ kubectl apply -f calico.yamlInstall calicoctl(可选)Note: Make sure you always install the version of calicoctl that matches the version of Calico running on your cluster.Install calicoctl as a binary on a single hostInstall calicoctl as a kubectl plugin on a single hostInstall calicoctl as a container on a single hostInstall calicoctl as a Kubernetes podInstall calicoctl as a binary on a single hostLinuxMac OSXWindowsLinux PPC64leLinux arm64Log into the host, open a terminal prompt, and navigate to the location where you want to install the binary.Tip: Consider navigating to a location that’s in your PATH. For example, /usr/local/bin/.Use the following command to download the calicoctl binary.$ curl -L https://github.com/projectcalico/calico/releases/download/v3.22.1/calicoctl-linux-amd64 -o calicoctlSet the file to be executable.$ chmod +x ./calicoctlNote: If the location of calicoctl is not already in your PATH, move the file to one that is or add its location to your PATH. This will allow you to invoke it without having to prepend its location.6.补充(可选)1.控制平面节点隔离1.默认情况下，出于安全原因，你的集群不会在控制平面节点上调度 Pod。 如果你希望能够在控制平面节点上调度 Pod， 例如用于开发的单机 Kubernetes 集群，请运行：kubectl taint nodes --all node-role.kubernetes.io/master-输出看起来像：node "test-01" untainted taint "node-role.kubernetes.io/master:" not found taint "node-role.kubernetes.io/master:" not found这将从任何拥有 node-role.kubernetes.io/master taint 标记的节点中移除该标记， 包括控制平面节点，这意味着调度程序将能够在任何地方调度 Pods。2.启用 kubectl 自动补全KubeKey 不会启用 kubectl 自动补全功能，请参见以下内容并将其打开：请确保已安装 bash-autocompletion 并可以正常工作。# Install bash-completion yum -y install bash-completion source <(kubectl completion bash) echo "source <(kubectl completion bash)" >> ~/.bashrc7.Summary:kubernetes 1.23.4部署过程中遇到的问题:#部署过程中遇到如下错误，可以尝试重启主机 --apiserver-advertise-address=< YOUR IP > \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version v1.23.4 \ --service-cidr=10.233.64.0/18 \ --pod-network-cidr=10.233.0.0/18 [init] Using Kubernetes version: v1.23.4 [preflight] Running pre-flight checks error execution phase preflight: [preflight] Some fatal errors occurred: [ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables does not exist [ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1 [preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=... To see the stack trace of this error execute with --v=5 or higher#检查api server的地址和hostname是否填写错误以及容器运行时的服务是否正常 [root@master manifests]# kubeadm init \ --apiserver-advertise-address=< YOUR IP > \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version v1.23.4 \ --service-cidr=10.233.64.0/18 \ --pod-network-cidr=10.233.0.0/18 [init] Using Kubernetes version: v1.23.4 [preflight] Running pre-flight checks [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Generating "ca" certificate and key [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local master] and IPs [10.233.64.1 < YOUR IP >] [certs] Generating "apiserver-kubelet-client" certificate and key [certs] Generating "front-proxy-ca" certificate and key [certs] Generating "front-proxy-client" certificate and key [certs] Generating "etcd/ca" certificate and key [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [localhost master] and IPs [< YOUR IP > 127.0.0.1 ::1] [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [localhost master] and IPs [< YOUR IP > 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key [certs] Generating "sa" key and public key [kubeconfig] Using kubeconfig folder "/etc/kubernetes" [kubeconfig] Writing "admin.conf" kubeconfig file [kubeconfig] Writing "kubelet.conf" kubeconfig file [kubeconfig] Writing "controller-manager.conf" kubeconfig file [kubeconfig] Writing "scheduler.conf" kubeconfig file [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Starting the kubelet [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" [control-plane] Creating static Pod manifest for "kube-controller-manager" [control-plane] Creating static Pod manifest for "kube-scheduler" [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s [kubelet-check] Initial timeout of 40s passed. Unfortunately, an error has occurred: timed out waiting for the condition This error is likely caused by: - The kubelet is not running - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled) If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands: - 'systemctl status kubelet' - 'journalctl -xeu kubelet' Additionally, a control plane component may have crashed or exited when started by the container runtime. To troubleshoot, list all containers using your preferred container runtimes CLI. Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl: - 'crictl --runtime-endpoint /var/run/crio/crio.sock ps -a | grep kube | grep -v pause' Once you have found the failing container, you can inspect its logs with: - 'crictl --runtime-endpoint /var/run/crio/crio.sock logs CONTAINERID' error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster To see the stack trace of this error execute with --v=5 or higher#Coredns i/o timeout,暂无解决办法 [root@master typecho]# kubectl logs coredns-6d8c4cb4d-8rtnf -n kube-system .:53 [INFO] plugin/reload: Running configuration MD5 = db32ca3650231d74073ff4cf814959a7 CoreDNS-1.8.6 linux/amd64, go1.17.1, 13a9191 [ERROR] plugin/errors: 2 8583400323192522356.6928627731897922297. HINFO: read udp 10.233.38.131:41807->183.60.83.19:53: i/o timeout [ERROR] plugin/errors: 2 8583400323192522356.6928627731897922297. HINFO: read udp 10.233.38.131:48047->183.60.82.98:53: i/o timeout [ERROR] plugin/errors: 2 8583400323192522356.6928627731897922297. HINFO: read udp 10.233.38.131:37963->183.60.82.98:53: i/o timeout [ERROR] plugin/errors: 2 8583400323192522356.6928627731897922297. HINFO: read udp 10.233.38.131:59708->183.60.82.98:53: i/o timeout [ERROR] plugin/errors: 2 8583400323192522356.6928627731897922297. HINFO: read udp 10.233.38.131:34816->183.60.82.98:53: i/o timeout [ERROR] plugin/errors: 2 8583400323192522356.6928627731897922297. HINFO: read udp 10.233.38.131:55215->183.60.83.19:53: i/o timeout [ERROR] plugin/errors: 2 8583400323192522356.6928627731897922297. HINFO: read udp 10.233.38.131:58251->183.60.82.98:53: i/o timeout [ERROR] plugin/errors: 2 8583400323192522356.6928627731897922297. HINFO: read udp 10.233.38.131:47995->183.60.83.19:53: i/o timeout [ERROR] plugin/errors: 2 8583400323192522356.6928627731897922297. HINFO: read udp 10.233.38.131:37868->183.60.82.98:53: i/o timeout [ERROR] plugin/errors: 2 8583400323192522356.6928627731897922297. HINFO: read udp 10.233.38.131:44159->183.60.82.98:53: i/o timeout