欢迎光临
我的个人博客网站

容器编排系统K8s之crd资源

  前文我们了解了k8s节点污点和pod的对节点污点容忍度相关话题,回顾请参考:https://www.cnblogs.com/qiuhom-1874/p/14255486.html;今天我们来聊一下扩展k8s相关话题;

  k8s上创建资源对象的过程

  我们知道在k8s上,资源是有类型的,不同类型的资源,其定义方式和使用的字段各不相同;用户创建一个资源,实际上就是把k8s抽象的资源做实例化,即把k8s抽象出来的资源,通过资源清单赋值,创建出来的对象就是我们实例化对应类型资源的结果;用户创建一个资源,首先会把请求发送给apiserver,通过apiserver的认证、授权、准入控制以后,对应创建资源的定义就存放在etcd中,控制器通过watch机制监视apiserver上的资源变动,通过对应资源变动事件触发对应类型资源的控制器将对应资源创建出来,并通过控制器内部的和解循环监控着对应资源状态是否和用户定义的期望状态一样,如果发现不一样,内部和解循环就会被触发,对应控制器会向apiserver发起创建资源的请求,将对应资源重建,让对应资源的状态始终满足用户期望的状态;从上述的过程来看,用户创建一个资源分两个步骤,第一步是将对应请求发送给apiserver,通过apiserver把对应资源定义的信息存放在etcd中;第二个步骤是对应资源类型的控制器通过apiserver从etcd中读取对应资源的定义,将其创建出来;对于etcd来说,它本是就一个kv数据库,可以存储任意类型的kv数据,但在在k8s上,apiserver将不同类型的资源定义抽象成不同的资源,使得用户创建对应资源,必须是满足对应类型资源定义的规范,然后将规范的定义存放在etcd中;简单讲apiserver就是把用户存入etcd中的数据做了一层抽象,使得用户不能够随意将任意数据存储到etcd中,存入etcd中的数据必须是满足对应apiserver接口定义的规范;这就好比我们在使用mysql数据库时,必须遵守对应库中的对应表的定义;

  在k8s上创建自定义资源类型

  在k8s上,资源的类型有很多,比如pod,service,PersistentVolume,PersistentVolumeClaim等等,这些都是一些基础的资源类型;我们要创建某种资源,直接使用对应的资源类型,实例化一个对象即可;假如我们要在k8s上创建一个集群,我们是不是可以直接使用某种类型的资源,实例化一个集群对象呢?理论上是可以的,但是前提是对应k8s上有对应类型的资源;有对应类型的资源,用户就可以把对应创建资源的定义存放在etcd中;除了有对应类型的资源,我们还需要有对应的控制器将对应资源创建出来;这样一来对于不同集群或应用来说,其组织方式和逻辑都不同,使用的资源类型和控制器也有所不同;用户要想实例化更高级的资源来,就必须自己手动实现定义其资源类型,将对应资源类型实例化为对象;除此之外必要时还需要自己实现对应资源的控制器;简单讲用户想要实现更高级的资源类型,就必须扩展现有k8s的资源类型和控制器;

  在k8s上扩展资源类型的方式有三种,第一种是crd,crd是k8s内建的资源类型,该类型资源主要用来创建用户自定义资源类型的资源;即通过crd资源,可以将用户自定义资源类型转换为k8s上资源类型;第二种是自定义apiserver;这种方式要比第一种方式要复杂一点,需要用户手动开发程序实现对应功能的apiserver,让其用户创建自定义类型资源能够通过自定义apiserver实现;第三种方式就是修改现有k8sapiserver,让其支持对应用户自定义资源类型;

  自定义控制器

  自定义资源类型我们可以使用crd资源实现,也可以使用自定义apiserver或修改原有apiserver代码实现,但是只有资源类型是不能够让对应自定义类型资源实例化为一个自定义资源对象,只有自定义资源类型,用户创建对应资源类型的资源对象时,只能把对应资源类型的定义信息写入到etcd中,它不能真正的跑起来,要想真正的跑起来,我们还需要一个自定义控制器,专门负责监听对应的资源类型的资源变化,将对应资源实例化为对应k8s上的资源对象;当然不是所有的自定义类型的资源都需要自定义控制器,如果对应自定义类型资源调用了底层基础控制器来管控对应自定义资源,那么对应自定义类型资源就不需要使用自定义控制器;我们知道控制器是k8s上的一个重要组件,它的工作逻辑是注册监听在apiserver上对应类型的资源变动,如果对应资源状态不满足用户期望状态,它就会根据内部的和解循环来请求apiserver将对应类型资源的定义发送给它,然后根据资源定义来重建对应的资源,让其状态始终和用户期望的状态保持一致;自定义控制器也是同样的逻辑,使用自定义控制器的目的也是让对应自定义类型资源能够被自定义控制器监听,一旦对应资源发生变动,它能够将其在k8s上创建出来,并一直保持和用户期望的状态吻合;自定义控制器和自定义资源类型可以分开实现,也可以合并在一起实现,即自定义控制器程序能够自动创建crd资源,让其对应自定义类型资源能够被k8s识别并将其创建出来;具体是分开实现还是合并在一起实现,取决开发自定义控制器程序员;

  crd资源定义帮助

[root@master01 ~]# kubectl explain crd KIND:     CustomResourceDefinition VERSION:  apiextensions.k8s.io/v1  DESCRIPTION:      CustomResourceDefinition represents a resource that should be exposed on      the API server. Its name MUST be in the format <.spec.name>.<.spec.group>.  FIELDS:    apiVersion   <string>      APIVersion defines the versioned schema of this representation of an      object. Servers should convert recognized schemas to the latest internal      value, and may reject unrecognized values. More info:      https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources     kind <string>      Kind is a string value representing the REST resource this object      represents. Servers may infer this from the endpoint the client submits      requests to. Cannot be updated. In CamelCase. More info:      https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds     metadata     <Object>     spec <Object> -required-      spec describes how the user wants the resources to appear     status       <Object>      status indicates the actual state of the CustomResourceDefinition  [root@master01 ~]# 

  提示:crd资源是k8s上的标准资源之一,它的定义主要有apiVersion,kind,metadata,spec和status;其中kind类型为CustomResourceDefinition,apiVersion是apiextensions.k8s.io/v1;这两个是固定格式;spec字段是用来定义对应指定资源类型资源的相关属性;

  crd.spec字段说明

[root@master01 ~]# kubectl explain crd.spec KIND:     CustomResourceDefinition VERSION:  apiextensions.k8s.io/v1  RESOURCE: spec <Object>  DESCRIPTION:      spec describes how the user wants the resources to appear       CustomResourceDefinitionSpec describes how a user wants their resource to      appear  FIELDS:    conversion   <Object>      conversion defines conversion settings for the CRD.     group        <string> -required-      group is the API group of the defined custom resource. The custom resources      are served under `/apis/<group>/...`. Must match the name of the      CustomResourceDefinition (in the form `<names.plural>.<group>`).     names        <Object> -required-      names specify the resource and kind names for the custom resource.     preserveUnknownFields        <boolean>      preserveUnknownFields indicates that object fields which are not specified      in the OpenAPI schema should be preserved when persisting to storage.      apiVersion, kind, metadata and known fields inside metadata are always      preserved. This field is deprecated in favor of setting      `x-preserve-unknown-fields` to true in      `spec.versions[*].schema.openAPIV3Schema`. See      https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#pruning-versus-preserving-unknown-fields      for details.     scope        <string> -required-      scope indicates whether the defined custom resource is cluster- or      namespace-scoped. Allowed values are `Cluster` and `Namespaced`.     versions     <[]Object> -required-      versions is the list of all API versions of the defined custom resource.      Version names are used to compute the order in which served versions are      listed in API discovery. If the version string is "kube-like", it will sort      above non "kube-like" version strings, which are ordered lexicographically.      "Kube-like" versions start with a "v", then are followed by a number (the      major version), then optionally the string "alpha" or "beta" and another      number (the minor version). These are sorted first by GA > beta > alpha      (where GA is a version with no suffix such as beta or alpha), and then by      comparing major version, then minor version. An example sorted list of      versions: v10, v2, v1, v11beta2, v10beta3, v3beta1, v12alpha1, v11alpha2,      foo1, foo10.  [root@master01 ~]#  

  提示:crd.spec中group字段使用来描述对应自定义类型资源的群组名称,其值为字符串;names字段是用来描述自定义类型资源对应类型,名称等等,其值为一个对象;scope字段用来定义对应自定义资源是那个级别的资源;该字段的值只能为Cluster或Namespaced;versions字段是用来指定对应自定义资源的版本,以及对应类型资源的属性字段等信息,该字段为一个列表对象;

  示例:定义一个自定义类型资源

[root@master01 ~]# cat crontab-crd.yaml apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata:   # 名字必需与下面的 spec 字段匹配,并且格式为 '<名称的复数形式>.<组名>'   name: crontabs.stable.example.com spec:   # 组名称,用于 REST API: /apis/<组>/<版本>   group: stable.example.com   # 列举此 CustomResourceDefinition 所支持的版本   versions:     - name: v1       # 每个版本都可以通过 served 标志来独立启用或禁止       served: true       # 其中一个且只有一个版本必需被标记为存储版本       storage: true       schema:         openAPIV3Schema:           type: object           properties:             spec:               type: object               properties:                 cronSpec:                   type: string                 image:                   type: string                 replicas:                   type: integer   # 可以是 Namespaced 或 Cluster   scope: Namespaced   names:     # 名称的复数形式,用于 URL:/apis/<组>/<版本>/<名称的复数形式>     plural: crontabs     # 名称的单数形式,作为命令行使用时和显示时的别名     singular: crontab     # kind 通常是单数形式的驼峰编码(CamelCased)形式。你的资源清单会使用这一形式。     kind: CronTab     # shortNames 允许你在命令行使用较短的字符串来匹配资源     shortNames:     - ct [root@master01 ~]#  

  应用资源清单前使用kubectl get crontab

[root@master01 ~]# kubectl get crontab error: the server doesn't have a resource type "crontab" [root@master01 ~]#  

  提示:在没有应用资源清单前使用kubectl get crontab,它会提示我们所没有类型为crontab的资源

  应用资源清单

[root@master01 ~]# kubectl apply -f crontab-crd.yaml customresourcedefinition.apiextensions.k8s.io/crontabs.stable.example.com created [root@master01 ~]# kubectl get crontab No resources found in default namespace. [root@master01 ~]#  

  提示:应用资源清单后,再次使用kubectl get crontab就没有报错了,只是提示在default名称空间没有对应类型资源;

  查看crd

[root@master01 ~]# kubectl get crd NAME                                                  CREATED AT bgpconfigurations.crd.projectcalico.org               2021-01-03T15:49:21Z bgppeers.crd.projectcalico.org                        2021-01-03T15:49:21Z blockaffinities.crd.projectcalico.org                 2021-01-03T15:49:21Z clusterinformations.crd.projectcalico.org             2021-01-03T15:49:21Z crontabs.stable.example.com                           2021-01-12T12:39:00Z felixconfigurations.crd.projectcalico.org             2021-01-03T15:49:21Z globalnetworkpolicies.crd.projectcalico.org           2021-01-03T15:49:21Z globalnetworksets.crd.projectcalico.org               2021-01-03T15:49:21Z hostendpoints.crd.projectcalico.org                   2021-01-03T15:49:21Z ipamblocks.crd.projectcalico.org                      2021-01-03T15:49:21Z ipamconfigs.crd.projectcalico.org                     2021-01-03T15:49:21Z ipamhandles.crd.projectcalico.org                     2021-01-03T15:49:21Z ippools.crd.projectcalico.org                         2021-01-03T15:49:21Z kubecontrollersconfigurations.crd.projectcalico.org   2021-01-03T15:49:21Z networkpolicies.crd.projectcalico.org                 2021-01-03T15:49:21Z networksets.crd.projectcalico.org                     2021-01-03T15:49:22Z [root@master01 ~]# kubectl get crd/crontabs.stable.example.com NAME                          CREATED AT crontabs.stable.example.com   2021-01-12T12:39:00Z [root@master01 ~]#  

  查看详情

[root@master01 ~]# kubectl get crd/crontabs.stable.example.com NAME                          CREATED AT crontabs.stable.example.com   2021-01-12T12:39:00Z [root@master01 ~]# kubectl describe crd/crontabs.stable.example.com Name:         crontabs.stable.example.com Namespace:     Labels:       <none> Annotations:  <none> API Version:  apiextensions.k8s.io/v1 Kind:         CustomResourceDefinition Metadata:   Creation Timestamp:  2021-01-12T12:39:00Z   Generation:          1   Managed Fields:     API Version:  apiextensions.k8s.io/v1     Fields Type:  FieldsV1     fieldsV1:       f:status:         f:acceptedNames:           f:kind:           f:listKind:           f:plural:           f:shortNames:           f:singular:         f:conditions:     Manager:      kube-apiserver     Operation:    Update     Time:         2021-01-12T12:39:00Z     API Version:  apiextensions.k8s.io/v1     Fields Type:  FieldsV1     fieldsV1:       f:metadata:         f:annotations:           .:           f:kubectl.kubernetes.io/last-applied-configuration:       f:spec:         f:conversion:           .:           f:strategy:         f:group:         f:names:           f:kind:           f:listKind:           f:plural:           f:shortNames:           f:singular:         f:scope:         f:versions:       f:status:         f:storedVersions:     Manager:         kubectl-client-side-apply     Operation:       Update     Time:            2021-01-12T12:39:00Z   Resource Version:  805506   UID:               b92a90f4-c953-4876-a496-030c9ba023fd Spec:   Conversion:     Strategy:  None   Group:       stable.example.com   Names:     Kind:       CronTab     List Kind:  CronTabList     Plural:     crontabs     Short Names:       ct     Singular:  crontab   Scope:       Namespaced   Versions:     Name:  v1     Schema:       openAPIV3Schema:         Properties:           Spec:             Properties:               Cron Spec:                 Type:  string               Image:                 Type:  string               Replicas:                 Type:  integer             Type:      object         Type:          object     Served:            true     Storage:           true Status:   Accepted Names:     Kind:       CronTab     List Kind:  CronTabList     Plural:     crontabs     Short Names:       ct     Singular:  crontab   Conditions:     Last Transition Time:  2021-01-12T12:39:00Z     Message:               no conflicts found     Reason:                NoConflicts     Status:                True     Type:                  NamesAccepted     Last Transition Time:  2021-01-12T12:39:00Z     Message:               the initial names have been accepted     Reason:                InitialNamesAccepted     Status:                True     Type:                  Established   Stored Versions:     v1 Events:  <none> [root@master01 ~]# 

  使用自定义资源类型crontab创建资源

[root@master01 ~]# cat my-crontab.yaml apiVersion: "stable.example.com/v1" kind: CronTab metadata:   name: my-new-cron-object spec:   cronSpec: "* * * * */5"   image: my-awesome-cron-image [root@master01 ~]#  

  提示:上述资源清单表示创建一个类型为Crontab的资源,该资源的群组版本为stable.example.com/v1;

  应用资源清单

[root@master01 ~]# kubectl apply -f my-crontab.yaml crontab.stable.example.com/my-new-cron-object created [root@master01 ~]# kubectl get ct NAME                 AGE my-new-cron-object   5s [root@master01 ~]# kubectl describe ct/my-new-cron-object Name:         my-new-cron-object Namespace:    default Labels:       <none> Annotations:  <none> API Version:  stable.example.com/v1 Kind:         CronTab Metadata:   Creation Timestamp:  2021-01-12T12:45:29Z   Generation:          1   Managed Fields:     API Version:  stable.example.com/v1     Fields Type:  FieldsV1     fieldsV1:       f:metadata:         f:annotations:           .:           f:kubectl.kubernetes.io/last-applied-configuration:       f:spec:         .:         f:cronSpec:         f:image:     Manager:         kubectl-client-side-apply     Operation:       Update     Time:            2021-01-12T12:45:29Z   Resource Version:  806182   UID:               31a88a3d-fa99-42b8-80f6-3e4559efdc40 Spec:   Cron Spec:  * * * * */5   Image:      my-awesome-cron-image Events:       <none> [root@master01 ~]#  

  提示:可以看到对应类型资源已经创建成功;以上示例只是单纯的crd的使用示例,没有任何实质的作用;

  部署自定义控制器

  示例:部署mongodb-aperator

  1、克隆项目

[root@master01 ~]# git clone https://github.com/mongodb/mongodb-kubernetes-operator.git Cloning into 'mongodb-kubernetes-operator'... remote: Enumerating objects: 95, done. remote: Counting objects: 100% (95/95), done. remote: Compressing objects: 100% (74/74), done. remote: Total 4506 (delta 30), reused 60 (delta 15), pack-reused 4411 Receiving objects: 100% (4506/4506), 18.04 MiB | 183.00 KiB/s, done. Resolving deltas: 100% (2621/2621), done. [root@master01 ~]#  

  2、创建名称空间mongodb,并进入到mongodb-kubernetes-operator目录应用crd资源,创建自定义资源类型

[root@master01 mongodb-kubernetes-operator]# kubectl create ns mongodb namespace/mongodb created [root@master01 mongodb-kubernetes-operator]# kubectl get ns NAME                   STATUS   AGE default                Active   35d ingress-nginx          Active   22d kube-node-lease        Active   35d kube-public            Active   35d kube-system            Active   35d kubernetes-dashboard   Active   11d mongodb                Active   4s [root@master01 mongodb-kubernetes-operator]# ls agent    build  deploy     docs    go.sum      pkg        release.json      scripts  testdata  version APACHE2  cmd    dev_notes  go.mod  LICENSE.md  README.md  requirements.txt  test     tools.go [root@master01 mongodb-kubernetes-operator]# kubectl apply -f deploy/crds/mongodb.com_mongodb_crd.yaml -n mongodb Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition customresourcedefinition.apiextensions.k8s.io/mongodb.mongodb.com created [root@master01 mongodb-kubernetes-operator]#  

  验证:查看mongodb类型资源是否已经创建成功?

[root@master01 mongodb-kubernetes-operator]# kubectl get crd NAME                                                  CREATED AT bgpconfigurations.crd.projectcalico.org               2021-01-03T15:49:21Z bgppeers.crd.projectcalico.org                        2021-01-03T15:49:21Z blockaffinities.crd.projectcalico.org                 2021-01-03T15:49:21Z clusterinformations.crd.projectcalico.org             2021-01-03T15:49:21Z crontabs.stable.example.com                           2021-01-12T12:39:00Z felixconfigurations.crd.projectcalico.org             2021-01-03T15:49:21Z globalnetworkpolicies.crd.projectcalico.org           2021-01-03T15:49:21Z globalnetworksets.crd.projectcalico.org               2021-01-03T15:49:21Z hostendpoints.crd.projectcalico.org                   2021-01-03T15:49:21Z ipamblocks.crd.projectcalico.org                      2021-01-03T15:49:21Z ipamconfigs.crd.projectcalico.org                     2021-01-03T15:49:21Z ipamhandles.crd.projectcalico.org                     2021-01-03T15:49:21Z ippools.crd.projectcalico.org                         2021-01-03T15:49:21Z kubecontrollersconfigurations.crd.projectcalico.org   2021-01-03T15:49:21Z mongodb.mongodb.com                                   2021-01-13T06:38:22Z networkpolicies.crd.projectcalico.org                 2021-01-03T15:49:21Z networksets.crd.projectcalico.org                     2021-01-03T15:49:22Z [root@master01 mongodb-kubernetes-operator]# kubectl get crd/mongodb.mongodb.com NAME                  CREATED AT mongodb.mongodb.com   2021-01-13T06:38:22Z [root@master01 mongodb-kubernetes-operator]# 

  3、安装operator

[root@master01 mongodb-kubernetes-operator]# kubectl apply -f deploy/operator/ -n mongodb deployment.apps/mongodb-kubernetes-operator created role.rbac.authorization.k8s.io/mongodb-kubernetes-operator created rolebinding.rbac.authorization.k8s.io/mongodb-kubernetes-operator created serviceaccount/mongodb-kubernetes-operator created [root@master01 mongodb-kubernetes-operator]#  

  提示:mongodb-kubernetes-operator这个项目是将自定义控制器和自定义资源类型分开实现的;其operator只负责创建和监听对应资源类型的变化,在资源有变化时,实例化为对应资源对象,并保持对应资源对象状态吻合用户期望状态;上述四个清单中主要是创建了一个sa账户,并对对应的sa用户授权;

  operator.yaml内容

[root@master01 mongodb-kubernetes-operator]# cat deploy/operator/operator.yaml  apiVersion: apps/v1 kind: Deployment metadata:   name: mongodb-kubernetes-operator spec:   replicas: 1   selector:     matchLabels:       name: mongodb-kubernetes-operator   template:     metadata:       labels:         name: mongodb-kubernetes-operator     spec:       serviceAccountName: mongodb-kubernetes-operator       containers:         - name: mongodb-kubernetes-operator           image: quay.io/mongodb/mongodb-kubernetes-operator:0.3.0           command:             - mongodb-kubernetes-operator           imagePullPolicy: Always           env:             - name: WATCH_NAMESPACE               valueFrom:                 fieldRef:                   fieldPath: metadata.namespace             - name: POD_NAME               valueFrom:                 fieldRef:                   fieldPath: metadata.name             - name: OPERATOR_NAME               value: "mongodb-kubernetes-operator"             - name: AGENT_IMAGE # The MongoDB Agent the operator will deploy to manage MongoDB deployments               value: quay.io/mongodb/mongodb-agent:10.19.0.6562-1             - name: VERSION_UPGRADE_HOOK_IMAGE               value: quay.io/mongodb/mongodb-kubernetes-operator-version-upgrade-post-start-hook:1.0.2             - name: MONGODB_IMAGE               value: "library/mongo"             - name: MONGODB_REPO_URL               value: "registry.hub.docker.com" [root@master01 mongodb-kubernetes-operator]#  

  提示:上述资源主要用deploy控制器运行对应自定义控制器为一个pod;

  验证:查看operator是否正常运行

[root@master01 mongodb-kubernetes-operator]# kubectl get pods -n mongodb NAME                                           READY   STATUS    RESTARTS   AGE mongodb-kubernetes-operator-7d557bcc95-th8js   1/1     Running   0          26s [root@master01 mongodb-kubernetes-operator]#  

  提示:能够看到operator正常运行,就表示operator已经安装成功;

  验证:使用自定义资源类型创建一个mongodb 副本集集群

[root@master01 mongodb-kubernetes-operator]# cat deploy/crds/mongodb.com_v1_mongodb_cr.yaml  --- apiVersion: mongodb.com/v1 kind: MongoDB metadata:   name: example-mongodb spec:   members: 3   type: ReplicaSet   version: "4.2.6"   security:     authentication:       modes: ["SCRAM"]   users:     - name: my-user       db: admin       passwordSecretRef: # a reference to the secret that will be used to generate the user's password         name: my-user-password       roles:         - name: clusterAdmin           db: admin         - name: userAdminAnyDatabase           db: admin       scramCredentialsSecretName: my-scram  # the user credentials will be generated from this secret # once the credentials are generated, this secret is no longer required --- apiVersion: v1 kind: Secret metadata:   name: my-user-password type: Opaque stringData:   password: 58LObjiMpxcjP1sMDW [root@master01 mongodb-kubernetes-operator]# kubectl apply -f deploy/crds/mongodb.com_v1_mongodb_cr.yaml mongodb.mongodb.com/example-mongodb created secret/my-user-password created [root@master01 mongodb-kubernetes-operator]#  

  应用清单

[root@master01 mongodb-kubernetes-operator]# kubectl apply -f deploy/crds/mongodb.com_v1_mongodb_cr.yaml -n mongodb mongodb.mongodb.com/example-mongodb created secret/my-user-password created [root@master01 mongodb-kubernetes-operator]# kubectl get pods -n mongodb NAME                                           READY   STATUS    RESTARTS   AGE example-mongodb-0                              0/2     Pending   0          9s mongodb-kubernetes-operator-7d557bcc95-th8js   1/1     Running   0          88s [root@master01 mongodb-kubernetes-operator]#  

  提示:这里可以看到对应pod处于pending状态;

  查看pod详细信息

[root@master01 mongodb-kubernetes-operator]# kubectl describe pod/example-mongodb-0 -n mongodb|grep -A 10 "Events" Events:   Type     Reason            Age                From               Message   ----     ------            ----               ----               -------   Warning  FailedScheduling  66s (x2 over 66s)  default-scheduler  0/5 nodes are available: 5 pod has unbound immediate PersistentVolumeClaims. [root@master01 mongodb-kubernetes-operator]#  

  提示:这里提示没有可以用的pvc;

  删除mongodb名称空间下pvc

[root@master01 mongodb-kubernetes-operator]# kubectl get pvc -n mongodb NAME                            STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE data-volume-example-mongodb-0   Pending                                                     92s [root@master01 mongodb-kubernetes-operator]# kubectl delete pvc --all -n mongodb persistentvolumeclaim "data-volume-example-mongodb-0" deleted [root@master01 mongodb-kubernetes-operator]# kubectl get pvc -n mongodb No resources found in mongodb namespace. [root@master01 mongodb-kubernetes-operator]#  

  创建pv和pvc

[root@master01 mongodb-kubernetes-operator]# cd [root@master01 ~]# cat pv-demo.yaml  apiVersion: v1 kind: PersistentVolume metadata:   name: nfs-pv-v1   labels:     app: example-mongodb-svc spec:   capacity:     storage: 1Gi   volumeMode: Filesystem   accessModes: ["ReadWriteOnce","ReadWriteMany","ReadOnlyMany"]   persistentVolumeReclaimPolicy: Retain   mountOptions:   - hard   - nfsvers=4.1   nfs:     path: /data/v1     server: 192.168.0.99 --- apiVersion: v1 kind: PersistentVolume metadata:   name: nfs-pv-v2   labels:     app: example-mongodb-svc spec:   capacity:     storage: 1Gi   volumeMode: Filesystem   accessModes: ["ReadWriteOnce","ReadWriteMany","ReadOnlyMany"]   persistentVolumeReclaimPolicy: Retain   mountOptions:   - hard   - nfsvers=4.1   nfs:     path: /data/v2     server: 192.168.0.99 ---  apiVersion: v1 kind: PersistentVolume metadata:   name: nfs-pv-v3   labels:     app: example-mongodb-svc spec:   capacity:     storage: 1Gi   volumeMode: Filesystem   accessModes: ["ReadWriteOnce","ReadWriteMany","ReadOnlyMany"]   persistentVolumeReclaimPolicy: Retain   mountOptions:   - hard   - nfsvers=4.1   nfs:     path: /data/v3     server: 192.168.0.99 [root@master01 ~]#  

  应用清单创建pv

[root@master01 ~]# kubectl apply -f pv-demo.yaml persistentvolume/nfs-pv-v1 created persistentvolume/nfs-pv-v2 created persistentvolume/nfs-pv-v3 created [root@master01 ~]# kubectl get pv NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE nfs-pv-v1   1Gi        RWO,ROX,RWX    Retain           Available                                   3s nfs-pv-v2   1Gi        RWO,ROX,RWX    Retain           Available                                   3s nfs-pv-v3   1Gi        RWO,ROX,RWX    Retain           Available                                   3s [root@master01 ~]#  

  创建pvc清单

[root@master01 ~]# cat pvc-demo.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata:   name: data-volume-example-mongodb-0    namespace: mongodb spec:   accessModes:     - ReadWriteMany   volumeMode: Filesystem   resources:     requests:       storage: 500Mi --- apiVersion: v1 kind: PersistentVolumeClaim metadata:   name: data-volume-example-mongodb-1   namespace: mongodb spec:   accessModes:     - ReadWriteMany   volumeMode: Filesystem   resources:     requests:       storage: 500Mi --- apiVersion: v1 kind: PersistentVolumeClaim metadata:   name: data-volume-example-mongodb-2   namespace: mongodb spec:   accessModes:     - ReadWriteMany   volumeMode: Filesystem   resources:     requests:       storage: 500Mi [root@master01 ~]#  

  应用清单创建pvc

[root@master01 ~]# kubectl get pvc -n mongodb No resources found in mongodb namespace. [root@master01 ~]# kubectl apply -f pvc-demo.yaml persistentvolumeclaim/data-volume-example-mongodb-0 created persistentvolumeclaim/data-volume-example-mongodb-1 created persistentvolumeclaim/data-volume-example-mongodb-2 created [root@master01 ~]# kubectl get pvc -n mongodb     NAME                            STATUS   VOLUME      CAPACITY   ACCESS MODES   STORAGECLASS   AGE data-volume-example-mongodb-0   Bound    nfs-pv-v1   1Gi        RWO,ROX,RWX                   6s data-volume-example-mongodb-1   Bound    nfs-pv-v2   1Gi        RWO,ROX,RWX                   6s data-volume-example-mongodb-2   Bound    nfs-pv-v3   1Gi        RWO,ROX,RWX                   6s [root@master01 ~]# kubectl get pv NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                   STORAGECLASS   REASON   AGE nfs-pv-v1   1Gi        RWO,ROX,RWX    Retain           Bound    mongodb/data-volume-example-mongodb-0                           102s nfs-pv-v2   1Gi        RWO,ROX,RWX    Retain           Bound    mongodb/data-volume-example-mongodb-1                           102s nfs-pv-v3   1Gi        RWO,ROX,RWX    Retain           Bound    mongodb/data-volume-example-mongodb-2                           102s [root@master01 ~]#  

  提示:可以看到对应pvc和pv已经绑定好了;

  验证:查看mongodb副本集集群是否跑起来了?

[root@master01 ~]# kubectl get pods -n mongodb NAME                                           READY   STATUS            RESTARTS   AGE example-mongodb-0                              2/2     Running           0          6m19s example-mongodb-1                              0/2     PodInitializing   0          10s mongodb-kubernetes-operator-7d557bcc95-th8js   1/1     Running           0          7m38s [root@master01 ~]# kubectl get pods -n mongodb -w NAME                                           READY   STATUS    RESTARTS   AGE example-mongodb-0                              2/2     Running   0          6m35s example-mongodb-1                              1/2     Running   0          26s mongodb-kubernetes-operator-7d557bcc95-th8js   1/1     Running   0          7m54s example-mongodb-1                              2/2     Running   0          43s example-mongodb-2                              0/2     Pending   0          0s example-mongodb-2                              0/2     Pending   0          0s example-mongodb-2                              0/2     Init:0/1   0          0s example-mongodb-2                              0/2     Init:0/1   0          1s example-mongodb-2                              0/2     Terminating   0          4s example-mongodb-2                              0/2     Terminating   0          6s example-mongodb-2                              0/2     Terminating   0          20s example-mongodb-2                              0/2     Terminating   0          20s example-mongodb-2                              0/2     Pending       0          0s example-mongodb-2                              0/2     Pending       0          0s example-mongodb-2                              0/2     Init:0/1      0          0s example-mongodb-2                              0/2     Init:0/1      0          1s example-mongodb-2                              0/2     PodInitializing   0          7s example-mongodb-2                              1/2     Running           0          14s example-mongodb-2                              2/2     Running           0          36s ^C[root@master01 ~]# kubectl get pods -n mongodb  NAME                                           READY   STATUS    RESTARTS   AGE example-mongodb-0                              2/2     Running   0          8m example-mongodb-1                              2/2     Running   0          111s example-mongodb-2                              2/2     Running   0          48s mongodb-kubernetes-operator-7d557bcc95-th8js   1/1     Running   0          9m19s [root@master01 ~]#  

  提示:可以看到对应的pod已经正常跑起来了;

  验证:使用mongo 连接mongodbpod,看看对应副本集集群是否工作正常?

[root@master01 ~]# kubectl get pods -n mongodb -o wide NAME                                           READY   STATUS    RESTARTS   AGE     IP             NODE             NOMINATED NODE   READINESS GATES example-mongodb-0                              2/2     Running   0          9m12s   10.244.4.101   node04.k8s.org   <none>           <none> example-mongodb-1                              2/2     Running   0          3m3s    10.244.2.130   node02.k8s.org   <none>           <none> example-mongodb-2                              2/2     Running   0          2m      10.244.1.130   node01.k8s.org   <none>           <none> mongodb-kubernetes-operator-7d557bcc95-th8js   1/1     Running   0          10m     10.244.3.116   node03.k8s.org   <none>           <none> [root@master01 ~]# mongo 10.244.4.101 MongoDB shell version v4.4.3 connecting to: mongodb://10.244.4.101:27017/test?compressors=disabled&gssapiServiceName=mongodb Implicit session: session { "id" : UUID("b9d16fe9-6a74-4638-96e6-70aaf3c83bfa") } MongoDB server version: 4.2.6 WARNING: shell and server versions do not match example-mongodb:PRIMARY> show dbs example-mongodb:PRIMARY> db.auth('my-user','58LObjiMpxcjP1sMDW') Error: Authentication failed. 0 example-mongodb:PRIMARY> use admin switched to db admin example-mongodb:PRIMARY> db.auth('my-user','58LObjiMpxcjP1sMDW') 1 example-mongodb:PRIMARY> show dbs admin   0.000GB config  0.000GB local   0.000GB example-mongodb:PRIMARY> db.isMaster() {         "hosts" : [                 "example-mongodb-0.example-mongodb-svc.mongodb.svc.cluster.local:27017",                 "example-mongodb-1.example-mongodb-svc.mongodb.svc.cluster.local:27017",                 "example-mongodb-2.example-mongodb-svc.mongodb.svc.cluster.local:27017"         ],         "setName" : "example-mongodb",         "setVersion" : 1,         "ismaster" : true,         "secondary" : false,         "primary" : "example-mongodb-0.example-mongodb-svc.mongodb.svc.cluster.local:27017",         "me" : "example-mongodb-0.example-mongodb-svc.mongodb.svc.cluster.local:27017",         "electionId" : ObjectId("7fffffff0000000000000003"),         "lastWrite" : {                 "opTime" : {                         "ts" : Timestamp(1610520741, 1),                         "t" : NumberLong(3)                 },                 "lastWriteDate" : ISODate("2021-01-13T06:52:21Z"),                 "majorityOpTime" : {                         "ts" : Timestamp(1610520741, 1),                         "t" : NumberLong(3)                 },                 "majorityWriteDate" : ISODate("2021-01-13T06:52:21Z")         },         "maxBsonObjectSize" : 16777216,         "maxMessageSizeBytes" : 48000000,         "maxWriteBatchSize" : 100000,         "localTime" : ISODate("2021-01-13T06:52:27.873Z"),         "logicalSessionTimeoutMinutes" : 30,         "connectionId" : 153,         "minWireVersion" : 0,         "maxWireVersion" : 8,         "readOnly" : false,         "ok" : 1,         "$clusterTime" : {                 "clusterTime" : Timestamp(1610520741, 1),                 "signature" : {                         "hash" : BinData(0,"EcWzL7O9Ue9kmm6cQ4FumkcIP6g="),                         "keyId" : NumberLong("6917119940596072451")                 }         },         "operationTime" : Timestamp(1610520741, 1) } example-mongodb:PRIMARY>  

  提示:可以看到三个mongodb pod是副本集关系;其中example-mongodb-0是主节点,其他两个是从节点;

  最后还是说一下,我在做上面的实验中,虽然mongodb operator已经正常工作,但是用mongo这个客户端工具连接到主节点没法写数据,提示没有权限;但对应的用户在对应库是有读写权限的;在admin库下创建用户能提示用户添加成功,但是过几秒查询用户信息,发现用户不存在,不知道什么原因,有知道的朋友麻烦告知下(博主邮箱:linux-1874@qq.com),博主将感激不尽。。

赞(0) 打赏
未经允许不得转载:张拓的天空 » 容器编排系统K8s之crd资源
分享到: 更多 (0)

评论 抢沙发

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址

专业的IT技术经验分享 更专业 更方便

联系我们本站主机

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏