Uploaded image for project: 'Red Hat 3scale API Management'
  1. Red Hat 3scale API Management
  2. THREESCALE-10518

Operator crash - OpenAPI CR doesn't create correct backend usage section in Product CR

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • 2.14.0 GA, 2.14.0-mas (0.11.8)
    • 3scale Operator
    • RHOAM Sprint 50, RHOAM Sprint 51, RHOAM Sprint 52
    • Important

    Description

      During work on https://issues.redhat.com/browse/THREESCALE-9573 I use this OpenAPI CR:

      apiVersion: capabilities.3scale.net/v1beta1
      kind: OpenAPI
      metadata:
        annotations:
          insecure_skip_verify: 'true'
        name: oasmkudleteststetgsy4wxa
        namespace: 3scale-epsql-eredis
      spec:
        oidc:
          issuerType: keycloak
        openapiRef:
          secretRef:
            name: oasmkudleteststetgsy4wxasecret
            namespace: 3scale-epsql-eredis
        prefixMatching: true
      

      and OpenAPI spec

      components:
        schemas:
          Error:
            properties:
              code:
                format: int32
                type: integer
              message:
                type: string
            required:
            - code
            - message
            type: object
          NewPet:
            properties:
              name:
                type: string
              tag:
                type: string
            required:
            - name
            type: object
          Pet:
            allOf:
            - $ref: '#/components/schemas/NewPet'
            - properties:
                id:
                  format: int64
                  type: integer
              required:
              - id
              type: object
        securitySchemes:
          myOauth:
            description: This API uses OAuth 2 with the implicit grant flow. [More info](https://api.example.com/docs/auth)
            flows:
              password:
                scopes:
                  read_pets: read your pets
                  write_pets: modify pets in your account
                tokenUrl: https://api.example.com/oauth2/token
            type: oauth2
      info:
        contact:
          email: apiteam@swagger.io
          name: Swagger API Team
          url: http://swagger.io
        description: A sample API that uses a petstore as an example to demonstrate features
          in the OpenAPI 3.0 specification
        license:
          name: Apache 2.0
          url: https://www.apache.org/licenses/LICENSE-2.0.html
        termsOfService: http://swagger.io/terms/
        title: Swagger Petstore
        version: 1.0.0
      openapi: 3.0.0
      paths:
        /pets:
          get:
            description: 'Returns all pets from the system that the user has access to
      
              Nam sed condimentum est. Maecenas tempor sagittis sapien, nec rhoncus sem
              sagittis sit amet. Aenean at gravida augue, ac iaculis sem. Curabitur odio
              lorem, ornare eget elementum nec, cursus id lectus. Duis mi turpis, pulvinar
              ac eros ac, tincidunt varius justo. In hac habitasse platea dictumst. Integer
              at adipiscing ante, a sagittis ligula. Aenean pharetra tempor ante molestie
              imperdiet. Vivamus id aliquam diam. Cras quis velit non tortor eleifend sagittis.
              Praesent at enim pharetra urna volutpat venenatis eget eget mauris. In eleifend
              fermentum facilisis. Praesent enim enim, gravida ac sodales sed, placerat
              id erat. Suspendisse lacus dolor, consectetur non augue vel, vehicula interdum
              libero. Morbi euismod sagittis libero sed lacinia.
      
      
              Sed tempus felis lobortis leo pulvinar rutrum. Nam mattis velit nisl, eu condimentum
              ligula luctus nec. Phasellus semper velit eget aliquet faucibus. In a mattis
              elit. Phasellus vel urna viverra, condimentum lorem id, rhoncus nibh. Ut pellentesque
              posuere elementum. Sed a varius odio. Morbi rhoncus ligula libero, vel eleifend
              nunc tristique vitae. Fusce et sem dui. Aenean nec scelerisque tortor. Fusce
              malesuada accumsan magna vel tempus. Quisque mollis felis eu dolor tristique,
              sit amet auctor felis gravida. Sed libero lorem, molestie sed nisl in, accumsan
              tempor nisi. Fusce sollicitudin massa ut lacinia mattis. Sed vel eleifend
              lorem. Pellentesque vitae felis pretium, pulvinar elit eu, euismod sapien.
      
              '
            operationId: findPets
            parameters:
            - description: tags to filter by
              in: query
              name: tags
              required: false
              schema:
                items:
                  type: string
                type: array
              style: form
            - description: maximum number of results to return
              in: query
              name: limit
              required: false
              schema:
                format: int32
                type: integer
            responses:
              '200':
                content:
                  application/json:
                    schema:
                      items:
                        $ref: '#/components/schemas/Pet'
                      type: array
                description: pet response
              default:
                content:
                  application/json:
                    schema:
                      $ref: '#/components/schemas/Error'
                description: unexpected error
            security:
            - myOauth:
              - read
              - write
          post:
            description: Creates a new pet in the store. Duplicates are allowed
            operationId: addPet
            requestBody:
              content:
                application/json:
                  schema:
                    $ref: '#/components/schemas/NewPet'
              description: Pet to add to the store
              required: true
            responses:
              '200':
                content:
                  application/json:
                    schema:
                      $ref: '#/components/schemas/Pet'
                description: pet response
              default:
                content:
                  application/json:
                    schema:
                      $ref: '#/components/schemas/Error'
                description: unexpected error
            security:
            - myOauth:
              - read
              - write
        /pets/{id}:
          delete:
            description: deletes a single pet based on the ID supplied
            operationId: deletePet
            parameters:
            - description: ID of pet to delete
              in: path
              name: id
              required: true
              schema:
                format: int64
                type: integer
            responses:
              '204':
                description: pet deleted
              default:
                content:
                  application/json:
                    schema:
                      $ref: '#/components/schemas/Error'
                description: unexpected error
            security:
            - myOauth:
              - read
              - write
          get:
            description: Returns a user based on a single ID, if the user does not have
              access to the pet
            operationId: find pet by id
            parameters:
            - description: ID of pet to fetch
              in: path
              name: id
              required: true
              schema:
                format: int64
                type: integer
            responses:
              '200':
                content:
                  application/json:
                    schema:
                      $ref: '#/components/schemas/Pet'
                description: pet response
              default:
                content:
                  application/json:
                    schema:
                      $ref: '#/components/schemas/Error'
                description: unexpected error
            security:
            - myOauth:
              - read
              - write
      security:
      - myOauth:
        - read
        - write
      servers:
      - url: https://httpbin.org:443/anything
      

      I see this error in Product

      spec.backendUsages[Swagger_Petstore]: Invalid value: v1beta1.BackendUsageSpec{Path:"/"}: backend usage does not have valid backend reference.
      

      So Product is in failed state

      	Conditions: Failed, Orphan
      

      and the Operator is in the crash loop.
      I see this error in Operator log:

      {"level":"info","ts":1701164406.6480556,"logger":"controllers.OpenAPI","msg":"Product/swaggerpetstore-daf5f8c5-48d9-4ff6-a0de-5f9b5fd3217b spec has changed:   v1beta1.ProductSpec{\n  \t... // 2 identical fields\n  \tDescription: \"A sample API that uses a petstore as an example to demonstrate f\"...,\n  \tDeployment:  &{ApicastHosted: &{Authentication: &{OIDC: &{IssuerType: \"keycloak\", AuthenticationFlow: &{DirectAccessGrantsEnabled: true}, CredentialsLoc: &\"headers\"}}}},\n  \tMappingRules: []v1beta1.MappingRuleSpec{\n  \t\t{HTTPMethod: \"POST\", Pattern: \"/anything/pets\", MetricMethodRef: \"addpet\", Increment: 1, ...},\n  \t\t{HTTPMethod: \"GET\", Pattern: \"/anything/pets\", MetricMethodRef: \"findpets\", Increment: 1, ...},\n  \t\t{\n- \t\t\tHTTPMethod:      \"GET\",\n+ \t\t\tHTTPMethod:      \"DELETE\",\n  \t\t\tPattern:         \"/anything/pets/{id}\",\n- \t\t\tMetricMethodRef: \"find_pet_by_id\",\n+ \t\t\tMetricMethodRef: \"deletepet\",\n  \t\t\tIncrement:       1,\n  \t\t\tLast:            nil,\n  \t\t},\n  \t\t{\n- \t\t\tHTTPMethod:      \"DELETE\",\n+ \t\t\tHTTPMethod:      \"GET\",\n  \t\t\tPattern:         \"/anything/pets/{id}\",\n- \t\t\tMetricMethodRef: \"deletepet\",\n+ \t\t\tMetricMethodRef: \"find_pet_by_id\",\n  \t\t\tIncrement:       1,\n  \t\t\tLast:            nil,\n  \t\t},\n  \t},\n  \tBackendUsages: {\"Swagger_Petstore\": {Path: \"/\"}},\n  \tMetrics:       {\"hits\": {Name: \"Hits\", Unit: \"hit\", Description: \"Number of API hits\"}},\n  \t... // 4 identical fields\n  }\n","openapi":"oasmkudleteststetgsy4wxa"}
      {"level":"info","ts":1701164406.6482427,"logger":"controllers.OpenAPI","msg":"Updated object 'v1beta1.Product/swaggerpetstore-daf5f8c5-48d9-4ff6-a0de-5f9b5fd3217b'"}
      {"level":"info","ts":1701164406.7457948,"msg":"Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference","controller":"product","controllerGroup":"capabilities.3scale.net","controllerKind":"Product","product":{"name":"swaggerpetstore-daf5f8c5-48d9-4ff6-a0de-5f9b5fd3217b","namespace":"3scale-epsql-eredis"},"namespace":"3scale-epsql-eredis","name":"swaggerpetstore-daf5f8c5-48d9-4ff6-a0de-5f9b5fd3217b","reconcileID":"622519b0-17d4-4e6c-9031-f2625145c6d3"}
      panic: runtime error: invalid memory address or nil pointer dereference [recovered]
      	panic: runtime error: invalid memory address or nil pointer dereference
      [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1646f2e]
      
      goroutine 288 [running]:
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
      	/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:118 +0x1f4
      panic({0x17e5660, 0x29a5fa0})
      	/opt/rh/go-toolset-1.19/root/usr/lib/go-toolset-1.19-golang/src/runtime/panic.go:884 +0x212
      github.com/3scale/3scale-operator/controllers/capabilities.(*ProductThreescaleReconciler).syncProxyOIDC(0x1826a40?, 0xc0013f2ed0?, 0xc0015da8c0)
      	/remote-source/app/controllers/capabilities/proxy.go:204 +0x16e
      github.com/3scale/3scale-operator/controllers/capabilities.(*ProductThreescaleReconciler).syncProxy(0xc001682340, {0xc004891948?, 0x48b417?})
      	/remote-source/app/controllers/capabilities/proxy.go:89 +0x839
      github.com/3scale/3scale-operator/pkg/helper.(*taskRunnerImpl).Run(0xc004891a68)
      	/remote-source/app/pkg/helper/task_runner.go:41 +0xd3
      github.com/3scale/3scale-operator/controllers/capabilities.(*ProductThreescaleReconciler).Reconcile(0xc001682340)
      	/remote-source/app/controllers/capabilities/product_threescale_reconciler.go:56 +0x38f
      github.com/3scale/3scale-operator/controllers/capabilities.(*ProductReconciler).reconcile(0xc00030c018, 0xc001652000)
      	/remote-source/app/controllers/capabilities/product_controller.go:224 +0x36a
      github.com/3scale/3scale-operator/controllers/capabilities.(*ProductReconciler).Reconcile(0xc00030c018, {0xc000100800?, 0xc000d25230?}, {{{0xc000a0a258, 0x13}, {0xc00098c780, 0x34}}})
      	/remote-source/app/controllers/capabilities/product_controller.go:151 +0x7a7
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x1ccdee8?, {0x1ccdf90?, 0xc000d25230?}, {{{0xc000a0a258?, 0x1921d00?}, {0xc00098c780?, 0x4045d4?}}})
      	/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:121 +0xc8
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0002928c0, {0x1ccdee8, 0xc000ceeb00}, {0x184be40?, 0xc000328940?})
      	/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:320 +0x33c
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0002928c0, {0x1ccdee8, 0xc000ceeb00})
      	/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:273 +0x1d9
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
      	/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:234 +0x85
      created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
      	/remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.2/pkg/internal/controller/controller.go:230 +0x333
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            mkudlej@redhat.com Martin Kudlej
            Martin Kudlej Martin Kudlej
            Valery Mogilevsky Valery Mogilevsky
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated: