Support alternate GVR in ServiceExport and ServiceImport in e2e tests

[lauralorenz]

Vendors today may be using a mirror of the ServiceExport and ServiceImport APIs and today's e2e tests make it impossible for them to run them as conformance tests because the mcsclient is ultimately configured to use the v1alpha1 API whose GVR (group version resource) is hardcoded as multicluster.x-k8s.io/v1alpha.

As discussed in SIG-MC today 11/15/2022 we'd like to accommodate vendors who may, due to their own API deprecation policy or during active dev, use a mirror of MCS until upstream is GA (GKE being one of these).

Two ways to potentially address this:

1. tests use a common struct and convert
2. parameterize the GVR

Ultimately the schema of the resources should still be the same for conformance.

Here's an example against GKE clusters of the e2e tests failing on this point:

 Unexpected error:
      <*errors.StatusError | 0xc000134640>: {
          ErrStatus: {
              TypeMeta: {Kind: "", APIVersion: ""},
              ListMeta: {
                  SelfLink: "",
                  ResourceVersion: "",
                  Continue: "",
                  RemainingItemCount: nil,
              },
              Status: "Failure",
              Message: "the server could not find the requested resource (post serviceimports.multicluster.x-k8s.io)",
              Reason: "NotFound",
              Details: {
                  Name: "",
                  Group: "multicluster.x-k8s.io",
                  Kind: "serviceimports",
                  UID: "",
                  Causes: [
                      {
                          Type: "UnexpectedServerResponse",
                          Message: "404 page not found",
                          Field: "",
                      },
                  ],
                  RetryAfterSeconds: 0,
              },
              Code: 404,
          },
      }
      the server could not find the requested resource (post serviceimports.multicluster.x-k8s.io)
  occurred
------------------------------

[lauralorenz]

/help

[k8s-ci-robot]

@lauralorenz:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

Details

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[lauralorenz]

/assign lauralorenz
@IshmeetMehta is helping with this one

[apelisse]

Where are the e2e tests that you're trying to run?

[IshmeetMehta]

@apelisse : The end to end test exists here : https://github.com/kubernetes-sigs/mcs-api/tree/master/e2etest

[IshmeetMehta]

Updating the issue with the findings so far.

1. The MCS API is using client-gen library to generate the clientset for this MCS API.

• Clientset are created for a specific (GVR) groupName and Verison only.

3. The generated client-set for this API is being used in the e2etest to create MCS resources ServiceImport and ServiceExport on clusters which have installed the MCS controller on their K8 cluster . Clientset expects the cluster to have the APIgroup for MCS "multicluster.x-k8s.io"

4. Vendors today may be using a mirror of the ServiceExport and ServiceImport APIs for example: GKE clusters use a different groupName for serviceImport and serviceExport resources.

• client-gen adds the groupName as a const in the generate d client set , which we are unable to override from outside in order to add multiple groupNames for Service Imports and ServiceExports which are registered to different groupNames in various vendor implementations of MCS API.

Hence its not possible to use the Client set generated by the client-gen to run conformance or end-to-end tests for various vendors who are using MCS API for their implementation of ServiceImports and ServiceExport resources.

kubectl api-resources | grep -ie "serviceimport\|serviceexports"
serviceexports                                 net.gke.io/v1                          true         ServiceExport
serviceimportconfigs                           net.gke.io/v1                          true         ServiceImportConfig
serviceimports                                 net.gke.io/v1                          true         ServiceImport

5. I tried to change the groupName in the generated in clientset to one which is used in GKE and was able to create the ServiceImport and ServiceExport resource on the gke cluster.

// Code generated by register-gen. DO NOT EDIT.

package v1alpha1

import (
	v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/apimachinery/pkg/runtime"
	"k8s.io/apimachinery/pkg/runtime/schema"
)

// GroupName specifies the group name used to register the objects.
//const GroupName = "multicluster.x-k8s.io"
const GroupName = "net.gke.io"

// GroupVersion specifies the group and the version used to register the objects.
//var GroupVersion = v1.GroupVersion{Group: GroupName, Version: "v1alpha1"}
var GroupVersion = v1.GroupVersion{Group: GroupName, Version: "v1"}

// SchemeGroupVersion is group version used to register these objects
// Deprecated: use GroupVersion instead.
//var SchemeGroupVersion = schema.GroupVersion{Group: GroupName, Version: "v1alpha1"}
var SchemeGroupVersion = schema.GroupVersion{Group: GroupName, Version: "v1"}

We need a way to override the groupName or provide multiple groupNames when generating clientset for multiple groupNames for MCS API so we can run the e2etests and conformance tests for the API using same clients for all vendors.

[apelisse]

3. The generated client-set for this API is being used in the e2etest to create MCS resources ServiceImport and ServiceExport on clusters which have installed the MCS controller on their K8 cluster . Clientset expects the cluster to have the APIgroup for MCS "multicluster.x-k8s.io"

where are the e2e tests?

[IshmeetMehta]

@apelisse : https://github.com/kubernetes-sigs/mcs-api/tree/master/e2etest

[tpantelis]

The way to do this is to modify the tests to use a dynamic client and the Unstructured type, allowing the group and version to be parameterized. The tests could also query the ServiceExport and ServiceImport CustomResourceDefinition resources to obtain the group name and stored version.

[skitt]

/sig multicluster