mirror of
https://github.com/kubesphere/kubesphere.git
synced 2026-10-16 21:03:06 +08:00
add kubesphere-gateway and kubesphere-gateway-api skills
Signed-off-by: junot <junotxiang@kubesphere.io>
This commit is contained in:
junot
@@ -0,0 +1,355 @@
|
||||
---
|
||||
name: kubesphere-gateway-api
|
||||
description: KubeSphere Gateway API extension management Skill (Traefik based, uses Kubernetes Gateway API + GatewayProxy CRD gatewayapi.kubesphere.io/v1alpha1). This is the newer Kubernetes Gateway API standard. For the older Ingress API based gateway (ingress-nginx + Gateway CRD gateway.kubesphere.io/v2alpha2), see the kubesphere-gateway skill instead. Covers installation, uninstallation, status checks, GatewayProxy status inspection, and troubleshooting.
|
||||
---
|
||||
# KubeSphere Gateway API
|
||||
|
||||
## Overview
|
||||
|
||||
Provides external access management using **Kubernetes Gateway API** with **Traefik** as the underlying proxy implementation. Supports three-tier gatewayproxy management:
|
||||
|
||||
| Tier | Scope | Name Pattern | Namespace | Description |
|
||||
| ------------------- | ------------------------ | -------------------------------------- | ------------------------------ | ----------------------------- |
|
||||
| **Cluster** | Entire cluster | `gatewayproxy-cluster` | `kubesphere-controls-system` | Cluster-scoped GatewayProxy |
|
||||
| **Workspace** | Single workspace | `gatewayproxy-workspace-{workspace}` | `kubesphere-controls-system` | Workspace-scoped GatewayProxy |
|
||||
| **Project** | Single project/namespace | `gatewayproxy-namespace-{namespace}` | `kubesphere-controls-system` | Namespace-scoped GatewayProxy |
|
||||
|
||||
Each **GatewayProxy** (`gatewayproxies.gatewayapi.kubesphere.io`) deploys a Traefik instance (the proxy implementation). It auto-creates a `GatewayClass`, and users can then create standard **Gateway** (`gateways.gateway.networking.k8s.io`) resources that reference that GatewayClass. The extension consists of three components:
|
||||
|
||||
- **backend-extension** — API server on the host cluster
|
||||
- **backend-agent** — API server + controller-manager on every cluster (including the host cluster if selected)
|
||||
- **frontend** — React SPA served via Nginx
|
||||
|
||||
### Key Differentiator from `kubesphere-gateway`
|
||||
|
||||
| Aspect | kubesphere-gateway | kubesphere-gateway-api |
|
||||
| -------------------- | ------------------------------------------------------- | ------------------------------------------------------------------------- |
|
||||
| Underlying proxy | ingress-nginx | Traefik |
|
||||
| API standard | Custom Gateway CRD (`gateway.kubesphere.io/v2alpha2`) | Kubernetes Gateway API (`gateway.networking.k8s.io`) + GatewayProxy CRD |
|
||||
| Core resource | `Gateway` + standard `IngressClass`/`Ingress` | `GatewayProxy` + standard `Gateway`/`GatewayClass`/`HTTPRoute` |
|
||||
| Lifecycle management | Helm release per gateway | Helm release per GatewayProxy |
|
||||
|
||||
### Core CRDs
|
||||
|
||||
- **`GatewayProxy`** (`gatewayapi.kubesphere.io/v1alpha1`) — the proxy implementation (e.g. Traefik). When created, the controller deploys Traefik via Helm SDK and auto-creates a `GatewayClass`. Key fields:
|
||||
|
||||
- `spec.type` — proxy type (currently only `Traefik`)
|
||||
- `spec.traefik.rawValues` — raw Helm values passed to the Traefik chart
|
||||
- `spec.traefik.deployment.replicas` — replica count
|
||||
- `spec.traefik.service.type` — Service type (ClusterIP, NodePort, LoadBalancer)
|
||||
- `spec.traefik.createDefaultGateway` — whether to auto-create a default Gateway
|
||||
- `spec.paused` — pause reconciliation
|
||||
- `status.conditions` — condition types: `Ready`, `Progressing`, `NewVersionDetected`
|
||||
- `status.service` — Service type, ports, external IPs, load balancer status
|
||||
- `status.entrypoints` — exposed entrypoints with ports and protocols
|
||||
- `status.gatewayClass.name` — auto-created GatewayClass name
|
||||
- `status.helmRelease.name` — Helm release name
|
||||
- **`GatewayClass`** (`gateway.networking.k8s.io/v1`) — standard Kubernetes Gateway API class, auto-created by the GatewayProxy controller
|
||||
- **`Gateway`** (`gateway.networking.k8s.io/v1`) — standard Kubernetes Gateway API gateway, associated with a GatewayClass
|
||||
- **`HTTPRoute`** / **`GRPCRoute`** / **`TLSRoute`** / **`TCPRoute`** / **`UDPRoute`** — standard Kubernetes Gateway API route resources
|
||||
|
||||
### Multi-Tenant Scoping
|
||||
|
||||
GatewayProxy and Gateway are scoped via labels:
|
||||
|
||||
- `gatewayapi.kubesphere.io/scope-type` — `cluster`, `workspace`, or `namespace`
|
||||
- `gatewayapi.kubesphere.io/scope-workspace` — workspace name (for workspace scope)
|
||||
- `gatewayapi.kubesphere.io/scope-namespace` — namespace name (for namespace scope)
|
||||
|
||||
### Monitoring Integration
|
||||
|
||||
GatewayProxy exposes Traefik metrics via Prometheus. Requires the `whizard-monitoring` extension (optional dependency). Log search requires the `whizard-logging` extension.
|
||||
|
||||
---
|
||||
|
||||
## Before You Start
|
||||
|
||||
Check if Gateway API extension is already installed:
|
||||
|
||||
```bash
|
||||
kubectl get installplans.kubesphere.io gateway-api --ignore-not-found
|
||||
```
|
||||
|
||||
If found, upgrading is supported — just select a newer version in Step 1.
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
### Step 1: Detect and Select Version
|
||||
|
||||
```bash
|
||||
ALL_VERSIONS=$(kubectl get extensionversions.kubesphere.io \
|
||||
-l kubesphere.io/extension-ref=gateway-api \
|
||||
-o jsonpath='{range .items[*]}{.spec.version}{"\n"}{end}' | sort -V)
|
||||
|
||||
LATEST_STABLE=$(echo "$ALL_VERSIONS" | grep -v -E 'alpha|beta|rc' | tail -1)
|
||||
if [ -z "$LATEST_STABLE" ]; then
|
||||
LATEST_STABLE=$(echo "$ALL_VERSIONS" | tail -1)
|
||||
fi
|
||||
|
||||
echo "Available versions:"
|
||||
echo "$ALL_VERSIONS"
|
||||
echo ""
|
||||
echo "Latest stable: $LATEST_STABLE"
|
||||
```
|
||||
|
||||
This sets `ALL_VERSIONS` and `LATEST_STABLE`. Use `SELECTED_VERSION` for the version chosen.
|
||||
|
||||
Use the `question` tool:
|
||||
|
||||
- **`$LATEST_STABLE (Recommended)`** — accept the auto-detected version
|
||||
- *(custom)* — type a specific version; validate it against the printed list
|
||||
|
||||
### Step 2: Detect and Select Clusters
|
||||
|
||||
```bash
|
||||
CLUSTER_DATA=$(kubectl get clusters.cluster.kubesphere.io \
|
||||
-o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.conditions[?(@.type=="Ready")].status}{"\n"}{end}')
|
||||
|
||||
READY_CLUSTERS=$(echo "$CLUSTER_DATA" | awk -F'\t' '$2 == "True" {print $1}')
|
||||
CLUSTER_COUNT=$(echo "$READY_CLUSTERS" | wc -l)
|
||||
|
||||
HOST_CLUSTER=$(kubectl get clusters.cluster.kubesphere.io \
|
||||
-l 'cluster-role.kubesphere.io/host' \
|
||||
-o jsonpath='{.items[0].metadata.name}' || echo "")
|
||||
|
||||
echo "Ready clusters:"
|
||||
echo "$READY_CLUSTERS"
|
||||
echo ""
|
||||
echo "Cluster count: $CLUSTER_COUNT"
|
||||
echo "Host cluster: $HOST_CLUSTER"
|
||||
```
|
||||
|
||||
This sets `READY_CLUSTERS`, `CLUSTER_COUNT`, `HOST_CLUSTER`.
|
||||
|
||||
- **1 cluster** → skip selection, auto-use it. Set `TARGET_CLUSTERS="$HOST_CLUSTER"`
|
||||
- **Multiple clusters** → use `question` with `multiple: true`:
|
||||
- **All clusters** → `TARGET_CLUSTERS="$READY_CLUSTERS"`
|
||||
- **Host cluster only** → `TARGET_CLUSTERS="$HOST_CLUSTER"`
|
||||
- *(custom)* — validate each name against `$READY_CLUSTERS`
|
||||
|
||||
### Step 3: Generate and Apply InstallPlan
|
||||
|
||||
```bash
|
||||
./scripts/generate-installplan.sh "$SELECTED_VERSION" "$TARGET_CLUSTERS"
|
||||
```
|
||||
|
||||
This generates the YAML to `/tmp/gateway-api-installplan.yaml`, runs `--dry-run=server`, then prints the apply command.
|
||||
|
||||
Apply it:
|
||||
|
||||
```bash
|
||||
kubectl apply -f /tmp/gateway-api-installplan.yaml
|
||||
```
|
||||
|
||||
Tell the user "Installing". Then ask if they want to check status. If yes:
|
||||
|
||||
```bash
|
||||
./scripts/check-status.sh poll
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Status Checking
|
||||
|
||||
| Purpose | Command |
|
||||
| ---------------------------------- | ----------------------------------- |
|
||||
| Single snapshot | `./scripts/check-status.sh quick` |
|
||||
| Wait until complete (5min timeout) | `./scripts/check-status.sh poll` |
|
||||
|
||||
Logic:
|
||||
|
||||
- **All `Installed`** → ✓ success
|
||||
- **Any `Failed`** → ✗ prints full status
|
||||
- **Timeout (300s)** → ⚠ prints current status
|
||||
- **In progress** → prints every 10s
|
||||
|
||||
---
|
||||
|
||||
## Uninstallation
|
||||
|
||||
> ⚠ **Always confirm with the user before proceeding.**
|
||||
|
||||
### Uninstall from all clusters
|
||||
|
||||
```bash
|
||||
if ! kubectl get installplans.kubesphere.io gateway-api &>/dev/null; then
|
||||
echo "Gateway API is not installed."
|
||||
exit 0
|
||||
fi
|
||||
```
|
||||
|
||||
Confirm with the user, then delete:
|
||||
|
||||
```bash
|
||||
kubectl delete installplans.kubesphere.io gateway-api --ignore-not-found
|
||||
```
|
||||
|
||||
Verify cleanup:
|
||||
|
||||
```bash
|
||||
./scripts/verify-uninstall.sh
|
||||
```
|
||||
|
||||
Success criteria:
|
||||
|
||||
1. InstallPlan is deleted
|
||||
2. No active pods remain in `extension-gateway-api` namespace
|
||||
|
||||
### Uninstall from specific clusters
|
||||
|
||||
> **WARNING**: Do NOT delete the InstallPlan. Only remove target clusters from the placement list.
|
||||
|
||||
Confirm which clusters to remove, compute remaining clusters, then patch:
|
||||
|
||||
```bash
|
||||
kubectl patch installplans.kubesphere.io gateway-api --type='json' \
|
||||
-p='[{"op": "replace", "path": "/spec/clusterScheduling/placement/clusters", "value": ["<REMAINING_CLUSTER_1>", "<REMAINING_CLUSTER_2>"]}]'
|
||||
```
|
||||
|
||||
Success: patch returns OK + removed clusters no longer in `.status.clusterSchedulingStatuses`.
|
||||
|
||||
---
|
||||
|
||||
## GatewayProxy Operations
|
||||
|
||||
### List GatewayProxies
|
||||
|
||||
GatewayProxies are organized by scope type. List them:
|
||||
|
||||
```bash
|
||||
echo "=== Cluster GatewayProxies ==="
|
||||
kubectl get gatewayproxies.gatewayapi.kubesphere.io -A \
|
||||
-l gatewayapi.kubesphere.io/scope-type=cluster
|
||||
|
||||
echo -e "\n=== Workspace GatewayProxies ==="
|
||||
kubectl get gatewayproxies.gatewayapi.kubesphere.io -A \
|
||||
-l gatewayapi.kubesphere.io/scope-type=workspace
|
||||
|
||||
echo -e "\n=== Namespace GatewayProxies ==="
|
||||
kubectl get gatewayproxies.gatewayapi.kubesphere.io -A \
|
||||
-l gatewayapi.kubesphere.io/scope-type=namespace
|
||||
```
|
||||
|
||||
### Check GatewayProxy Status
|
||||
|
||||
Pick a gateway proxy name from the list above and run:
|
||||
|
||||
```bash
|
||||
GWP_NS="kubesphere-controls-system"
|
||||
GWP_NAME="<gatewayproxy-name-from-list>"
|
||||
|
||||
# app.kubernetes.io/instance uses the Helm release name if available, otherwise the GatewayProxy name
|
||||
GWP_INSTANCE=$(kubectl get gatewayproxies.gatewayapi.kubesphere.io -n $GWP_NS $GWP_NAME -o jsonpath='{.status.helmRelease.name}' 2>/dev/null)
|
||||
if [ -z "$GWP_INSTANCE" ]; then
|
||||
GWP_INSTANCE="$GWP_NAME"
|
||||
fi
|
||||
|
||||
kubectl get gatewayproxies.gatewayapi.kubesphere.io -n $GWP_NS $GWP_NAME -o wide
|
||||
kubectl describe gatewayproxies.gatewayapi.kubesphere.io -n $GWP_NS $GWP_NAME
|
||||
kubectl get gatewayproxies.gatewayapi.kubesphere.io -n $GWP_NS $GWP_NAME -o yaml
|
||||
kubectl get pods -n $GWP_NS -l "app.kubernetes.io/instance=$GWP_INSTANCE"
|
||||
```
|
||||
|
||||
GatewayProxy conditions:
|
||||
|
||||
| Condition Type | Status True | Meaning |
|
||||
| ---------------------- | ----------- | --------------------------- |
|
||||
| `Ready` | True | Fully operational |
|
||||
| `Progressing` | True | Being created or updated |
|
||||
| `NewVersionDetected` | True | New chart version available |
|
||||
|
||||
### List Associated Gateways
|
||||
|
||||
Each GatewayProxy may have associated standard Gateways and GatewayClasses. The GatewayProxy creates GatewayClasses with the label `gatewayapi.kubesphere.io/gateway-class-name`, and auto-created Gateways carry scope labels:
|
||||
|
||||
```bash
|
||||
# List GatewayClasses created by any GatewayProxy
|
||||
kubectl get gatewayclass -l "gatewayapi.kubesphere.io/gateway-class-name"
|
||||
|
||||
# List Gateways associated with a specific GatewayProxy (by scope label)
|
||||
kubectl get gateways.gateway.networking.k8s.io -n $GWP_NS \
|
||||
-l "gatewayapi.kubesphere.io/scope-type"
|
||||
|
||||
# Alternatively, list Gateways by the GatewayClass name they reference
|
||||
GW_CLASS=$(kubectl get gatewayproxies.gatewayapi.kubesphere.io -n $GWP_NS $GWP_NAME \
|
||||
-o jsonpath='{.status.gatewayClass.name}')
|
||||
kubectl get gateways.gateway.networking.k8s.io -A \
|
||||
-o jsonpath='{range .items[?(@.spec.gatewayClassName=="'"$GW_CLASS"'")]}{.metadata.namespace}{"\t"}{.metadata.name}{"\t"}{.spec.gatewayClassName}{"\n"}{end}'
|
||||
```
|
||||
|
||||
### View GatewayProxy Pods and Logs
|
||||
|
||||
```bash
|
||||
kubectl get pods -n $GWP_NS -l "app.kubernetes.io/instance=$GWP_INSTANCE"
|
||||
kubectl logs -n $GWP_NS -l "app.kubernetes.io/instance=$GWP_INSTANCE" --tail=100
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
> Set `$GWP_NAME` and `$GWP_NS` to the target GatewayProxy name and namespace (`kubesphere-controls-system` for all tiers). `$GWP_INSTANCE` is auto-resolved from `status.helmRelease.name` (falls back to `$GWP_NAME`).
|
||||
|
||||
### GatewayProxy stuck in `Progressing` state
|
||||
|
||||
```bash
|
||||
kubectl describe gatewayproxies.gatewayapi.kubesphere.io -n $GWP_NS $GWP_NAME
|
||||
|
||||
kubectl logs -n extension-gateway-api -l app=gateway-api-controller-manager --tail=200 | grep -iE "(error|helm|install|upgrade|reconcile)"
|
||||
|
||||
kubectl get pods -n $GWP_NS -l "app.kubernetes.io/instance=$GWP_INSTANCE"
|
||||
```
|
||||
|
||||
Common causes: Helm chart not found, invalid `rawValues`, Traefik image pull failure, Helm SDK timeout.
|
||||
|
||||
### GatewayProxy shows `Ready=False` state
|
||||
|
||||
```bash
|
||||
kubectl describe gatewayproxies.gatewayapi.kubesphere.io -n $GWP_NS $GWP_NAME
|
||||
|
||||
kubectl get deployment -n $GWP_NS -l "app.kubernetes.io/instance=$GWP_INSTANCE" -o wide
|
||||
kubectl describe deployment -n $GWP_NS -l "app.kubernetes.io/instance=$GWP_INSTANCE"
|
||||
kubectl get pods -n $GWP_NS -l "app.kubernetes.io/instance=$GWP_INSTANCE" -o wide
|
||||
|
||||
POD_NAME=$(kubectl get pods -n $GWP_NS -l "app.kubernetes.io/instance=$GWP_INSTANCE" -o jsonpath='{.items[0].metadata.name}')
|
||||
kubectl describe pod -n $GWP_NS $POD_NAME
|
||||
kubectl logs -n $GWP_NS $POD_NAME --tail=100
|
||||
```
|
||||
|
||||
Common causes: Image pull failure, resource constraints, port conflicts, missing ConfigMap/Secret, Helm release abnormal.
|
||||
|
||||
### GatewayProxy pod crash-looping / CrashLoopBackOff
|
||||
|
||||
```bash
|
||||
kubectl logs -n $GWP_NS -l "app.kubernetes.io/instance=$GWP_INSTANCE" --tail=100 --previous
|
||||
kubectl get events -n $GWP_NS --sort-by='.lastTimestamp' | tail -20
|
||||
kubectl exec -n $GWP_NS -l "app.kubernetes.io/instance=$GWP_INSTANCE" -- cat /etc/traefik/traefik.yaml 2>/dev/null | head -50
|
||||
kubectl get configmap -n $GWP_NS -l "app.kubernetes.io/instance=$GWP_INSTANCE" -o yaml
|
||||
```
|
||||
|
||||
Common causes: Misconfigured Traefik config, port conflicts, resource limits (OOMKilled), missing dependencies (ConfigMap/Secret).
|
||||
|
||||
### Controller not reconciling
|
||||
|
||||
```bash
|
||||
kubectl get pods -n extension-gateway-api -l app=gateway-api-controller-manager
|
||||
kubectl logs -n extension-gateway-api -l app=gateway-api-controller-manager --tail=200
|
||||
kubectl get validatingwebhookconfiguration -l "app.kubernetes.io/managed-by=Helm,kubesphere.io/extension-ref=gateway-api"
|
||||
kubectl get deployment -n extension-gateway-api -l app=gateway-api-controller-manager -o yaml
|
||||
```
|
||||
|
||||
Common causes: Controller pod not running, webhook configuration blocking updates, Helm release state mismatch, RBAC permission issues.
|
||||
|
||||
### Log search / metrics not working
|
||||
|
||||
GatewayProxy observability proxies to `whizard-telemetry-apiserver`:
|
||||
|
||||
```bash
|
||||
kubectl get pods -n extension-whizard-telemetry
|
||||
kubectl get svc -n extension-whizard-telemetry whizard-telemetry-apiserver
|
||||
kubectl logs -n extension-gateway-api -l app=gateway-api-apiserver --tail=100 | grep -iE "(log|search|whizard|proxy|metric)"
|
||||
```
|
||||
|
||||
Common causes: Whizard-telemetry not installed or not running, network policy blocking cross-namespace traffic.
|
||||
+77
@@ -0,0 +1,77 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
MODE="${1:-quick}"
|
||||
POLL_INTERVAL="${POLL_INTERVAL:-10}"
|
||||
TIMEOUT="${TIMEOUT:-300}"
|
||||
|
||||
if ! kubectl get installplans.kubesphere.io gateway-api &>/dev/null; then
|
||||
echo "Gateway API InstallPlan not found."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$MODE" = "quick" ]; then
|
||||
echo "=== Extension State ==="
|
||||
echo "State: $(kubectl get installplans.kubesphere.io gateway-api -o jsonpath='{.status.state}')"
|
||||
echo "Reason: $(kubectl get installplans.kubesphere.io gateway-api -o jsonpath='{.status.conditions[-1].reason}')"
|
||||
echo "Message: $(kubectl get installplans.kubesphere.io gateway-api -o jsonpath='{.status.conditions[-1].message}')"
|
||||
|
||||
echo ""
|
||||
echo "=== Per-Cluster Agent States ==="
|
||||
kubectl get installplans.kubesphere.io gateway-api -o go-template='
|
||||
{{- range $cluster, $status := .status.clusterSchedulingStatuses}}
|
||||
{{$cluster}}{{"\t"}}{{$status.state}}{{"\t"}}{{if $status.conditions}}{{(index $status.conditions 0).reason}}{{end}}{{"\n"}}
|
||||
{{- end}}' | column -t
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Polling mode
|
||||
ELAPSED=0
|
||||
|
||||
while [ $ELAPSED -lt $TIMEOUT ]; do
|
||||
EXT_STATE=$(kubectl get installplans.kubesphere.io gateway-api -o jsonpath='{.status.state}' 2>/dev/null)
|
||||
|
||||
if [ -z "$EXT_STATE" ]; then
|
||||
echo "[$ELAPSED s] InstallPlan not found yet..."
|
||||
else
|
||||
AGENT_STATES=$(kubectl get installplans.kubesphere.io gateway-api \
|
||||
-o jsonpath='{.status.clusterSchedulingStatuses.*.state}' 2>/dev/null || true)
|
||||
|
||||
echo "[$ELAPSED s] Extension: $EXT_STATE | Agents: $AGENT_STATES"
|
||||
|
||||
ALL_STATES="$EXT_STATE $AGENT_STATES"
|
||||
|
||||
ALL_INSTALLED=true
|
||||
for state in $ALL_STATES; do
|
||||
if [ "$state" != "Installed" ]; then
|
||||
ALL_INSTALLED=false
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
if [ "$ALL_INSTALLED" = true ]; then
|
||||
echo ""
|
||||
echo "✓ All Gateway API components installed successfully!"
|
||||
kubectl get installplans.kubesphere.io gateway-api -o go-template='
|
||||
{{- range $cluster, $status := .status.clusterSchedulingStatuses}}
|
||||
{{$cluster}}{{"\t"}}{{$status.state}}{{"\n"}}
|
||||
{{- end}}' | column -t
|
||||
exit 0
|
||||
fi
|
||||
|
||||
for state in $ALL_STATES; do
|
||||
if [ "$state" = "Failed" ]; then
|
||||
echo "✗ Installation failed! Check details:"
|
||||
kubectl get installplans.kubesphere.io gateway-api -o yaml
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
sleep $POLL_INTERVAL
|
||||
ELAPSED=$((ELAPSED + POLL_INTERVAL))
|
||||
done
|
||||
|
||||
echo "⚠ Timeout reached ($TIMEOUT s). Current status:"
|
||||
kubectl get installplans.kubesphere.io gateway-api -o yaml
|
||||
exit 1
|
||||
@@ -0,0 +1,50 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SELECTED_VERSION="${1:-${SELECTED_VERSION:-}}"
|
||||
TARGET_CLUSTERS="${2:-${TARGET_CLUSTERS:-}}"
|
||||
|
||||
if [ -z "$SELECTED_VERSION" ] || [ -z "$TARGET_CLUSTERS" ]; then
|
||||
echo "Usage: $0 <version> <clusters>"
|
||||
echo " clusters: space-separated list of cluster names (e.g. 'host' or 'host member1')"
|
||||
echo " or set SELECTED_VERSION and TARGET_CLUSTERS env vars"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! command -v kubectl &>/dev/null; then
|
||||
echo "Error: kubectl not found in PATH"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
INSTALLPLAN_FILE="/tmp/gateway-api-installplan.yaml"
|
||||
|
||||
cat > "$INSTALLPLAN_FILE" <<YAML
|
||||
apiVersion: kubesphere.io/v1alpha1
|
||||
kind: InstallPlan
|
||||
metadata:
|
||||
name: gateway-api
|
||||
spec:
|
||||
extension:
|
||||
name: gateway-api
|
||||
version: "${SELECTED_VERSION}"
|
||||
clusterScheduling:
|
||||
placement:
|
||||
clusters:
|
||||
YAML
|
||||
|
||||
for cluster in $TARGET_CLUSTERS; do
|
||||
echo " - \"${cluster}\"" >> "$INSTALLPLAN_FILE"
|
||||
done
|
||||
|
||||
echo "=== Generated InstallPlan ==="
|
||||
cat "$INSTALLPLAN_FILE"
|
||||
echo ""
|
||||
|
||||
echo "=== Dry-run validation ==="
|
||||
if ! kubectl apply --dry-run=server -f "$INSTALLPLAN_FILE"; then
|
||||
echo "✗ Dry-run failed — check version '${SELECTED_VERSION}' and cluster list."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "✓ InstallPlan validated. Apply with: kubectl apply -f $INSTALLPLAN_FILE"
|
||||
@@ -0,0 +1,33 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
NAMESPACE="${NAMESPACE:-extension-gateway-api}"
|
||||
TIMEOUT="${TIMEOUT:-300}"
|
||||
ELAPSED=0
|
||||
|
||||
while [ $ELAPSED -lt $TIMEOUT ]; do
|
||||
if kubectl get installplans.kubesphere.io gateway-api &>/dev/null; then
|
||||
echo " Waiting for InstallPlan to be deleted... (${ELAPSED}s)"
|
||||
sleep 10
|
||||
ELAPSED=$((ELAPSED + 10))
|
||||
continue
|
||||
fi
|
||||
|
||||
REMAINING=$(kubectl get pods -n "$NAMESPACE" 2>/dev/null | grep -v -E 'uninstaller|Completed|NAME' | wc -l) || true
|
||||
|
||||
if [ "$REMAINING" -eq 0 ]; then
|
||||
echo ""
|
||||
echo "✓ Uninstallation complete — all Gateway API components removed from $NAMESPACE."
|
||||
echo " InstallPlan: deleted"
|
||||
echo " Remaining pods: none"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo " Waiting for components to be removed... (${ELAPSED}s, $REMAINING pods remaining)"
|
||||
sleep 10
|
||||
ELAPSED=$((ELAPSED + 10))
|
||||
done
|
||||
|
||||
echo "⚠ Timeout reached. Remaining pods in $NAMESPACE:"
|
||||
kubectl get pods -n "$NAMESPACE" 2>/dev/null | grep -v -E 'uninstaller|Completed'
|
||||
exit 1
|
||||
@@ -0,0 +1,320 @@
|
||||
---
|
||||
name: kubesphere-gateway
|
||||
description: KubeSphere Gateway extension management Skill (ingress-nginx based, uses Kubernetes Ingress API + Gateway CRD gateway.kubesphere.io/v2alpha2). For the newer Kubernetes Gateway API (Traefik + GatewayProxy CRD), see the kubesphere-gateway-api skill instead. Covers installation, uninstallation, status checks, gateway status inspection, and troubleshooting (gateway stuck states, Helm failures, pod issues).
|
||||
---
|
||||
|
||||
# KubeSphere Gateway
|
||||
|
||||
## Overview
|
||||
|
||||
Provides external access management (ingress) for KubeSphere using **ingress-nginx**. Supports three-tier gateway management:
|
||||
|
||||
| Tier | Scope | Name Pattern | Namespace | Label |
|
||||
|---|---|---|---|
|
||||
| **Cluster** | Entire cluster | `kubesphere-router-cluster` | `kubesphere-controls-system` | `kubesphere.io/gateway-type=cluster` |
|
||||
| **Workspace** | Single workspace | `kubesphere-router-workspace-{workspace}` | `kubesphere-controls-system` | `kubesphere.io/gateway-type=workspace` |
|
||||
| **Project** | Single project/namespace | `kubesphere-router-{namespace}` | `kubesphere-controls-system` | `kubesphere.io/gateway-type=project` |
|
||||
|
||||
Each gateway is a standalone Helm release of ingress-nginx. The gateway-controller-manager manages the lifecycle (install/upgrade/uninstall) via Helm.
|
||||
|
||||
### Core CRDs
|
||||
|
||||
- **`Gateway`** (`gateway.kubesphere.io/v2alpha2`) — represents a single ingress-nginx deployment. Key fields:
|
||||
- `spec.appVersion` — the Helm chart version (e.g. `kubesphere-nginx-ingress-<version>`)
|
||||
- `spec.values` — Helm values for ingress-nginx (controller config, service type, resources, etc.)
|
||||
- `status.state` — `Creating`, `Updating`, `Running`, `Faulted`, `Stopped`
|
||||
- `status.conditions[].type=GatewayReady` — True when fully operational
|
||||
- `status.loadBalancer` — LB ingress IPs/hostnames
|
||||
- `status.service` — Service type, ports, external IPs
|
||||
|
||||
- **`UpgradePlan`** (`gateway.kubesphere.io/v2alpha2`) — batch gateway upgrade job. Key fields:
|
||||
- `spec.gatewayReferences` — list of `{name, namespace}` to upgrade
|
||||
- `spec.targetAppVersion` — target version
|
||||
- `status.state` — `Pending`, `Running`, `Succeeded`, `Failed`
|
||||
|
||||
### Monitoring Integration
|
||||
|
||||
Gateway exposes NGINX metrics (requests, 4xx/5xx, latency P50/P90/P99) via Prometheus. Requires the `whizard-monitoring` extension (optional dependency).
|
||||
|
||||
---
|
||||
|
||||
## Before You Start
|
||||
|
||||
Check if Gateway extension is already installed:
|
||||
|
||||
```bash
|
||||
kubectl get installplans.kubesphere.io gateway --ignore-not-found
|
||||
```
|
||||
|
||||
If found, upgrading is supported — just select a newer version in Step 1.
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
### Step 1: Detect and Select Version
|
||||
|
||||
```bash
|
||||
ALL_VERSIONS=$(kubectl get extensionversions.kubesphere.io \
|
||||
-l kubesphere.io/extension-ref=gateway \
|
||||
-o jsonpath='{range .items[*]}{.spec.version}{"\n"}{end}' | sort -V)
|
||||
|
||||
LATEST_STABLE=$(echo "$ALL_VERSIONS" | grep -v -E 'alpha|beta|rc' | tail -1)
|
||||
if [ -z "$LATEST_STABLE" ]; then
|
||||
LATEST_STABLE=$(echo "$ALL_VERSIONS" | tail -1)
|
||||
fi
|
||||
|
||||
echo "Available versions:"
|
||||
echo "$ALL_VERSIONS"
|
||||
echo ""
|
||||
echo "Latest stable: $LATEST_STABLE"
|
||||
```
|
||||
|
||||
This sets `ALL_VERSIONS` and `LATEST_STABLE`. Use `SELECTED_VERSION` for the version chosen.
|
||||
|
||||
Use the `question` tool:
|
||||
- **`$LATEST_STABLE (Recommended)`** — accept the auto-detected version
|
||||
- *(custom)* — type a specific version; validate it against the printed list
|
||||
|
||||
### Step 2: Detect and Select Clusters
|
||||
|
||||
```bash
|
||||
CLUSTER_DATA=$(kubectl get clusters.cluster.kubesphere.io \
|
||||
-o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.conditions[?(@.type=="Ready")].status}{"\n"}{end}')
|
||||
|
||||
READY_CLUSTERS=$(echo "$CLUSTER_DATA" | awk -F'\t' '$2 == "True" {print $1}')
|
||||
CLUSTER_COUNT=$(echo "$READY_CLUSTERS" | wc -l)
|
||||
|
||||
HOST_CLUSTER=$(kubectl get clusters.cluster.kubesphere.io \
|
||||
-l 'cluster-role.kubesphere.io/host' \
|
||||
-o jsonpath='{.items[0].metadata.name}' || echo "")
|
||||
|
||||
echo "Ready clusters:"
|
||||
echo "$READY_CLUSTERS"
|
||||
echo ""
|
||||
echo "Cluster count: $CLUSTER_COUNT"
|
||||
echo "Host cluster: $HOST_CLUSTER"
|
||||
```
|
||||
|
||||
This sets `READY_CLUSTERS`, `CLUSTER_COUNT`, `HOST_CLUSTER`.
|
||||
|
||||
- **1 cluster** → skip selection, auto-use it. Set `TARGET_CLUSTERS="$HOST_CLUSTER"`
|
||||
- **Multiple clusters** → use `question` with `multiple: true`:
|
||||
- **All clusters** → `TARGET_CLUSTERS="$READY_CLUSTERS"`
|
||||
- **Host cluster only** → `TARGET_CLUSTERS="$HOST_CLUSTER"`
|
||||
- *(custom)* — validate each name against `$READY_CLUSTERS`
|
||||
|
||||
### Step 3: Generate and Apply InstallPlan
|
||||
|
||||
```bash
|
||||
./scripts/generate-installplan.sh "$SELECTED_VERSION" "$TARGET_CLUSTERS"
|
||||
```
|
||||
|
||||
This generates the YAML to `/tmp/gateway-installplan.yaml`, runs `--dry-run=server`, then prints the apply command.
|
||||
|
||||
> For configurable extension values (ingress-nginx default settings, image registry, upgrade tool config, etc.), see [references/extension-values.md](references/extension-values.md).
|
||||
|
||||
Apply it:
|
||||
|
||||
```bash
|
||||
kubectl apply -f /tmp/gateway-installplan.yaml
|
||||
```
|
||||
|
||||
Tell the user "Installing". Then ask if they want to check status. If yes:
|
||||
|
||||
```bash
|
||||
./scripts/check-status.sh poll
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Status Checking
|
||||
|
||||
| Purpose | Command |
|
||||
|---|---|
|
||||
| Single snapshot | `./scripts/check-status.sh quick` |
|
||||
| Wait until complete (5min timeout) | `./scripts/check-status.sh poll` |
|
||||
|
||||
Logic:
|
||||
- **All `Installed`** → ✓ success
|
||||
- **Any `Failed`** → ✗ prints full status
|
||||
- **Timeout (300s)** → ⚠ prints current status
|
||||
- **In progress** → prints every 10s
|
||||
|
||||
---
|
||||
|
||||
## Uninstallation
|
||||
|
||||
> ⚠ **Always confirm with the user before proceeding.**
|
||||
|
||||
### Uninstall from all clusters
|
||||
|
||||
```bash
|
||||
if ! kubectl get installplans.kubesphere.io gateway &>/dev/null; then
|
||||
echo "Gateway is not installed."
|
||||
exit 0
|
||||
fi
|
||||
```
|
||||
|
||||
Confirm with the user, then delete:
|
||||
|
||||
```bash
|
||||
kubectl delete installplans.kubesphere.io gateway --ignore-not-found
|
||||
```
|
||||
|
||||
Verify cleanup:
|
||||
|
||||
```bash
|
||||
./scripts/verify-uninstall.sh
|
||||
```
|
||||
|
||||
Success criteria:
|
||||
1. InstallPlan is deleted
|
||||
2. No active pods remain in `extension-gateway` namespace
|
||||
|
||||
### Uninstall from specific clusters
|
||||
|
||||
> **WARNING**: Do NOT delete the InstallPlan. Only remove target clusters from the placement list.
|
||||
|
||||
Confirm which clusters to remove, compute remaining clusters, then patch:
|
||||
|
||||
```bash
|
||||
kubectl patch installplans.kubesphere.io gateway --type='json' \
|
||||
-p='[{"op": "replace", "path": "/spec/clusterScheduling/placement/clusters", "value": ["<REMAINING_CLUSTER_1>", "<REMAINING_CLUSTER_2>"]}]'
|
||||
```
|
||||
|
||||
Success: patch returns OK + removed clusters no longer in `.status.clusterSchedulingStatuses`.
|
||||
|
||||
---
|
||||
|
||||
## Gateway Operations
|
||||
|
||||
### List Gateways
|
||||
|
||||
Gateways are organized by tier (see [Overview](#overview)), with each tier identified by the label `kubesphere.io/gateway-type`. List them grouped by tier:
|
||||
|
||||
```bash
|
||||
echo "=== Cluster Gateway ==="
|
||||
kubectl get gateways.gateway.kubesphere.io -n kubesphere-controls-system \
|
||||
-l kubesphere.io/gateway-type=cluster
|
||||
|
||||
echo -e "\n=== Workspace Gateways ==="
|
||||
kubectl get gateways.gateway.kubesphere.io -n kubesphere-controls-system \
|
||||
-l kubesphere.io/gateway-type=workspace
|
||||
|
||||
echo -e "\n=== Project Gateways ==="
|
||||
kubectl get gateways.gateway.kubesphere.io -n kubesphere-controls-system \
|
||||
-l kubesphere.io/gateway-type=project
|
||||
```
|
||||
|
||||
### Check Gateway Status
|
||||
|
||||
Pick a gateway name from the [List Gateways](#list-gateways) output and run:
|
||||
|
||||
```bash
|
||||
GW_NS="kubesphere-controls-system"
|
||||
GW_NAME="<gateway-name-from-list>"
|
||||
|
||||
# app.kubernetes.io/instance uses the Helm release name if available, otherwise the Gateway name
|
||||
GW_INSTANCE=$(kubectl get gateways.gateway.kubesphere.io -n $GW_NS $GW_NAME -o jsonpath='{.status.helmRelease.name}' 2>/dev/null)
|
||||
if [ -z "$GW_INSTANCE" ]; then
|
||||
GW_INSTANCE="$GW_NAME"
|
||||
fi
|
||||
|
||||
kubectl get gateways.gateway.kubesphere.io -n $GW_NS $GW_NAME -o wide
|
||||
kubectl describe gateways.gateway.kubesphere.io -n $GW_NS $GW_NAME
|
||||
kubectl get gateways.gateway.kubesphere.io -n $GW_NS $GW_NAME -o yaml
|
||||
kubectl get pods -n $GW_NS -l "app.kubernetes.io/instance=$GW_INSTANCE"
|
||||
```
|
||||
|
||||
Gateway states:
|
||||
| State | Meaning |
|
||||
|---|---|
|
||||
| `Creating` | First-time Helm install in progress |
|
||||
| `Updating` | Helm upgrade in progress (spec changed) |
|
||||
| `Running` | Fully operational (all replicas available) |
|
||||
| `Faulted` | Deployment missing, stopped unexpectedly, or health probe timeout |
|
||||
| `Stopped` | Scaled to zero replicas intentionally |
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
> Set `$GW_NAME` according to the gateway tier being troubleshot (see naming rules in [Overview](#overview)):
|
||||
> - Cluster → `GW_NAME=kubesphere-router-cluster`
|
||||
> - Workspace → `GW_NAME=kubesphere-router-workspace-${WORKSPACE}`
|
||||
> - Project → `GW_NAME=kubesphere-router-${NAMESPACE}`
|
||||
>
|
||||
> Common namespace: `GW_NS=kubesphere-controls-system`
|
||||
>
|
||||
> `$GW_INSTANCE` is auto-resolved from `status.helmRelease.name` (falls back to `$GW_NAME`). If not yet set, run:
|
||||
> ```bash
|
||||
> GW_INSTANCE=$(kubectl get gateways.gateway.kubesphere.io -n $GW_NS $GW_NAME -o jsonpath='{.status.helmRelease.name}' 2>/dev/null)
|
||||
> if [ -z "$GW_INSTANCE" ]; then
|
||||
> GW_INSTANCE="$GW_NAME"
|
||||
> fi
|
||||
> ```
|
||||
|
||||
### Gateway stuck in `Creating` or `Updating` state
|
||||
|
||||
```bash
|
||||
kubectl describe gateways.gateway.kubesphere.io -n $GW_NS $GW_NAME
|
||||
|
||||
kubectl logs -n extension-gateway -l app=gateway-controller-manager --tail=200 | grep -iE "(error|helm|install|upgrade|reconcile)"
|
||||
|
||||
kubectl get deployment -n $GW_NS -l "app.kubernetes.io/instance=$GW_INSTANCE,app.kubernetes.io/component=controller"
|
||||
|
||||
kubectl get configmap -n $GW_NS $GW_NAME -o yaml
|
||||
```
|
||||
|
||||
Common causes: Chart ConfigMap missing/corrupted, Helm wrapper timeout, invalid `spec.values`.
|
||||
|
||||
### Gateway shows `Faulted` state
|
||||
|
||||
```bash
|
||||
kubectl get deployment -n $GW_NS $GW_NAME -o wide
|
||||
kubectl describe deployment -n $GW_NS $GW_NAME
|
||||
kubectl get pods -n $GW_NS -l "app.kubernetes.io/instance=$GW_INSTANCE" -o wide
|
||||
|
||||
POD_NAME=$(kubectl get pods -n $GW_NS -l "app.kubernetes.io/instance=$GW_INSTANCE" -o jsonpath='{.items[0].metadata.name}')
|
||||
kubectl describe pod -n $GW_NS $POD_NAME
|
||||
kubectl logs -n $GW_NS $POD_NAME --tail=100
|
||||
```
|
||||
|
||||
Common causes: Image pull failure, resource constraints, port conflicts, missing ConfigMap/Secret.
|
||||
|
||||
### Gateway pod crash-looping / CrashLoopBackOff
|
||||
|
||||
```bash
|
||||
kubectl logs -n $GW_NS -l "app.kubernetes.io/instance=$GW_INSTANCE" --tail=100 --previous
|
||||
kubectl get events -n $GW_NS --sort-by='.lastTimestamp' | tail -20
|
||||
kubectl exec -n $GW_NS -l "app.kubernetes.io/instance=$GW_INSTANCE" -- cat /etc/nginx/nginx.conf 2>/dev/null | head -50
|
||||
kubectl get configmap -n $GW_NS -l "app.kubernetes.io/instance=$GW_INSTANCE" -o yaml
|
||||
```
|
||||
|
||||
Common causes: Misconfigured nginx config, port conflicts, resource limits (OOMKilled), missing dependencies (ConfigMap/Secret).
|
||||
|
||||
### Log search not working
|
||||
|
||||
Gateway log search proxies to `whizard-telemetry-apiserver`:
|
||||
|
||||
```bash
|
||||
kubectl get configmap -n extension-gateway gateway-agent-backend-config -o yaml
|
||||
kubectl get pods -n extension-whizard-telemetry
|
||||
kubectl get svc -n extension-whizard-telemetry whizard-telemetry-apiserver
|
||||
kubectl logs -n extension-gateway -l app=gateway-apiserver --tail=100 | grep -iE "(log|search|whizard|proxy)"
|
||||
```
|
||||
|
||||
Common causes: Whizard-telemetry not installed or not running, misconfigured `gateway-agent-backend-config`, network policy blocking cross-namespace traffic.
|
||||
|
||||
### Gateway controller not reconciling
|
||||
|
||||
```bash
|
||||
kubectl get pods -n extension-gateway -l app=gateway-controller-manager
|
||||
kubectl logs -n extension-gateway -l app=gateway-controller-manager --tail=200
|
||||
kubectl get validatingwebhookconfiguration -l "app.kubernetes.io/managed-by=Helm,kubesphere.io/extension-ref=gateway"
|
||||
kubectl get deployment -n extension-gateway -l app=gateway-controller-manager -o yaml
|
||||
```
|
||||
|
||||
Common causes: Controller pod not running, webhook configuration blocking updates, Helm release state mismatch, RBAC permission issues.
|
||||
```
|
||||
@@ -0,0 +1,100 @@
|
||||
## Extension Values
|
||||
|
||||
These are configurable values in the InstallPlan's `spec.config` for the Gateway extension.
|
||||
|
||||
To check the actual latest default versions from the installed extension:
|
||||
```bash
|
||||
kubectl get configmap -n kubesphere-system -l kubesphere.io/extension-ref=gateway -o json | python3 -c "import json,sys; [print(c['metadata']['name']) for c in json.load(sys.stdin)['items']]"
|
||||
```
|
||||
|
||||
| Name | Meaning | Default | Range |
|
||||
| --- | --- | --- | --- |
|
||||
| `config.gateway.namespace` | Namespace where gateway components are deployed | `kubesphere-controls-system` | |
|
||||
| `config.gateway.exposeNodeLabelKey` | Node label key for gateway pod scheduling | `node-role.kubernetes.io/control-plane` | |
|
||||
| `config.gateway.versionConstraint` | Allowed ingress-nginx chart version range | — | (e.g. `>= 4.3.0, <= 4.13.9`) |
|
||||
| `config.gateway.logSearchEndpoint` | Endpoint for log search proxy | `http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:9090` | |
|
||||
| `config.gateway.valuesOverride.controller.image.registry` | Container image registry for ingress-nginx | `docker.io` | |
|
||||
| `config.gateway.valuesOverride.controller.image.tag` | ingress-nginx controller image tag | `v1.13.9` | |
|
||||
| `config.upgradeTool.image.repository` | Upgrade tool image repository | `kubesphere/gateway-upgrade-tool` | |
|
||||
| `config.upgradeTool.image.tag` | Upgrade tool image tag | `v1.1.6` | |
|
||||
| `config.upgradeTool.backup.enable` | Enable automatic backup before upgrade | `true` | `true` / `false` |
|
||||
| `config.upgradeTool.backup.persistentVolumeClaim.storageCapacity` | Backup PVC size | `1Gi` | |
|
||||
| `config.helmExecutor.image.registry` | Helm executor image registry | `registry.cn-beijing.aliyuncs.com` | |
|
||||
| `config.helmExecutor.image.tag` | Helm executor image tag | `v1.33.1` | |
|
||||
| `config.helmExecutor.timeout` | Helm operation timeout | `10m` | |
|
||||
| `config.helmExecutor.resources.limits.cpu` | Helm executor CPU limit | `500m` | |
|
||||
| `config.helmExecutor.resources.limits.memory` | Helm executor memory limit | `500Mi` | |
|
||||
|
||||
### Example: Customize node scheduling and resources
|
||||
|
||||
```yaml
|
||||
apiVersion: kubesphere.io/v1alpha1
|
||||
kind: InstallPlan
|
||||
metadata:
|
||||
name: gateway
|
||||
spec:
|
||||
extension:
|
||||
name: gateway
|
||||
version: "1.1.7"
|
||||
enabled: true
|
||||
upgradeStrategy: Manual
|
||||
clusterScheduling:
|
||||
placement:
|
||||
clusters:
|
||||
- "host"
|
||||
config: |
|
||||
config:
|
||||
gateway:
|
||||
namespace: kubesphere-controls-system
|
||||
exposeNodeLabelKey: "node-role.kubernetes.io/ingress"
|
||||
valuesOverride:
|
||||
controller:
|
||||
image:
|
||||
registry: docker.io
|
||||
tag: v1.13.9
|
||||
nodeSelector:
|
||||
kubernetes.io/os: linux
|
||||
tolerations:
|
||||
- key: "node-role.kubernetes.io/ingress"
|
||||
operator: "Exists"
|
||||
effect: "NoSchedule"
|
||||
resources:
|
||||
requests:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
limits:
|
||||
cpu: 2
|
||||
memory: 2Gi
|
||||
upgradeTool:
|
||||
backup:
|
||||
enable: "true"
|
||||
persistentVolumeClaim:
|
||||
storageCapacity: "5Gi"
|
||||
helmExecutor:
|
||||
timeout: 15m
|
||||
resources:
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: 1Gi
|
||||
```
|
||||
|
||||
### Example: Minimal override (image registry only)
|
||||
|
||||
```yaml
|
||||
config: |
|
||||
config:
|
||||
gateway:
|
||||
valuesOverride:
|
||||
controller:
|
||||
image:
|
||||
registry: registry.cn-beijing.aliyuncs.com
|
||||
```
|
||||
|
||||
### Example: Custom log search endpoint
|
||||
|
||||
```yaml
|
||||
config: |
|
||||
config:
|
||||
gateway:
|
||||
logSearchEndpoint: "http://my-custom-logstack.ns.svc:9090"
|
||||
```
|
||||
+77
@@ -0,0 +1,77 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
MODE="${1:-quick}"
|
||||
POLL_INTERVAL="${POLL_INTERVAL:-10}"
|
||||
TIMEOUT="${TIMEOUT:-300}"
|
||||
|
||||
if ! kubectl get installplans.kubesphere.io gateway &>/dev/null; then
|
||||
echo "Gateway InstallPlan not found."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$MODE" = "quick" ]; then
|
||||
echo "=== Extension State ==="
|
||||
echo "State: $(kubectl get installplans.kubesphere.io gateway -o jsonpath='{.status.state}')"
|
||||
echo "Reason: $(kubectl get installplans.kubesphere.io gateway -o jsonpath='{.status.conditions[-1].reason}')"
|
||||
echo "Message: $(kubectl get installplans.kubesphere.io gateway -o jsonpath='{.status.conditions[-1].message}')"
|
||||
|
||||
echo ""
|
||||
echo "=== Per-Cluster Agent States ==="
|
||||
kubectl get installplans.kubesphere.io gateway -o go-template='
|
||||
{{- range $cluster, $status := .status.clusterSchedulingStatuses}}
|
||||
{{$cluster}}{{"\t"}}{{$status.state}}{{"\t"}}{{if $status.conditions}}{{(index $status.conditions 0).reason}}{{end}}{{"\n"}}
|
||||
{{- end}}' | column -t
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Polling mode
|
||||
ELAPSED=0
|
||||
|
||||
while [ $ELAPSED -lt $TIMEOUT ]; do
|
||||
EXT_STATE=$(kubectl get installplans.kubesphere.io gateway -o jsonpath='{.status.state}' 2>/dev/null)
|
||||
|
||||
if [ -z "$EXT_STATE" ]; then
|
||||
echo "[$ELAPSED s] InstallPlan not found yet..."
|
||||
else
|
||||
AGENT_STATES=$(kubectl get installplans.kubesphere.io gateway \
|
||||
-o jsonpath='{.status.clusterSchedulingStatuses.*.state}' 2>/dev/null || true)
|
||||
|
||||
echo "[$ELAPSED s] Extension: $EXT_STATE | Agents: $AGENT_STATES"
|
||||
|
||||
ALL_STATES="$EXT_STATE $AGENT_STATES"
|
||||
|
||||
ALL_INSTALLED=true
|
||||
for state in $ALL_STATES; do
|
||||
if [ "$state" != "Installed" ]; then
|
||||
ALL_INSTALLED=false
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
if [ "$ALL_INSTALLED" = true ]; then
|
||||
echo ""
|
||||
echo "✓ All Gateway components installed successfully!"
|
||||
kubectl get installplans.kubesphere.io gateway -o go-template='
|
||||
{{- range $cluster, $status := .status.clusterSchedulingStatuses}}
|
||||
{{$cluster}}{{"\t"}}{{$status.state}}{{"\n"}}
|
||||
{{- end}}' | column -t
|
||||
exit 0
|
||||
fi
|
||||
|
||||
for state in $ALL_STATES; do
|
||||
if [ "$state" = "Failed" ]; then
|
||||
echo "✗ Installation failed! Check details:"
|
||||
kubectl get installplans.kubesphere.io gateway -o yaml
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
sleep $POLL_INTERVAL
|
||||
ELAPSED=$((ELAPSED + POLL_INTERVAL))
|
||||
done
|
||||
|
||||
echo "⚠ Timeout reached ($TIMEOUT s). Current status:"
|
||||
kubectl get installplans.kubesphere.io gateway -o yaml
|
||||
exit 1
|
||||
@@ -0,0 +1,52 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SELECTED_VERSION="${1:-${SELECTED_VERSION:-}}"
|
||||
TARGET_CLUSTERS="${2:-${TARGET_CLUSTERS:-}}"
|
||||
|
||||
if [ -z "$SELECTED_VERSION" ] || [ -z "$TARGET_CLUSTERS" ]; then
|
||||
echo "Usage: $0 <version> <clusters>"
|
||||
echo " clusters: space-separated list of cluster names (e.g. 'host' or 'host member1')"
|
||||
echo " or set SELECTED_VERSION and TARGET_CLUSTERS env vars"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! command -v kubectl &>/dev/null; then
|
||||
echo "Error: kubectl not found in PATH"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
INSTALLPLAN_FILE="/tmp/gateway-installplan.yaml"
|
||||
|
||||
cat > "$INSTALLPLAN_FILE" <<YAML
|
||||
apiVersion: kubesphere.io/v1alpha1
|
||||
kind: InstallPlan
|
||||
metadata:
|
||||
name: gateway
|
||||
spec:
|
||||
extension:
|
||||
name: gateway
|
||||
version: "${SELECTED_VERSION}"
|
||||
enabled: true
|
||||
upgradeStrategy: Manual
|
||||
clusterScheduling:
|
||||
placement:
|
||||
clusters:
|
||||
YAML
|
||||
|
||||
for cluster in $TARGET_CLUSTERS; do
|
||||
echo " - \"${cluster}\"" >> "$INSTALLPLAN_FILE"
|
||||
done
|
||||
|
||||
echo "=== Generated InstallPlan ==="
|
||||
cat "$INSTALLPLAN_FILE"
|
||||
echo ""
|
||||
|
||||
echo "=== Dry-run validation ==="
|
||||
if ! kubectl apply --dry-run=server -f "$INSTALLPLAN_FILE"; then
|
||||
echo "✗ Dry-run failed — check version '${SELECTED_VERSION}' and cluster list."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "✓ InstallPlan validated. Apply with: kubectl apply -f $INSTALLPLAN_FILE"
|
||||
+33
@@ -0,0 +1,33 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
NAMESPACE="${NAMESPACE:-extension-gateway}"
|
||||
TIMEOUT="${TIMEOUT:-300}"
|
||||
ELAPSED=0
|
||||
|
||||
while [ $ELAPSED -lt $TIMEOUT ]; do
|
||||
if kubectl get installplans.kubesphere.io gateway &>/dev/null; then
|
||||
echo " Waiting for InstallPlan to be deleted... (${ELAPSED}s)"
|
||||
sleep 10
|
||||
ELAPSED=$((ELAPSED + 10))
|
||||
continue
|
||||
fi
|
||||
|
||||
REMAINING=$(kubectl get pods -n "$NAMESPACE" 2>/dev/null | grep -v -E 'uninstaller|Completed|NAME' | wc -l) || true
|
||||
|
||||
if [ "$REMAINING" -eq 0 ]; then
|
||||
echo ""
|
||||
echo "✓ Uninstallation complete — all Gateway components removed from $NAMESPACE."
|
||||
echo " InstallPlan: deleted"
|
||||
echo " Remaining pods: none"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo " Waiting for components to be removed... (${ELAPSED}s, $REMAINING pods remaining)"
|
||||
sleep 10
|
||||
ELAPSED=$((ELAPSED + 10))
|
||||
done
|
||||
|
||||
echo "⚠ Timeout reached. Remaining pods in $NAMESPACE:"
|
||||
kubectl get pods -n "$NAMESPACE" 2>/dev/null | grep -v -E 'uninstaller|Completed'
|
||||
exit 1
|
||||
Reference in New Issue
Block a user