How to install DS3 Gateway

Welcome to the DS3 Gateway installation guide. The DS3 Gateway is the primary interface between external clients and the DS3 Swarm, providing an S3-compatible API for seamless integration with existing applications and tools. Each gateway instance is a lightweight, stateless service that handles:

S3 API handling: Terminates and processes client requests according to DS3 semantics.
Encryption: Performs client-side encryption, ensuring data is secure before transmission to the Swarm.
Data chunking & redundancy: Splits objects into chunks, applies erasure coding based on your Redundancy Class, and prepares shards for distribution via the Offloader component.
Shard distribution: Routes encoded shards to Agents in selected Rings across one or more Nexuses.
Optional caching: Accelerates workloads with local or shared caches for repeated or latency-sensitive operations.

Multiple gateways can run in parallel—on-premises, in the cloud, or at the edge—scaling independently of the storage backend and easily load-balanced for high throughput and high availability. To read more about DS3 Gateway, please refer to this section

Before you begin

Before you begin, verify the following system prerequisites

Kubernetes cluster A conformant cluster (on-prem, cloud, or edge).
tip
If you don’t have one, see the the section below.
DNS & SSL (recommended)
- Custom domain (e.g., example.com)
- Valid TLS certificate(s)

You will configure DNS & SSL in the Composer UI under Domain & TLS.

How to install

0. Provision a DS3 Gateway

In the Composer UI, navigate to Tenant → Settings → Gateway → Create your private gateway.
(Optional, but recommended) In Composer UI under Tenant → Settings → Gateway → DNS, add your custom domain if you plan to expose the gateway on a friendly hostname. Then follow the guide on the UI and create the following DNS records.

Composer UI will prompt you to create a DNS TXT record to verify domain ownership:

<your-domain>  TXT  <verification-code>

After verification, create these A records pointing to your Kubernetes ingress IP(s):

console.<domain>     A    <ingress-ip>
s3.<domain>          A    <ingress-ip>
*.s3.<domain>        A    <ingress-ip>

Complete the wizard; at the end you’ll receive the exact installer command (a curl | bash line). Copy this for the next step.

1. Run Installer

Run the installer script to deploy the DS3 Gateway software onto your Kubernetes cluster, creating all necessary Kubernetes resources, services, and ingress rules. Paste and execute the command you obtained from Composer. It should resemble:

curl -fSsL https://installer.s3.cubbit.eu/gateway/installer.sh | bash -s -- \
  --tenant-name <TENANT_NAME> \
  --tenant-id <TENANT_ID> \
  --domain <YOUR_DOMAIN> \
  --gateway-id <GATEWAY_ID> \
  --gateway-secret <GATEWAY_SECRET>

Argument	Description
`--domain`	The base domain under which the DS3 Gateway will be exposed (e.g., `example.com`). Ingress rules will use `console.<domain>`, `s3.<domain>`, and `*.s3.<domain>`.
`--gateway-id`	A UUID that uniquely identifies this gateway instance.
`--gateway-secret`	A secret key associated with the gateway for authentication between the gateway and the Coordinator.
`--tenant-name`	The human-readable name of the tenant you are installing the gateway for.
`--tenant-id`	A UUID that uniquely identifies the tenant within your organization.

Lost the installer command or the gateway secret? See Recovering lost installer parameters.

note

The installer will use the Kubernetes context currently set. If you want to override this behavior, you can target a custom kubeconfig using the KUBECONFIG env variable like:

export KUBECONFIG=/path/to/your/kubeconfig

What Happens During Installation

The installer script will:

Connect to Kubernetes using your kubeconfig
Create or use a dedicated namespace
Deploy Cubbit Kubernetes Operator resources
Configure necessary ConfigMaps and Secrets
Set up ingress resources for:
- console.<domain>
- s3.<domain>
- *.s3.<domain>
Apply any cache or override configurations
Configure TLS certificates for HTTPS endpoints

Installer Help & Flags

Run the installation script with --help to view all options:

curl -fSsL https://installer.s3.cubbit.eu/gateway/installer.sh | bash -s -- --help
Usage: bash [OPTIONS]
  --help                      Display this help message
  --version                   Print version
  --domain DOMAIN             Set the domain
  --tenant-name NAME          Set the tenant name
  --tenant-id ID              Set the tenant ID
  --gateway-id ID             Set the gateway ID
  --gateway-secret SECRET     Set the gateway secret
  --cache PATH                Set the cache
  --force                     Force removal
  --uninstall                 Uninstall the application
  --no-tls                    Disable TLS
  --no-init                   Disable init node
  --no-infra                  Disable infrastructure setup
  --no-app                    Disable application setup
  --ingress                   Install only the ingress
  --no-console                Disable console setup
  --no-offloader              Disable offloader setup
  --no-s3                     Disable S3 setup
  --coordinator-domain DOMAIN Set the coordinator domain [eu00wi.cubbit.services]
  --cert-root PATH            Set the certificate root path [./cert]

Recovering lost installer parameters

The installer command, including parameters such as --gateway-secret, is shown only once when the gateway is provisioned, and the installer stores no configuration file on disk. You will need these parameters again to run --uninstall, to update certificates with --ingress, or to re-install on another cluster — and they must match the values used for the original installation exactly. If you no longer have the command, you can recover each parameter from the Kubernetes cluster itself via kubectl.

While the gateway is installed, all the values are readable from the objects the installer created on the cluster. Export the kubeconfig of the target cluster first:

export KUBECONFIG=/path/to/your/kubeconfig

Then run:

# gateway ID, gateway secret and coordinator domain
kubectl -n cubbit get deployment diplomat \
  -o jsonpath='{range .spec.template.spec.containers[*].env[*]}{.name}={.value}{"\n"}{end}' \
  | grep -E 'GATEWAY_(ID|SECRET)|COMPOSER_HUB_SERVICE_ADDRESS'

# tenant name, tenant ID, domain, coordinator domain
kubectl -n cubbit get ingress -o yaml \
  | grep -E 'x-cbt-tenant-(id|name)|cors-allow-origin|- host:'

Each installer flag maps to a value on the cluster as follows:

Installer flag	Where the value is found
`--gateway-id`	The `GATEWAY_ID` environment variable of the `diplomat` deployment in the `cubbit` namespace.
`--gateway-secret`	The `GATEWAY_SECRET` environment variable of the same deployment (stored in clear text).
`--tenant-id`	The `x-cbt-tenant-id` value, set inside the `configuration-snippet` annotation of the `console.<domain>` and `s3.<domain>` Ingresses (visible in their `-o yaml` output).
`--tenant-name`	The `x-cbt-tenant-name` value, set inside the `configuration-snippet` annotation of the same Ingresses. It is also the suffix of the `cert-console-<tenant-name>`, `cert-gw-<tenant-name>` and `cert-gw-wc-<tenant-name>` Secrets.
`--domain`	The `console.<domain>` and `s3.<domain>` Ingress names, without the `console.` or `s3.` prefix. The `s3.<domain>` Ingress also has a host rule for `*.s3.<domain>` — ignore it when reading the values.
`--coordinator-domain`	Only required when the gateway runs on a non-default coordinator — it renders the CORS annotation on regenerated Ingresses, so pass the original value for `--ingress`, uninstall and re-installations to avoid a wrong CORS setting. Found as `COMPOSER_HUB_SERVICE_ADDRESS=https://api.<coordinator-domain>/composer-hub` in the `diplomat` deployment, or in the second entry of the `cors-allow-origin` annotation (`https://.cubbit.eu,https://.<coordinator-domain>`), dropping the `https://*.` prefix.

note

The values stored in the cluster are the ones the resources were actually created with, so they are authoritative. If a lookup returns nothing, check that KUBECONFIG points to the right cluster and that you are querying the cubbit namespace.

warning

These lookups only work while the gateway is installed. Recover the values before uninstalling — after the gateway is removed they are no longer available on the cluster.

2. Required Dependencies

The installer sandbox‑downloads the following binaries into a local folder (it does not install them system‑wide):

kubectl
helm
yq

This behavior is controlled via the following environment variables:

Env Var	Behavior
SKIP_BINARY=1	Use versions of required utilities already installed on your machine.
FORCE_BINARY=1	Re-download binaries into the sandbox even if they exist locally.

3. Configuration

Domain & TLS (recommended)

In the Composer UI under Tenant → Settings → Gateway → DNS, add your custom domain.

The installer script will create ingress rules for:

console.<domain>
s3.<domain>
*.s3.<domain> (for virtual‑host style bucket URLs)

Place your TLS certificate(s) and private key(s) in a folder called ./cert/. You have two options here:

Option A: Single Certificate
Option B: Service-Specific Certificates

Use one certificate for all services:

cert/
  ├─ fullchain.pem   # Combined certificate
  └─ privkey.pem     # Private key

Separate certificates per service:

cert/
  ├─ console/           # For console.<domain>
  │   ├─ fullchain.pem
  │   └─ privkey.pem
  ├─ gateway/           # For s3.<domain>
  │   ├─ fullchain.pem
  │   └─ privkey.pem
  └─ gateway-wildcard/  # For *.s3.<domain>
      ├─ fullchain.pem
      └─ privkey.pem

tip

For SSL certificate generation:

Use any method you prefer (Let’s Encrypt, CA, etc.).

For a quick Certbot DNS example:

sudo certbot certonly --manual \
  -d '*.s3.<your-domain>' \
  -d 's3.<your-domain>' \
  -d 'console.<your-domain>' \
  --preferred-challenges dns

This command will generate a unique certificate covering the 3 subdomains. You can then copy the resulting keypair to the ./cert/ folder following theOption A.above. If you prefer to create 3 separate certificates, you can run the command above 3 times, one for each subdomain. In this case, make sure to useOption B` above instead.

To disable TLS completely you can use the flag.

--no-tls

warning

If you disable SSL, the private Web Console will not work. You will have to generate keys and interact with the gateway from the CLI.

Updating TLS Certificates

TLS certificates expire and need periodic renewal. To update your certificates without a full re-installation, the installer provides a dedicated --ingress flag that regenerates only the ingress resources and TLS Secrets, without affecting the running gateway software.

Place your new certificate files in the ./cert/ folder using the same structure as during initial installation (see Domain & TLS above), or recover them from the cluster if you no longer have them — see Recovering the certificate from the cluster.
Run the installer with the --ingress flag. Include --coordinator-domain with the value recovered above when the gateway is not on the default coordinator, otherwise the regenerated Ingresses get the wrong CORS setting. --yes skips the interactive confirmation prompt, which cannot appear when the installer is piped from curl:
```
curl -fSsL https://installer.s3.cubbit.eu/gateway/installer.sh | bash -s -- \
  --tenant-name <TENANT_NAME> \
  --tenant-id <TENANT_ID> \
  --domain <YOUR_DOMAIN> \
  --coordinator-domain <YOUR_COORDINATOR_DOMAIN> \
  --gateway-id <GATEWAY_ID> \
  --gateway-secret <GATEWAY_SECRET> \
  --ingress --yes
```
This will update the Kubernetes TLS Secrets and re-apply the Ingress configuration with the new certificates.

Recovering the certificate from the cluster

If you no longer have the original fullchain.pem and privkey.pem files, you can extract the certificate the installer deployed from the Kubernetes TLS Secrets — one Secret per endpoint:

cert-console-<tenant-name> — the console.<domain> endpoint
cert-gw-<tenant-name> — the s3.<domain> endpoint
cert-gw-wc-<tenant-name> — the *.s3.<domain> hostnames

List the Secrets and inspect the certificate each one holds:

kubectl -n cubbit get secret | grep '^cert-'

for secret in $(kubectl -n cubbit get secret -o name | grep '/cert-'); do
  echo "== ${secret#secret/}"
  kubectl -n cubbit get "$secret" -o jsonpath='{.data.tls\.crt}' \
    | base64 -d | openssl x509 -noout -subject -ext subjectAltName
done

Use the Secret whose subjectAltName covers your hostnames, then write the files into the ./cert/ structure the installer expects — the installer reads fullchain.pem and privkey.pem from its --cert-root folder (default ./cert, with the Option B sub-folders taking precedence), so recovering them anywhere else has no effect (see Domain & TLS above for the Option A / Option B layouts):

umask 077
mkdir -p ./cert
kubectl -n cubbit get secret cert-gw-<tenant-name> -o jsonpath='{.data.tls\.crt}' \
  | base64 -d > ./cert/fullchain.pem
kubectl -n cubbit get secret cert-gw-<tenant-name> -o jsonpath='{.data.tls\.key}' \
  | base64 -d > ./cert/privkey.pem

The umask 077 line restricts the created files to the current user only; without it the private key would be created world-readable with a typical default umask. If ./cert/privkey.pem already exists, also verify its permissions afterwards:

chmod 600 ./cert/privkey.pem ./cert/fullchain.pem

The commands above extract a single Secret into the Option A root layout: that is only correct when the chosen Secret's subjectAltName covers console.<domain>, s3.<domain> and *.s3.<domain>. When a separate Secret exists for each endpoint, extract each one into its Option B sub-folder instead (cert-console-<tenant-name> into ./cert/console/, cert-gw-<tenant-name> into ./cert/gateway/, and cert-gw-wc-<tenant-name> into ./cert/gateway-wildcard/), replacing cert-gw-<tenant-name> and the output path in the commands above.

note

The decoded tls.crt is exactly the certificate chain that was supplied to the installer at install time, so write it as fullchain.pem. The Secret names map to the Option B folders: cert-console-<tenant-name> to console/, cert-gw-<tenant-name> to gateway/, and cert-gw-wc-<tenant-name> to gateway-wildcard/.

warning

The private key is sensitive — treat the recovered privkey.pem with the same care as the original. Also note that removing the gateway or the namespace removes these Secrets as well: extract the files before uninstalling.

If the recovered certificate is expired or does not cover your domain, issue a fresh one following Domain & TLS (recommended).

Cache Configuration

To optimize the upload process to storage nodes, include the --cache argument in your installer command:

--cache /path/to/cache

When specified, this path serves as a local disk cache, reducing the need for a full sync each time the gateway software transfers data to or from the Swarm. If you omit --cache, the gateway defaults to performing a full sync for every operation.

Key considerations:

Disk space & permissions: Ensure the cache path has sufficient capacity and that the gateway process can read/write to it.
Per-node local cache: In multi-node clusters, each node maintains its own cache. Because these caches are not shared, you may encounter inconsistent data availability or synchronization issues between nodes.
Shared cluster cache: For consistent caching across the cluster, mount a shared volume (e.g., NFS, GlusterFS) at the same path on every node, and point --cache to that network-mounted directory.

For more advanced setups or troubleshooting cache consistency, please contact our support team for guidance.

note

In multi‑node clusters, local caches are not shared. Consider a network‑mounted cache for consistency.

Custom Overrides

To customize service settings, drop YAML files in an overrides/ folder where you run the installer script. These act like Helm values:

# overrides/gods3.yaml
hpa:
  enabled: false  # Enables or disables Horizontal Pod Autoscaling for the service
resources:        # Kubernetes resource limits and requests
  requests:       # Minimum resources required for the service
    memory: 1Gi   # Memory requested by the service
    cpu: 1000m    # CPU requested by the service
  limits:         # Maximum resources the service can consume
    memory: 1Gi   # Memory limit for the service

# overrides/offloader.yaml
offset: 5         # Sets the number of offloader instances to be deployed
env:              # Environment variables for additional configuration
  - name: UPLOADER_OPERATION_QUEUE_MAX_SIZE
    value: "16"   # Maximum size of the uploader operation queue. A higher value allows more operations to be queued but requires more resources and is also limited by the number of open connections
  - name: CCCP_CONCURRENCY
    value: "128"  # Number of concurrent operations, ideally set as total redundancy connections multiplied by `UPLOADER_OPERATION_QUEUE_MAX_SIZE`

How to uninstall

You can uninstall using the installer script. The installer validates its five required parameters (everything except --coordinator-domain, which defaults to eu00wi.cubbit.services) on every invocation, so pass the same values used at install time, recovered with the steps above if needed. --yes skips the interactive confirmation prompt, which cannot appear when the installer is piped from curl:

curl -fSsL https://installer.s3.cubbit.eu/gateway/installer.sh | bash -s -- \
  --uninstall --yes \
  --tenant-name <tenant-name> \
  --tenant-id <tenant-id> \
  --domain <domain> \
  --coordinator-domain <coordinator-domain> \
  --gateway-id <gateway-id> \
  --gateway-secret <gateway-secret>

tip

If you used our k3s bootstrap script to install your Kubernetes cluster (see Extra: Installing Kubernetes (k3s)), be sure to follow the steps in that section to tear down k3s and any Cilium interfaces.

Setting Up Kubernetes

If you don’t already have a Kubernetes cluster (required to install the DS3 Gateway software), you can quickly bootstrap one using our k3s setup script. Simply copy and paste the command below:

curl -fSsL https://installer.s3.cubbit.eu/k8s-setup.sh | bash -s -- \
  --user ubuntu \
  --ssh-key ./id_rsa \
  --contexts ./cluster-contexts.txt

tip

Refer to the Context File section below for the syntax of cluster-contexts.txt.

This installer supports both single-node and multi-node clusters. For alternative topologies or advanced configuration, refer to the k3s documentation or contact support.

The installer will:

Read the context file to determine clusters.
Download or reuse k3sup and kubectl per env settings.
Bootstrap k3s on each node via k3sup.
Configure the selected CNI plugin.

Output

After completion, look for a config/ folder containing:

Per-context kubeconfigs: kubeconfig-<context> for each cluster.
Merged kubeconfig: A single file named kubeconfig including all contexts, enabling unified cluster management.

Prerequisites

Passwordless sudo user: The target machine(s) must have a user with passwordless sudo.
SSH key authentication: SSH keys set up between your control machine and targets. (Local installs don’t require SSH.)

CNI Plugin Selection

Choose the Container Network Interface plugin at install time:

Flannel (default)
Cilium (add --cilium)

note

Once k3s is installed with a given CNI, switching later isn’t supported.

Dependencies

The script sandbox-downloads two binaries into a local folder (does not install system-wide):

k3sup (for bootstrapping over SSH)
kubectl

Control behavior with environment variables:

Env variable	Description
`SKIP_BINARY=1`	Use system-installed binaries instead of sandbox downloads.
`FORCE_BINARY=1`	Re-download binaries even if they already exist locally.

Usage

Run with --help to list all options:

Usage: bash --contexts CONTEXT_FILE [--ssh-key SSH_KEY_PATH] [--user USER]
  --help                      Display this help message
  --contexts CONTEXT_FILE     The file with the list of contexts to create
  --ssh-key SSH_KEY_PATH      The path to the SSH private key
  --user USER                 The username for SSH connections
  --cilium                    Use cilium for CNI
  --no-test                   Skip the connectivity test

Flag	Description
`--contexts`	Path to a context file defining clusters (see format below).
`--user USER`	SSH user with passwordless sudo (omit if running locally).
`--ssh-key PATH`	SSH private key for authentication (omit if running locally).
`--cilium`	Use Cilium CNI instead of Flannel.
`--no-test`	Skip pre-flight connectivity tests.

Context file

The context file is a plain text file, one cluster node per line:

<k8s-context>,<ip>[,<ssh-port>]

<k8s-context>: A name for the cluster context (e.g., ds3-gateway).
<ip>: Node IP (use localhost for local installs).
<ssh-port>: Optional SSH port (defaults to 22).

Example

# Two nodes in 'ds3-gateway' cluster, one local single-node cluster
ds3-gateway,192.168.1.1,22
ds3-gateway,192.168.1.2,2222
localhost,192.168.1.3,22

Nodes sharing the same context join into one cluster; any localhost entry always creates a single-node cluster.

Uninstall Kubernetes

If you installed k3s using Cilium as the CNI, you may need to remove any related network interfaces. Run these commands only if Cilium was your chosen CNI:

sudo ip link delete cilium_host
sudo ip link delete cilium_net
sudo ip link delete cilium_vxlan

To remove the Kubernetes cluster (k3s) from the target machine, k3s provides a built-in uninstallation script called k3s-uninstall.sh. Run the following command with appropriate permissions on the target machine:

sudo /usr/local/bin/k3s-uninstall.sh

These steps stop and remove all k3s-related services, configurations, and data, restoring the machine to its pre-installation state.

Before you begin​

How to install​

0. Provision a DS3 Gateway​

1. Run Installer​

What Happens During Installation​

Installer Help & Flags​

Recovering lost installer parameters​

2. Required Dependencies​

3. Configuration​

Domain & TLS (recommended)​

Updating TLS Certificates​

Recovering the certificate from the cluster​

Cache Configuration​

Custom Overrides​

How to uninstall​

Setting Up Kubernetes​

Output​

Prerequisites​

CNI Plugin Selection​

Dependencies​

Usage​

Context file​

Example​

Uninstall Kubernetes​

Before you begin

How to install

0. Provision a DS3 Gateway

1. Run Installer

What Happens During Installation

Installer Help & Flags

Recovering lost installer parameters

2. Required Dependencies

3. Configuration

Domain & TLS (recommended)

Updating TLS Certificates

Recovering the certificate from the cluster

Cache Configuration

Custom Overrides

How to uninstall

Setting Up Kubernetes

Output

Prerequisites

CNI Plugin Selection

Dependencies

Usage

Context file

Example

Uninstall Kubernetes