DS3 Bridge

DS3 Bridge lets you move your buckets from another S3-compatible provider (AWS, Wasabi, etc.) into Cubbit with zero downtime. You create a bridge for the buckets you want to migrate, and from that moment you can point your applications at Cubbit and keep working normally — reads, writes, and deletes all behave as if the bucket were already fully on Cubbit, even while the underlying data is still being migrated over in the background.

Experimental feature

DS3 Bridge is currently experimental. It has a limited, well-defined scope (see Limitations below).

How it works

You create a bridge for one or more buckets, providing read/write credentials for the source S3 provider and your Cubbit credentials. This creates the matching destination buckets on Cubbit, but does not start copying any objects yet — the bridge and its buckets are created in a ready state, waiting for you to start the migration.
Switch your applications to Cubbit right away — point them at your Cubbit endpoint instead of the source provider. Cubbit transparently serves your data no matter where it currently lives, even before migration has started: reads and listings are served straight from the source, and new uploads go to Cubbit directly. Because the migration hasn't started yet, nothing has been deleted from the source at this point — this is a safe window to verify the switch works as expected before committing to the actual migration.
When you're ready, start the migration by calling the start migration endpoint. This is the point where DS3 Bridge begins copying objects from the source, one by one, in the background. Each object is deleted from the source as soon as it has been copied to Cubbit — the source bucket is progressively emptied as the migration advances, so make sure you no longer need direct access to it once the migration is started.
From then on, Cubbit keeps transparently serving your data no matter where it currently lives:
- Reading or listing an object that hasn't been migrated yet is served straight from the source, so nothing is ever missing.
- Once an object has been copied to Cubbit, it's removed from the source and served from Cubbit from then on.
- New uploads always go to Cubbit directly.
The migration finishes on its own. Once every object in the bucket has been walked through, that bucket is marked completed. Objects that failed to migrate are not retried automatically and are not deleted from the source, so a bucket can reach completed while a few objects remain un-migrated — check the failed count (see Tracking migration progress) to spot this.

Create the bridge
        │
        ▼
Switch your applications to Cubbit right away
(You may validate the switch works seamlessly)
        │
        ▼
Start the migration
        │
        ▼
DS3 Bridge migrates objects from the source, bucket by bucket
        │
        ▼
 Reads: served by Cubbit (proxied from source S3 if not migrated)
 Writes: go straight to Cubbit
        │
        ▼
 Bucket is marked "completed" once every object has been processed

important

Once the migration is started for a bucket, stop using the source bucket directly and rely on Cubbit exclusively. Continuing to write to the source bucket outside of Cubbit after the migration has started can break the migration.

Prerequisites

A Cubbit project and Cubbit API credentials (access key and secret).
The source buckets must not have Object Lock enabled. Bridges cannot be created for buckets that don't meet this condition.

Permanent IAM credentials for the source S3 provider — temporary/STS credentials are not supported (no session token field is sent). The identity needs the following permissions:

Level	Permissions
Account	`ListAllMyBuckets`
Bucket (per bucket to migrate)	`GetBucketLocation`, `GetBucketVersioning`, `GetBucketObjectLockConfiguration`, `GetBucketOwnershipControls`, `ListBucketVersions`, `ListBucket`
Object (per bucket to migrate)	`GetObject`, `GetObjectTagging`, `GetObjectAcl`, `PutObjectAcl`, `PutObjectTagging`, `DeleteObject`, `DeleteObjectTagging`

Creating a bridge

tip

Prefer a guided flow? Migrating with the CLI walks you through credential entry, bucket selection, verification, migration start and live monitoring from the terminal — no API calls by hand.

Once DS3 Bridge is enabled for your gateway, you create a migration bridge with a single API call:

info

For Cubbit's public gateway, the DS3 Bridge API is available at https://s3bridge.cubbit.eu.

Endpoint: POST /bridges

Request body:

{
    "source_connection": {
        "api_key": "string",
        "secret": "string",
        "endpoint": "string",
        "region": "string",
        "use_path_style": false
    },
    "cubbit_credentials": {
        "api_key": "string",
        "secret": "string"
    },
    "tenant_id": "string",
    "bucket_names": ["string"]
}

source_connection.use_path_style is required and tells DS3 Bridge how to address buckets on the source provider:

false is the normal choice — it lets the SDK use virtual-hosted-style addressing (https://bucket.endpoint/key) when possible, falling back to path-style automatically if needed. This is how AWS S3 and most S3-compatible providers work.
true forces path-style URLs (https://endpoint/bucket/key) instead. This is only needed in special cases — some, typically older or self-hosted, S3-compatible providers only support path-style requests.

Response:

A successful call returns 201 Created with the ID of the new bridge:

{ "id": "<uuid>" }

note

You can only have one bridge per project, if you need another one delete the current bridge first.

Starting migration

Start the migration process for an already created bridge.

Endpoint: POST /bridges/start-migration

Request headers:

Header	Required	Description
x-cbt-api-key	Yes	Cubbit API key
x-cbt-api-secret	Yes	Cubbit API secret
x-cbt-tenant-id	Yes	Tenant UUID

A successful call returns 204 No Content with no body. This moves every bucket in the bridge from ready to in-progress, making them eligible for migration. This operation is idempotent.

Tracking migration progress

Get information about migration progress:

Endpoint: GET /bridges

Request headers:

Header	Required	Description
x-cbt-api-key	Yes	Cubbit API key
x-cbt-api-secret	Yes	Cubbit API secret
x-cbt-tenant-id	Yes	Tenant UUID

Response body:

{
    "status": "in-progress",
    "buckets": [
        {
            "bucket_name": "my-bucket",
            "status": "in-progress",
            "migrated": 1204,
            "locked": 0,
            "failed": 0
        }
    ]
}

Each bucket's status is ready before the migration is started, in-progress while it's being migrated, and completed once it has finished.
The bridge-level status follows the same three values, based on its buckets: in-progress if any bucket is in-progress; otherwise ready if any bucket is still ready; otherwise completed (only once every bucket has finished).
migrated counts objects that were copied to Cubbit successfully. failed counts objects that could not be migrated — they remain on the source. locked is reserved for future use and is currently always 0.

Deleting a bridge

Remove a bridge and all of its tracked bucket state:

Endpoint: DELETE /bridges

Request headers:

Header	Required	Description
x-cbt-api-key	Yes	Cubbit API key
x-cbt-api-secret	Yes	Cubbit API secret
x-cbt-tenant-id	Yes	Tenant UUID

A successful call returns 204 No Content with no body. This operation is idempotent. You can delete a bridge even while migration is in progress.

important

Deleting a bridge only removes DS3 Bridge's own tracking of it — it does not undo any migration already performed. Objects already copied to Cubbit stay there, and anything already deleted from the source is not restored.

Limitations

These fall into two groups: permanent limits, which are architectural constraints and are not planned to change, and items on the roadmap, which reflect the current state and will be addressed in future.

Permanent limits:

Existing bucket name conflicts — if a bucket with the same name already exists in your Cubbit tenant it cannot be migrated (unless the existing bucket has been created by the Bridge migrator).
Source bucket must not be used after migration is started — switch to Cubbit for the bucket as soon as the bridge is created, and stop using the source directly once you start the migration.
Objects in Glacier (or Glacier Deep Archive) on the source are not restored automatically, so they will fail to migrate until you restore them yourself.
Deleting a bridge doesn't undo migration — it only removes DS3 Bridge's own tracking; objects already copied to Cubbit stay there, and anything already deleted from the source isn't restored.
Object Lock is not supported — buckets with Object Lock enabled (now or in the past) cannot be migrated.
While a bucket is mid-migration:
- Pending multipart uploads on the source are not migrated and can't be resumed after the bridge is created.
- CopyObject / UploadPartCopy are blocked.
- Changing bucket versioning/Object Lock settings, or deleting the bucket, is blocked.

On the roadmap:

One bridge per project.
Metadata can look inconsistent while a bucket is mid-migration — fetching the same object twice during migration can return either its original source metadata or its final Cubbit metadata, depending on whether it has been migrated yet.
Cubbit-to-Cubbit migrations are not supported. This will fail when the source is the very same gateway you're migrating into.
Path-style access only — the migrator always addresses the source using path-style URLs. Most providers support this, but some newer AWS regions restrict it, which can prevent migrating buckets from those regions.

Mixed: permanent & roadmap:

Some categories contain a mix of permanent limits and items on the roadmap:

Bucket properties that aren't carried over:
- Permanent: any bucket feature Cubbit doesn't support, and region (always mapped to Cubbit's eu-west region).
- On the roadmap: lifecycle rules and ACLs (Cubbit applies a default private ACL instead).
Object properties that aren't carried over:
- Permanent: ETag (large/multipart objects may end up with a different ETag on Cubbit than on the source), storage class (objects land in the default STANDARD class), and some headers/features (Content-Language, Expires, website redirects, server-side encryption).
- On the roadmap: ACLs (default ACLs are used instead).

Migrating with the CLI

The DS3 Bridge CLI (ds3bridge) is an optional front end to the same API: a terminal UI that covers the whole journey — credentials, checks, bucket selection, bridge creation, verification, migration start and live monitoring — plus classic subcommands for scripting. The prerequisites and limitations above apply to the CLI as well.

Install

curl -fsSL https://cubbit-public.s3.cubbit.eu/ds3bridge-cli/latest/install.sh | bash

Linux and macOS on amd64/arm64; the script checks the SHA-256 checksum and installs to /usr/local/bin. Use --prefix <dir> to install elsewhere — for example ~/.local/bin to avoid sudo — or --version <v> to pin a build. Uninstall with uninstall.sh from the same URL.

Quick start

ds3bridge            # opens the menu on a terminal, prints the help in a pipe
ds3bridge --version  # check what you installed

The menu offers three flows:

Entry	What it is for
Single project	Guided wizard: one project, from credentials to a monitored migration.
Bulk from CSV	Many projects and buckets described in one CSV file, monitored together.
Existing bridge	Resume or monitor the bridge of the project already in your configuration file.

When a saved bulk session exists, Resume bulk also appears and continues that run without re-importing the CSV or re-entering credentials.

Single project

API endpoint — the bridge-migrator API; https://s3bridge.cubbit.eu for the public gateway, pre-filled.
Source credentials — access key, secret, endpoint, region and addressing style, with an optional prefill from ~/.aws/credentials.
Buckets — the source buckets are listed and classified ELEGIBLE, NOT ELEGIBLE (with the limit shown) or ? not determined; objects in cold storage raise a warning, but the bucket stays selectable.
Cubbit credentials — API key and secret for the destination tenant, plus the gateway endpoint (https://s3.cubbit.eu).
Preflight — the CLI validates your Cubbit credentials and shows the selection once more; if a bridge already exists you are taken to it instead.
Verification — before anything is copied, the CLI waits for the gateways to pick up the bridge, samples objects from the source and reads one through Cubbit to compare the two, then writes, reads back and deletes a test object under _ds3bridge-check/. It is mandatory: a failing check can only be retried, or resolved by deleting the bridge. If the check is interrupted, a leftover test object may remain and can be deleted manually.
Start and monitor — after you confirm you stopped writing to the source, the CLI starts the migration and shows a live per-bucket dashboard, from which you can delete the bridge when done.

Bulk from CSV

One row per bucket, , or ; as separator, with this header:

bucket,tenant_id,cubbit_api_key,cubbit_api_secret,source_endpoint,source_region,source_api_key,source_secret,use_path_style

Rows are grouped into bridges by tenant, Cubbit credentials and source connection: the same tenant with different Cubbit credentials is a different project, and rows of one project with different source connections are rejected in the preview — a bridge has exactly one source connection. The flow is: pick the CSV (or browse to it), review the error-free preview, create one bridge per project, choose which projects to start (starting a project starts all of its ready buckets), then monitor them together. Every run is saved to ~/.config/ds3bridge/bulk-<timestamp>.json — credentials included — and can be resumed later from Resume bulk.

Existing bridge

A project can only have one bridge. When it already exists, the CLI shows its status and per-bucket counters and offers the live dashboard (v, while in progress), the same mandatory verification and start (s, when ready), deletion (d), back and quit.

Subcommands

setup, create, start, status, watch and delete mirror the menu for non-interactive use, with --config <path> to point at another configuration file and --verbose to log the HTTP traffic with credentials redacted:

ds3bridge status --json     # raw API response, for scripts
ds3bridge watch --interval 5s

setup and delete need an interactive terminal: delete always asks you to type bridge to confirm, and no flag skips that prompt. It removes only DS3 Bridge's tracking, exactly as described above.

Configuration file

The CLI keeps its settings in ~/.config/ds3bridge/config.json, created with owner-only permissions (0600) and holding credentials in clear text — keep it private and out of any repository.

How it works​

Prerequisites​

Creating a bridge​

Starting migration​

Tracking migration progress​

Deleting a bridge​

Limitations​

Migrating with the CLI​

Install​

Quick start​

Single project​

Bulk from CSV​

Existing bridge​

Subcommands​

Configuration file​

How it works

Prerequisites

Creating a bridge

Starting migration

Tracking migration progress

Deleting a bridge

Limitations

Migrating with the CLI

Install

Quick start

Single project

Bulk from CSV

Existing bridge

Subcommands

Configuration file