DS3 Bridge

DS3 Bridge lets you move your buckets from another S3-compatible provider (AWS, Wasabi, etc.) into Cubbit with zero downtime. You create a bridge for the buckets you want to migrate, and from that moment you can point your applications at Cubbit and keep working normally — reads, writes, and deletes all behave as if the bucket were already fully on Cubbit, even while the underlying data is still being migrated over in the background.

Experimental feature

DS3 Bridge is currently experimental. It has a limited, well-defined scope (see Limitations below).

How it works

1. You create a bridge for one or more buckets, providing read/write credentials for the source S3 provider and your Cubbit credentials. This creates the matching destination buckets on Cubbit, but does not start copying any objects yet — the bridge and its buckets are created in a ready state, waiting for you to start the migration.
2. Switch your applications to Cubbit right away — point them at your Cubbit endpoint instead of the source provider. Cubbit transparently serves your data no matter where it currently lives, even before migration has started: reads and listings are served straight from the source, and new uploads go to Cubbit directly. Because the migration hasn't started yet, nothing has been deleted from the source at this point — this is a safe window to verify the switch works as expected before committing to the actual migration.
3. When you're ready, start the migration by calling the start migration endpoint. This is the point where DS3 Bridge begins copying objects from the source, one by one, in the background. Each object is deleted from the source as soon as it has been copied to Cubbit — the source bucket is progressively emptied as the migration advances, so make sure you no longer need direct access to it once the migration is started.
4. From then on, Cubbit keeps transparently serving your data no matter where it currently lives:
   • Reading or listing an object that hasn't been migrated yet is served straight from the source, so nothing is ever missing.
   • Once an object has been copied to Cubbit, it's removed from the source and served from Cubbit from then on.
   • New uploads always go to Cubbit directly.
5. The migration finishes on its own. Once every object in the bucket has been walked through, that bucket is marked completed. Objects that failed to migrate are not retried automatically and are not deleted from the source, so a bucket can reach completed while a few objects remain un-migrated — check the failed count (see Tracking migration progress) to spot this.

Create the bridge
        │
        ▼
Switch your applications to Cubbit right away
(You may validate the switch works seamlessly)
        │
        ▼
Start the migration
        │
        ▼
DS3 Bridge migrates objects from the source, bucket by bucket
        │
        ▼
 Reads: served by Cubbit (proxied from source S3 if not migrated)
 Writes: go straight to Cubbit
        │
        ▼
 Bucket is marked "completed" once every object has been processed

important

Once the migration is started for a bucket, stop using the source bucket directly and rely on Cubbit exclusively. Continuing to write to the source bucket outside of Cubbit after the migration has started can break the migration.

Prerequisites

• A Cubbit project and Cubbit API credentials (access key and secret).

• The source buckets must be unversioned (versioning disabled) and must not have Object Lock enabled. Bridges cannot be created for buckets that don't meet these conditions.

• Permanent IAM credentials for the source S3 provider — temporary/STS credentials are not supported (no session token field is sent). The identity needs the following permissions:

  Level	Permissions
  Account	ListAllMyBuckets
  Bucket (per bucket to migrate)	GetBucketLocation, GetBucketVersioning, GetBucketObjectLockConfiguration, GetBucketOwnershipControls, ListBucketVersions, ListBucket
  Object (per bucket to migrate)	GetObject, GetObjectTagging, GetObjectAcl, PutObjectAcl, PutObjectTagging, DeleteObject, DeleteObjectTagging

Creating a bridge

Once DS3 Bridge is enabled for your gateway, you create a migration bridge with a single API call:

info

For Cubbit's public gateway, the DS3 Bridge API is available at https://s3bridge.cubbit.eu.

Endpoint: POST /bridges

Request body:

{
    "source_connection": {
        "api_key": "string",
        "secret": "string",
        "endpoint": "string",
        "region": "string",
        "use_path_style": false
    },
    "cubbit_credentials": {
        "api_key": "string",
        "secret": "string"
    },
    "tenant_id": "string",
    "bucket_names": ["string"]
}

source_connection.use_path_style is required and tells DS3 Bridge how to address buckets on the source provider:

• false is the normal choice — it lets the SDK use virtual-hosted-style addressing (https://bucket.endpoint/key) when possible, falling back to path-style automatically if needed. This is how AWS S3 and most S3-compatible providers work.
• true forces path-style URLs (https://endpoint/bucket/key) instead. This is only needed in special cases — some, typically older or self-hosted, S3-compatible providers only support path-style requests.

Response:

A successful call returns 201 Created with the ID of the new bridge:

{ "id": "<uuid>" }

note

You can only have one bridge per project, if you need another one delete the current bridge first.

Starting migration

Start the migration process for an already created bridge.

Endpoint: POST /bridges/start-migration

Request headers:

Header	Required	Description
x-cbt-api-key	Yes	Cubbit API key
x-cbt-api-secret	Yes	Cubbit API secret
x-cbt-tenant-id	Yes	Tenant UUID

A successful call returns 204 No Content with no body. This moves every bucket in the bridge from ready to in-progress, making them eligible for migration. This operation is idempotent.

Tracking migration progress

Get information about migration progress:

Endpoint: GET /bridges

Request headers:

Header	Required	Description
x-cbt-api-key	Yes	Cubbit API key
x-cbt-api-secret	Yes	Cubbit API secret
x-cbt-tenant-id	Yes	Tenant UUID

Response body:

{
    "status": "in-progress",
    "buckets": [
        {
            "bucket_name": "my-bucket",
            "status": "in-progress",
            "migrated": 1204,
            "locked": 0,
            "failed": 0
        }
    ]
}

• Each bucket's status is ready before the migration is started, in-progress while it's being migrated, and completed once it has finished.
• The bridge-level status follows the same three values, based on its buckets: in-progress if any bucket is in-progress; otherwise ready if any bucket is still ready; otherwise completed (only once every bucket has finished).
• migrated counts objects that were copied to Cubbit successfully. failed counts objects that could not be migrated — they remain on the source. locked is reserved for future use and is currently always 0.

Deleting a bridge

Remove a bridge and all of its tracked bucket state:

Endpoint: DELETE /bridges

Request headers:

Header	Required	Description
x-cbt-api-key	Yes	Cubbit API key
x-cbt-api-secret	Yes	Cubbit API secret
x-cbt-tenant-id	Yes	Tenant UUID

A successful call returns 204 No Content with no body. This operation is idempotent. You can delete a bridge even while migration is in progress.

important

Deleting a bridge only removes DS3 Bridge's own tracking of it — it does not undo any migration already performed. Objects already copied to Cubbit stay there, and anything already deleted from the source is not restored.

Limitations

These fall into two groups: permanent limits, which are architectural constraints and are not planned to change, and items on the roadmap, which reflect the current state and will be addressed in future.

Permanent limits:

• Existing bucket name conflicts — if a bucket with the same name already exists in your Cubbit tenant it cannot be migrated (unless the existing bucket has been created by the Bridge migrator).
• Source bucket must not be used after migration is started — switch to Cubbit for the bucket as soon as the bridge is created, and stop using the source directly once you start the migration.
• Objects in Glacier (or Glacier Deep Archive) on the source are not restored automatically, so they will fail to migrate until you restore them yourself.
• Deleting a bridge doesn't undo migration — it only removes DS3 Bridge's own tracking; objects already copied to Cubbit stay there, and anything already deleted from the source isn't restored.
• While a bucket is mid-migration:
  • Pending multipart uploads on the source are not migrated and can't be resumed after the bridge is created.
  • CopyObject / UploadPartCopy are blocked.
  • Changing bucket versioning/Object Lock settings, or deleting the bucket, is blocked.

On the roadmap:

• Unversioned buckets only — buckets with versioning or Object Lock enabled are rejected when the bridge is created.
• One bridge per project.
• Metadata can look inconsistent while a bucket is mid-migration — fetching the same object twice during migration can return either its original source metadata or its final Cubbit metadata, depending on whether it has been migrated yet.
• Cubbit-to-Cubbit migrations are not supported. This will fail when the source is the very same gateway you're migrating into.
• Path-style access only — the migrator always addresses the source using path-style URLs. Most providers support this, but some newer AWS regions restrict it, which can prevent migrating buckets from those regions.

Mixed: permanent & roadmap:

Some categories contain a mix of permanent limits and items on the roadmap:

• Bucket properties that aren't carried over:
  • Permanent: any bucket feature Cubbit doesn't support, and region (always mapped to Cubbit's eu-west region).
  • On the roadmap: lifecycle rules and ACLs (Cubbit applies a default private ACL instead).
• Object properties that aren't carried over:
  • Permanent: ETag (large/multipart objects may end up with a different ETag on Cubbit than on the source), storage class (objects land in the default STANDARD class), and some headers/features (Content-Language, Expires, website redirects, server-side encryption).
  • On the roadmap: ACLs (default ACLs are used instead).