DS3 Bridge

DS3 Bridge lets you move your buckets from another S3-compatible provider (AWS, Wasabi, etc.) into Cubbit with zero downtime. You create a bridge for the buckets you want to migrate, and from that moment you can point your applications at Cubbit and keep working normally — reads, writes, and deletes all behave as if the bucket were already fully on Cubbit, even while the underlying data is still being migrated over in the background.

Experimental feature

DS3 Bridge is currently experimental. It has a limited, well-defined scope (see Limitations below).

How it works

1. You create a bridge for one or more buckets, providing read/write credentials for the source S3 provider and your Cubbit credentials.
2. Cubbit creates matching destination buckets and begins copying objects from the source, one by one, in the background. Each object is deleted from the source as soon as it has been copied to Cubbit — the source bucket is progressively emptied as the migration advances, so make sure you no longer need direct access to it once the bridge is created.
3. From that point on, use Cubbit only — point your applications at your Cubbit endpoint instead of the source provider. Cubbit transparently serves your data no matter where it currently lives:
   • Reading or listing an object that hasn't been migrated yet is served straight from the source, so nothing is ever missing.
   • Once an object has been copied to Cubbit, it's removed from the source and served from Cubbit from then on.
   • New uploads always go to Cubbit directly.
4. The migration finishes on its own. Once every object in the bucket has been walked through, that bucket is marked completed. Objects that failed to migrate are not retried automatically and are not deleted from the source, so a bucket can reach completed while a few objects remain un-migrated — check the failed count (see Tracking migration progress) to spot this.

Create the bridge
        │
        ▼
Switch your applications to Cubbit right away
        │
        ▼
DS3 Bridge migrates objects from the source, bucket by bucket
        │
        ▼
 Reads: served by Cubbit (proxied from source S3 if not migrated)
 Writes: go straight to Cubbit
        │
        ▼
 Bucket is marked "completed" once every object has been processed

important

As soon as a bridge is created for a bucket, stop using the source bucket directly and switch to Cubbit immediately. Continuing to write to the source bucket outside of Cubbit after the bridge is opened can break the migration.

Prerequisites

• A Cubbit project and Cubbit API credentials (access key and secret).

• The source buckets must be unversioned (versioning disabled) and must not have Object Lock enabled. Bridges cannot be created for buckets that don't meet these conditions.

• Permanent IAM credentials for the source S3 provider — temporary/STS credentials are not supported (no session token field is sent). The identity needs the following permissions:

  Level	Permissions
  Account	ListAllMyBuckets
  Bucket (per bucket to migrate)	GetBucketLocation, GetBucketVersioning, GetBucketObjectLockConfiguration, GetBucketOwnershipControls, ListBucketVersions, ListBucket
  Object (per bucket to migrate)	GetObject, GetObjectTagging, GetObjectAcl, PutObjectAcl, PutObjectTagging, DeleteObject, DeleteObjectTagging

Creating a bridge

Once DS3 Bridge is enabled for your gateway, you create a migration bridge with a single API call:

info

For Cubbit's public gateway, the DS3 Bridge API is available at https://s3bridge.cubbit.eu.

Endpoint: POST /bridges

Request body:

{
    "source_connection": {
        "api_key": "string",
        "secret": "string",
        "endpoint": "string",
        "region": "string"
    },
    "cubbit_credentials": {
        "api_key": "string",
        "secret": "string"
    },
    "tenant_id": "string",
    "bucket_names": ["string"]
}

Response:

A successful call returns 201 Created with the ID of the new bridge:

{ "id": "<uuid>" }

nota

You can only have one bridge per project, and a bridge cannot be deleted once created.

Tracking migration progress

Get information about migration progress:

Endpoint: GET /bridges

Request headers:

Header	Required	Description
x-cbt-api-key	Yes	Cubbit API key
x-cbt-api-secret	Yes	Cubbit API secret
x-cbt-tenant-id	Yes	Tenant UUID

Response body:

{
    "status": "in-progress",
    "buckets": [
        {
            "bucket_name": "my-bucket",
            "status": "in-progress",
            "migrated": 1204,
            "locked": 0,
            "failed": 0
        }
    ]
}

• status is completed once every bucket in the bridge has finished migrating, and in-progress otherwise.
• migrated counts objects that were copied to Cubbit successfully. failed counts objects that could not be migrated — they remain on the source. locked is reserved for future use and is currently always 0.

Limitations

These fall into two groups: permanent limits, which are architectural constraints and are not planned to change, and items on the roadmap, which reflect the current state and will be addressed in future.

Permanent limits:

• Existing bucket name conflicts — if a bucket with the same name already exists in your Cubbit tenant it cannot be migrated (unless the existing bucket has been created by the Bridge migrator).
• Source bucket must not be used after the bridge is opened — once a bridge is created for a bucket, you should immediately switch to Cubbit for that bucket and stop using the source directly.
• Objects in Glacier (or Glacier Deep Archive) on the source are not restored automatically, so they will fail to migrate until you restore them yourself.
• While a bucket is mid-migration:
  • Pending multipart uploads on the source are not migrated and can't be resumed after the bridge is created.
  • CopyObject / UploadPartCopy are blocked.
  • Changing bucket versioning/Object Lock settings, or deleting the bucket, is blocked.

On the roadmap:

• Unversioned buckets only — buckets with versioning or Object Lock enabled are rejected when the bridge is created.
• One bridge per project, and it cannot be removed once created.
• Metadata can look inconsistent while a bucket is mid-migration — fetching the same object twice during migration can return either its original source metadata or its final Cubbit metadata, depending on whether it has been migrated yet.
• Cubbit-to-Cubbit migrations are not supported. This will fail when the source is the very same gateway you're migrating into.
• Path-style access only — the migrator always addresses the source using path-style URLs. Most providers support this, but some newer AWS regions restrict it, which can prevent migrating buckets from those regions.

Mixed: permanent & roadmap:

Some categories contain a mix of permanent limits and items on the roadmap:

• Bucket properties that aren't carried over:
  • Permanent: any bucket feature Cubbit doesn't support, and region (always mapped to Cubbit's eu-west region).
  • On the roadmap: lifecycle rules and ACLs (Cubbit applies a default private ACL instead).
• Object properties that aren't carried over:
  • Permanent: ETag (large/multipart objects may end up with a different ETag on Cubbit than on the source), storage class (objects land in the default STANDARD class), and some headers/features (Content-Language, Expires, website redirects, server-side encryption).
  • On the roadmap: ACLs (default ACLs are used instead).