Cloud storage setup

1. Sign in to the AWS S3 Console.
2. Click Create bucket.
3. Choose a region and give the bucket a name.
4. Leave the default settings and click Create bucket.

1. Open the GCS Console.
2. Click Create bucket.
3. Choose a location and pick a storage class (Standard is fine for most use cases).
4. Click Create.

1. Open the Azure Portal.
2. Create a Storage Account if you don't have one.
3. Inside the storage account, go to Containers and click + Container.
4. Give it a name and click Create.

1. Sign in to the Oracle Cloud Console.
2. Navigate to Object Storage & Archive Storage.
3. Click Create Bucket, choose a compartment and name.
4. Click Create Bucket.

1. Install MinIO on your server or use an existing deployment.
2. Open the MinIO Console or use the CLI:

3. mc mb myminio/my-bucket

1. Sign in to the Cloudflare dashboard.
2. Go to R2 Object Storage and click Create bucket.
3. Give it a name and choose a location (or a jurisdiction, if your data must stay in the EU).
4. Copy your Account ID from the R2 overview page. Blixt uses it to build the S3 endpoint.

Change notifications

R2 has no S3 notification API. A bucket’s object events are routed by a notification rule to a Cloudflare Queue, and Blixt drains that queue over the Queues HTTP pull endpoint. Those calls go to api.cloudflare.com on a Cloudflare API token — a different credential from the R2 access keys, which sign S3 requests only.

Two properties of R2 queues shape how this behaves, and both differ from other object stores (S3, GCS, etc.):

• One notification queue per bucket. R2 rejects overlapping notification rules, so all of a bucket’s events go to exactly one queue. There is no per-server subscription to create, and pointing a second Blixt server at the same bucket adds no second stream.
• Every notification is delivered once. Consumers of a queue compete for its messages: whichever consumer pulls a message is the only one that sees it. A Blixt deployment consumes the queue from a single place — the updater runs one replica, and the embedded path runs in the indexing server — so a deployment sees every event exactly once. Two separate deployments pointed at the same R2 bucket would split the stream between them, each seeing roughly half the events, so run one deployment per bucket.

Blixt discovers the queue rather than assuming it: it asks R2 which queue the bucket’s events reach and binds to that one, so a bucket repointed at a different queue keeps working instead of quietly polling an empty queue. CLOUDFLARE_QUEUE names the queue to use only when Blixt has to create a rule for a bucket that has none.

Create the queue, give it an HTTP pull consumer, and route the bucket’s events to it. Blixt logs these same commands for any bucket it finds unconfigured:

wrangler queues create bfs-queue
wrangler queues consumer http add bfs-queue
wrangler r2 bucket notification create my-bucket --queue bfs-queue \
  --event-type object-create --event-type object-delete

For a bucket in a jurisdiction, add --jurisdiction eu (or fedramp) to the queue and notification commands: a jurisdiction is a separate namespace, and commands that omit it address the default one and appear to succeed. Blixt creates a missing notification rule itself unless rule management is turned off; the queue and its pull consumer are always yours to create.

Without an API token, R2 serves normally but delivers no notifications. Everything written through Blixt is visible immediately; changes made to the bucket outside Blixt — from the Cloudflare dashboard, wrangler or another S3 client — are picked up by the next filesystem scan instead of straight away.

1. In the IAM Console, create a user (or role) with s3:* permissions on your bucket.
2. Go to Security credentials and click Create access key.
3. Copy the Access key ID and Secret access key.
4. Enter these in Blixt during setup.

Running inside GCE or GKE? Blixt supports Application Default Credentials (ADC) out of the box. GCE instances and GKE pods automatically authenticate to GCS via the metadata server, using the service account attached to the VM or workload. No credential files to manage — just make sure the service account has the Storage Object Admin role on your bucket.

Using gcloud on your machine? If you've run gcloud auth application-default login, Blixt will pick up your default credentials automatically. This is convenient for development and testing.

Other environments? You'll need to create and manage a service account key:

1. In the Service Accounts page, create a new service account.
2. Grant it the Storage Object Admin role on your bucket.
3. Click Keys → Add Key → JSON to download a key file.
4. Point Blixt at this JSON key file during setup.

Pub/Sub permissions

GCS requires Pub/Sub permissions to be configured for the project. Blixt creates a Pub/Sub channel to receive updates from GCS, but by default GCS doesn't have permission to publish to Pub/Sub. The following commands authorise GCS to send Pub/Sub messages to Blixt. See the GCS documentation for background.

1. Find the GCS service account for your project. Replace PROJECT with your GCP project name. Use the output as SERVICE-ACCOUNT in the next command.

gcloud storage service-agent --project=PROJECT

2. Grant the roles/pubsub.publisher role to the GCS service account:

gcloud pubsub topics add-iam-policy-binding projects/PROJECT/topics/bfs \
  --member="serviceAccount:SERVICE-ACCOUNT" \
  --role="roles/pubsub.publisher"

1. In the Azure Portal, open your Storage Account.
2. Go to Security + networking → Access keys.
3. Copy the Storage account name and one of the Keys.
4. Enter these in Blixt during setup.

1. In the Oracle Cloud Console, go to your User Settings.
2. Under Customer Secret Keys, click Generate Secret Key.
3. Copy the access key and secret key (S3-compatible).
4. Note your tenancy namespace and region. Enter all of these in Blixt.

1. Use your MinIO access key and secret key. These work the same as S3 credentials.
2. Set the endpoint URL to your MinIO server address (e.g. https://minio.example.com).
3. Enter these in Blixt during setup.

1. In the Cloudflare dashboard, go to R2 Object Storage → API → Manage API tokens.
2. Create an Account API token with Object Read & Write permission for your bucket.
3. Copy the Access Key ID and Secret Access Key. These are S3-compatible credentials.
4. Enter these, along with your account ID, in Blixt during setup.
5. For change notifications, also create a user API token with Account:Queues (Edit) and Account:Workers R2 Storage (Edit). This is a separate credential: the Cloudflare API accepts no S3 key.

Environment variables

R2 configuration does not use a credentials file. Account id and credentials are supplied as environment variables, or command-line flags.

Required to serve a bucket

• CLOUDFLARE_ACCOUNT_ID (--cloudflare_account_id) — the R2 account ID. The S3 endpoint is derived from it.
• CLOUDFLARE_ACCESS_KEY (--cloudflare_access_key) — the R2 API token’s Access Key ID.
• CLOUDFLARE_SECRET_KEY (--cloudflare_secret_key) — the R2 API token’s Secret Access Key.

Required for change notifications

Notifications arrive over the Cloudflare Queues API, which accepts no S3 credential, so this is a second credential rather than the keys above. The R2 access key ID is a token’s id and the secret key is the SHA-256 of the token value, so the S3 pair cannot be turned back into anything api.cloudflare.com will take. Set the token, and the R2 bucket stays in step with changes made outside Blixt.

• CLOUDFLARE_API_TOKEN (--cloudflare_api_token) — an API token with Account:Queues (Edit) and Account:Workers R2 Storage (Edit). Leave it unset and R2 serves normally, with out-of-band changes picked up by the next scan.
• CLOUDFLARE_QUEUE (--cloudflare_queue) — the queue to route a bucket’s events to when Blixt has to create a notification rule. An existing rule’s queue is discovered, not taken from here.

Optional

• CLOUDFLARE_JURISDICTION (--cloudflare_jurisdiction) — eu or fedramp. A bucket created in a jurisdiction is only reachable through that jurisdiction’s endpoint.
• CLOUDFLARE_PROJECT (--cloudflare_project) — defaults to the account ID, which is what R2 uses as the project.
• CLOUDFLARE_BUCKETS (CLOUDFLARE_BUCKET, or R2_BUCKETS / R2_BUCKET) — buckets to serve, instead of listing them as arguments.

Summary

export CLOUDFLARE_ACCOUNT_ID=your-account-id
export CLOUDFLARE_ACCESS_KEY=your-access-key-id
export CLOUDFLARE_SECRET_KEY=your-secret-access-key

# Optional: change notifications, over a Cloudflare Queue.
export CLOUDFLARE_API_TOKEN=your-api-token
export CLOUDFLARE_QUEUE=bfs-queue

In Docker, pass them with -e; in Kubernetes, the Helm chart reads them from the bfs-cloudflare Secret. Keep the secret key and the API token out of shell history and out of version control.