Crate krabka_object_store 

Unified object-store construction for Krabka.

Krabka’s KIP-405 tiered storage, krabka-remote-storage, and the observability blockstore, krabka-blockstore, share krabka-object-store.

The scope is the object-store access and plumbing layer only. The crate turns a typed ObjectStoreConfig into an object_store::ObjectStore handle. The data representation stays in the respective consumer crates. That representation is verbatim Kafka segment bytes or Parquet blocks.

On wasm32-wasip1 the crate has only the in-memory backend. The S3, GCS and local-filesystem backends need an HTTP client stack or a filesystem walk that object_store does not build for that target, so their configurations fail with ObjectStoreError::InvalidConfig there, and so do the WORM bucket checks. The multipart-upload listing is not exported.

Modules

fault

A fault-injecting [ObjectStore] wrapper, for drills against a store that misbehaves.

Structs

GcsConfig

Connection and bucket parameters for native Google Cloud Storage.

IncompleteMultipartUpload

One upload that S3 has initiated but not completed or aborted.

ObjectStoreClient

The single concrete ObjectOps implementation.

PutOutcome

What a put produced.

PutRequest

How one put should behave.

S3Config

Connection and bucket parameters for an S3-compatible backend.

Enums

ObjectStoreConfig

Selects and parameterises the object-store backend to construct.

ObjectStoreError

Errors raised by the object-store substrate.

PutMode

Precondition for a write, re-exported so callers need not depend on object_store directly. Configure preconditions for the put operation

Constants

DEFAULT_CONNECT_TIMEOUT

Default ceiling on the connect phase alone. It is 5 seconds, matching object_store.

DEFAULT_MAX_RETRIES

Default number of times a single object-store request is retried before the error reaches the caller. It matches object_store’s own default of 10.

DEFAULT_MULTIPART_CHUNK_SIZE

Default per-part size for multipart uploads. The size is 16 MiB. AWS requires >= 5 MiB per non-final part and caps the parts at 10 000, so 16 MiB scales past any real segment.

DEFAULT_MULTIPART_THRESHOLD

Default threshold above which a segment upload changes from a single PUT to a streaming multipart upload. The threshold is 100 MiB, well below AWS’s 5 GiB single-PUT cap.

DEFAULT_REQUEST_TIMEOUT

Default ceiling on one HTTP request, connect phase included. It is 30 seconds, object_store’s default, chosen because an object-store request may transfer a whole segment.

DEFAULT_RETRY_TIMEOUT

Default ceiling on the wall-clock time one request may spend across all of its retries.

Traits

ObjectOps

Async object-store operations. The trait is Send + Sync, so tasks can share it.

Functions

build_object_store

Build an object_store handle for cfg.

list_s3_multipart_uploads

Lists every incomplete S3 multipart upload under prefix.

normalize_prefix

Trim whitespace and separators copied from a console URL so /tier/, tier, and tier/ address the same root. Empty prefixes mean no prefix.

prefixed_key

Build the absolute object key for a path relative to an archive root.

read_capped

Read a whole object, and reject it if its size exceeds max_bytes.

verify_gcs_worm_bucket

Confirms that a GCS bucket can protect multipart WORM objects.

verify_s3_worm_bucket

Confirms that an S3 bucket can protect multipart WORM objects.