mirror of
https://github.com/s2-streamstore/s2.git
synced 2026-09-27 18:19:36 +00:00
1e954f014aLite stores WAL writes and LSM data in the same object store, so WAL latency can be affected by memtable flushes and compaction output. Add `--wal-bucket` and `--wal-local-root` to configure a dedicated WAL store through SlateDB's existing builder API. The README leads with a local filesystem WAL and a remote S3 bucket for the main database. `--wal-bucket` reuses the main S3 connection configuration by default, including its endpoint, region and credentials. Optional `S2LITE_WAL_AWS_*` overrides select a different server or credentials; the WAL endpoint takes precedence over an inherited `AWS_ENDPOINT_URL_S3`. Both buckets use the same S3 builder; a WAL-specific key pair replaces the complete credential set, including its optional session token. Omitting both WAL selectors preserves the shared-store default. Filesystem WAL storage retains fsync. The default flush interval follows the WAL store type, and `SL8_FLUSH_INTERVAL` still takes precedence. Separate WAL storage requires an explicitly configured persistent main store. Both stores use `--path` and must be reopened at the same locations; these options do not migrate existing WAL data. Validation: `just fmt`, `just test` (840 passed), and `just clippy`, using locked dependencies. Tests cover CLI constraints, inherited S3 connection settings, generic and S3-specific endpoint inheritance, endpoint-only and credential overrides, session-token isolation, physical file routing, and recovery of acknowledged records after killing and restarting S2. No dependency changes. No performance improvement is claimed. Fixes #673. |
History
|
|
|---|---|---|
| .. | ||
| cli.rs | ||
| integration.rs | ||
| lite_wal.rs | ||