diff --git a/docs/configuration/provisioning.mdx b/docs/configuration/provisioning.mdx index ee6a5d5b..08d76c0a 100644 --- a/docs/configuration/provisioning.mdx +++ b/docs/configuration/provisioning.mdx @@ -81,3 +81,37 @@ For more information about the replication settings, see the **[Data Replication TLS verification for replication destinations is configured globally for the source instance with `RS_REPLICATION_VERIFY_SSL` and `RS_REPLICATION_CA_PATH`. These settings apply to all replication tasks created through provisioning, the HTTP API, SDKs, CLI, or the Web Console. + +## Lifecycle Provisioning + +You can provision lifecycle policies by setting environment variables. Lifecycle policies run in the background and currently support deleting old records from a bucket. + +| Name | Default | Description | +| ---------------------------- | ------- | ------------------------------------------------------------------------------------------------------------- | +| `RS_LIFECYCLE__NAME` | | Provisioned lifecycle policy name (required) | +| `RS_LIFECYCLE__TYPE` | delete | Lifecycle action type. Currently only `delete` is supported. | +| `RS_LIFECYCLE__BUCKET` | | Bucket to apply the lifecycle policy to (required) | +| `RS_LIFECYCLE__MAX_AGE` | | Maximum record age before deletion, for example `30d`, `24h`, or `3600s` (required). Minimum value is `1h`. | +| `RS_LIFECYCLE__INTERVAL` | 3600s | Interval between lifecycle runs, for example `10m`, `1h`, or `3600s`. Minimum value is `10m`. | +| `RS_LIFECYCLE__ENTRIES` | | Comma-separated list of entries to clean. If empty, all entries are matched. Prefix wildcards are supported. | +| `RS_LIFECYCLE__WHEN` | | JSON condition for records to delete. Uses the same condition syntax as queries, but `#ext` is not supported. | +| `RS_LIFECYCLE__MODE` | enabled | Policy mode: `enabled`, `disabled`, or `dry_run`. | + +Example: + +```env +RS_LIFECYCLE_A_NAME=purge-sensors-30d +RS_LIFECYCLE_A_TYPE=delete +RS_LIFECYCLE_A_BUCKET=telemetry +RS_LIFECYCLE_A_ENTRIES=sensors/*,env/temp,env/humidity +RS_LIFECYCLE_A_MAX_AGE=30d +RS_LIFECYCLE_A_INTERVAL=10m +RS_LIFECYCLE_A_WHEN={"$eq":["&label","true"]} +RS_LIFECYCLE_A_MODE=dry_run +``` + +Use `dry_run` mode to count matching records and write lifecycle audit events without deleting data. This is useful for validating `max_age`, `entries`, and `when` filters before switching the policy to `enabled`. + +Provisioned lifecycle policies cannot be updated or removed through the API. Change their environment variables and restart ReductStore to update them. If `RS_LIFECYCLE__MODE` is not set, ReductStore preserves the existing stored mode for an already provisioned lifecycle policy. + +For more information about lifecycle policies, see the **[Lifecycle Policies Guide](../guides/lifecycle-policies.mdx)**. diff --git a/docs/configuration/settings.mdx b/docs/configuration/settings.mdx index 28b5bd59..ae2e6c80 100644 --- a/docs/configuration/settings.mdx +++ b/docs/configuration/settings.mdx @@ -177,7 +177,10 @@ ReductStore can persist aggregated API interactions in the `$audit` system bucke Notes: -- If `RS_AUDIT_ENABLED` is unset, audit defaults to `true` when `RS_API_TOKEN` is set, otherwise `false`. +- If `RS_AUDIT_ENABLED` is unset, audit defaults to `true` when `RS_API_TOKEN` is set or lifecycle policies are configured, otherwise `false`. - `RS_AUDIT_ENABLED` explicitly overrides the token-based default. - `RS_AUDIT_QUOTA_SIZE` is applied only when audit is enabled. - If `RS_AUDIT_QUOTA_SIZE` is unset, `$audit` capacity is unlimited. +- Audit records are stored as structured events with common fields: `type`, `timestamp`, `instance`, `entry_name`, `status`, `message`, and `payload`. +- API audit events use `type: "api_call"` and keep request details (`token_name`, `method`, `path`, `client_ip`, `call_count`, and `duration`) in `payload`. +- Lifecycle audit events use `type: "lifecycle_run"` and keep run details (`policy_name`, `action_type`, `bucket`, `duration`, `processed_records`, `error_code`, and `error_message`) in `payload`. diff --git a/docs/examples/cli/lifecycle_policy_browse.sh b/docs/examples/cli/lifecycle_policy_browse.sh new file mode 100644 index 00000000..24d7a33a --- /dev/null +++ b/docs/examples/cli/lifecycle_policy_browse.sh @@ -0,0 +1,3 @@ +reduct-cli alias add local -L http://localhost:8383 -t "my-token" +reduct-cli lifecycle ls local --full +reduct-cli lifecycle show local/my-lifecycle diff --git a/docs/examples/cli/lifecycle_policy_create.sh b/docs/examples/cli/lifecycle_policy_create.sh new file mode 100644 index 00000000..5594657d --- /dev/null +++ b/docs/examples/cli/lifecycle_policy_create.sh @@ -0,0 +1,3 @@ +reduct-cli alias add local -L http://localhost:8383 -t "my-token" +reduct-cli bucket create local/my-bucket +reduct-cli lifecycle create local/my-lifecycle my-bucket --max-age 30d --interval 1h --entries cli-example --when '{"&anomaly":{"$eq":1}}' diff --git a/docs/examples/cli/lifecycle_policy_mode.sh b/docs/examples/cli/lifecycle_policy_mode.sh new file mode 100644 index 00000000..7dfacc09 --- /dev/null +++ b/docs/examples/cli/lifecycle_policy_mode.sh @@ -0,0 +1,4 @@ +reduct-cli alias add local -L http://localhost:8383 -t "my-token" +reduct-cli lifecycle dry-run local/my-lifecycle +reduct-cli lifecycle disable local/my-lifecycle +reduct-cli lifecycle enable local/my-lifecycle diff --git a/docs/examples/cli/lifecycle_policy_remove.sh b/docs/examples/cli/lifecycle_policy_remove.sh new file mode 100644 index 00000000..ac94fa14 --- /dev/null +++ b/docs/examples/cli/lifecycle_policy_remove.sh @@ -0,0 +1,2 @@ +reduct-cli alias add local -L http://localhost:8383 -t "my-token" +reduct-cli lifecycle rm local/lifecycle-to-remove --yes diff --git a/docs/examples/cpp/src/lifecycle_policy_browse.cc b/docs/examples/cpp/src/lifecycle_policy_browse.cc new file mode 100644 index 00000000..d8eb1640 --- /dev/null +++ b/docs/examples/cpp/src/lifecycle_policy_browse.cc @@ -0,0 +1,27 @@ +#include +#include + +using reduct::IClient; + +int main() { + auto client = IClient::Build("http://127.0.0.1:8383", {.api_token = "my-token"}); + + auto [lifecycles, list_err] = client->GetLifecycleList(); + if (list_err != reduct::Error::kOk) { + return 1; + } + + for (const auto& lifecycle : lifecycles) { + std::cout << "Lifecycle: " << lifecycle.name << std::endl; + std::cout << "Running: " << lifecycle.is_running << std::endl; + std::cout << "Provisioned: " << lifecycle.is_provisioned << std::endl; + } + + auto [detail, detail_err] = client->GetLifecycle("my-lifecycle"); + if (detail_err != reduct::Error::kOk) { + return 1; + } + + std::cout << "Lifecycle: " << detail.info.name << std::endl; + std::cout << "Bucket: " << detail.settings.bucket << std::endl; +} diff --git a/docs/examples/cpp/src/lifecycle_policy_create.cc b/docs/examples/cpp/src/lifecycle_policy_create.cc new file mode 100644 index 00000000..e1520931 --- /dev/null +++ b/docs/examples/cpp/src/lifecycle_policy_create.cc @@ -0,0 +1,22 @@ +#include +#include + +using reduct::Error; +using reduct::IClient; + +int main() { + auto client = IClient::Build("http://127.0.0.1:8383", {.api_token = "my-token"}); + + auto [bucket, bucket_err] = client->GetOrCreateBucket("my-bucket"); + assert(bucket_err == Error::kOk); + + auto err = client->CreateLifecycle("my-lifecycle", IClient::LifecycleSettings{ + .type = IClient::LifecycleType::kDelete, + .bucket = "my-bucket", + .entries = {"cpp-example"}, + .max_age = "30d", + .interval = "1h", + .when = R"({"&anomaly":{"$eq":1}})", + }); + assert(err == Error::kOk); +} diff --git a/docs/examples/cpp/src/lifecycle_policy_mode.cc b/docs/examples/cpp/src/lifecycle_policy_mode.cc new file mode 100644 index 00000000..5fc3cfba --- /dev/null +++ b/docs/examples/cpp/src/lifecycle_policy_mode.cc @@ -0,0 +1,18 @@ +#include +#include + +using reduct::Error; +using reduct::IClient; + +int main() { + auto client = IClient::Build("http://127.0.0.1:8383", {.api_token = "my-token"}); + + auto err = client->SetLifecycleMode("my-lifecycle", IClient::LifecycleMode::kDryRun); + assert(err == Error::kOk); + + err = client->SetLifecycleMode("my-lifecycle", IClient::LifecycleMode::kDisabled); + assert(err == Error::kOk); + + err = client->SetLifecycleMode("my-lifecycle", IClient::LifecycleMode::kEnabled); + assert(err == Error::kOk); +} diff --git a/docs/examples/cpp/src/lifecycle_policy_remove.cc b/docs/examples/cpp/src/lifecycle_policy_remove.cc new file mode 100644 index 00000000..55cdf05e --- /dev/null +++ b/docs/examples/cpp/src/lifecycle_policy_remove.cc @@ -0,0 +1,12 @@ +#include +#include + +using reduct::Error; +using reduct::IClient; + +int main() { + auto client = IClient::Build("http://127.0.0.1:8383", {.api_token = "my-token"}); + + auto err = client->DeleteLifecycle("lifecycle-to-remove"); + assert(err == Error::kOk); +} diff --git a/docs/examples/curl/lifecycle_policy_browse.sh b/docs/examples/curl/lifecycle_policy_browse.sh new file mode 100644 index 00000000..83181f67 --- /dev/null +++ b/docs/examples/curl/lifecycle_policy_browse.sh @@ -0,0 +1,11 @@ +#!/bin/bash +set -e -x + +API_PATH="http://127.0.0.1:8383/api/v1" +AUTH_HEADER="Authorization: Bearer my-token" + +curl -H "${AUTH_HEADER}" \ + "${API_PATH}"/lifecycles + +curl -H "${AUTH_HEADER}" \ + "${API_PATH}"/lifecycles/my-lifecycle diff --git a/docs/examples/curl/lifecycle_policy_create.sh b/docs/examples/curl/lifecycle_policy_create.sh new file mode 100644 index 00000000..95dd954f --- /dev/null +++ b/docs/examples/curl/lifecycle_policy_create.sh @@ -0,0 +1,10 @@ +#!/bin/bash +set -e -x + +API_PATH="http://127.0.0.1:8383/api/v1" +AUTH_HEADER="Authorization: Bearer my-token" + +curl -X POST \ + -H "${AUTH_HEADER}" \ + -d '{"bucket":"my-bucket","entries":["curl-example"],"max_age":"30d","interval":"1h","when":{"&anomaly":{"$eq":1}}}' \ + "${API_PATH}"/lifecycles/my-lifecycle diff --git a/docs/examples/curl/lifecycle_policy_mode.sh b/docs/examples/curl/lifecycle_policy_mode.sh new file mode 100644 index 00000000..bf58bb7a --- /dev/null +++ b/docs/examples/curl/lifecycle_policy_mode.sh @@ -0,0 +1,20 @@ +#!/bin/bash +set -e -x + +API_PATH="http://127.0.0.1:8383/api/v1" +AUTH_HEADER="Authorization: Bearer my-token" + +curl -X PATCH \ + -H "${AUTH_HEADER}" \ + -d '{"mode":"dry_run"}' \ + "${API_PATH}"/lifecycles/my-lifecycle/mode + +curl -X PATCH \ + -H "${AUTH_HEADER}" \ + -d '{"mode":"disabled"}' \ + "${API_PATH}"/lifecycles/my-lifecycle/mode + +curl -X PATCH \ + -H "${AUTH_HEADER}" \ + -d '{"mode":"enabled"}' \ + "${API_PATH}"/lifecycles/my-lifecycle/mode diff --git a/docs/examples/curl/lifecycle_policy_remove.sh b/docs/examples/curl/lifecycle_policy_remove.sh new file mode 100644 index 00000000..6c989f2d --- /dev/null +++ b/docs/examples/curl/lifecycle_policy_remove.sh @@ -0,0 +1,9 @@ +#!/bin/bash +set -e -x + +API_PATH="http://127.0.0.1:8383/api/v1" +AUTH_HEADER="Authorization: Bearer my-token" + +curl -X DELETE \ + -H "${AUTH_HEADER}" \ + "${API_PATH}"/lifecycles/lifecycle-to-remove diff --git a/docs/examples/go/src/lifecycle_policy_browse.go b/docs/examples/go/src/lifecycle_policy_browse.go new file mode 100644 index 00000000..fdad81ae --- /dev/null +++ b/docs/examples/go/src/lifecycle_policy_browse.go @@ -0,0 +1,30 @@ +package main + +import ( + "context" + + reduct "github.com/reductstore/reduct-go" +) + +func main() { + client := reduct.NewClient("http://localhost:8383", reduct.ClientOptions{APIToken: "my-token"}) + + lifecycles, err := client.GetLifecycles(context.Background()) + if err != nil { + panic(err) + } + + for _, lifecycle := range lifecycles { + println("Lifecycle:", lifecycle.Name) + println("Mode:", string(lifecycle.Mode)) + println("Running:", lifecycle.IsRunning) + println("Provisioned:", lifecycle.IsProvisioned) + } + + detail, err := client.GetLifecycle(context.Background(), "my-lifecycle") + if err != nil { + panic(err) + } + println("Lifecycle:", detail.Info.Name) + println("Bucket:", detail.Settings.Bucket) +} diff --git a/docs/examples/go/src/lifecycle_policy_create.go b/docs/examples/go/src/lifecycle_policy_create.go new file mode 100644 index 00000000..818766e8 --- /dev/null +++ b/docs/examples/go/src/lifecycle_policy_create.go @@ -0,0 +1,32 @@ +package main + +import ( + "context" + + reduct "github.com/reductstore/reduct-go" + model "github.com/reductstore/reduct-go/model" +) + +func main() { + client := reduct.NewClient("http://localhost:8383", reduct.ClientOptions{APIToken: "my-token"}) + _, err := client.CreateOrGetBucket(context.Background(), "my-bucket", nil) + if err != nil { + panic(err) + } + + settings := model.LifecycleSettings{ + LifecycleType: model.LifecycleTypeDelete, + Bucket: "my-bucket", + Entries: []string{"go-example"}, + MaxAge: "30d", + Interval: "1h", + When: map[string]any{ + "&anomaly": map[string]any{"$eq": 1}, + }, + } + + err = client.CreateLifecycle(context.Background(), "my-lifecycle", settings) + if err != nil { + panic(err) + } +} diff --git a/docs/examples/go/src/lifecycle_policy_mode.go b/docs/examples/go/src/lifecycle_policy_mode.go new file mode 100644 index 00000000..25265eb6 --- /dev/null +++ b/docs/examples/go/src/lifecycle_policy_mode.go @@ -0,0 +1,22 @@ +package main + +import ( + "context" + + reduct "github.com/reductstore/reduct-go" + model "github.com/reductstore/reduct-go/model" +) + +func main() { + client := reduct.NewClient("http://localhost:8383", reduct.ClientOptions{APIToken: "my-token"}) + + if err := client.SetLifecycleMode(context.Background(), "my-lifecycle", model.LifecycleModeDryRun); err != nil { + panic(err) + } + if err := client.SetLifecycleMode(context.Background(), "my-lifecycle", model.LifecycleModeDisabled); err != nil { + panic(err) + } + if err := client.SetLifecycleMode(context.Background(), "my-lifecycle", model.LifecycleModeEnabled); err != nil { + panic(err) + } +} diff --git a/docs/examples/go/src/lifecycle_policy_remove.go b/docs/examples/go/src/lifecycle_policy_remove.go new file mode 100644 index 00000000..13ac078c --- /dev/null +++ b/docs/examples/go/src/lifecycle_policy_remove.go @@ -0,0 +1,15 @@ +package main + +import ( + "context" + + reduct "github.com/reductstore/reduct-go" +) + +func main() { + client := reduct.NewClient("http://localhost:8383", reduct.ClientOptions{APIToken: "my-token"}) + + if err := client.DeleteLifecycle(context.Background(), "lifecycle-to-remove"); err != nil { + panic(err) + } +} diff --git a/docs/examples/js/src/lifecycle_policy_browse.mjs b/docs/examples/js/src/lifecycle_policy_browse.mjs new file mode 100644 index 00000000..fe959760 --- /dev/null +++ b/docs/examples/js/src/lifecycle_policy_browse.mjs @@ -0,0 +1,14 @@ +import { Client } from "reduct-js"; + +const client = new Client("http://127.0.0.1:8383", { apiToken: "my-token" }); + +for (const lifecycle of await client.getLifecycleList()) { + console.log("Lifecycle:", lifecycle.name); + console.log("Mode:", lifecycle.mode); + console.log("Running:", lifecycle.isRunning); + console.log("Provisioned:", lifecycle.isProvisioned); +} + +const details = await client.getLifecycle("my-lifecycle"); +console.log("Lifecycle:", details.info.name); +console.log("Settings:", details.settings); diff --git a/docs/examples/js/src/lifecycle_policy_create.mjs b/docs/examples/js/src/lifecycle_policy_create.mjs new file mode 100644 index 00000000..a913c59e --- /dev/null +++ b/docs/examples/js/src/lifecycle_policy_create.mjs @@ -0,0 +1,15 @@ +import { Client, LifecycleType } from "reduct-js"; + +const client = new Client("http://127.0.0.1:8383", { apiToken: "my-token" }); +await client.getOrCreateBucket("my-bucket"); + +const settings = { + lifecycleType: LifecycleType.DELETE, + bucket: "my-bucket", + entries: ["js-example"], + maxAge: "30d", + interval: "1h", + when: { "&anomaly": { $eq: 1 } }, +}; + +await client.createLifecycle("my-lifecycle", settings); diff --git a/docs/examples/js/src/lifecycle_policy_mode.mjs b/docs/examples/js/src/lifecycle_policy_mode.mjs new file mode 100644 index 00000000..5faed484 --- /dev/null +++ b/docs/examples/js/src/lifecycle_policy_mode.mjs @@ -0,0 +1,7 @@ +import { Client, LifecycleMode } from "reduct-js"; + +const client = new Client("http://127.0.0.1:8383", { apiToken: "my-token" }); + +await client.setLifecycleMode("my-lifecycle", LifecycleMode.DRY_RUN); +await client.setLifecycleMode("my-lifecycle", LifecycleMode.DISABLED); +await client.setLifecycleMode("my-lifecycle", LifecycleMode.ENABLED); diff --git a/docs/examples/js/src/lifecycle_policy_remove.mjs b/docs/examples/js/src/lifecycle_policy_remove.mjs new file mode 100644 index 00000000..a19e1777 --- /dev/null +++ b/docs/examples/js/src/lifecycle_policy_remove.mjs @@ -0,0 +1,4 @@ +import { Client } from "reduct-js"; + +const client = new Client("http://127.0.0.1:8383", { apiToken: "my-token" }); +await client.deleteLifecycle("lifecycle-to-remove"); diff --git a/docs/examples/provisioning/lifecycle_create.yml b/docs/examples/provisioning/lifecycle_create.yml new file mode 100644 index 00000000..cb98d054 --- /dev/null +++ b/docs/examples/provisioning/lifecycle_create.yml @@ -0,0 +1,22 @@ +version: "3" +services: + reductstore: + image: reduct/store:latest + ports: + - "8383:8383" + volumes: + - ./data:/data + environment: + RS_API_TOKEN: my-api-token + RS_BUCKET_1_NAME: telemetry + RS_LIFECYCLE_1_NAME: purge-sensors-30d + RS_LIFECYCLE_1_BUCKET: telemetry + RS_LIFECYCLE_1_TYPE: delete + RS_LIFECYCLE_1_MAX_AGE: 30d + RS_LIFECYCLE_1_INTERVAL: 10m + RS_LIFECYCLE_1_ENTRIES: "sensor-1,sensor-2" + + RS_LIFECYCLE_1_WHEN: | + { + "&sensor_type": { "$eq": "temperature" } + } diff --git a/docs/examples/py/src/lifecycle_policy_browse.py b/docs/examples/py/src/lifecycle_policy_browse.py new file mode 100644 index 00000000..3d16dbad --- /dev/null +++ b/docs/examples/py/src/lifecycle_policy_browse.py @@ -0,0 +1,19 @@ +import asyncio + +from reduct import Client + + +async def main(): + async with Client("http://127.0.0.1:8383", api_token="my-token") as client: + for lifecycle in await client.get_lifecycles(): + print("Lifecycle:", lifecycle.name) + print("Mode:", lifecycle.mode) + print("Running:", lifecycle.is_running) + print("Provisioned:", lifecycle.is_provisioned) + + details = await client.get_lifecycle_detail("my-lifecycle") + print("Lifecycle:", details.info.name) + print("Settings:", details.settings) + + +asyncio.run(main()) diff --git a/docs/examples/py/src/lifecycle_policy_create.py b/docs/examples/py/src/lifecycle_policy_create.py new file mode 100644 index 00000000..a0e8d610 --- /dev/null +++ b/docs/examples/py/src/lifecycle_policy_create.py @@ -0,0 +1,21 @@ +import asyncio + +from reduct import Client, LifecycleSettings, LifecycleType + + +async def main(): + async with Client("http://127.0.0.1:8383", api_token="my-token") as client: + await client.create_bucket("my-bucket", exist_ok=True) + + settings = LifecycleSettings( + type=LifecycleType.DELETE, + bucket="my-bucket", + entries=["py-example"], + max_age="30d", + interval="1h", + when={"&anomaly": {"$eq": 1}}, + ) + await client.create_lifecycle("my-lifecycle", settings) + + +asyncio.run(main()) diff --git a/docs/examples/py/src/lifecycle_policy_mode.py b/docs/examples/py/src/lifecycle_policy_mode.py new file mode 100644 index 00000000..00a19c22 --- /dev/null +++ b/docs/examples/py/src/lifecycle_policy_mode.py @@ -0,0 +1,13 @@ +import asyncio + +from reduct import Client, LifecycleMode + + +async def main(): + async with Client("http://127.0.0.1:8383", api_token="my-token") as client: + await client.set_lifecycle_mode("my-lifecycle", LifecycleMode.DRY_RUN) + await client.set_lifecycle_mode("my-lifecycle", LifecycleMode.DISABLED) + await client.set_lifecycle_mode("my-lifecycle", LifecycleMode.ENABLED) + + +asyncio.run(main()) diff --git a/docs/examples/py/src/lifecycle_policy_remove.py b/docs/examples/py/src/lifecycle_policy_remove.py new file mode 100644 index 00000000..19dd41a2 --- /dev/null +++ b/docs/examples/py/src/lifecycle_policy_remove.py @@ -0,0 +1,11 @@ +import asyncio + +from reduct import Client + + +async def main(): + async with Client("http://127.0.0.1:8383", api_token="my-token") as client: + await client.delete_lifecycle("lifecycle-to-remove") + + +asyncio.run(main()) diff --git a/docs/examples/rs/examples/lifecycle_policy_browse.rs b/docs/examples/rs/examples/lifecycle_policy_browse.rs new file mode 100644 index 00000000..28884ec1 --- /dev/null +++ b/docs/examples/rs/examples/lifecycle_policy_browse.rs @@ -0,0 +1,22 @@ +use reduct_rs::{ReductClient, ReductError}; + +#[tokio::main] +async fn main() -> Result<(), ReductError> { + let client = ReductClient::builder() + .url("http://127.0.0.1:8383") + .api_token("my-token") + .build(); + + for lifecycle in client.list_lifecycles().await? { + println!("Lifecycle: {}", lifecycle.name); + println!("Mode: {:?}", lifecycle.mode); + println!("Running: {}", lifecycle.is_running); + println!("Provisioned: {}", lifecycle.is_provisioned); + } + + let detail = client.get_lifecycle("my-lifecycle").await?; + println!("Lifecycle: {}", detail.info.name); + println!("Settings: {:?}", detail.settings); + + Ok(()) +} diff --git a/docs/examples/rs/examples/lifecycle_policy_create.rs b/docs/examples/rs/examples/lifecycle_policy_create.rs new file mode 100644 index 00000000..f09881cb --- /dev/null +++ b/docs/examples/rs/examples/lifecycle_policy_create.rs @@ -0,0 +1,28 @@ +use reduct_rs::{condition, LifecycleType, ReductClient, ReductError}; + +#[tokio::main] +async fn main() -> Result<(), ReductError> { + let client = ReductClient::builder() + .url("http://127.0.0.1:8383") + .api_token("my-token") + .build(); + + let _ = client + .create_bucket("my-bucket") + .exist_ok(true) + .send() + .await?; + + client + .create_lifecycle("my-lifecycle") + .lifecycle_type(LifecycleType::Delete) + .bucket("my-bucket") + .entries(vec!["rs-example".to_string()]) + .max_age("30d") + .interval("1h") + .when(condition!({ "&anomaly": { "$eq": 1 } })) + .send() + .await?; + + Ok(()) +} diff --git a/docs/examples/rs/examples/lifecycle_policy_mode.rs b/docs/examples/rs/examples/lifecycle_policy_mode.rs new file mode 100644 index 00000000..df2e0572 --- /dev/null +++ b/docs/examples/rs/examples/lifecycle_policy_mode.rs @@ -0,0 +1,21 @@ +use reduct_rs::{LifecycleMode, ReductClient, ReductError}; + +#[tokio::main] +async fn main() -> Result<(), ReductError> { + let client = ReductClient::builder() + .url("http://127.0.0.1:8383") + .api_token("my-token") + .build(); + + client + .set_lifecycle_mode("my-lifecycle", LifecycleMode::DryRun) + .await?; + client + .set_lifecycle_mode("my-lifecycle", LifecycleMode::Disabled) + .await?; + client + .set_lifecycle_mode("my-lifecycle", LifecycleMode::Enabled) + .await?; + + Ok(()) +} diff --git a/docs/examples/rs/examples/lifecycle_policy_remove.rs b/docs/examples/rs/examples/lifecycle_policy_remove.rs new file mode 100644 index 00000000..f2defe4b --- /dev/null +++ b/docs/examples/rs/examples/lifecycle_policy_remove.rs @@ -0,0 +1,12 @@ +use reduct_rs::{ReductClient, ReductError}; + +#[tokio::main] +async fn main() -> Result<(), ReductError> { + let client = ReductClient::builder() + .url("http://127.0.0.1:8383") + .api_token("my-token") + .build(); + + client.delete_lifecycle("lifecycle-to-remove").await?; + Ok(()) +} diff --git a/docs/glossary.mdx b/docs/glossary.mdx index 1cf33ead..fc1918ef 100644 --- a/docs/glossary.mdx +++ b/docs/glossary.mdx @@ -124,3 +124,9 @@ A special instruction in a conditional query that modifies how the query engine ## query link A secure way to share data from ReductStore without using access tokens. Query links can be created with a specific query and expiration time, allowing controlled access to data. + +## lifecycle policy + +A background task that periodically applies an action, such as deleting old records, to a bucket. Lifecycle policies can be scoped to specific entries, filtered with a conditional query, disabled, or run in `dry_run` preview mode. + +_Related entries:_ bucket, record, conditional query diff --git a/docs/guides/data-management.mdx b/docs/guides/data-management.mdx index 5ba8db43..85fac11f 100644 --- a/docs/guides/data-management.mdx +++ b/docs/guides/data-management.mdx @@ -128,6 +128,16 @@ Attachment keys starting with `$` are reserved for official ReductStore plugins The ReductStore API allows you to delete specific records from an entry, an entire entry, or an entire **[bucket](../glossary#bucket)**. You can do this using the Client SDKs, the Reduct CLI, or the HTTP API. +### Automatic Deletion With Lifecycle Policies + +For automatic cleanup, use lifecycle policies. A delete lifecycle policy periodically +removes records older than `max_age` from a bucket. You can limit cleanup to selected +entries with `entries` and to matching records with a `when` condition. Use `dry_run` +mode to preview how many records match before enabling deletion. + +For a full explanation and provisioning examples, see the +**[Lifecycle Policies](lifecycle-policies.mdx)** guide. + ### Deleting Entry or Bucket The following examples show how to delete the entire entry or the entire bucket: diff --git a/docs/guides/lifecycle-policies.mdx b/docs/guides/lifecycle-policies.mdx new file mode 100644 index 00000000..367dbbe9 --- /dev/null +++ b/docs/guides/lifecycle-policies.mdx @@ -0,0 +1,269 @@ +--- +title: Lifecycle Policies +description: Complete guide to managing lifecycle policies in ReductStore +--- + +import CodeBlock from "@theme/CodeBlock"; +import Tabs from "@theme/Tabs"; +import TabItem from "@theme/TabItem"; +import lifecycleCreate from "!!raw-loader!@site/docs/examples/provisioning/lifecycle_create.yml"; + + + + + +# Lifecycle Policies + +**[Lifecycle policies](../glossary#lifecycle-policy)** are background tasks that automatically clean up old records from your **[buckets](../glossary#bucket)**. They help you manage storage efficiently by removing outdated data based on rules you define. + +## How It Works + +A lifecycle policy runs at a configurable interval and scans the target bucket for records older than `max_age`. It removes these records and logs the operation in the `$audit` system bucket. You can optionally scope policies to specific **[entries](../glossary#entry)** or filter records using conditions. + +Policies are stored on disk and remain active after restarts. However, read-only (replica) instances cannot execute lifecycle policies. Every policy run is recorded as a `lifecycle_run` event in the `$audit` bucket for transparency and debugging. + +## Supported Actions + +Currently, ReductStore supports one action type for lifecycle policies: + +| Type | Description | +| -------- | ---------------------------------------------------- | +| `delete` | Removes records older than `max_age` from the bucket | + +More action types may be added in future releases. + +## Policy Settings + +You can configure lifecycle policies using the following settings: + +| Setting | Description | Required | Default | +| ---------- | ----------------------------------------------------------------------------------------------------------- | -------- | --------- | +| `type` | Action type (e.g., `delete`) | No | `delete` | +| `bucket` | Target bucket name | Yes | - | +| `entries` | List of entries to scope the policy (wildcard `*` supported) | No | - | +| `max_age` | Maximum age of records before action is applied (e.g., `30d`, `1h`, `5m`) | Yes | - | +| `interval` | Policy execution interval (e.g., `3600s`, `10m`) | No | `3600s` | +| `when` | JSON condition to filter records (see **[Conditional Query Reference](/docs/conditional-query/index.mdx)**) | No | - | +| `mode` | Policy mode: `enabled` (default), `dry_run`, or `disabled` | No | `enabled` | + +## Conditional Policies + +You can refine lifecycle policies using the `entries` and `when` parameters: + +- **`entries`**: Limit the policy to specific entries (e.g., `sensor-1,sensor-2` or `sensor-*`) +- **`when`**: Apply the policy only to records matching a condition (e.g., `{"&sensor_type": {"$eq": "temperature"}}`) + +For more details, see the **[Conditional Query Reference](/docs/conditional-query/index.mdx)**. + +## Example + +Imagine you store telemetry data from temperature sensors in a bucket called `telemetry`. You want to automatically delete records older than 30 days, but only for specific sensors (`sensor-1` and `sensor-2`) and only if the `sensor_type` label equals `temperature`. Here’s how you’d configure the policy: + +{lifecycleCreate} + +## Managing Lifecycle Policies + +Here you will find examples of how to create, list, retrieve, update modes, and delete lifecycle policies using the ReductStore SDKs, REST API, **[CLI](../glossary#cli)** and **[Web Console](../glossary#web-console)**. + +All examples are written for a local ReductStore instance available at **`http://127.0.0.1:8383`** with API token `my-token`. + +For more information on setting up a local ReductStore instance, see the **[Getting Started](../getting-started/index.mdx)** guide. + +### Creating a Lifecycle Policy + +To create a lifecycle policy, you must provide at least the target bucket and `max_age`. The examples below also include optional filters (`entries` and `when`) and a custom `interval`. + +import CreateLifecycleCli from "!!raw-loader!../examples/cli/lifecycle_policy_create.sh"; +import CreateLifecyclePy from "!!raw-loader!../examples/py/src/lifecycle_policy_create.py"; +import CreateLifecycleJs from "!!raw-loader!../examples/js/src/lifecycle_policy_create.mjs"; +import CreateLifecycleGo from "!!raw-loader!../examples/go/src/lifecycle_policy_create.go"; +import CreateLifecycleRs from "!!raw-loader!../examples/rs/examples/lifecycle_policy_create.rs"; +import CreateLifecycleCpp from "!!raw-loader!../examples/cpp/src/lifecycle_policy_create.cc"; +import CreateLifecycleCurl from "!!raw-loader!../examples/curl/lifecycle_policy_create.sh"; + + + + {CreateLifecycleCli} + + + + Steps to create a lifecycle policy using the Web Console: + 1. Open the Web Console at `http://127.0.0.1:8383` in your browser. + 2. Enter the API token if authorization is enabled. + 3. Click **"Lifecycles"** in the left sidebar. + 4. Click the plus icon in the top-right corner. + 5. Fill in the lifecycle name, bucket, `max_age`, optional `interval`, and optional filters (`entries` and `when`). + 6. Click **"Create Lifecycle"**. + + + + {CreateLifecyclePy} + + + {CreateLifecycleJs} + + + {CreateLifecycleGo} + + + {CreateLifecycleRs} + + + {CreateLifecycleCpp} + + + {CreateLifecycleCurl} + + + +### Browsing Lifecycle Policies + +You can list all lifecycle policies and inspect full settings/details of a specific policy. + +import BrowseLifecycleCli from "!!raw-loader!../examples/cli/lifecycle_policy_browse.sh"; +import BrowseLifecyclePy from "!!raw-loader!../examples/py/src/lifecycle_policy_browse.py"; +import BrowseLifecycleJs from "!!raw-loader!../examples/js/src/lifecycle_policy_browse.mjs"; +import BrowseLifecycleGo from "!!raw-loader!../examples/go/src/lifecycle_policy_browse.go"; +import BrowseLifecycleRs from "!!raw-loader!../examples/rs/examples/lifecycle_policy_browse.rs"; +import BrowseLifecycleCpp from "!!raw-loader!../examples/cpp/src/lifecycle_policy_browse.cc"; +import BrowseLifecycleCurl from "!!raw-loader!../examples/curl/lifecycle_policy_browse.sh"; + + + + {BrowseLifecycleCli} + + + + Steps to browse lifecycle policies in the Web Console: + 1. Open the Web Console at `http://127.0.0.1:8383` in your browser. + 2. Enter the API token if authorization is enabled. + 3. Click **"Lifecycles"** in the left sidebar. + 4. Open any lifecycle policy in the list to inspect settings and status. + + + + {BrowseLifecyclePy} + + + {BrowseLifecycleJs} + + + {BrowseLifecycleGo} + + + {BrowseLifecycleRs} + + + {BrowseLifecycleCpp} + + + {BrowseLifecycleCurl} + + + +### Lifecycle Modes + +Lifecycle policies can be switched between modes without recreating the policy. + +import UpdateLifecycleModeCli from "!!raw-loader!../examples/cli/lifecycle_policy_mode.sh"; +import UpdateLifecycleModePy from "!!raw-loader!../examples/py/src/lifecycle_policy_mode.py"; +import UpdateLifecycleModeJs from "!!raw-loader!../examples/js/src/lifecycle_policy_mode.mjs"; +import UpdateLifecycleModeGo from "!!raw-loader!../examples/go/src/lifecycle_policy_mode.go"; +import UpdateLifecycleModeRs from "!!raw-loader!../examples/rs/examples/lifecycle_policy_mode.rs"; +import UpdateLifecycleModeCpp from "!!raw-loader!../examples/cpp/src/lifecycle_policy_mode.cc"; +import UpdateLifecycleModeCurl from "!!raw-loader!../examples/curl/lifecycle_policy_mode.sh"; + + + + {UpdateLifecycleModeCli} + + + + Steps to update lifecycle policy mode using the Web Console: + 1. Open the Web Console at `http://127.0.0.1:8383` in your browser. + 2. Enter the API token if authorization is enabled. + 3. Click **"Lifecycles"** in the left sidebar. + 4. Update the policy mode by clicking the dropdown button next to the policy name and selecting the desired mode (`enabled`, `dry_run`, or `disabled`). + + + + {UpdateLifecycleModePy} + + + {UpdateLifecycleModeJs} + + + {UpdateLifecycleModeGo} + + + {UpdateLifecycleModeRs} + + + {UpdateLifecycleModeCpp} + + + {UpdateLifecycleModeCurl} + + + +### Removing a Lifecycle Policy + +You can remove a lifecycle policy when it is no longer needed. + +:::info +Provisioned lifecycle policies cannot be modified or deleted through the API. To change or remove them, update environment variables and restart ReductStore. +::: + +import RemoveLifecycleCli from "!!raw-loader!../examples/cli/lifecycle_policy_remove.sh"; +import RemoveLifecyclePy from "!!raw-loader!../examples/py/src/lifecycle_policy_remove.py"; +import RemoveLifecycleJs from "!!raw-loader!../examples/js/src/lifecycle_policy_remove.mjs"; +import RemoveLifecycleGo from "!!raw-loader!../examples/go/src/lifecycle_policy_remove.go"; +import RemoveLifecycleRs from "!!raw-loader!../examples/rs/examples/lifecycle_policy_remove.rs"; +import RemoveLifecycleCpp from "!!raw-loader!../examples/cpp/src/lifecycle_policy_remove.cc"; +import RemoveLifecycleCurl from "!!raw-loader!../examples/curl/lifecycle_policy_remove.sh"; + + + + {RemoveLifecycleCli} + + + + Steps to remove a lifecycle policy using the Web Console: + 1. Open the Web Console at `http://127.0.0.1:8383` in your browser. + 2. Enter the API token if authorization is enabled. + 3. Click **"Lifecycles"** in the left sidebar. + 4. Open the lifecycle policy details. + 5. Click the trash icon in the bottom-right corner and confirm by typing the policy name. + + + + {RemoveLifecyclePy} + + + {RemoveLifecycleJs} + + + {RemoveLifecycleGo} + + + {RemoveLifecycleRs} + + + {RemoveLifecycleCpp} + + + {RemoveLifecycleCurl} + + + +## Audit Logs + +Every lifecycle run generates a `lifecycle_run` event in the `$audit` bucket. The event includes: + +- Policy name and action type +- Target bucket and duration of the run +- Number of records processed +- Error details (if any) diff --git a/docs/reduct-bridge/dev/index.mdx b/docs/reduct-bridge/dev/index.mdx new file mode 100644 index 00000000..af128d8d --- /dev/null +++ b/docs/reduct-bridge/dev/index.mdx @@ -0,0 +1,25 @@ +--- +title: "Dev Environment" +description: "Documentation for Dev Environment in ReductBridge." +--- +# Dev Environment + +## Start + +```sh +cargo build --features mqtt +cd dev && docker compose up -d +``` + +## Watch (auto-restart on rebuild) + +```sh +cargo watch -x 'build --features mqtt' -s 'docker compose -f dev/docker-compose.yml restart bridge' +``` + +## Regenerate proto files + +```sh +protoc --proto_path=dev/mqtt --include_imports --descriptor_set_out=dev/mqtt/factory.desc factory.proto +protoc --proto_path=dev/mqtt --python_out=dev/mqtt factory.proto +``` diff --git a/docs/reduct-bridge/index.mdx b/docs/reduct-bridge/index.mdx index f64d696a..d106642a 100644 --- a/docs/reduct-bridge/index.mdx +++ b/docs/reduct-bridge/index.mdx @@ -24,8 +24,17 @@ Supported input types include: - **[Metrics](./input/metrics/index.mdx)** - collect host CPU, memory, and disk metrics as JSON records. - **[MQTT](./input/mqtt/index.mdx)** - subscribe to MQTT v3/v5 topics over `mqtt://` or `mqtts://` and store raw payloads with optional payload/property label mapping. - **[Shell](./input/shell/index.mdx)** - run shell commands on a fixed interval and store output lines as records. -- **[ROS1](./input/ros1/index.mdx)** - subscribe to ROS1 topics and store ROS messages as records. -- **[ROS2](./input/ros2/index.mdx)** - subscribe to ROS2 topics and store serialized CDR payloads as records. +- **[ROS1](./input/ros1/index.mdx)** - subscribe to ROS1 topics, store ROS messages, and extract labels from message fields. +- **[ROS2](./input/ros2/index.mdx)** - subscribe to ROS2 topics, store serialized CDR payloads, and extract labels from decoded message fields. + +## Formats + +ReductBridge supports different payload formats. +Support means ReductBridge can parse payloads, extract labels, and, when schema information is available, store that schema in ReductStore. + +- JSON: find values by field path (example: **[examples/mqtt_config.toml](https://github.com/reductstore/reduct-bridge/blob/main/examples/mqtt_config.toml)**). +- Protobuf: find values by field path (with schema) or by field ID/type (example: **[examples/mqtt_protobuf_config.toml](https://github.com/reductstore/reduct-bridge/blob/main/examples/mqtt_protobuf_config.toml)**). +- ROS formats: decode ROS message payloads for labels and store the payloads as records (example: **[examples/ros_config.toml](https://github.com/reductstore/reduct-bridge/blob/main/examples/ros_config.toml)**). ## Remotes @@ -65,6 +74,50 @@ labels = [ ## Installation +ReductBridge is published in build types named after Cargo feature bundles: `ros1`, `ros2`, and `iot`. +Choose the build type for the input family you need. + +### Build Types + +Published packages and Docker images are split into these build types so each artifact only includes the dependencies needed for its input family. + +| Build type | Description | Included inputs | Published artifact names | +| --- | --- | --- | --- | +| `ros1` | ROS1 robotics bundle. | **[ROS1](./input/ros1/index.mdx)**, **[Shell](./input/shell/index.mdx)**, **[Metrics](./input/metrics/index.mdx)** | Snap: `reduct-bridge-ros1`; Docker build type: `ros1`; binary: `bridge-ros1.x86_64-unknown-linux-gnu.tar.gz` | +| `ros2` | ROS2 robotics bundle. | **[ROS2](./input/ros2/index.mdx)**, **[Shell](./input/shell/index.mdx)**, **[Metrics](./input/metrics/index.mdx)** | Snap: `reduct-bridge-ros2`; Docker build types: `ros2-jazzy`, `ros2-humble`; binaries: `bridge-ros2-jazzy.x86_64-unknown-linux-gnu.tar.gz`, `bridge-ros2-humble.x86_64-unknown-linux-gnu.tar.gz` | +| `iot` | MQTT/IIoT bundle. | **[MQTT](./input/mqtt/index.mdx)**, **[Shell](./input/shell/index.mdx)**, **[Metrics](./input/metrics/index.mdx)** | Snap: `reduct-bridge-iot`; Docker build type: `iot`; binary: `bridge-iot.x86_64-unknown-linux-gnu.tar.gz` | + +Note: ROS2 binary and Docker artifacts are published per ROS distribution: `ros2-jazzy` and `ros2-humble`. The ROS2 snap is published as `reduct-bridge-ros2` and currently uses the Jazzy build. + +Input names in configuration use their input table names, for example `[inputs.ros.*]`, `[inputs.ros2.*]`, `[inputs.mqtt.*]`, `[inputs.shell.*]`, and `[inputs.metrics.*]`. + +### Docker + +Docker images are published as `reduct/bridge:`, where `` combines a release channel or version with the build type. +For ROS2 Docker images, use the ROS distribution-specific build type: `ros2-jazzy` or `ros2-humble`. + +```bash +docker pull reduct/bridge:main- # development builds from main +docker pull reduct/bridge:latest- # stable builds +docker pull reduct/bridge:v- # release builds +``` + +For example: + +```bash +docker pull reduct/bridge:main-iot +docker pull reduct/bridge:main-ros2-jazzy +docker pull reduct/bridge:main-ros2-humble +``` + +Run the container with a mounted config file: + +```bash +docker run --rm \ + -v "$PWD/config.toml:/etc/reduct-bridge/config.toml:ro" \ + reduct/bridge:main-iot /etc/reduct-bridge/config.toml +``` + ### Snap (Ubuntu 22.04+) ```bash @@ -100,8 +153,8 @@ To build additional inputs explicitly from source: ```bash cargo build --no-default-features --features ros1 -cargo build --no-default-features --features shell,metrics,iot -cargo build --no-default-features --features all-inputs +cargo build --no-default-features --features ros2 +cargo build --no-default-features --features iot ``` ## Usage diff --git a/docs/reduct-bridge/input/metrics/index.mdx b/docs/reduct-bridge/input/metrics/index.mdx index e990330f..a69bb30b 100644 --- a/docs/reduct-bridge/input/metrics/index.mdx +++ b/docs/reduct-bridge/input/metrics/index.mdx @@ -98,12 +98,6 @@ Build only metrics input support: cargo build --no-default-features --features metrics ``` -Build all inputs in one command: - -```bash -cargo build --no-default-features --features all-inputs -``` - ## Runtime Notes - If `metrics` is empty or omitted, the input collects all supported metrics. diff --git a/docs/reduct-bridge/input/mqtt/index.mdx b/docs/reduct-bridge/input/mqtt/index.mdx index 0afbc756..5e3f0639 100644 --- a/docs/reduct-bridge/input/mqtt/index.mdx +++ b/docs/reduct-bridge/input/mqtt/index.mdx @@ -7,43 +7,64 @@ description: "Documentation for MQTT input in ReductBridge." The MQTT input subscribes to one or more MQTT topics and forwards messages into a pipeline. Both `mqtt://` and `mqtts://` brokers are supported. -MQTT label rules follow the same ordered model as the ROS inputs: +Labels are applied with the following rules: -- static labels are applied first -- payload field labels are applied after static labels and can override them -- MQTT v5 property labels are also applied in order and can override earlier labels +1. Labels are processed in the order they appear in `labels` in TOML. +2. For conflicts, later rules override earlier rules for the same label key. +3. This ordering behavior applies to all rule types: `static`, `field`, and `property`. +4. Payload field labels (`field = "..."`) work with JSON and protobuf payloads. +5. MQTT v5 property labels: + - `property = "content_type"` reads the built-in MQTT v5 `content_type` property. + - `property = ""` reads the MQTT v5 user property named ``. -For MQTT v5 property labels: +## Protobuf support -- `property = "content_type"` reads the built-in MQTT v5 `content_type` property -- any other `property = ""` reads the MQTT v5 user property named `` +MQTT input supports protobuf payloads and can map protobuf fields to labels. +Choose one of these modes. + +1. Schema mode (recommended) + +- `schema_path` must point to a protobuf descriptor set file generated by `protoc`. + Example command: `protoc --include_imports --descriptor_set_out=schema.desc schema.proto`. +- `schema_name` is the protobuf message type to decode from the payload, e.g. `my.package.Message`. +- Use `{ field = "...", label = "..." }` to extract labels by field path. +- The schema is emitted as attachment `$schema` in JSON with fields: + `encoding` (payload encoding, `protobuf`), + `topic` (MQTT publish topic that produced the record), + `schema_name` (same as configured), and + `schema` (the configured schema file content, base64-encoded). + +Use this mode when you want self-describing data and easier downstream inspection. + +2. No-schema mode + +- Omit both `schema_path` and `schema_name`. +- Use `{ field_id = , field_type = "", label = "..." }` to extract labels from raw protobuf wire fields. + +Use this mode for lightweight setups when you only need a few known fields. ## Configuration +### Shared input settings + ```toml [inputs.mqtt.main] broker = "mqtts://broker.example.com:8883" client_id = "reduct-bridge" version = "v5" qos = 1 - username = "bridge" password = "${MQTT_PASSWORD}" - entry_prefix = "/mqtt" +``` +### JSON topic + +```toml [[inputs.mqtt.main.topics]] name = "factory/+/telemetry" entry_name = "telemetry" content_type = "application/json" -# Optional label rules (default = []): -# 1) Dynamic field label: -# { field = "device_id", label = "device" } -# 2) Static labels: -# { static = { source = "mqtt", site = "lab" } } -# 3) MQTT v5 property label: -# { property = "content_type", label = "mime" } -# { property = "tenant", label = "tenant" } # MQTT v5 user property labels = [ { field = "device_id", label = "device" }, { field = "site", label = "site" }, @@ -51,7 +72,37 @@ labels = [ { property = "content_type", label = "mime" }, { property = "tenant", label = "tenant" } ] +``` + +### Protobuf topic with schema (recommended) +```toml +[[inputs.mqtt.main.topics]] +name = "factory/electrical/+/power" +entry_name = "power" +content_type = "application/protobuf" +schema_path = "./factory.desc" +schema_name = "factory.PowerReading" +labels = [ + { field = "device_id", label = "meter_id" }, + { field = "panel", label = "panel" } +] +``` + +### Protobuf topic without schema + +```toml +[[inputs.mqtt.main.topics]] +name = "factory/electrical/+/power/raw" +content_type = "application/protobuf" +labels = [ + { field_id = 1, field_type = "string", label = "meter_id" } +] +``` + +### Static-only topic + +```toml [[inputs.mqtt.main.topics]] name = "factory/+/events" labels = [ @@ -59,8 +110,11 @@ labels = [ ] ``` -`entry_name`, `content_type`, and `labels` are configured per topic. `content_type` is used as the default record content type when an MQTT v5 publish does not provide a `content_type` property; for MQTT v3 it is used directly. -Use bare names such as `tenant`, `status`, or `id` for MQTT v5 user properties. +`entry_name`, `content_type`, and `labels` are configured per topic. `content_type` +is used as the default record content type when an MQTT v5 publish does not provide +a `content_type` property; for MQTT v3 it is used directly. Use bare names such as +`tenant`, `status`, or `id` for MQTT v5 user properties. +`schema_path` and `schema_name` must be set together. ### MQTT v3 example diff --git a/docs/reduct-bridge/input/ros1/index.mdx b/docs/reduct-bridge/input/ros1/index.mdx index a9fa79cc..a03d612c 100644 --- a/docs/reduct-bridge/input/ros1/index.mdx +++ b/docs/reduct-bridge/input/ros1/index.mdx @@ -57,12 +57,6 @@ Build only ROS1 input support: cargo build --no-default-features --features ros1 ``` -Build all inputs in one command: - -```bash -cargo build --no-default-features --features all-inputs -``` - ## Runtime Notes - Empty topic names are not allowed. diff --git a/docs/reduct-bridge/input/ros2/index.mdx b/docs/reduct-bridge/input/ros2/index.mdx index db6cae8d..4469b2f6 100644 --- a/docs/reduct-bridge/input/ros2/index.mdx +++ b/docs/reduct-bridge/input/ros2/index.mdx @@ -70,13 +70,6 @@ cargo build --no-default-features --features ros2 The ROS2 installation must include the standard interface packages that `rclrs` links against. In CI we install `ros-jazzy-example-interfaces` and `ros-jazzy-test-msgs` explicitly. -Build all inputs in one command: - -```bash -source /opt/ros/jazzy/setup.bash -cargo build --no-default-features --features all-inputs -``` - ## Runtime Notes - Empty topic names are not allowed. diff --git a/docs/reduct-bridge/remote/reduct/index.mdx b/docs/reduct-bridge/remote/reduct/index.mdx index 9cf5bf42..d72b74d8 100644 --- a/docs/reduct-bridge/remote/reduct/index.mdx +++ b/docs/reduct-bridge/remote/reduct/index.mdx @@ -37,12 +37,26 @@ batch_max_size_bytes = 8388608 # Periodic flush interval in milliseconds (default: 1000, must be > 0). batch_max_interval_ms = 1000 + +# Optional: create the bucket on startup if it does not exist. +# Omit this table to require the bucket to exist before starting reduct-bridge. +[remotes.reduct.create_bucket] +# Required when create_bucket is configured: NONE, FIFO, or HARD. +quota_type = "FIFO" + +# Required when create_bucket is configured: quota size as bytes or a unit string. +# Accepted examples: 1073741824, "1GB", "512MB", "4GiB", "1.5TiB", "500 B". +# Accepted decimal units: B, K/KB, M/MB, G/GB, T/TB, P/PB, E/EB. +# Accepted binary units: Ki/KiB, Mi/MiB, Gi/GiB, Ti/TiB, Pi/PiB, Ei/EiB. +quota_size = "1GB" ``` ## Runtime Notes - Records are buffered and flushed by count, size, or interval. - Invalid batching values (`0`) are not allowed. +- Buckets are not created automatically unless `[remotes.reduct.create_bucket]` is configured. +- Automatic bucket creation only applies when the configured bucket is missing; existing bucket settings are not changed. ## Changes diff --git a/docs/sdk/js/api/classes/Bucket.md b/docs/sdk/js/api/classes/Bucket.md index 3fc3f77e..52abad20 100644 --- a/docs/sdk/js/api/classes/Bucket.md +++ b/docs/sdk/js/api/classes/Bucket.md @@ -9,7 +9,7 @@ description: "API reference for Bucket in the ReductStore Client SDK for JavaScr # Bucket -Defined in: [Bucket.ts:27](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L27) +Defined in: [Bucket.ts:28](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L28) Represents a bucket in ReductStore @@ -19,7 +19,7 @@ Represents a bucket in ReductStore > **new Bucket**(`name`, `httpClient`): `Bucket` -Defined in: [Bucket.ts:38](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L38) +Defined in: [Bucket.ts:39](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L39) Create a bucket. Use Client.creatBucket or Client.getBucket instead it @@ -45,7 +45,7 @@ Create a bucket. Use Client.creatBucket or Client.getBucket instead it > **beginRead**(`entry`, `ts?`, `head?`): `Promise`\<`ReadableRecord`\> -Defined in: [Bucket.ts:233](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L233) +Defined in: [Bucket.ts:234](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L234) Start reading a record from an entry @@ -81,7 +81,7 @@ Promise<ReadableRecord> > **beginRemoveBatch**(`entry`): `Promise`\<[`Batch`](Batch.md)\> -Defined in: [Bucket.ts:121](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L121) +Defined in: [Bucket.ts:122](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L122) Remove a batch of records @@ -103,7 +103,7 @@ Remove a batch of records > **beginRemoveRecordBatch**(): [`RecordBatch`](RecordBatch.md) -Defined in: [Bucket.ts:478](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L478) +Defined in: [Bucket.ts:479](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L479) Create a new batch for removing records across multiple entries. @@ -117,7 +117,7 @@ Create a new batch for removing records across multiple entries. > **beginUpdateBatch**(`entry`): `Promise`\<[`Batch`](Batch.md)\> -Defined in: [Bucket.ts:459](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L459) +Defined in: [Bucket.ts:460](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L460) Create a new batch for updating records in the database. @@ -137,7 +137,7 @@ Create a new batch for updating records in the database. > **beginUpdateRecordBatch**(): [`RecordBatch`](RecordBatch.md) -Defined in: [Bucket.ts:471](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L471) +Defined in: [Bucket.ts:472](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L472) Create a new batch for updating records across multiple entries. @@ -160,7 +160,7 @@ await batch.send(); > **beginWrite**(`entry`, `options?`): `Promise`\<`WritableRecord`\> -Defined in: [Bucket.ts:184](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L184) +Defined in: [Bucket.ts:185](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L185) Start writing a record into an entry @@ -201,7 +201,7 @@ await record.write("Hello!"); > **beginWriteBatch**(`entry`): `Promise`\<[`Batch`](Batch.md)\> -Defined in: [Bucket.ts:439](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L439) +Defined in: [Bucket.ts:440](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L440) Create a new batch for writing records to the database. @@ -221,7 +221,7 @@ Create a new batch for writing records to the database. > **beginWriteRecordBatch**(): [`RecordBatch`](RecordBatch.md) -Defined in: [Bucket.ts:451](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L451) +Defined in: [Bucket.ts:452](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L452) Create a new batch for writing records to multiple entries. @@ -244,7 +244,7 @@ await batch.send(); > **createQueryLink**(`entry`, `start?`, `stop?`, `query?`, `record?`, `expireAt?`, `fileName?`, `baseUrl?`): `Promise`\<`string`\> -Defined in: [Bucket.ts:495](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L495) +Defined in: [Bucket.ts:496](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L496) Create a query link for downloading records @@ -310,7 +310,7 @@ base url for link generation. If not set, the server's base url will be used > **getEntryList**(): `Promise`\<[`EntryInfo`](EntryInfo.md)[]\> -Defined in: [Bucket.ts:82](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L82) +Defined in: [Bucket.ts:83](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L83) Get entry list @@ -326,7 +326,7 @@ Get entry list > **getInfo**(): `Promise`\<[`BucketInfo`](BucketInfo.md)\> -Defined in: [Bucket.ts:72](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L72) +Defined in: [Bucket.ts:73](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L73) Get information about a bucket @@ -342,7 +342,7 @@ Get information about a bucket > **getName**(): `string` -Defined in: [Bucket.ts:350](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L350) +Defined in: [Bucket.ts:351](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L351) #### Returns @@ -354,7 +354,7 @@ Defined in: [Bucket.ts:350](https://github.com/reductstore/reduct-js/blob/main/s > **getSettings**(): `Promise`\<[`BucketSettings`](BucketSettings.md)\> -Defined in: [Bucket.ts:50](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L50) +Defined in: [Bucket.ts:51](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L51) Get bucket settings @@ -370,7 +370,7 @@ Get bucket settings > **query**(`entry`, `start?`, `stop?`, `options?`): `AsyncGenerator`\<`ReadableRecord`\> -Defined in: [Bucket.ts:295](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L295) +Defined in: [Bucket.ts:296](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L296) Query records for a time interval as generator @@ -423,7 +423,7 @@ for await (const record in bucket.query("entry-1", start, stop)) { > **readAttachments**(`entry`): `Promise`\<`Record`\<`string`, `unknown`\>\> -Defined in: [Bucket.ts:616](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L616) +Defined in: [Bucket.ts:617](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L617) Read attachments from an entry. @@ -447,7 +447,7 @@ map of attachment key to decoded JSON value > **remove**(): `Promise`\<`void`\> -Defined in: [Bucket.ts:94](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L94) +Defined in: [Bucket.ts:95](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L95) Remove bucket @@ -463,7 +463,7 @@ Remove bucket > **removeAttachments**(`entry`, `attachmentKeys?`): `Promise`\<`void`\> -Defined in: [Bucket.ts:640](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L640) +Defined in: [Bucket.ts:646](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L646) Remove attachments from an entry. @@ -493,7 +493,7 @@ list of keys to remove > **removeEntry**(`entry`): `Promise`\<`void`\> -Defined in: [Bucket.ts:104](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L104) +Defined in: [Bucket.ts:105](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L105) Remove an entry @@ -517,7 +517,7 @@ Remove an entry > **removeQuery**(`entry`, `start?`, `stop?`, `options?`): `Promise`\<`number`\> -Defined in: [Bucket.ts:132](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L132) +Defined in: [Bucket.ts:133](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L133) Remove records by query @@ -557,7 +557,7 @@ Remove records by query > **removeRecord**(`entry`, `ts`): `Promise`\<`void`\> -Defined in: [Bucket.ts:113](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L113) +Defined in: [Bucket.ts:114](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L114) Remove a record @@ -585,7 +585,7 @@ Remove a record > **rename**(`newName`): `Promise`\<`void`\> -Defined in: [Bucket.ts:266](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L266) +Defined in: [Bucket.ts:267](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L267) Rename a bucket @@ -607,7 +607,7 @@ new name of the bucket > **renameEntry**(`entry`, `newEntry`): `Promise`\<`void`\> -Defined in: [Bucket.ts:250](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L250) +Defined in: [Bucket.ts:251](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L251) Rename an entry @@ -635,7 +635,7 @@ new entry name > **setSettings**(`settings`): `Promise`\<`void`\> -Defined in: [Bucket.ts:60](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L60) +Defined in: [Bucket.ts:61](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L61) Set bucket settings @@ -659,7 +659,7 @@ Set bucket settings > **update**(`entry`, `ts`, `labels`): `Promise`\<`void`\> -Defined in: [Bucket.ts:212](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L212) +Defined in: [Bucket.ts:213](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L213) Update labels of an existing record @@ -693,9 +693,9 @@ If a label has empty string value, it will be removed. ### writeAttachments() -> **writeAttachments**(`entry`, `attachments`): `Promise`\<`void`\> +> **writeAttachments**(`entry`, `attachments`, `contentType?`): `Promise`\<`void`\> -Defined in: [Bucket.ts:587](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L587) +Defined in: [Bucket.ts:588](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L588) Write attachments to an entry. @@ -715,6 +715,10 @@ name of the source entry map of attachment key to JSON-serializable content +##### contentType? + +`string` + #### Returns `Promise`\<`void`\> diff --git a/docs/sdk/js/api/interfaces/WriteOptions.md b/docs/sdk/js/api/interfaces/WriteOptions.md index 45d2e5d8..32c45614 100644 --- a/docs/sdk/js/api/interfaces/WriteOptions.md +++ b/docs/sdk/js/api/interfaces/WriteOptions.md @@ -9,7 +9,7 @@ description: "API reference for WriteOptions in the ReductStore Client SDK for J # WriteOptions -Defined in: [Bucket.ts:18](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L18) +Defined in: [Bucket.ts:19](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L19) Options for writing records @@ -19,7 +19,7 @@ Options for writing records > `optional` **contentType**: `string` -Defined in: [Bucket.ts:21](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L21) +Defined in: [Bucket.ts:22](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L22) *** @@ -27,7 +27,7 @@ Defined in: [Bucket.ts:21](https://github.com/reductstore/reduct-js/blob/main/sr > `optional` **labels**: `LabelMap` -Defined in: [Bucket.ts:20](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L20) +Defined in: [Bucket.ts:21](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L21) *** @@ -35,4 +35,4 @@ Defined in: [Bucket.ts:20](https://github.com/reductstore/reduct-js/blob/main/sr > `optional` **ts**: `bigint` -Defined in: [Bucket.ts:19](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L19) +Defined in: [Bucket.ts:20](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L20) diff --git a/docs/sdk/js/bucket/index.mdx b/docs/sdk/js/bucket/index.mdx index d2ba58c0..01cfda93 100644 --- a/docs/sdk/js/bucket/index.mdx +++ b/docs/sdk/js/bucket/index.mdx @@ -12,7 +12,7 @@ description: API reference for the Bucket Module of the ReductStore Client SDK f ## Bucket -Defined in: [Bucket.ts:27](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L27) +Defined in: [Bucket.ts:28](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L28) Represents a bucket in ReductStore @@ -22,7 +22,7 @@ Represents a bucket in ReductStore > **new Bucket**(`name`, `httpClient`): `Bucket` -Defined in: [Bucket.ts:38](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L38) +Defined in: [Bucket.ts:39](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L39) Create a bucket. Use Client.creatBucket or Client.getBucket instead it @@ -48,7 +48,7 @@ Create a bucket. Use Client.creatBucket or Client.getBucket instead it > **beginRead**(`entry`, `ts?`, `head?`): `Promise`<`ReadableRecord`> -Defined in: [Bucket.ts:233](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L233) +Defined in: [Bucket.ts:234](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L234) Start reading a record from an entry @@ -84,7 +84,7 @@ Promise<ReadableRecord> > **beginRemoveBatch**(`entry`): `Promise`<[`Batch`](api/classes/Batch)> -Defined in: [Bucket.ts:121](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L121) +Defined in: [Bucket.ts:122](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L122) Remove a batch of records @@ -106,7 +106,7 @@ Remove a batch of records > **beginRemoveRecordBatch**(): [`RecordBatch`](api/classes/RecordBatch) -Defined in: [Bucket.ts:478](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L478) +Defined in: [Bucket.ts:479](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L479) Create a new batch for removing records across multiple entries. @@ -120,7 +120,7 @@ Create a new batch for removing records across multiple entries. > **beginUpdateBatch**(`entry`): `Promise`<[`Batch`](api/classes/Batch)> -Defined in: [Bucket.ts:459](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L459) +Defined in: [Bucket.ts:460](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L460) Create a new batch for updating records in the database. @@ -140,7 +140,7 @@ Create a new batch for updating records in the database. > **beginUpdateRecordBatch**(): [`RecordBatch`](api/classes/RecordBatch) -Defined in: [Bucket.ts:471](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L471) +Defined in: [Bucket.ts:472](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L472) Create a new batch for updating records across multiple entries. @@ -163,7 +163,7 @@ await batch.send(); > **beginWrite**(`entry`, `options?`): `Promise`<`WritableRecord`> -Defined in: [Bucket.ts:184](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L184) +Defined in: [Bucket.ts:185](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L185) Start writing a record into an entry @@ -204,7 +204,7 @@ await record.write("Hello!"); > **beginWriteBatch**(`entry`): `Promise`<[`Batch`](api/classes/Batch)> -Defined in: [Bucket.ts:439](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L439) +Defined in: [Bucket.ts:440](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L440) Create a new batch for writing records to the database. @@ -224,7 +224,7 @@ Create a new batch for writing records to the database. > **beginWriteRecordBatch**(): [`RecordBatch`](api/classes/RecordBatch) -Defined in: [Bucket.ts:451](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L451) +Defined in: [Bucket.ts:452](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L452) Create a new batch for writing records to multiple entries. @@ -247,7 +247,7 @@ await batch.send(); > **createQueryLink**(`entry`, `start?`, `stop?`, `query?`, `record?`, `expireAt?`, `fileName?`, `baseUrl?`): `Promise`<`string`> -Defined in: [Bucket.ts:495](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L495) +Defined in: [Bucket.ts:496](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L496) Create a query link for downloading records @@ -313,7 +313,7 @@ base url for link generation. If not set, the server's base url will be used > **getEntryList**(): `Promise`<[`EntryInfo`](api/classes/EntryInfo)[]> -Defined in: [Bucket.ts:82](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L82) +Defined in: [Bucket.ts:83](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L83) Get entry list @@ -329,7 +329,7 @@ Get entry list > **getInfo**(): `Promise`<[`BucketInfo`](api/classes/BucketInfo)> -Defined in: [Bucket.ts:72](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L72) +Defined in: [Bucket.ts:73](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L73) Get information about a bucket @@ -345,7 +345,7 @@ Get information about a bucket > **getName**(): `string` -Defined in: [Bucket.ts:350](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L350) +Defined in: [Bucket.ts:351](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L351) ##### Returns @@ -357,7 +357,7 @@ Defined in: [Bucket.ts:350](https://github.com/reductstore/reduct-js/blob/main/s > **getSettings**(): `Promise`<[`BucketSettings`](api/classes/BucketSettings)> -Defined in: [Bucket.ts:50](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L50) +Defined in: [Bucket.ts:51](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L51) Get bucket settings @@ -373,7 +373,7 @@ Get bucket settings > **query**(`entry`, `start?`, `stop?`, `options?`): `AsyncGenerator`<`ReadableRecord`> -Defined in: [Bucket.ts:295](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L295) +Defined in: [Bucket.ts:296](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L296) Query records for a time interval as generator @@ -426,7 +426,7 @@ for await (const record in bucket.query("entry-1", start, stop)) { > **readAttachments**(`entry`): `Promise`<`Record`<`string`, `unknown`>> -Defined in: [Bucket.ts:616](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L616) +Defined in: [Bucket.ts:617](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L617) Read attachments from an entry. @@ -450,7 +450,7 @@ map of attachment key to decoded JSON value > **remove**(): `Promise`<`void`> -Defined in: [Bucket.ts:94](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L94) +Defined in: [Bucket.ts:95](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L95) Remove bucket @@ -466,7 +466,7 @@ Remove bucket > **removeAttachments**(`entry`, `attachmentKeys?`): `Promise`<`void`> -Defined in: [Bucket.ts:640](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L640) +Defined in: [Bucket.ts:646](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L646) Remove attachments from an entry. @@ -496,7 +496,7 @@ list of keys to remove > **removeEntry**(`entry`): `Promise`<`void`> -Defined in: [Bucket.ts:104](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L104) +Defined in: [Bucket.ts:105](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L105) Remove an entry @@ -520,7 +520,7 @@ Remove an entry > **removeQuery**(`entry`, `start?`, `stop?`, `options?`): `Promise`<`number`> -Defined in: [Bucket.ts:132](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L132) +Defined in: [Bucket.ts:133](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L133) Remove records by query @@ -560,7 +560,7 @@ Remove records by query > **removeRecord**(`entry`, `ts`): `Promise`<`void`> -Defined in: [Bucket.ts:113](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L113) +Defined in: [Bucket.ts:114](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L114) Remove a record @@ -588,7 +588,7 @@ Remove a record > **rename**(`newName`): `Promise`<`void`> -Defined in: [Bucket.ts:266](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L266) +Defined in: [Bucket.ts:267](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L267) Rename a bucket @@ -610,7 +610,7 @@ new name of the bucket > **renameEntry**(`entry`, `newEntry`): `Promise`<`void`> -Defined in: [Bucket.ts:250](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L250) +Defined in: [Bucket.ts:251](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L251) Rename an entry @@ -638,7 +638,7 @@ new entry name > **setSettings**(`settings`): `Promise`<`void`> -Defined in: [Bucket.ts:60](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L60) +Defined in: [Bucket.ts:61](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L61) Set bucket settings @@ -662,7 +662,7 @@ Set bucket settings > **update**(`entry`, `ts`, `labels`): `Promise`<`void`> -Defined in: [Bucket.ts:212](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L212) +Defined in: [Bucket.ts:213](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L213) Update labels of an existing record @@ -696,9 +696,9 @@ If a label has empty string value, it will be removed. #### writeAttachments() -> **writeAttachments**(`entry`, `attachments`): `Promise`<`void`> +> **writeAttachments**(`entry`, `attachments`, `contentType?`): `Promise`<`void`> -Defined in: [Bucket.ts:587](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L587) +Defined in: [Bucket.ts:588](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L588) Write attachments to an entry. @@ -718,6 +718,10 @@ name of the source entry map of attachment key to JSON-serializable content +###### contentType? + +`string` + ##### Returns `Promise`<`void`> @@ -727,7 +731,7 @@ map of attachment key to JSON-serializable content ## WriteOptions -Defined in: [Bucket.ts:18](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L18) +Defined in: [Bucket.ts:19](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L19) Options for writing records @@ -737,7 +741,7 @@ Options for writing records > `optional` **contentType**: `string` -Defined in: [Bucket.ts:21](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L21) +Defined in: [Bucket.ts:22](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L22) *** @@ -745,7 +749,7 @@ Defined in: [Bucket.ts:21](https://github.com/reductstore/reduct-js/blob/main/sr > `optional` **labels**: `LabelMap` -Defined in: [Bucket.ts:20](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L20) +Defined in: [Bucket.ts:21](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L21) *** @@ -753,5 +757,5 @@ Defined in: [Bucket.ts:20](https://github.com/reductstore/reduct-js/blob/main/sr > `optional` **ts**: `bigint` -Defined in: [Bucket.ts:19](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L19) +Defined in: [Bucket.ts:20](https://github.com/reductstore/reduct-js/blob/main/src/Bucket.ts#L20) diff --git a/docs/sdk/py/client/index.mdx b/docs/sdk/py/client/index.mdx index 14657a2a..f972456c 100644 --- a/docs/sdk/py/client/index.mdx +++ b/docs/sdk/py/client/index.mdx @@ -387,4 +387,121 @@ Delete a replication - `ReductError` - if there is an HTTP error + + +#### get\_lifecycles + +```python +async def get_lifecycles() -> list[LifecycleInfo] +``` + +Get a list of lifecycle policies + +**Returns**: + +- `List[LifecycleInfo]` - List of lifecycle policies with their statuses + +**Raises**: + +- `ReductError` - if there is an HTTP error + + + +#### get\_lifecycle\_detail + +```python +async def get_lifecycle_detail(lifecycle_name: str) -> LifecycleDetailInfo +``` + +Get detailed information about a lifecycle policy + +**Arguments**: + +- `lifecycle_name` - Name of the lifecycle policy to show details + +**Returns**: + +- `LifecycleDetailInfo` - Detailed information about the lifecycle policy + +**Raises**: + +- `ReductError` - if there is an HTTP error + + + +#### create\_lifecycle + +```python +async def create_lifecycle(lifecycle_name: str, + settings: LifecycleSettings) -> None +``` + +Create a new lifecycle policy + +**Arguments**: + +- `lifecycle_name` - Name of the new lifecycle policy +- `settings` - Settings for the new lifecycle policy + +**Raises**: + +- `ReductError` - if there is an HTTP error + + + +#### update\_lifecycle + +```python +async def update_lifecycle(lifecycle_name: str, + settings: LifecycleSettings) -> None +``` + +Update an existing lifecycle policy + +**Arguments**: + +- `lifecycle_name` - Name of the lifecycle policy to update +- `settings` - New settings for the lifecycle policy + +**Raises**: + +- `ReductError` - if there is an HTTP error + + + +#### set\_lifecycle\_mode + +```python +async def set_lifecycle_mode(lifecycle_name: str, mode: LifecycleMode) -> None +``` + +Update mode for an existing lifecycle policy without overriding settings + +**Arguments**: + +- `lifecycle_name` - Name of the lifecycle policy to update +- `mode` - New lifecycle mode + +**Raises**: + +- `ReductError` - if there is an HTTP error + + + +#### delete\_lifecycle + +```python +async def delete_lifecycle(lifecycle_name: str) -> None +``` + +Delete a lifecycle policy + +**Arguments**: + +- `lifecycle_name` - Name of the lifecycle policy to delete + +**Raises**: + +- `ReductError` - if there is an HTTP error + diff --git a/sidebars.json b/sidebars.json index 23da0202..91bc3aef 100644 --- a/sidebars.json +++ b/sidebars.json @@ -63,6 +63,7 @@ "guides/data-ingestion", "guides/data-querying", "guides/data-management", + "guides/lifecycle-policies", "guides/data-replication", "guides/access-control", "guides/disaster-recovery"