diff --git a/pip/pip-317.md b/pip/pip-317.md new file mode 100644 index 0000000000000..03ddae41838a2 --- /dev/null +++ b/pip/pip-317.md @@ -0,0 +1,75 @@ +# PIP-317: Add `bookkeeperDeleted` field to show whether a ledger is deleted from the Bookie while using tiered storage + +# Background knowledge + +Pulsar's Tiered Storage feature allows older backlog data to be moved from BookKeeper to long-term and cheaper storage, while still allowing clients to access the backlog as if nothing has changed. + +The Pulsar Admin CLI provides a command called `./bin/pulsar-admin topics stats-internal`. This command returns internal statistics of a topic, including information about the ledgers associated with the topic. + +# Motivation + +Suppose we are using Pulsar's Tiered Storage feature. The returned `ledgers` in `./bin/pulsar-admin topics stats-internal` does not provide any information about whether a ledger has been deleted from the Bookie or not. This information can be useful for administrators and users to better understand the state of the ledgers and the overall storage usage. + +To address this, we propose adding a new field called `bookkeeperDeleted` to the `ManagedLedgerInternalStats.LedgerInfo` class. This field will indicate whether a ledger has been deleted from the Bookie, providing more insight into the state of the ledgers. + +# Goals + +## In Scope + +- Add a new field `bookkeeperDeleted` to the `ManagedLedgerInternalStats.LedgerInfo` class. +- While using Pulsar's Tiered Storage feature, we are able to use `./bin/pulsar-admin topics stats-internal` command to show whether the ledger is deleted in Bookie. + +## Out of Scope + +None. + +# High Level Design + +Before this, there is no way to know whether one ledger is still in bookie if the topic has been offloaded. + +We will add a new boolean field `bookkeeperDeleted`, which will indicate whether a ledger has been deleted from the Bookie. + + +# Detailed Design + +## Design & Implementation Details + +Add a new boolean field `bookkeeperDeleted` to the `ManagedLedgerInternalStats.LedgerInfo` class. + +## Public-facing Changes + +### Public API + +### Binary protocol + +### Configuration + +### CLI +The `./bin/pulsar-admin topics stats-internal` command will be updated to include the `bookkeeperDeleted` field in the returned `ledgers` array. + +### Metrics +None. + +# Monitoring +None. + +# Security Considerations +None. + +# Backward & Forward Compatibility + +## Revert + +## Upgrade + +# Alternatives + +# General Notes + +# Links + + +* Mailing List discussion thread: https://lists.apache.org/thread/9kjgg06oj384myxqls73dwn840thtrrc +* Mailing List voting thread: https://lists.apache.org/thread/5nfm3qkbwqylk1zp77zn9tfokhy212tb