diff --git a/dir.yaml b/dir.yaml index 4782887e..c7a11310 100644 --- a/dir.yaml +++ b/dir.yaml @@ -117,6 +117,9 @@ - title_en: Query Parallelism Control title_cn: 查询并行度控制 path: development-guide/query-optimization/hints/parallel-degree + - title_en: Hybrid Cache Control + title_cn: 混合缓存控制 + path: development-guide/query-optimization/hints/hybrid-cache - title_en: Index Hints title_cn: Index Hints path: development-guide/query-optimization/hints/index diff --git a/en_US/development-guide/query-optimization/hints/hybrid-cache.md b/en_US/development-guide/query-optimization/hints/hybrid-cache.md new file mode 100644 index 00000000..bc8085b0 --- /dev/null +++ b/en_US/development-guide/query-optimization/hints/hybrid-cache.md @@ -0,0 +1,52 @@ +--- +title: "Datalayers Hybrid Cache Hint Guide" +description: "Control object store cache behavior per query using the hybrid_cache hint." +--- +# Hybrid Cache Hint Guide + +## Overview + +Datalayers uses a Hybrid Cache (memory + disk cache) to cache Parquet file data and object metadata from object stores, accelerating queries. However, in some scenarios, bypassing the cache is desirable. The `hybrid_cache` hint temporarily disables the hybrid cache for an individual query. + +## Syntax + +```sql +-- Disable cache for the current query +SELECT /*+ SET_VAR(hybrid_cache=off) */ * FROM t; + +-- Explicitly enable cache (default behavior, no need to set). Note: Hybrid Cache must be configured in the config file for it to actually take effect +SELECT /*+ SET_VAR(hybrid_cache=on) */ * FROM t; +``` + +Supported values (case insensitive): + +| Value | Meaning | +|-------|---------| +| `on` or `1` | Enable cache (default) | +| `off` or `0` | Disable cache | + +Can be combined with other hints: + +```sql +SELECT /*+ SET_VAR(parallel_degree=4), SET_VAR(hybrid_cache=off) */ * FROM t; +``` + +## Use Cases + +- **Benchmarking after cache warm-up**: compare cached vs. non-cached query performance +- **Debugging cache issues**: disable cache to verify data consistency when suspecting stale cache entries +- **Large scan queries**: when a single query would evict hot data from the cache, temporarily disable it + +## Verification + +Use `EXPLAIN` or `EXPLAIN ANALYZE` to inspect the physical plan output for the hybrid cache state: + +```sql +-- If Hybrid Cache is configured, the PartitionScanExec operator shows hybrid_cache=on +EXPLAIN SELECT * FROM t; + +-- If Hybrid Cache is not configured, or disabled by hint, the PartitionScanExec operator shows hybrid_cache=off +EXPLAIN SELECT /*+ SET_VAR(hybrid_cache=off) */ * FROM t; +``` + +> **Note**: If Hybrid Cache is not configured at startup (`object_store.file_cache` and `object_store.metadata_cache` are both empty), the output always shows `hybrid_cache=off` regardless of the hint value. diff --git a/zh_CN/development-guide/query-optimization/hints/hybrid-cache.md b/zh_CN/development-guide/query-optimization/hints/hybrid-cache.md new file mode 100644 index 00000000..3562df7d --- /dev/null +++ b/zh_CN/development-guide/query-optimization/hints/hybrid-cache.md @@ -0,0 +1,52 @@ +--- +title: "Datalayers Hybrid Cache Hint 使用指南" +description: "介绍如何使用 hybrid_cache Hint 按查询控制是否使用混合缓存。" +--- +# Hybrid Cache Hint 使用指南 + +## 概述 + +Datalayers 在配置文件中打开混合缓存后,在查询时默认使用 Hybrid Cache(内存加磁盘的混合缓存)来加速查询。在某些场景下,绕过缓存会更合适(如对非热点数据进行一次性的分析查询)。`hybrid_cache` Hint 可用于控制单个查询时选择是否使用混合缓存。 + +## 语法 + +```sql +-- 禁用缓存(当前查询不走 Hybrid Cache) +SELECT /*+ SET_VAR(hybrid_cache=off) */ * FROM t; + +-- 显式启用缓存(默认行为) +-- 注意:只有在配置文件中启用了 Hybrid Cache,该设置才会生效 +SELECT /*+ SET_VAR(hybrid_cache=on) */ * FROM t; +``` + +支持的值(大小写不敏感): + +| 值 | 含义 | +| :----------- | :--------------- | +| `on` 或 `1` | 启用缓存(默认) | +| `off` 或 `0` | 禁用缓存 | + +可与其他 Hint 组合使用: + +```sql +SELECT /*+ SET_VAR(parallel_degree=4), SET_VAR(hybrid_cache=off) */ * FROM t; +``` + +## 适用场景 + +- **缓存预热后的基准测试**:对比缓存命中与绕过缓存的性能差异 +- **大范围扫描查询**:单条查询扫描的数据量远超缓存容量、可能频繁淘汰热点数据时,可临时禁用缓存 + +## 如何验证 + +使用 `EXPLAIN` 或 `EXPLAIN ANALYZE` 查看物理执行计划,确认 Hybrid Cache 的状态: + +```sql +-- 如果已配置 Hybrid Cache,PartitionScanExec 算子会显示 hybrid_cache=on +EXPLAIN SELECT * FROM t; + +-- 如果未配置 Hybrid Cache,或通过 hint 禁用了缓存,PartitionScanExec 算子会显示 hybrid_cache=off +EXPLAIN SELECT /*+ SET_VAR(hybrid_cache=off) */ * FROM t; +``` + +> **注意**:如果在启动 Datalayers 时未配置 Hybrid Cache(`object_store.file_cache` 和 `object_store.metadata_cache` 均为空),那么无论 Hint 如何设置,输出都会显示 `hybrid_cache=off`。 diff --git a/zh_CN/development-guide/query-optimization/hints/overview.md b/zh_CN/development-guide/query-optimization/hints/overview.md index ba077295..7dd6754d 100644 --- a/zh_CN/development-guide/query-optimization/hints/overview.md +++ b/zh_CN/development-guide/query-optimization/hints/overview.md @@ -10,10 +10,12 @@ SQL Hints 是嵌入在 SQL 中的特殊注释指令,用于向优化器提供 目前 Datalayers 支持以下 SQL Hints: -- 查询并行度(parallel_degree)控制 +- 查询并行度(parallel_degree)控制 +- 混合缓存控制(hybrid_cache) - Index Hints ## 相关文档 - 了解并行度控制,请参考 [查询并行度优化指南](./parallel-degree.md) -- 了解索引提示,请参考 [Index Hints 使用指南](./index.md) +- 了解混合缓存控制,请参考 [混合缓存控制使用指南](./hybrid-cache.md) +- 了解 Index Hints,请参考 [Index Hints 使用指南](./index.md)