Hierarchy/MDX Filter Support & Value Handling Modernization#445
Open
PPN-OL wants to merge 4 commits into
Open
Conversation
Hierarchy/MDX Filter Support & Value Handling Modernization
## What was changed?
This PR delivers two independent sets of changes across four Bedrock TI processes.
---
### 1. Hierarchy & MDX Filter Support
#### `}bedrock.cube.view.create`
- **Hierarchy syntax:** `pFilter` now accepts `DimName:HierName¦elem1+elem2`. The dimension token is split on `:` to extract dimension and hierarchy name separately. All subsequent operations (`HierarchySubsetCreate`, `HierarchySubsetDeleteAllElements`, `ViewSubsetAssign`) use the correct hierarchy.
- **MDX syntax:** `pFilter` now accepts `DimName¦MDX:{<expression>}`. The brace-depth counter (`nBraceDepth`) prevents all delimiter characters (`+`, `&`, `¦`) inside `{...}` from being interpreted as delimiters. The MDX subset is created by delegating to `}bedrock.hier.sub.create.bymdx`.
- **Element validation:** Replaced `DIMIX` with `HIERARCHYELEMENTINDEX` — works correctly for both default and non-default hierarchies.
- **Parsed filter output:** `sTargetFilter` preserves `Dim:Hier` notation for downstream consumers.
- **Validation:** Added checks for unmatched braces, invalid hierarchy names, and use of `:` as a delimiter character.
#### `}bedrock.cube.data.copy.intercube`
- **Filter parser extended:** The existing character-by-character parser now:
- Splits `DimName:HierName` tokens — extracts the bare dimension name for target-cube dimension matching while preserving the full token in the reconstructed target filter.
- Handles `MDX:{...}` blocks as atomic units — brace-depth counter prevents delimiters inside the block from being interpreted as filter structure.
- Validates element membership via `HIERARCHYELEMENTINDEX` (with `sHierarchy = sDimension` as fallback for the default hierarchy).
- Raises an error for unmatched braces after parsing.
- **`pFilterParallel` validation:** The conflict check for `pFilterParallel` vs. `pFilter` now also detects `Dim:Hier¦...` notation, preventing false negatives when the parallel dimension is specified with hierarchy syntax.
#### `}bedrock.cube.data.copy`
- **`pFilterParallel` validation:** Extended the conflict check to detect `Dim:Hier¦...` notation in `pFilter` (added `% Scan(Lower(sDimParallel) | ':', Lower(pFilter)) > 0`).
#### `}bedrock.cube.data.export`
- **`pFilterParallel` validation:** Identical fix as `data.copy` — one-line change at the same validation point.
#### `}bedrock.cube.data.clear`
- **`pFilterParallel` validation:** Identical fix as `data.copy` and `data.export` — one-line change extending the conflict check to detect `Dim:Hier¦...` notation in `pFilter`.
---
### 2. Value Handling Modernization (`data.copy` and `data.copy.intercube`)
#### `}bedrock.cube.data.copy` and `}bedrock.cube.data.copy.intercube` — Prolog
- **`CubeDimensionCountGet`** replaces the existing `TabDim`-based while-loop for counting cube dimensions. One API call instead of a loop.
- **Precomputed index strings:** `sDimCount` and `sDimCountP1` are computed once from `nDimensionCount` for use in `Expand`-based DataProcedure lookups.
- **`sLastDimName`:** The name of the cube's last dimension is computed once in the Prolog via `TabDim(pCube, nDimensionCount)` (or `TabDim(pTgtCube, ...)` for intercube). Constant across all data rows — avoids a repeated lookup per row.
#### `}bedrock.cube.data.copy` — DataProcedure
- **Single-pass value preparation:** A unified block before the `nDimensionCount` branch now handles all value and type logic once per data row:
- `sLastEle` is read via `Expand('%v' | sDimCount | '%')`.
- `sElType = DType(sLastDimName, sLastEle)` replaces the per-block `DType(sDimN, vN)` call.
- VIEW mode (`pFile = 0`): numeric value from `NValue * nFactor`; string value from `SValue`.
- CSV mode (`pFile > 0`): value from `Expand('%v' | sDimCountP1 | '%')` with `StringToNumberEx`.
- **Simplified dimension blocks (n = 3..27):** Each block now contains only a `CellIsUpdateable` check and a `CellPutN`/`CellPutS`/`CellIncrementN` call — all type detection and value preparation removed.
- **`CellIncrementN`** replaces `CellGetN + add + CellPutN` for `pCumulate = 1` — atomic operation, eliminates race conditions under parallel execution.
- **`nDimensionCount = 2` block:** Normal string/numeric cells modernized to use `CellIncrementN` and the pre-computed `vString`/`nCbal`. Special-case handling (`}ElementSecurity_*`, attribute cubes `AA`/`AS`/`AN`) is preserved unchanged.
#### `}bedrock.cube.data.copy.intercube` — DataProcedure
- Same single-pass value preparation pattern, adapted for `sV` variables: `sLastEle = Expand('%sV' | sDimCount | '%')`.
- All 26 dimension blocks (n = 2..27) simplified to `CellIsUpdateable` + `CellPutS(sVString)` / `CellIncrementN(nCbal)`.
- `nDimensionCount = 2` block: `ElementSecurityPut` and `AttrPut*` special cases updated to use pre-computed `sVString`/`nCbal`.
---
## Why?
### Hierarchy & MDX Filter Support
TM1/Planning Analytics supports non-default hierarchies and MDX set expressions in view definitions. Without these changes:
- A `pFilter` value of `Year:FiscalYear¦2024` would either fail silently or raise a spurious error because the parser did not understand the `:` separator.
- An MDX expression in `pFilter` would be corrupted by the character-by-character parser treating `+`, `&`, and `¦` inside the MDX block as structural delimiters.
- The `pFilterParallel` conflict check would produce a **false negative** when `pFilter` used `Dim:Hier` notation — the same dimension could be specified in both parameters without triggering a validation error, leading to silent double-filtering or incorrect parallel splits.
### Value Handling Modernization
- **Correctness:** Reading numeric values via `StringToNumberEx` in VIEW mode (`pFile = 0`) is unnecessary — the value is already available as a native number in `NValue`. Any mismatch between the server's actual decimal separator and `pDecimalSeparator` would cause silent wrong values.
- **Performance:** `DType(sDimN, vN)` and `StringToNumberEx(vN, ...)` were repeated in every one of the 27 dimension blocks, even though the result is identical for all blocks at the same data row. Moving this to a single pre-computation eliminates 26 redundant lookups per row.
- **Concurrency safety:** `CellGetN + add + CellPutN` is not atomic — under parallel execution with multiple threads writing to the same cell, intermediate reads can produce incorrect results. `CellIncrementN` is an atomic server-side operation.
---
## Affected Processes
| Process | Changes |
|---|---|
| `}bedrock.cube.view.create` | Hierarchy syntax, MDX syntax, `HIERARCHYELEMENTINDEX`, brace-depth parser, `sTargetFilter` output |
| `}bedrock.cube.data.copy.intercube` | Filter parser extension (Hierarchy + MDX), `pFilterParallel` validation fix, `NValue`/`CellIncrementN`/single-pass value prep |
| `}bedrock.cube.data.copy` | `pFilterParallel` validation fix, `NValue`/`CellIncrementN`/single-pass value prep |
| `}bedrock.cube.data.export` | `pFilterParallel` validation fix only |
| `}bedrock.cube.data.clear` | `pFilterParallel` validation fix only |
`}bedrock.cube.clone`
---
## Backward Compatibility
All changes are fully backward compatible.
| Concern | Status |
|---|---|
| Existing `pFilter` calls without hierarchy syntax | ✅ Unchanged — the parser falls back to `sHierarchy = sDimension` (default hierarchy) when no `:` is present |
| Existing `pFilter` calls without MDX | ✅ Unchanged — the brace-depth counter activates only when `{` is encountered |
| `pDecimalSeparator` / `pThousandSeparator` parameters | ✅ Still used in CSV mode (`pFile > 0`) — no parameters removed or renamed |
| `pCumulate` semantics | ✅ Identical — `CellIncrementN` produces the same result as the previous `CellGetN + add + CellPutN` sequence |
| `DType` result for `nDimensionCount = 2` special cases | ✅ `sElType` is still derived from `DType` — attribute cube detection (`AA`/`AS`/`AN`) and `}ElementSecurity_*` handling are unchanged |
| Calls using plain `DimName¦elem` syntax | ✅ Identical behavior — no syntax changes required for existing callers |
| `}bedrock.cube.data.clear` callers | ✅ No interface change — one-line internal validation fix only |
| `}bedrock.cube.clone` callers | ✅ No interface change — `clone` is not modified |
| Process parameter interfaces | ✅ No parameters added, removed, or renamed in any of the four processes |
…entries for the changed processes.
…entries for the changed processes.
Contributor
|
Nice work! In particular I do like the mdx keyword to provide an MDX. For completeness, would it be possible to add a subset keyword and provide a public subset within the hierarchy/dimension? That would be useful since without it, we either need additional Bedrock calls, or a SubsetGetMDX and then supply with the mdx keyword and maybe other implications. But for now, thanks a lot! |
Author
|
Hi. |
Contributor
|
That’s a good solution too. Support for MDX is the big improvement here and
yes it allows for public subsets. Thanks.
…------
Best regards / Beste groeten,
Wim Gielis
Current IBM Champion 2024-2026
Former MS Excel MVP 2011-2014
https://www.wimgielis.com <http://www.wimgielis.be>
Op ma 18 mei 2026 om 07:58 schreef PPN-OL ***@***.***>
*PPN-OL* left a comment (cubewise-code/bedrock#445)
<#445 (comment)>
Hi.
Thanks. I was thinking about adding subset in filter but came up with that
MDX will cover this as well by TM1SubsetToSet([Dimension].[Hierarchy],
"SubsetName")
—
Reply to this email directly, view it on GitHub
<#445 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEDHULKTOV42AHXRUEBZWFD43KQ7JAVCNFSM6AAAAACY7RUXN6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DINZUHAYDCNBWGM>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request: Bedrock v4.1 — Hierarchy/MDX Filter Support & Value Handling Modernization
What was changed?
This PR delivers two independent sets of changes across four Bedrock TI processes.
1. Hierarchy & MDX Filter Support
}bedrock.cube.view.createpFilternow acceptsDimName:HierName¦elem1+elem2. The dimension token is split on:to extract dimension and hierarchy name separately. All subsequent operations (HierarchySubsetCreate,HierarchySubsetDeleteAllElements,ViewSubsetAssign) use the correct hierarchy.pFilternow acceptsDimName¦MDX:{<expression>}. The brace-depth counter (nBraceDepth) prevents all delimiter characters (+,&,¦) inside{...}from being interpreted as delimiters. The MDX subset is created by delegating to}bedrock.hier.sub.create.bymdx.DIMIXwithHIERARCHYELEMENTINDEX— works correctly for both default and non-default hierarchies.sTargetFilterpreservesDim:Hiernotation for downstream consumers.:as a delimiter character.}bedrock.cube.data.copy.intercubeDimName:HierNametokens — extracts the bare dimension name for target-cube dimension matching while preserving the full token in the reconstructed target filter.MDX:{...}blocks as atomic units — brace-depth counter prevents delimiters inside the block from being interpreted as filter structure.HIERARCHYELEMENTINDEX(withsHierarchy = sDimensionas fallback for the default hierarchy).pFilterParallelvalidation: The conflict check forpFilterParallelvs.pFilternow also detectsDim:Hier¦...notation, preventing false negatives when the parallel dimension is specified with hierarchy syntax.}bedrock.cube.data.copypFilterParallelvalidation: Extended the conflict check to detectDim:Hier¦...notation inpFilter(added% Scan(Lower(sDimParallel) | ':', Lower(pFilter)) > 0).}bedrock.cube.data.exportpFilterParallelvalidation: Identical fix asdata.copy— one-line change at the same validation point.}bedrock.cube.data.clearpFilterParallelvalidation: Identical fix asdata.copyanddata.export— one-line change extending the conflict check to detectDim:Hier¦...notation inpFilter.2. Value Handling Modernization (
data.copyanddata.copy.intercube)}bedrock.cube.data.copyand}bedrock.cube.data.copy.intercube— PrologCubeDimensionCountGetreplaces the existingTabDim-based while-loop for counting cube dimensions. One API call instead of a loop.sDimCountandsDimCountP1are computed once fromnDimensionCountfor use inExpand-based DataProcedure lookups.sLastDimName: The name of the cube's last dimension is computed once in the Prolog viaTabDim(pCube, nDimensionCount)(orTabDim(pTgtCube, ...)for intercube). Constant across all data rows — avoids a repeated lookup per row.}bedrock.cube.data.copy— DataProcedurenDimensionCountbranch now handles all value and type logic once per data row:sLastEleis read viaExpand('%v' | sDimCount | '%').sElType = DType(sLastDimName, sLastEle)replaces the per-blockDType(sDimN, vN)call.pFile = 0): numeric value fromNValue * nFactor; string value fromSValue.pFile > 0): value fromExpand('%v' | sDimCountP1 | '%')withStringToNumberEx.CellIsUpdateablecheck and aCellPutN/CellPutS/CellIncrementNcall — all type detection and value preparation removed.CellIncrementNreplacesCellGetN + add + CellPutNforpCumulate = 1— atomic operation, eliminates race conditions under parallel execution.nDimensionCount = 2block: Normal string/numeric cells modernized to useCellIncrementNand the pre-computedvString/nCbal. Special-case handling (}ElementSecurity_*, attribute cubesAA/AS/AN) is preserved unchanged.}bedrock.cube.data.copy.intercube— DataProceduresVvariables:sLastEle = Expand('%sV' | sDimCount | '%').CellIsUpdateable+CellPutS(sVString)/CellIncrementN(nCbal).nDimensionCount = 2block:ElementSecurityPutandAttrPut*special cases updated to use pre-computedsVString/nCbal.Why?
Hierarchy & MDX Filter Support
TM1/Planning Analytics supports non-default hierarchies and MDX set expressions in view definitions. Without these changes:
pFiltervalue ofYear:FiscalYear¦2024would either fail silently or raise a spurious error because the parser did not understand the:separator.pFilterwould be corrupted by the character-by-character parser treating+,&, and¦inside the MDX block as structural delimiters.pFilterParallelconflict check would produce a false negative whenpFilterusedDim:Hiernotation — the same dimension could be specified in both parameters without triggering a validation error, leading to silent double-filtering or incorrect parallel splits.Value Handling Modernization
StringToNumberExin VIEW mode (pFile = 0) is unnecessary — the value is already available as a native number inNValue. Any mismatch between the server's actual decimal separator andpDecimalSeparatorwould cause silent wrong values.DType(sDimN, vN)andStringToNumberEx(vN, ...)were repeated in every one of the 27 dimension blocks, even though the result is identical for all blocks at the same data row. Moving this to a single pre-computation eliminates 26 redundant lookups per row.CellGetN + add + CellPutNis not atomic — under parallel execution with multiple threads writing to the same cell, intermediate reads can produce incorrect results.CellIncrementNis an atomic server-side operation.Affected Processes
}bedrock.cube.view.createHIERARCHYELEMENTINDEX, brace-depth parser,sTargetFilteroutput}bedrock.cube.data.copy.intercubepFilterParallelvalidation fix,NValue/CellIncrementN/single-pass value prep}bedrock.cube.data.copypFilterParallelvalidation fix,NValue/CellIncrementN/single-pass value prep}bedrock.cube.data.exportpFilterParallelvalidation fix only}bedrock.cube.data.clearpFilterParallelvalidation fix onlyBackward Compatibility
All changes are fully backward compatible.
pFiltercalls without hierarchy syntaxsHierarchy = sDimension(default hierarchy) when no:is presentpFiltercalls without MDX{is encounteredpDecimalSeparator/pThousandSeparatorparameterspFile > 0) — no parameters removed or renamedpCumulatesemanticsCellIncrementNproduces the same result as the previousCellGetN + add + CellPutNsequenceDTyperesult fornDimensionCount = 2special casessElTypeis still derived fromDType— attribute cube detection (AA/AS/AN) and}ElementSecurity_*handling are unchangedDimName¦elemsyntax}bedrock.cube.data.clearcallers}bedrock.cube.clonecallerscloneis not modifiedDeliverables
main_asJSON/Contains all Bedrock processes as ready-to-import JSON files. Each file is a self-contained, patchable process definition that can be uploaded directly to a TM1/Planning Analytics server via REST API or TM1py.
The
Attributesblock ("Caption") has been stripped from every file in this folder to prevent accidental overwrite of server-side captions during import.docs/wiki/Contains Markdown documentation for each process modified in this PR:
bedrock.cube.view.create.md}bedrock.cube.view.createbedrock.cube.data.copy.md}bedrock.cube.data.copybedrock.cube.data.copy.intercube.md}bedrock.cube.data.copy.intercubebedrock.cube.data.export.md}bedrock.cube.data.exportbedrock.cube.data.clear.md}bedrock.cube.data.clearprocess-dependencies.md}bedrock.*processes