Feature suggestion: model matrices for hierarchical aggregation

[jnlb]

This issue is to describe, discuss, and implement a feature that could replace this package's reliance on sdcTable tools. In place of sdcTable, reliance on flexible methods from the SSBtools package should be enough for this package's aggregation needs. To a certain extent, I will also be documenting under this issue. This should make it easier to update the relevant vignettes or docs when or if changes are eventually merged into the main branch.

What sdcTable is currently used for

Lines 55-62 of class.R:

private$prob_object <- sdcTable::makeProblem(
        data = dt,
        dimList = dimList,
        numVarInd = numVars
 )

 # Extract the table skeleton
private$data_summary <- as.data.table(sdcProb2df(private$prob_object, addDups = TRUE, addNumVars = TRUE, dimCodes = "original"))

The code above aggregates the table itself without perturbation as a part of initialization. It is a part of generating the structural mapping used throughout. The structural mapping is also used to identify the leaf nodes, which uses the S4 object sdcProblem and its internal attributes. See lines 69-71 of class.R:

min_info <- lapply(private$prob_object@dimInfo@dimInfo, function(x) {
        data.table(code = slot(x, "codesOriginal"), is_minimal = slot(x, "codesMinimal"))
 })

Finally, sdcTable::makeProblem is used a second time to aggregate the perturbed values in the perturb() function. See lines 158-161 of class.R:

tmp_prob <- makeProblem(data = as.data.frame(dt[strID %in% private$base_cell_ids]),
                              dimList = self$dimList, numVarInd = all_target_cols)

 full_res <- as.data.table(sdcProb2df(tmp_prob, addDups = TRUE, addNumVars = TRUE, dimCodes = "original"))

How model matrix methods can replace each of the above

The ModelMatrix function of SSBtools creates a sparse dummy matrix where each column represents a cell in the output table, and each row corresponds to a microdata record. (In case this becomes too efficient, there are also routines for pre-aggregating records according to the desired table structure, but more on that later. For the time being, let us assume that the model matrix can be left at the record-level granularity.)

The simplest possible function call would be:

mm <- ModelMatrix(dt, hierarchies = dimList, crossTable = TRUE)
# mm$modelMatrix is the sparse matrix (rows = microdata, cols = cells)
# mm$crossTable is the output table skeleton (dimension codes per cell)

Using mm, one could then easily aggregate by using a matrix multiplication z = t(x) %*% y, for a numeric vector y. Let us say dtcontains a numeric vector v. Then aggregation would simply be done by:

aggregated <- as.numeric(Matrix::crossprod(mm_result$modelMatrix, dt[[v]]))

Furthermore, I believe that the model matrix structure allows easy detection of base cells since it is simply a cell whose column in the model matrix has exactly one 1 per microdata record that belongs to it. It is probably also possible to detect base cells with SSBtools::AutoHierarchies, though. IMO, this is a miniscule amount of work either way.

Another point: the current code relies on row identifiers strID. This is not an issue, and can be replaced by the crossTable rows. SSBtools handles the equivalent functionality, it only needs to be ported over to work with this package.

How I intend to proceed

Everything I described above is simple to add or drop in as replacements for sdcTable parts. I will make a minimal branch that implements such a change as a kind of baseline. However, moving over to ModelMatrix could bring significant further benefits. Of these, I am most interested in:

• defining tables with the SSBtools formula and hierarchies interfaces
• possibility of more complex hierarchies
• storing the model matrix once and reusing
• from GaussSuppression: arbitrary user-coded sensitivity rules

The authoritative reference on hierarchical aggregation with model matrices is https://journal.r-project.org/articles/RJ-2023-088/

Furthermore, there is a poster about the formula interface (which is quite nice) on here: https://langsrud.com/stat/A0_poster_Barcelona_Oyvind_Langsrud_2025.pdf

I will return shortly, with more contributions, and also a branch with a more substantial rewrite than the so far discussed measures.