Adding support for per-observation loss weighting (instead of subsampling)

[novaktim]

Summary

This pull request is submitted in response to the exchange between Philipp Doebler and Marvin N. Wright. The code itself was developed as part of my bachelor thesis (Tim Novak, 2023) supervised by Philipp Doebler and Susanne Frick. Rather than subsampling observations to achieve weighting effects, users can now supply a loss.weights vector that is applied in the split‐rule and node‐impurity calculations of regression/probability trees.

Changes

• New loss.weights argument in ranger()
• Data classes extended: all Data* types gain a w field with get_w/set_w to hold per‑obs weights.
• Forests now have bool use_loss_weights automatically derived and passed to trees for algorithm extensions.
• For compatibility, not supplying loss.weights will provide loss.weights = (1, 1, 1, ...).
• Weighted splits & estimates: TreeRegression and TreeProbability use data->get_w(...) in impurity, split search and terminal estimation.
• Test added: "test_loss_weights.R" verifies regression and classification behavior under varying weights.

Next Steps

• So far, only the defaults Regression with Variance and Classification with Gini splitrules implemented with a custom splitting function "findBestSplitValueLossWeights"
• So far, no implementation for Classification with probability = FALSE
• So far, no implementation for save.memory = TRUE

Compatibility & Testing

Built on top of past commit SHA 38560f2 (March 31, 2023)
All existing tests including the additional one passed on Windows 10

Risks

• Forward compatibility with the current main branch version
• Backward compatibility via the unit‐weight vector that overrides some calculus code lines
• Correct derivation and implementation of weighted estimators

[novaktim]

Hello Mr. Wright and team,

thank you for taking an interest in my work. I’ve noticed there are some merge conflicts on the up-to-date master branch—I’d be also happy to resolve those once you’ve agreed on the overall approach.
I look forward to your feedback on the feature itself. Please let me know if I can help with any next steps!

Best regards,
Tim Novak

[mayer79]

This would be a great feature to have!

[mnwright]

Thanks, this looks great and we should definitely have it! A few thoughts:

• The name "loss.weights" might confuse people? In ranger, we usually use splitrule, impurity, etc. and not loss; maybe that's confusing for some people? Also we now have class weights, case weights and loss weights and even without the new weights many people were confused. Maybe give them all new names to confuse people even more? :)
• I see the point of adding the weights to the Data* classes. However, the other weights etc. are saved in the Forest object. I think I would have put them there as well for consistency.
• Not so sure about adding new split functions. We already have too many of them with quite some code redundancy. I'm thinking about a re-design of the splitting functions with less redundancy, including missing value handling, randomized splitting and weights.
• Could we merge it with class weights? Shouldn't they do the same thing but for all observations with a given class?
• We should have it for all splitrules and outcomes types.

[mayer79]

@mnwright : What about replacing case.weights by loss weights? (Keeping the name case.weights, but using the new logic.) In XGB/LGB, weighting is done via loss weights.

[mnwright]

I think we should keep the bootstrap weights. Also replacing is not a good idea for consistency over versions.