Stable hash()

NEWS.md

@@ -1,7 +1,10 @@
 # rlang (development version)
 
-* Fixed `env_get()` issue causing double evaluation of active bindings on older R versions <= 4.4 (#1893).
+* `hash()` now uses its own walking strategy to make it independent of pecularities of the R serialiser. This fixes stability issues with function bytecode and shrinkable vectors on R 4.6.0 (#1681).
+
+  This does mean that with this version all hash values will now be different. In general we gain stability across versions of R, but may lose some stability across versions of rlang. We'll try and avoid changes to our hash algorithm for the sake of stability but you should assume it's always possible for a new version to invalidate existing hashes.
 
+* Fixed `env_get()` issue causing double evaluation of active bindings on older R versions <= 4.4 (#1893).
 
 # rlang 1.2.0
 

R/hash.R

@@ -5,20 +5,37 @@
 #'
 #' - `hash_file()` hashes the data contained in a file.
 #'
-#' The generated hash is guaranteed to be reproducible across platforms that
-#' have the same endianness and are using the same R version.
+#' For ordinary data objects (vectors, lists, data frames, etc.), the generated
+#' hash is reproducible across sessions and R versions on platforms that have
+#' the same endianness. However, hash values may change between rlang versions,
+#' although that should be rare. Reference-like objects (environments, external
+#' pointers, builtins) are hashed by identity, so their hashes are only stable
+#' within a session. Closures hash their formals, body, and environment
+#' identity. Byte-compiled and uncompiled closures hash identically
+#' because the body is always hashed from the original language tree.
+#'
+#' By default, source references are stripped before hashing so that
+#' closures and calls that are textually identical produce the same
+#' hash regardless of where they were parsed. Set `zap_srcref` to
+#' `FALSE` to include source references in the hash.
 #'
 #' @details
 #' These hashers use the XXH128 hash algorithm of the xxHash library, which
 #' generates a 128-bit hash. Both are implemented as streaming hashes, which
 #' generate the hash with minimal extra memory usage.
 #'
-#' For `hash()`, objects are converted to binary using R's native serialization
-#' tools. Serialization version 3 is used. See [serialize()] for more
-#' information about the serialization version.
+#' For `hash()`, a custom object walker feeds the object's type, length, data
+#' bytes, and attributes directly into the hash algorithm. This avoids
+#' dependency on R's serialization format, making the hash immune to internal
+#' representation details (e.g. ALTREP compact forms, the growable vector bit).
 #'
 #' @param x An object.
 #'
+#' @param zap_srcref Whether to ignore source references when hashing
+#'   (default `TRUE`). Source references depend on parse location, so
+#'   including them makes hashes of closures and calls
+#'   non-reproducible across sessions.
+#'
 #' @param path A character vector of paths to the files to be hashed.
 #'
 #' @return
@@ -39,8 +56,8 @@
 #' # If you need a single hash for multiple files,
 #' # hash the result of `hash_file()`
 #' hash(hashes)
-hash <- function(x) {
-  .Call(ffi_hash, x)
+hash <- function(x, zap_srcref = TRUE) {
+  .Call(ffi_hash, x, zap_srcref)
 }
 
 # Keep this alias for a while

src/internal/decl/hash-decl.h

@@ -1 +1,11 @@
+static void hash_object(struct hash_ctx* ctx, r_obj* x);
+static r_obj* hash_attribs_cb(r_obj* tag, r_obj* value, void* data);
+static void hash_feed_vector_data(
+    struct hash_ctx* ctx,
+    r_obj* x,
+    int type,
+    r_ssize n
+);
+static r_obj* hash_impl(void* p_data);
+static void hash_cleanup(void* p_data);
 static r_obj* hash_file_impl(void* p_data);