Skip to content

DPL Analysis: Dynamic extraction matchers#15402

Open
aalkin wants to merge 15 commits into
AliceO2Group:devfrom
aalkin:dynamic-extraction-matchers
Open

DPL Analysis: Dynamic extraction matchers#15402
aalkin wants to merge 15 commits into
AliceO2Group:devfrom
aalkin:dynamic-extraction-matchers

Conversation

@aalkin
Copy link
Copy Markdown
Member

@aalkin aalkin commented May 15, 2026

  • Rework workflow option handling (supersedes DPL: streamline option building #15320)
  • Add --aod-origin-replace <newOrigin> workflow option, that is used in adaptAnalysisTask to replace all of the default origins (i.e. AOD) with the new origin - both in inputs (including implicit) and outputs
  • Rework the input extraction, grouping and handling of Spawns/Defines/Builds to account for origin replacement
  • Track the origin replacement using InputSpec/OutputSpec metadata for correct handling by the reader
  • Update the grouping errors to use the actual matcher in the error message
  • Restrict Preslice to only accept tables and not iterators as its template argument (O2Physics PR [Common,PWGDQ,PWGJE,PWGLF] Preslice requires table type rather than iterator type O2Physics#16258)

Using the provided option, the common support tasks, like event selection and track propagation, can be duplicated in the workflow to process separate origin chains.

@alibuild
Copy link
Copy Markdown
Collaborator

alibuild commented May 15, 2026

Error while checking build/O2/fullCI_slc9 for 43ea930 at 2026-05-18 07:04:

## sw/BUILD/O2Physics-latest/log
ninja: build stopped: subcommand failed.


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
grep: error-log.txt: binary file matches
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/142da419b25e089969064e13a5e8c0db4ce34f8a/slc9_x86-64/o2checkcode/1.0-local170/etc/modulefiles
++ alibuild-generate-module
+ HAS_BIN=
+ HAS_LIB=
+ HAS_CMAKE=
+ HAS_ROOT=
+ '[' 0 '!=' 0 ']'
+ cat
+ printf 'if ![ is-loaded '\''BASE/1.0'\'' ] {\n module load BASE/1.0\n}'
+ echo 'BUILD_REQUIRES=CMake alibuild-recipe-tools defaults-release'
BUILD_REQUIRES=CMake alibuild-recipe-tools defaults-release
+ FULL_BUILD_REQUIRES='bz2 safe_int googlebenchmark MPFR defaults-release UUID json-c ninja-fortran CMake Alice-GRID-Utils FairCMakeModules pytorch_cpuinfo Python-modules-list O2-customization alibuild-recipe-tools double-conversion date ninja'
+ echo 'FULL_BUILD_REQUIRES=bz2 safe_int googlebenchmark MPFR defaults-release UUID json-c ninja-fortran CMake Alice-GRID-Utils FairCMakeModules pytorch_cpuinfo Python-modules-list O2-customization alibuild-recipe-tools double-conversion date ninja'
FULL_BUILD_REQUIRES=bz2 safe_int googlebenchmark MPFR defaults-release UUID json-c ninja-fortran CMake Alice-GRID-Utils FairCMakeModules pytorch_cpuinfo Python-modules-list O2-customization alibuild-recipe-tools double-conversion date ninja
++ env
++ cut -f1 -d=
++ grep -v '^DEFAULT_'
++ grep -v PKGREVISION
++ grep -v ALIBUILD_RECIPE_TOOLS
++ grep REVISION
++ sed -e s/_REVISION//
+ for x in $(env | cut -f1 -d= | grep -v "^DEFAULT_" | grep -v PKGREVISION | grep -v ALIBUILD_RECIPE_TOOLS | grep REVISION | sed -e 's/_REVISION//')
++ eval 'echo $LIBFFI_REVISION'
+++ echo 10
+ REVISION_VALUE=10
++ eval 'echo $LIBFFI_VERSION'
+++ echo v3.2.1-alice1
+ VERSION_VALUE=v3.2.1-alice1
++ eval 'echo $LIBFFI_ROOT'
+++ echo /sw/slc9_x86-64/libffi/v3.2.1-alice1-10
+ ROOT_PATH_VALUE=/sw/slc9_x86-64/libffi/v3.2.1-alice1-10
+ '[' -z 10 ']'
+ echo 'bz2 safe_int googlebenchmark MPFR defaults-release UUID json-c ninja-fortran CMake Alice-GRID-Utils FairCMakeModules pytorch_cpuinfo Python-modules-list O2-customization alibuild-recipe-tools double-conversion date ninja'
+ tr '[:lower:]' '[:upper:]'
+ tr - _
+ tr ' ' '\n'
[0 more errors; see full log]

Full log here.

@aalkin aalkin force-pushed the dynamic-extraction-matchers branch from 43ea930 to d2f616b Compare May 18, 2026 06:26
@aalkin aalkin force-pushed the dynamic-extraction-matchers branch from b264fad to f4b7582 Compare May 18, 2026 09:36
@alibuild
Copy link
Copy Markdown
Collaborator

Error while checking build/O2/fullCI_slc9 for f4b7582 at 2026-05-18 15:51:

## sw/BUILD/O2Physics-latest/log
c++: fatal error: Killed signal terminated program cc1plus
c++: fatal error: Killed signal terminated program cc1plus
ninja: build stopped: subcommand failed.


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep ' error:'
++ grep -v clang-diagnostic-error error-log.txt
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/c682c3ce163f3903974e1f158461ba96b31ab597/slc9_x86-64/o2checkcode/1.0-local172/etc/modulefiles
++ alibuild-generate-module
+ HAS_BIN=
+ HAS_LIB=
+ HAS_CMAKE=
+ HAS_ROOT=
+ '[' 0 '!=' 0 ']'
+ cat
+ printf 'if ![ is-loaded '\''BASE/1.0'\'' ] {\n module load BASE/1.0\n}'
+ echo 'BUILD_REQUIRES=CMake alibuild-recipe-tools defaults-release'
BUILD_REQUIRES=CMake alibuild-recipe-tools defaults-release
+ FULL_BUILD_REQUIRES='ninja defaults-release pytorch_cpuinfo ninja-fortran UUID safe_int json-c bz2 date Python-modules-list alibuild-recipe-tools googlebenchmark FairCMakeModules MPFR double-conversion CMake O2-customization Alice-GRID-Utils'
+ echo 'FULL_BUILD_REQUIRES=ninja defaults-release pytorch_cpuinfo ninja-fortran UUID safe_int json-c bz2 date Python-modules-list alibuild-recipe-tools googlebenchmark FairCMakeModules MPFR double-conversion CMake O2-customization Alice-GRID-Utils'
FULL_BUILD_REQUIRES=ninja defaults-release pytorch_cpuinfo ninja-fortran UUID safe_int json-c bz2 date Python-modules-list alibuild-recipe-tools googlebenchmark FairCMakeModules MPFR double-conversion CMake O2-customization Alice-GRID-Utils
++ env
++ cut -f1 -d=
++ grep -v '^DEFAULT_'
++ grep -v PKGREVISION
++ grep -v ALIBUILD_RECIPE_TOOLS
++ grep REVISION
++ sed -e s/_REVISION//
+ for x in $(env | cut -f1 -d= | grep -v "^DEFAULT_" | grep -v PKGREVISION | grep -v ALIBUILD_RECIPE_TOOLS | grep REVISION | sed -e 's/_REVISION//')
++ eval 'echo $LIBFFI_REVISION'
+++ echo 10
+ REVISION_VALUE=10
++ eval 'echo $LIBFFI_VERSION'
+++ echo v3.2.1-alice1
+ VERSION_VALUE=v3.2.1-alice1
++ eval 'echo $LIBFFI_ROOT'
+++ echo /sw/slc9_x86-64/libffi/v3.2.1-alice1-10
+ ROOT_PATH_VALUE=/sw/slc9_x86-64/libffi/v3.2.1-alice1-10
+ '[' -z 10 ']'
+ echo 'ninja defaults-release pytorch_cpuinfo ninja-fortran UUID safe_int json-c bz2 date Python-modules-list alibuild-recipe-tools googlebenchmark FairCMakeModules MPFR double-conversion CMake O2-customization Alice-GRID-Utils'
+ tr '[:lower:]' '[:upper:]'
+ tr - _
[0 more errors; see full log]

Full log here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants