Currently micro-merging is done by each CMSSW process appending their output to the merged file in CMSSW end of lumi event. As events are not buffered by the process and instead written to disk during the lumisection, this requires reading data back from the disk and writing it back to the merged file, thus creating additional disk I/O.
This step could be skipped by delegating the merging to hltd esCopy function. Internally it opens a file for writing and appends contents of the single merged file from local disk. Instead the function could be modified to read contents of multiple files and merge to BU mount point on the fly (similar to option "B" in mini/macro merger)
Currently micro-merging is done by each CMSSW process appending their output to the merged file in CMSSW end of lumi event. As events are not buffered by the process and instead written to disk during the lumisection, this requires reading data back from the disk and writing it back to the merged file, thus creating additional disk I/O.
This step could be skipped by delegating the merging to hltd esCopy function. Internally it opens a file for writing and appends contents of the single merged file from local disk. Instead the function could be modified to read contents of multiple files and merge to BU mount point on the fly (similar to option "B" in mini/macro merger)