Double-event creation

[wcpettus]

In rare cases, the filesystem appears to send two events for file creation.

• The file is scheduled twice, evident in the logs by INFO [Scheduler] ▶ Sending <{file}> to the classifier event timestamps are identical to the millisecond
• Most often this results in hornet completing the process for the first time before the second time is classified. The file is migrated to DIRAC intact, but a classifier error is then logged to slack:
  • CRIT [Classifier] ▶ [classifier] file <{file_path}/{file}> does not exist
• In rarer instances, both versions are classified, and two moves are attempted; the first move will succeed, the second move will fail, but will remove the completed file from warm and the data will be lost:
  • ERRO [Copy] ▶ File copy failed! ({file_path}/{file} -> {warm_path}/{file}) [open {file_path}/{file}: no such file or directory]

Lesson for future upgrade (in dragonfly migration):

• gracefully handle double-scheduling (probably still slack warning so we know it's happening)
• don't overwrite files and lose data