cgatpipelines.tasks.preprocess still uses semi-colons to seperate out its commands. This has not been suitable for some time, as when commands are seperated by ;, if one errors, then:
- The subsequent commands will run anyway
- I far as I can tell, only the stderr of the last command in the chain returned.
We used to have checkpoint; to run between commands for this reason, but that has been retired, long ago, in favour of seperating commands by && instead of ; checkpoint;.
Except, it seems in cgatpipeline.tasks.preprocess.
I started changing over the statements to use ' && '.join () rather than ';'.join, but it seems that there are three lines in the MasterProcessor class that actively check for ; at the end of statement, which fails when things are joined by &&. Any idea what the purpose of this is?
https://github.com/cgat-developers/cgat-flow/blob/7ae2e893a41f952c07f35b5cebb4c3c408d8477b/cgatpipelines/tasks/preprocess.py#L290C9-L292C47
Any reason I shouldn't just remove them?
cgatpipelines.tasks.preprocessstill uses semi-colons to seperate out its commands. This has not been suitable for some time, as when commands are seperated by;, if one errors, then:We used to have
checkpoint;to run between commands for this reason, but that has been retired, long ago, in favour of seperating commands by&&instead of; checkpoint;.Except, it seems in
cgatpipeline.tasks.preprocess.I started changing over the statements to use
' && '.join ()rather than';'.join, but it seems that there are three lines in theMasterProcessorclass that actively check for;at the end of statement, which fails when things are joined by&&. Any idea what the purpose of this is?https://github.com/cgat-developers/cgat-flow/blob/7ae2e893a41f952c07f35b5cebb4c3c408d8477b/cgatpipelines/tasks/preprocess.py#L290C9-L292C47
Any reason I shouldn't just remove them?