Issue Description:
Hello.
I have discovered a performance degradation in the read_csv function of pandas version 1.3.3. And I notice some parts of the repository depend on pandas 1.3.3 in dadmatools/requirements.txt and some other dependencies require pandas below 1.4. I am not sure whether this performance problem in pandas will affect this repository. I found some discussions on pandas GitHub related to this issue, including #44158 and #44610.
I also found that dadmatools/pipeline/informal2formal/utils.py and dadmatools/pipeline/informal2formal/VerbHandler.py used the influenced api. There may be more files using the influenced api.
Suggestion
I would recommend considering an upgrade to a different version of pandas >= 1.4 or exploring other solutions to optimize the performance of read_csv.
Any other workarounds or solutions would be greatly appreciated.
Thank you!
Issue Description:
Hello.
I have discovered a performance degradation in the
read_csvfunction of pandas version 1.3.3. And I notice some parts of the repository depend on pandas 1.3.3 indadmatools/requirements.txtand some other dependencies require pandas below 1.4. I am not sure whether this performance problem in pandas will affect this repository. I found some discussions on pandas GitHub related to this issue, including #44158 and #44610.I also found that
dadmatools/pipeline/informal2formal/utils.pyanddadmatools/pipeline/informal2formal/VerbHandler.pyused the influenced api. There may be more files using the influenced api.Suggestion
I would recommend considering an upgrade to a different version of pandas >= 1.4 or exploring other solutions to optimize the performance of
read_csv.Any other workarounds or solutions would be greatly appreciated.
Thank you!