CI/CD in this repo can be used to ensure that the many download scripts successfully reach the expected data.
The best way to achieve this may be an architectural change in the downloader, as suggested in #15:
Downloaders should be split into their respective websites, and centralized functions should be used as the base call for that site. For e.g., each GEO downloader will use a centralized download_geo script which will perform the initial generic steps of downloading from GEO. This is important because if a site changes its download schema we will (hopefully) only need to change a single script. It's also useful for continuously checking if the download structure matches our expectations for a test case (CI/CD).
CI/CD can be used for more future applications of this repo. E.g., we could add code for API calls to common bioinformatic websites. Tests can be added to the CI/CD to ensure each API hasn't changed its expected structure.
CI/CD in this repo can be used to ensure that the many download scripts successfully reach the expected data.
The best way to achieve this may be an architectural change in the downloader, as suggested in #15:
CI/CD can be used for more future applications of this repo. E.g., we could add code for API calls to common bioinformatic websites. Tests can be added to the CI/CD to ensure each API hasn't changed its expected structure.