Update artifactory backend to use REST API

[hagenw]

Removes dohq-artifactory as dependency and uses the REST API of Artifactory instead.

It introduces two breaking changes:

• audbackend.backend.Artifactory.path() now returns a string (as the other backends) and no longer an artifaactory.ArtifactoryPath object
• The install option audbackend[artifactory] is no longer available

Besides making sure that we still have the same functionality (by the unit tests), we also compare execution time to see we do not regress with the new implementation:

Operation	size	workers	main-dohq	rest-api	Δ	%
ls	100 files	1	8.1720	0.0451	−8.127	−99.4%
ls	10 files	1	0.9218	0.0403	−0.882	−95.6%
remove_file	1MB	1	0.1186	0.0767	−0.0419	−35.3%
copy_file	1MB	1	0.1449	0.0994	−0.0455	−31.4%
move_file	1MB	1	0.1577	0.1160	−0.0417	−26.4%
put_file	1KB	1	0.2612	0.2240	−0.0372	−14.2%
put_file	1MB	1	0.3061	0.2678	−0.0383	−12.5%
put_archive	1MB zip	1	0.3758	0.3313	−0.0445	−11.8%
checksum	1KB	1	0.0389	0.0364	−0.0025	−6.4%
owner	1KB	1	0.0386	0.0362	−0.0024	−6.2%
exists	1KB	1	0.0381	0.0363	−0.0018	−4.7%
put_file	50MB	1	1.1703	1.1407	−0.0296	−2.5%
date	1KB	1	0.0382	0.0373	−0.0009	−2.4%
get_file	50MB	1	0.6323	0.6334	+0.0011	+0.2%
get_file	50MB	6	0.6304	0.6315	+0.0011	+0.2%
get_file	1KB	1	0.0762	0.0766	+0.0004	+0.5%
get_archive	1MB zip	1	0.0944	0.0954	+0.0010	+1.1%
get_file	1MB	6	0.1308	0.1335	+0.0027	+2.1%
get_file	1MB	1	0.1288	0.1323	+0.0035	+2.7%
get_file	1KB	6	0.0754	0.0775	+0.0021	+2.8%

[sourcery-ai]

Reviewer's Guide

Replaces the Artifactory backend’s dohq-artifactory dependency with a thin requests-based REST client while preserving the public Artifactory API, adjusts authentication/config loading and test cleanup, and adds a reusable benchmark suite plus recorded baseline results for validating performance of the new implementation.

Sequence diagram for REST-based file download with progress

sequenceDiagram
    actor Developer
    participant BenchmarkScript
    participant ArtifactoryBackend as Artifactory
    participant RestClient as ArtifactoryRestClient
    participant ArtifactoryServer

    Developer->>BenchmarkScript: run benchmark_artifactory_main()
    BenchmarkScript->>ArtifactoryBackend: get_file(src_path, dst_path, num_workers)
    activate ArtifactoryBackend
    ArtifactoryBackend->>ArtifactoryBackend: _get_file(src_path, dst_path, verbose)
    ArtifactoryBackend->>DownloadHelper: _download_with_progress(_client, src_path, dst_path, verbose)
    activate DownloadHelper
    DownloadHelper->>RestClient: stat(src_path)
    activate RestClient
    RestClient->>ArtifactoryServer: GET /api/storage/{repo}/{path}
    ArtifactoryServer-->>RestClient: 200 size, checksums, mtime, owner
    RestClient-->>DownloadHelper: metadata dict
    DownloadHelper->>DownloadHelper: create progress_bar(total=size)
    loop stream chunks
        DownloadHelper->>RestClient: download(src_path, dst_path, on_chunk=pbar.update)
        RestClient->>ArtifactoryServer: GET /{repo}/{path} (stream)
        ArtifactoryServer-->>RestClient: chunk bytes
        RestClient-->>DownloadHelper: on_chunk(len)
        DownloadHelper->>DownloadHelper: pbar.update(len)
    end
    deactivate RestClient
    deactivate DownloadHelper
    ArtifactoryBackend-->>BenchmarkScript: return
    deactivate ArtifactoryBackend

Loading

Class diagram for Artifactory backend using REST client

classDiagram
    class Artifactory {
        - str host
        - str repository
        - tuple authentication
        - ArtifactoryRestClient _client
        - requests_Session _session
        + Artifactory(str host, str repository, tuple authentication)
        + get_authentication(str host) tuple
        + path(str path) str
        - _open()
        - _close()
        - _checksum(str path) str
        - _date(str path) str
        - _exists(str path) bool
        - _size(str path) int
        - _ls(str path) list~str~
        - _get_file(str src_path, str dst_path, bool verbose)
        - _get_file_stream(str src_path) Iterator~bytes~
        - _put_file(str src_path, str dst_path, str checksum, bool verbose)
        - _copy_file(str src_path, str dst_path, bool verbose)
        - _move_file(str src_path, str dst_path, bool verbose)
        - _remove_file(str path)
        - _create()
        - _delete()
        - _owner(str path) str
    }

    class ArtifactoryRestClient {
        - str host
        - str repository
        - requests_Session session
        + ArtifactoryRestClient(str host, str repository, requests_Session session)
        + stat(str path) dict
        + exists(str path) bool
        + download(str path, str dst_path, int chunk_size, on_chunk)
        + stream(str path, int chunk_size) Iterator~bytes~
        + upload(str src_path, str dst_path, str md5)
        + delete(str path)
        + copy(str src_path, str dst_path)
        + move(str src_path, str dst_path)
        + list_files(str sub_path) list~str~
        + repository_exists() bool
        + create_repository(str package_type)
        + delete_repository()
        - _file_url(str path) str
        - _storage_url(str path) str
        - _repo_url() str
        - _copy_or_move(str action, str src_path, str dst_path)
    }

    class DownloadHelpers {
        + _download_with_progress(ArtifactoryRestClient client, str src_path, str dst_path, bool verbose)
    }

    class ConfigHelpers {
        + _find_config_entry(configparser_ConfigParser config, str host) dict
    }

    Artifactory --> ArtifactoryRestClient : uses
    DownloadHelpers --> ArtifactoryRestClient : calls
    Artifactory ..> DownloadHelpers : uses
    Artifactory ..> ConfigHelpers : uses

Loading

File-Level Changes

Change	Details	Files
Replace dohq-artifactory usage in the Artifactory backend with a custom REST client built on requests while keeping the public Artifactory interface stable.	Introduce ArtifactoryRestClient to wrap the subset of Artifactory REST endpoints needed for metadata, file transfer, listing, and repository admin Refactor Artifactory backend methods (open/close, path, checksum, exists, date, owner, size, ls, copy/move/delete, get/put file/archive) to delegate to ArtifactoryRestClient instead of ArtifactoryPath/RepositoryLocal Rework internal download/upload helpers to operate on URLs via the REST client, including streaming download with a progress bar callback Simplify path handling to operate on URL strings rather than ArtifactoryPath objects, returning full URLs from path()	audbackend/core/backend/_artifactory_rest.py audbackend/core/backend/artifactory.py
Reimplement Artifactory authentication config resolution without artifactory helpers, aligning it with a local config file and environment variables.	Add DEFAULT_CONFIG_PATH and use configparser to read ~/.artifactory_python.cfg (or ARTIFACTORY_CONFIG_FILE) directly Add _find_config_entry to resolve host entries with or without scheme and trailing slashes, mirroring previous behavior Update get_authentication to use _find_config_entry and fall back to anonymous user when no credentials are found	audbackend/core/backend/artifactory.py
Adjust repository lifecycle management and test cleanup to use the new REST-based backend API instead of direct dohq-artifactory types.	Change Artifactory._create and _delete to use ArtifactoryRestClient.repository_exists/create_repository/delete_repository instead of RepositoryLocal Update tests’ interface fixture to call backend_cls.delete(host, repository) for cleanup and rely on backend delete exceptions rather than dohq-artifactory-specific exceptions Rewrite tests/misc/cleanup_artifactory.py to delete leftover repositories via raw requests.delete against /api/repositories/{repo}	audbackend/core/backend/artifactory.py tests/conftest.py tests/misc/cleanup_artifactory.py
Promote Artifactory backend to a non-optional dependency using requests and remove dohq-artifactory from all dependency sets.	Add requests to core project dependencies Remove artifactory extra and dohq-artifactory entries from optional dependencies and dev dependencies Import Artifactory unconditionally in audbackend.init and audbackend.backend.init instead of wrapping imports in try/except	pyproject.toml audbackend/__init__.py audbackend/backend/__init__.py
Add a benchmark suite for the Artifactory backend and check in baseline results for the dohq and REST implementations.	Introduce benchmark_artifactory.py which uses only the public Artifactory backend API to benchmark put/get, metadata operations, ls, copy/move/remove, and archive operations over configurable sizes and worker counts Add benchmark/README.md documenting how to run benchmarks, compare branches, interpret results, and criteria for accepting the REST migration Check in markdown and CSV benchmark result files for the main-dohq and rest-api implementations to document performance characteristics	benchmark/benchmark_artifactory.py benchmark/README.md benchmark/results/main-dohq.md benchmark/results/rest-api.md benchmark/results/main-dohq.csv benchmark/results/rest-api.csv

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.0%. Comparing base (be13a49) to head (59bc97a).

Additional details and impacted files

Files with missing lines	Coverage Δ
audbackend/__init__.py	100.0% <100.0%> (ø)
audbackend/backend/__init__.py	100.0% <100.0%> (ø)
audbackend/core/backend/_artifactory_rest.py	100.0% <100.0%> (ø)
audbackend/core/backend/artifactory.py	100.0% <100.0%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.