Skip to content

fix: use Bearer for token credentials with Basic-only challenge#792

Closed
hrhee wants to merge 2 commits into
datalad:mainfrom
hrhee:fix/handle-401-token-bearer-fallback
Closed

fix: use Bearer for token credentials with Basic-only challenge#792
hrhee wants to merge 2 commits into
datalad:mainfrom
hrhee:fix/handle-401-token-bearer-fallback

Conversation

@hrhee

@hrhee hrhee commented May 15, 2026

Copy link
Copy Markdown

Problem

When a server responds to an unauthenticated request with
WWW-Authenticate: Basic realm="" and the stored credential is of type
token (Bearer), handle_401 crashes with IndexError: list index out of range.

The root cause is in the if ascheme is None block:

ascheme = cred.get('http_auth_scheme')   # set, but immediately overwritten
ascheme = [
    c for c in auth_schemes
    if c in DataladAuth._supported_auth_schemes
    and cred.get('type') == DataladAuth._supported_auth_schemes[c]
][0]   # IndexError when list is empty (Basic != token)

Two bugs:

  1. The http_auth_scheme value from the credential is overwritten
    unconditionally by the list comprehension below it.
  2. The list comprehension result is indexed with [0] without guarding
    against an empty list.

Error

$ datalad --dbg download --credential mytoken https://example.com/file

download(error): file [download failure] [list index out of range]
...
'exception': list index out of range
  [requests_auth.py:handle_401:290],

Fix

  • Split into two separate if ascheme is None checks so http_auth_scheme
    from the credential is honoured before the list comprehension runs.
  • Guard the list comprehension with if matching: instead of bare [0].
  • Add fallback: if the credential type is token and no matching scheme
    was found, try Bearer anyway.

Steps to reproduce

  1. Store a credential of type token for an HTTP endpoint:
    datalad credentials set mytoken type=token \
      realm=https://example.com \
      secret=<some-token>
    
  2. Download from a server that responds to unauthenticated requests with
    WWW-Authenticate: Basic realm="" but accepts Bearer tokens:
    datalad download --credential mytoken https://example.com/file
    
  3. IndexError: list index out of range in handle_401

This combination occurs with dCache WebDAV endpoints using OIDC
authentication: dCache advertises only Basic in its 401 challenge but
requires a Bearer token.

TODO

  • Add a unit test for the case where a type=token credential meets a Basic-only WWW-Authenticate header
    Tested with
    uv run --with pytest pytest datalad_next/utils/tests/test_requests_auth.py  datalad_next/utils/tests/test_parse_www_authenticate.py
    

Franz Rhee added 2 commits May 15, 2026 14:40
…server

When a server responds to an unauthenticated request with
'WWW-Authenticate: Basic realm=\"\"' and the stored credential is of type
'token' (Bearer), handle_401 crashes with IndexError: list index out of range.

Fix by:
- Splitting the ascheme-is-None block so http_auth_scheme from the
  credential is checked first and not immediately overwritten
- Guarding the list comprehension against an empty result
- Adding a fallback: if credential type is 'token' and no matching
  scheme was advertised, try Bearer"
@hrhee
hrhee requested a review from mih as a code owner May 15, 2026 13:11
@mih

mih commented May 17, 2026

Copy link
Copy Markdown
Member

Thanks! I cherry-picked these commits into the main branch. I am planning to release a new version soon.

@mih mih closed this May 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants