Skip to content
This repository was archived by the owner on Apr 7, 2025. It is now read-only.
This repository was archived by the owner on Apr 7, 2025. It is now read-only.

error on /subset #8

Description

@brandomr

My guess is that this is because of an ESGF API error not being handled File "/opt/climate search/api/search/providers/esgf.py", line 314, in run_esgf_query not returning JSON

INFO:     10.244.19.33:36402 - "GET /subset/esgf?dataset_id=CMIP6.CMIP.NASA-GISS.GISS-E2-1-G.historical.r203i1p1f2.Amon.tasmax.gn.v20231004%7Cdpesgf03.nccs.nasa.gov&parent_id=46e2e26f-dedb-46ee-a1ad-973e0b6b192c&envelope=-22.724%2C0%2C-11.176%2C0 HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/fastapi/applications.py", line 1106, in __call__
    await super().__call__(scope, receive, send)
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/fastapi/routing.py", line 274, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/climate-search/api/server.py", line 61, in esgf_subset
    urls = esgf.get_all_access_paths_by_id(dataset_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/climate-search/api/search/providers/esgf.py", line 166, in get_all_access_paths_by_id
    for id in self.get_mirrors_for_dataset(dataset_id)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/climate-search/api/search/providers/esgf.py", line 172, in get_mirrors_for_dataset
    response = self.run_esgf_query(f"id:{dataset_id}*", 1, {})
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/climate-search/api/search/providers/esgf.py", line 314, in run_esgf_query
    response = r.json()
               ^^^^^^^^
  File "/home/svc/.cache/pypoetry/virtualenvs/climate-data-MFNGOi4f-py3.11/lib/python3.11/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions