Skip to content

USE 373 - reduce TIM bulk update logging #381

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ghukill merged 2 commits into from
Mar 12, 2026
Merged

USE 373 - reduce TIM bulk update logging #381

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
531170d
Update dependencies, pickup TDA v4
ghukill Mar 11, 2026
4b0159b
Filter opensearchpy DEBUG API request logging
ghukill Mar 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions tim/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
import sentry_sdk

OPENSEARCH_BULK_CONFIG_DEFAULTS = {
"OPENSEARCH_BULK_MAX_CHUNK_BYTES": 100 * 1024 * 1024,
"OPENSEARCH_BULK_MAX_CHUNK_BYTES": 20 * 1024 * 1024,
"OPENSEARCH_BULK_MAX_RETRIES": 50,
"OPENSEARCH_REQUEST_TIMEOUT": 120,
}
Expand Down Expand Up @@ -41,7 +41,7 @@ def configure_logger(
"""Configure application via passed application root logger.

If verbose=True, 3rd party libraries can be quite chatty. For convenience, they can
be set to WARNING level by either passing a comma seperated list of logger names to
be set to WARNING level by either passing a comma separated list of logger names to
'warning_only_loggers' or by setting the env var WARNING_ONLY_LOGGERS.
"""
if verbose:
Expand All @@ -50,6 +50,14 @@ def configure_logger(
"%(asctime)s %(levelname)s %(name)s.%(funcName)s() "
"line %(lineno)d: %(message)s"
)

# Explicitly filter out opensearchpy's DEBUG logging of all API request payloads.
# This private method logs both the request + response payloads. If we need to
# know, we can both log data sent (request) or received (response).
logging.getLogger("opensearch").addFilter(
lambda record: record.funcName != "_log_request_response"
)

else:
root_logger.setLevel(logging.INFO)
logging_format = "%(asctime)s %(levelname)s %(name)s.%(funcName)s(): %(message)s"
Expand Down
40 changes: 20 additions & 20 deletions uv.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading