Skip to content

catch exceptions from urllib and handling gracefully #266

Description

@eformo

When using a remote_url with a selenium grid or node there are times where the server is down and a connection cannot be established. Can we add a catch in pylenium somewhere to catch and log the exception instead of throwing out massive urllib stack traces. When looking at the order of operations, this might have to be done in WDM library, not pylenium.

Currently in this situation pylenium and wdm are set to retry 3 times, it looks like there is a teardown that is attempted 6 times.

The captured setup and teardown logs show the following WARNING: (EDIT: pasted wrong message)
WARNING 14:00:23 - connectionpool.py:urlopen:810 - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x11e95c4f0>: Failed to establish a new connection: [Errno 61] Connection refused')': /wd/hub/session

The root of the error comes from:

venv/lib/python3.9/site-packages/urllib3/util/connection.py:95:
if err is not None:
    raise err

This error that is raised is:

venv/lib/python3.9/site-packages/urllib3/util/connection.py:85: ConnectionRefusedError
>               sock.connect(sa)
E               ConnectionRefusedError: [Errno 61] Connection refused

here is the traceback:

fixtures/intelligence/app/ui/intelligence_ui.py:21: in login_steps
    frontend.py.visit(environment.url)
venv/lib/python3.9/site-packages/pylenium/driver.py:263: in visit
    self.webdriver.get(url)
venv/lib/python3.9/site-packages/pylenium/driver.py:231: in webdriver
    return self.init_webdriver() if self._webdriver is None else self._webdriver
venv/lib/python3.9/site-packages/pylenium/driver.py:201: in init_webdriver
    self._webdriver = webdriver_factory.build_from_config(self.config)
venv/lib/python3.9/site-packages/pylenium/webdriver_factory.py:139: in build_from_config
    return build_remote(browser, remote_url, **_config)
venv/lib/python3.9/site-packages/pylenium/webdriver_factory.py:376: in build_remote
    return webdriver.Remote(
venv/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py:275: in __init__
    self.start_session(capabilities, browser_profile)
venv/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py:365: in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
venv/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py:428: in execute
    response = self.command_executor.execute(driver_command, params)
venv/lib/python3.9/site-packages/selenium/webdriver/remote/remote_connection.py:347: in execute
    return self._request(command_info[0], url, body=data)
venv/lib/python3.9/site-packages/selenium/webdriver/remote/remote_connection.py:369: in _request
    response = self._conn.request(method, url, body=body, headers=headers)
venv/lib/python3.9/site-packages/urllib3/request.py:78: in request
    return self.request_encode_body(
venv/lib/python3.9/site-packages/urllib3/request.py:170: in request_encode_body
    return self.urlopen(method, url, **extra_kw)
venv/lib/python3.9/site-packages/urllib3/poolmanager.py:376: in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
venv/lib/python3.9/site-packages/urllib3/connectionpool.py:813: in urlopen
    return self.urlopen(
venv/lib/python3.9/site-packages/urllib3/connectionpool.py:813: in urlopen
    return self.urlopen(
venv/lib/python3.9/site-packages/urllib3/connectionpool.py:813: in urlopen
    return self.urlopen(
venv/lib/python3.9/site-packages/urllib3/connectionpool.py:785: in urlopen
    retries = retries.increment(

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions