Skip to content

refactor: route SDK and CLI requests through shared HTTP clients - #733

Open
KAJdev wants to merge 2 commits into
mainfrom
zeke/standardize-http-and-auth
Open

KAJdev wants to merge 2 commits into
mainfrom
zeke/standardize-http-and-auth

Conversation

@KAJdev

@KAJdev KAJdev commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Adding an SDK or CLI request involves several URL builders, HTTP session wrappers, and credential locations. Shared route templates and scoped clients give these requests consistent parameter escaping, service configuration, and response cleanup. Related REST requests can reuse one connection pool:

from runpod.api.rest import AsyncRESTClient
from runpod.api.routes import Route

async with AsyncRESTClient() as client:
    for pod_id in pod_ids:
        await client.request(Route("GET", "/v2/pods/{pod_id}", pod_id=pod_id))

Moved out auth stuff into a more top level auth module. and now both rp and runpod use the same CLI package

Comment thread tests/test_auth/test_agent_login.py Resolved
Comment thread tests/test_auth/test_package.py Resolved
Comment thread tests/test_auth/test_package.py Resolved
Comment thread runpod/api/rest.py Resolved
Comment thread runpod/api/rest.py Resolved
Comment thread runpod/api/routes.py Resolved
Comment thread tests/test_endpoint/test_transport.py Resolved

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

REST and Apps clients still treat unfollowed 3xx redirects as success, allowing deletions or uploads to be reported complete without reaching their destination.

1 open issue
What changed in this PR

This PR centralizes HTTP routing and connection management across the SDK and CLI, separates authentication from Apps, and consolidates both CLI entry points under runpod.cli.

Changes:

  • Adds escaped route templates and scoped HTTP/REST clients with connection reuse and response cleanup.
  • Migrates endpoint, Apps, GraphQL, and authentication requests to shared infrastructure.
  • Updates CLI imports, compatibility aliases, tests, and contributor guidance.
File Description
tests/​test_shared/​test_auth.py Updates credential mock location.
tests/​test_profile.py Uses shared credential storage.
tests/​test_network_volume_transfer.py Updates CLI and authentication imports.
tests/​test_endpoint/​test_transport.py Adds routing and response-lifetime coverage.
tests/​test_endpoint/​test_failed_jobs.py Adapts async response mocks.
tests/​test_endpoint/​test_asyncio_runner.py Adapts mocks to shared requests.
tests/​test_cli/​test_volume_command.py Updates volume CLI imports and patches.
tests/​test_cli/​test_update_command.py Uses consolidated update command.
tests/​test_cli/​test_stream_encoding.py Updates CLI subprocess imports.
tests/​test_cli/​test_project_commands.py Updates project-command imports.
tests/​test_cli/​test_pod_command.py Updates CLI and transport mocks.
tests/​test_cli/​test_management_commands.py Updates management and login mocks.
tests/​test_cli/​test_init_command.py Uses consolidated CLI entry point.
tests/​test_cli/​test_endpoint_command.py Updates endpoint CLI imports.
tests/​test_cli/​test_deploy_command.py Uses consolidated deploy command.
tests/​test_cli/​test_console.py Uses relocated console module.
tests/​test_cli/​test_compute.py Uses relocated compute module.
tests/​test_cli/​test_cli_output.py Updates output helper imports.
tests/​test_cli/​test_cli_options.py Updates option helper imports.
tests/​test_cli/​test_cli_logs.py Uses relocated log commands.
tests/​test_cli/​test_cli_groups/​test_ssh_dispatch.py Uses consolidated SSH entry point.
tests/​test_cli/​test_cli_api.py Uses relocated CLI API helpers.
tests/​test_cli/​test_catalog_command.py Updates catalog imports and mocks.
tests/​test_cli/​test_agent_login.py Updates shared login and credential references.
tests/​test_auth/​test_package.py Tests authentication package boundaries and compatibility.
tests/​test_auth/​test_login.py Uses relocated browser login.
tests/​test_auth/​test_credentials.py Targets shared credential storage.
tests/​test_auth/​test_api.py Adds dedicated authentication client tests.
tests/​test_auth/​test_agent_login.py Adapts agent-login transport tests.
tests/​test_auth/​__init__.py Defines authentication test package.
tests/​test_apps/​test_user_agent.py Updates transport attribution tests.
tests/​test_apps/​test_transport_parity.py Mocks shared HTTP clients.
tests/​test_apps/​test_tasks.py Adapts task transport mocks.
tests/​test_apps/​test_targets.py Tests shared load-balancer URL helper.
tests/​test_apps/​test_retry.py Tests session reuse across retries.
tests/​test_apps/​test_logs.py Adapts log streaming mocks.
tests/​test_apps/​test_http_transport.py Adds Apps routing and credential-boundary coverage.
tests/​test_apps/​test_http_errors.py Uses host API environment configuration.
tests/​test_apps/​test_entrypoint_execution.py Updates embedded CLI import.
tests/​test_apps/​test_app.py Updates catalog transport mocks.
tests/​test_apps/​test_api_client.py Moves authentication coverage to its own suite.
tests/​test_api/​test_rate_limit.py Updates session-level request mock.
tests/​test_api/​test_http.py Tests routes, ownership, pooling, and credentials.
setup.py Points both commands to shared CLI.
runpod/​rp_cli/​__init__.py Removes old package docstring.
runpod/​network_volume_transfer.py Loads credentials from authentication package.
runpod/​http_client.py Updates credential import.
runpod/​endpoint/​runner.py Uses shared transport and adds pool cleanup.
runpod/​endpoint/​asyncio/​asyncio_runner.py Uses shared routes with borrowed sessions.
runpod/​cli/​volume.py Updates CLI and credential dependencies.
runpod/​cli/​update.py Houses CLI update functionality.
runpod/​cli/​pod.py Uses consolidated CLI helpers.
runpod/​cli/​output.py Uses relocated console renderer.
runpod/​cli/​options.py Uses consolidated API and compute helpers.
runpod/​cli/​main.py Integrates shared CLI and authentication modules.
runpod/​cli/​logs.py Updates shared CLI dependencies.
runpod/​cli/​groups/​config/​__init__.py Aliases legacy credential imports.
runpod/​cli/​endpoint.py Uses consolidated endpoint CLI helpers.
runpod/​cli/​endpoint_config.py Updates configuration helper imports.
runpod/​cli/​console.py Houses shared CLI rendering.
runpod/​cli/​compute.py Houses catalog resolution helpers.
runpod/​cli/​catalog.py Uses consolidated catalog dependencies.
runpod/​cli/​api.py Updates compute error import.
runpod/​auth/​login.py Separates login flows and adopts shared transport.
runpod/​auth/​credentials.py Updates credential module documentation.
runpod/​auth/​api.py Adds dedicated authentication API client.
runpod/​auth/​__init__.py Centralizes API-key selection.
runpod/​apps/​utils/​network.py Uses shared HTTP and endpoint helpers.
runpod/​apps/​tasks.py Migrates task requests to scoped clients.
runpod/​apps/​targets.py Shares routing and retry connection pools.
runpod/​apps/​logs.py Uses shared routes and configurable log service.
runpod/​apps/​api.py Separates authentication and reuses upload sessions.
runpod/​api/​routes.py Adds escaped routes and service URL helpers.
runpod/​api/​rest.py Adds reusable REST clients with quota handling.
runpod/​api/​queries/​apps.py Removes relocated authentication query.
runpod/​api/​mutations/​apps.py Removes relocated authentication mutation.
runpod/​api/​http.py Adds shared synchronous and asynchronous transports.
runpod/​api/​graphql.py Uses shared transports and key resolution.
runpod/​_hosts.py Centralizes additional service configuration.
runpod/​__init__.py Routes credential exports through authentication package.
pyproject.toml Updates both CLI entry points.
CONTRIBUTING.md Documents transport and package conventions.

Review effort: Balanced

Comment thread runpod/api/http.py
self.session._retry_connection = False
if not isinstance(timeout, aiohttp.ClientTimeout):
timeout = aiohttp.ClientTimeout(total=timeout)
kwargs.setdefault("allow_redirects", False)
Comment thread runpod/api/routes.py
parameters[name] = (
encoded.replace(".", "%2E") if encoded in {".", ".."} else encoded
)
return self.path.format_map(parameters)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolved_path calls format_map even when the Route has no parameters, and run_rest_request, RESTClient, QueueClient.http and Endpoint._request all wrap raw paths in Route(method, path). A path containing { now raises KeyError: run_rest_request("GET", "/v2/pods/a{b}"), rp pod get 'a{b}', or Route("GET", '/search?q={"a":1}'). api_errors in cli/api.py does not catch KeyError, so the CLI prints a traceback where it used to give a clean 404. Could you skip format_map when there are no parameters (or escape braces first) and add a test for braces in a raw path?

Comment thread runpod/api/http.py
self.session._retry_connection = False
if not isinstance(timeout, aiohttp.ClientTimeout):
timeout = aiohttp.ClientTimeout(total=timeout)
kwargs.setdefault("allow_redirects", False)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stream() sets allow_redirects=False, but the 3xx check lives only in HTTPClient.request(). RESTClient.request and AsyncRESTClient.request override request() without that check, and stream() callers (artifact upload via Route("PUT", "") + raise_for_status, targets._request_json, get_json, tasks.poll_result) never see it. A 302 with an empty body makes run_rest_request return None (callers then fail with TypeError), the upload is treated as success because raise_for_status only fires at 400+, and the JSON paths raise a decode error instead of AppHTTPError. The old requests.post GraphQL path followed redirects, so this is also a behavior change. Could the 3xx check move into stream() or a shared response-check helper, with tests for a 3xx on REST and on the upload?

Comment thread runpod/api/http.py
) as response:
yield response
return
self.session._retry_connection = False

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This runs on every stream() call, and __aenter__ does the same at line 36, including when the caller passed their own aiohttp session (the asyncio runner wraps the caller's session at asyncio_runner.py 50, 131, 183, 204, 219). That permanently changes private state on a session we don't own. As I read the aiohttp source, the flag turns off the automatic retry of idempotent requests on a stale keep-alive connection, so a long-lived status()/output() poll could now fail with ServerDisconnectedError where it used to be retried. I have not reproduced this against a live server. Is disabling retry intentional for non-idempotent POSTs only? If so, could it be set only on sessions the client owns and only for non-idempotent methods?

Comment thread runpod/_hosts.py
import runpod

module_base = (runpod.endpoint_url_base or "").strip()
return _env(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

endpoint_base_url() gives RUNPOD_ENDPOINT_BASE_URL priority over a value assigned in code to runpod.endpoint_url_base. Before this PR, RunPodClient and the asyncio runner read runpod.endpoint_url_base directly. A user who sets runpod.endpoint_url_base = "https://custom" in a shell that also has the env var set now silently hits the env var's URL. Is that precedence intended? If so, a note in the docs/release notes would help; otherwise an explicit assignment in code should probably win.

Comment thread runpod/auth/login.py


class LoginError(AppError):
class LoginError(AuthenticationError):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runpod.rp_cli.* and runpod.apps.auth shipped in 2.3.0 and are deleted with no compatibility alias, while runpod.cli.groups.config.functions does get a shim. from runpod.apps.auth import ... and from runpod.rp_cli.main import run now raise ImportError. Also, LoginError now subclasses AuthenticationError instead of AppError, so existing except AppError: blocks stop catching login failures. Could you add sys.modules aliases like the existing one (or keep AppError as a second base of LoginError) and call the change out in the release notes?

)


def test_service_bases_respect_configuration(monkeypatch):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sets runpod.endpoint_url_base but does not clear RUNPOD_ENDPOINT_BASE_URL or RUNPOD_PROXY_DOMAIN, and I don't see a conftest that isolates them, so it fails on any machine or CI job with those set. monkeypatch.delenv(..., raising=False) for both would fix it. While you are in this file, there are no tests yet for an invalid RUNPOD_API_BASE_URL (now a ValueError out of Route.url, uncaught by api_errors) or the RuntimeError on a closed borrowed session.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants