Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions python/quadrants/lang/ast/ast_transformer_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,11 @@ def __call__(self, ctx: "ASTTransformerFuncContext", node: ast.AST):
if method is None:
error_msg = f'Unsupported node "{node.__class__.__name__}"'
raise QuadrantsSyntaxError(error_msg)
info = ctx.get_pos_info(node) if isinstance(node, (ast.stmt, ast.expr)) else ""
# Attach only the cheap file/line/function header to every IR node. Building the full source-line
# hint (get_pos_info) here means running TextWrapper for every AST node of every kernel compilation,
# which dominates kernel build time. The full hint is still produced on the actual compile-error path
# below (get_pos_info in the except handler), so error messages are unchanged.
info = ctx.get_pos_header(node) if isinstance(node, (ast.stmt, ast.expr)) else ""
with impl.get_runtime().src_info_guard(info):
res = method(ctx, node)
if not hasattr(node, "violates_pure"):
Expand Down Expand Up @@ -387,8 +391,11 @@ def get_var_by_name(self, name: str) -> tuple[bool, Any, str | None]:
except AttributeError:
raise QuadrantsNameError(f'Name "{name}" is not defined')

def get_pos_header(self, node: ast.AST) -> str:
return f'File "{self.file}", line {node.lineno + self.lineno_offset}, in {self.func.func.__name__}:\n'

def get_pos_info(self, node: ast.AST) -> str:
msg = f'File "{self.file}", line {node.lineno + self.lineno_offset}, in {self.func.func.__name__}:\n'
msg = self.get_pos_header(node)
col_offset = self.indent + node.col_offset
end_col_offset = self.indent + node.end_col_offset

Expand Down
11 changes: 7 additions & 4 deletions python/quadrants/lang/kernel_impl.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import inspect
import linecache
import re
import sys
import typing
Expand Down Expand Up @@ -141,10 +141,13 @@ def pyfunc(fn: Callable) -> QuadrantsCallable:
def _inside_class(level_of_class_stackframe: int) -> bool:
try:
maybe_class_frame = sys._getframe(level_of_class_stackframe)
statement_list = inspect.getframeinfo(maybe_class_frame)[3]
if statement_list is None:
# Read the decoration-site source line via linecache rather than inspect.getframeinfo: getframeinfo
# resolves the frame's module through inspect.getmodule, an O(len(sys.modules)) scan run once per kernel
# creation. With thousands of modules loaded this dominates kernel build time; linecache.getline returns
# the same source line with no such scan.
first_statment = linecache.getline(maybe_class_frame.f_code.co_filename, maybe_class_frame.f_lineno).strip()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve loader-backed source lookup

When user kernels are defined in modules loaded from zipimport/zipapps/PEX-style archives, co_filename is a virtual path like /app.pyz/pkg/mod.py, and linecache.getline(filename, lineno) returns an empty string unless module globals/loader state are supplied. The previous inspect.getframeinfo path routed through inspect.findsource()/linecache.getlines(file, module.__dict__), so the loader could provide the source; with this change _inside_class returns False for @qd.kernel/@qd.func methods in those packaged modules, causing @qd.data_oriented not to wrap class kernels and self to be treated as a normal kernel argument.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has this been addressed? If so, how?

I have mixed feelings about this codex comment. On the one hand, we dont use any zipimport style archives. On the other hand, being silently wrong seems not great. We should either explicitly forbid this and throw an exception, or somehow ensure this is correct (eg falling back on the old method), I feel.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Refresh linecache before reading decorator lines

When a long-lived process redefines kernels from a filename that is already in linecache after the file has changed (for example, importlib.reload()/hot-reload after inserting lines above a @qd.data_oriented class), this can read the stale cached line. The old inspect.getframeinfo() path invalidated the cache via linecache.checkcache() before returning context, but linecache.getline() does not, so _inside_class() can match an old class/decorator line or miss the new one and misclassify kernels, breaking method binding through data_oriented. Refresh this filename before getline() or otherwise avoid stale cache reads.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, this seems like not something we do typically, but silently being incorrect seems not great?

if not first_statment:
return False
first_statment = statement_list[0].strip()
for pat in _KERNEL_CLASS_STACKFRAME_STMT_RES:
if pat.match(first_statment):
return True
Expand Down
Loading