Add Node example validator - #253
Conversation
fce337b to
f5069c5
Compare
db2cc54 to
ea8c3c6
Compare
|
|
||
| // Retrieve statistics | ||
| const stats = await client.getStatistics(); | ||
| const stats = await client.getStatistics() as Record<string, any>; |
There was a problem hiding this comment.
Hmmm. Why is this cast necessary? Are we not properly specifying the return type for getStatistics? 🤔
There was a problem hiding this comment.
getStatistics return type is an Object so the example need to cast it to use the . notation.
| on: | ||
| pull_request: | ||
| branches: [main] | ||
| workflow_dispatch: |
There was a problem hiding this comment.
workflow_dispatch isn't included in the corresponding C# workflow. Should we add it for consistency?
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout valkey-glide-docs | ||
| uses: actions/checkout@v4 |
There was a problem hiding this comment.
One of the security findings for the C# repo was that we should use commit hashes rather than version numbers, e.g.
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
(As I understand it, a commit hash cannot be changed, but a malicious actor who was able to gain control of a repo could update the tag to point to a new commit and so download different code).
Are you able to raise an issue for this on the docs repo? Should be a very easy issue to fix with AI.
| - name: Checkout valkey-glide | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| repository: valkey-io/valkey-glide | ||
| path: valkey-glide |
There was a problem hiding this comment.
Would there be any advantages to doing a limited checkout of the few directories (e.g. node/, ffi/, and glide-core) that are actually needed for this job?
(The C# validator workflow doesn't checkout the valkey-glide submodule for same reason!)
There was a problem hiding this comment.
We could restrict it but I don't think it would net us much improvements.
The main bottleneck currently is during the Node client build phase. It has to build the Rust layer and Node, essentially building the full client. The C# checker only has to build the C# layer so it doesn't need valkey-glide submodule, and is a lot quicker.
| - name: Build Node client | ||
| working-directory: valkey-glide/node | ||
| run: | | ||
| npm ci | ||
| npm run build:release |
There was a problem hiding this comment.
In the C# example, we are able to skip building the Rust core because it actually isn't needed to validate the examples syntax. Are there any similar optimizations that we could perform here? I know that building the Rust core can be quite slow. What about installing all of the node modules?
There was a problem hiding this comment.
Yup this is currently the biggest bottleneck here. Updating this to match C# would require a code change on Node side although I haven't looked at the scope of this.
| def _extract_examples() -> dict[str, str]: | ||
| """Extract TypeScript/JS code blocks from all MDX files.""" | ||
| examples: dict[str, str] = {} |
There was a problem hiding this comment.
Some more documentation of the return type (similar to what exists in the current C# script) might be useful, I think.
There was a problem hiding this comment.
Moved to _common.py and updated docstring.
See 244a629
| _DEFAULT_IMPORTS = ( | ||
| "GlideClient, GlideClusterClient, GlideClientConfiguration, " | ||
| "GlideClusterClientConfiguration, Batch, ClusterBatch, BatchOptions, " | ||
| "ClusterBatchOptions, ClusterBatchRetryStrategy, Script, Transaction, " | ||
| "Routes, Logger, RequestError, ServerCredentials, GlideFt, " | ||
| "GlideJson, OpenTelemetry, ClientSideCache, EvictionPolicy, TimeUnit, " | ||
| "Field, FtSearchOptions, FtSearchReturnType, Decoder, " | ||
| "AdvancedGlideClusterClientConfiguration, CompressionBackend, " | ||
| "OpenTelemetryConfig, OpenTelemetryMetricsConfig, OpenTelemetryTracesConfig, " | ||
| "ClusterScanCursor, ReadFrom, GlideString, PubSubMsg, ALL_CHANNELS, ALL_PATTERNS" | ||
| ) |
There was a problem hiding this comment.
Why are these grouped into multi-entry strings like this?
There was a problem hiding this comment.
It is a formatted form from a single line import.
| line = lines[i] | ||
| if _IMPORT_LINE.match(line): | ||
| import_lines = [line] | ||
| while not re.search(r"""from\s+["'][^"']+["']\s*;?\s*$""", line) and i + 1 < len(lines): |
There was a problem hiding this comment.
Perhaps we should extract this regex? Otherwise, I think it might re-compile on each pass through the loop (not sure whether a typical Python interpreter would optimize this?)
| for imp in imports: | ||
| m = re.search(r"\{([^}]+)\}", imp) | ||
| if m: | ||
| for name in m.group(1).split(","): | ||
| existing_names.add(name.strip()) | ||
| defaults = [ | ||
| n.strip() for n in _DEFAULT_IMPORTS.split(",") | ||
| if n.strip() not in existing_names | ||
| ] | ||
| if defaults: | ||
| parts.append(f"import {{ {', '.join(defaults)} }} from \"@valkey/valkey-glide\";\n") |
There was a problem hiding this comment.
It's not completely clear to me how the logic in here works, or what the re.search call is doing. Maybe some comments would be useful?
There was a problem hiding this comment.
Refactored in 8a7a4fb.
We are trying to build the import statement, but have to take care and remove duplicated imports as it counts as errors for tsc.
| if source: | ||
| result[source] = msgs |
There was a problem hiding this comment.
Under what circumstances would this not be part of the dict?
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
ea8c3c6 to
7ac52e6
Compare
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
Signed-off-by: Alex Le <alex.le@improving.com>
Summary
This PR implement the Node example validator.
Issue link
#237
Content Changes
scripts/validators/node.pyto validator Node examples.validator/folder.Implementation
Follows the implementation per #237.