-
Notifications
You must be signed in to change notification settings - Fork 25
Speedscope export #49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mjambon
wants to merge
28
commits into
LexiFi:master
Choose a base branch
from
mjambon:speedscope-export
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
239b3a4
Add Speedscope export format
mjambon 53f3869
Address PR review comments
mjambon 3340d0b
Copy TS spec comments into ATD file as <doc text="..."> annotations
mjambon 97a3a3d
Wrap speedscope_fmt.atd at 80 columns
mjambon 0c14944
Fix imported doc comments
mjambon 4274e1c
Update changelog; regenerate from revised ATD file
mjambon 892f860
Add intro comments
mjambon 6650d86
Move Speedscope export to new landmarks-exports lib + support custom …
mjambon 85142f1
Fix double-init crash; add example to test suite
mjambon 25a29e0
Don't call init twice
mjambon 7513a4d
Write example output to a file
mjambon cd9d984
Add comment
mjambon f475fad
Remove 'flags' option that is now inherited from the root 'dune' file
mjambon 99106f0
Update changelog
mjambon 001c059
Don't fail on unknown exporters
mjambon ce07b8a
Update readme
mjambon 32842d2
Formatting
mjambon c4dd372
Improve warning
mjambon 8043cce
Typo
mjambon 7fa87d6
Use -linkall flag to register this library's modules
mjambon 64a6a34
Rename lib landmarks-exports -> landmarks_speedscope
mjambon d76ed5d
Rename package landmarks-exporters -> landmarks_speedscope
mjambon 8c3f33c
Undo minor changes
mjambon 8fd37b8
Use Graph.dfs instead of own implementation
mjambon fb48c4f
Avoid warning 44 by not using 'let open'
mjambon adea397
Add missing line break
mjambon c1ea837
Settle on "landmarks-speedscope" for the name of the new package and …
mjambon e22e736
Add 'make speedscope-demo', remove stale tests/speedscope/profile.json
mjambon File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,3 +2,6 @@ _build | |
| .merlin | ||
| *.install | ||
| admin/website | ||
|
|
||
| # Speedscope demo | ||
| /profile.speedscope | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| .PHONY: build | ||
| build: | ||
| dune build | ||
|
|
||
| .PHONY: test | ||
| test: | ||
| dune test | ||
|
|
||
| # Speedscope demo and sanity check | ||
| .PHONY: speedscope-demo | ||
| speedscope-demo: | ||
| dune build tests/speedscope/example.exe | ||
| OCAML_LANDMARKS="format=speedscope,output=profile.speedscope,time" \ | ||
| ./_build/default/tests/speedscope/example.exe | ||
| @echo "👉 Upload profile.speedscope at https://www.speedscope.app/" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| (env | ||
| ; '_' targets any profile | ||
| (_ | ||
| (flags (:standard -w +A-30-42-41-40-4-70 -safe-string -strict-sequence)))) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| # This file is generated by dune, edit dune-project instead | ||
| opam-version: "2.0" | ||
| version: "1.6" | ||
| synopsis: "Additional export formats for the Landmarks profiling library" | ||
| description: | ||
| "This provides export to the Speedscope format and possibly more formats in the future." | ||
| maintainer: ["Marc Lasson <marc.lasson@lexifi.com>"] | ||
| authors: ["Marc Lasson <marc.lasson@lexifi.com>"] | ||
| license: "MIT" | ||
| homepage: "https://github.com/LexiFi/landmarks" | ||
| bug-reports: "https://github.com/LexiFi/landmarks/issues" | ||
| depends: [ | ||
| "dune" {>= "3.16"} | ||
| "landmarks" {= version} | ||
| "yojson" {>= "3.0.0"} | ||
| "odoc" {with-doc} | ||
| ] | ||
| build: [ | ||
| ["dune" "subst"] {dev} | ||
| [ | ||
| "dune" | ||
| "build" | ||
| "-p" | ||
| name | ||
| "-j" | ||
| jobs | ||
| "@install" | ||
| "@runtest" {with-test} | ||
| "@doc" {with-doc} | ||
| ] | ||
| ] | ||
| dev-repo: "git+https://github.com/LexiFi/landmarks.git" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| ; A library for exporting profiling data to various formats | ||
| (library | ||
| (name landmarks_speedscope) | ||
| (public_name landmarks-speedscope) | ||
| (libraries | ||
| landmarks | ||
| yojson | ||
| ) | ||
| ; Force linking and evaluation of this library's modules | ||
| (library_flags -linkall) | ||
| ) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| (* | ||
| Export to the Speedscope format | ||
| *) | ||
|
|
||
| open Landmark | ||
|
|
||
| let schema_url = "https://www.speedscope.app/file-format-schema.json" | ||
| let exporter_name = "landmarks" | ||
|
|
||
| let parse_location loc = | ||
| match String.rindex_opt loc ':' with | ||
| | None -> loc, None | ||
| | Some i -> | ||
| let file = String.sub loc 0 i in | ||
| let rest = String.sub loc (i + 1) (String.length loc - i - 1) in | ||
| (match int_of_string_opt rest with | ||
| | Some n -> file, Some n | ||
| | None -> loc, None) | ||
|
|
||
| (* One Speedscope frame per unique landmark (by landmark_id), skipping Root. *) | ||
| let make_frames (graph : Graph.graph) = | ||
| let tbl = Hashtbl.create 16 in | ||
| let frames = ref [] in | ||
| let next_idx = ref 0 in | ||
| Array.iter (fun (node : Graph.node) -> | ||
| if node.kind <> Graph.Root && not (Hashtbl.mem tbl node.landmark_id) then begin | ||
| let file, line = parse_location node.location in | ||
| let frame = Speedscope_fmt.create_frame ~name:node.name ~file ?line () in | ||
| Hashtbl.add tbl node.landmark_id !next_idx; | ||
| frames := frame :: !frames; | ||
| incr next_idx | ||
| end | ||
| ) graph.nodes; | ||
| List.rev !frames, tbl | ||
|
|
||
| (* DFS producing one sample per call-graph node with positive self-time. | ||
| Each sample is a stack of frame indices from outermost to innermost | ||
| caller (Speedscope's "bottom to top" convention). | ||
| Counter and Sampler nodes are skipped. *) | ||
| let collect_samples ~use_sys_time (graph : Graph.graph) frame_idx = | ||
| let samples = ref [] in | ||
| let weights = ref [] in | ||
| let node_time (n : Graph.node) = if use_sys_time then n.sys_time else n.time in | ||
| Graph.dfs | ||
| (fun ancestors (node : Graph.node) -> | ||
| match node.kind with | ||
| | Root -> true | ||
| | Counter | Sampler -> false | ||
| | Normal -> | ||
| let fidx = Hashtbl.find frame_idx node.landmark_id in | ||
| let child_list = Graph.children graph node in | ||
| let child_time = | ||
| List.fold_left (fun acc c -> acc +. node_time c) 0.0 child_list | ||
| in | ||
| let self_time = node_time node -. child_time in | ||
| if self_time > 0.0 then begin | ||
| let stack = | ||
| fidx :: | ||
| List.filter_map | ||
| (fun (a : Graph.node) -> | ||
| match a.kind with | ||
| | Normal -> Some (Hashtbl.find frame_idx a.landmark_id) | ||
| | Root | Counter | Sampler -> None) | ||
| ancestors | ||
| in | ||
| samples := List.rev stack :: !samples; | ||
| weights := self_time :: !weights | ||
| end; | ||
| true) | ||
| (fun _ _ -> ()) | ||
| graph; | ||
| List.rev !samples, List.rev !weights | ||
|
|
||
| let exporter oc (graph : Graph.graph) = | ||
| let frames, frame_idx = make_frames graph in | ||
| let use_sys_time = | ||
| Array.exists (fun (n : Graph.node) -> n.sys_time > 0.0) graph.nodes | ||
| in | ||
| let samples, weights = collect_samples ~use_sys_time graph frame_idx in | ||
| let end_value = List.fold_left ( +. ) 0.0 weights in | ||
| let weight_unit = | ||
| if use_sys_time then Speedscope_fmt.Seconds else Speedscope_fmt.None_ | ||
| in | ||
| let profile = Speedscope_fmt.create_sampled_profile | ||
| ~type_:"sampled" | ||
| ~name:graph.label | ||
| ~unit:weight_unit | ||
| ~start_value:0.0 | ||
| ~end_value | ||
| ~samples | ||
| ~weights | ||
| () | ||
| in | ||
| let shared = Speedscope_fmt.create_profile_shared ~frames () in | ||
| let file = Speedscope_fmt.create_file_format | ||
| ~schema:schema_url | ||
| ?name:(if graph.label = "" then None else Some graph.label) | ||
| ~exporter:exporter_name | ||
| ~profiles:[profile] | ||
| ~shared | ||
| () | ||
| in | ||
| Yojson.Safe.pretty_to_channel ~std:true oc | ||
| (Speedscope_fmt.yojson_of_file_format file); | ||
| output_char oc '\n' | ||
|
|
||
| (* This relies on the [-linkall] flag passed with [-a] when building | ||
| the library to ensure the registration takes place. *) | ||
| let () = Landmark.register_exporter "speedscope" exporter |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| (** Export to the Speedscope format | ||
|
|
||
| See https://www.speedscope.app for using the visualization app | ||
| and https://github.com/jlfwong/speedscope/blob/main/src/lib/file-format-spec.ts | ||
| for the annotated format specification. | ||
| *) | ||
|
|
||
| val exporter : out_channel -> Landmark.Graph.graph -> unit | ||
| (** Write a Speedscope sampled profile to [out_channel]. | ||
|
|
||
| If [sys_time] was collected during profiling, weights are in seconds; | ||
| otherwise raw CPU-cycle counts are used with unit "none". | ||
|
|
||
| The resulting JSON can be opened at | ||
| {{: https://www.speedscope.app } speedscope.app}. | ||
|
|
||
| This exporter is automatically registered with the Landmarks library | ||
| to provide support for [format=speedscope]. | ||
| *) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| <doc text=" | ||
| Speedscope file-format types. | ||
|
|
||
| Schema: https://www.speedscope.app/file-format-schema.json | ||
| Spec (TS): https://github.com/jlfwong/speedscope/blob/main/src/lib/file-format-spec.ts | ||
| Import docs: https://github.com/jlfwong/speedscope/wiki/Importing-from-custom-sources | ||
|
|
||
| To regenerate speedscope_fmt.ml and speedscope_fmt.mli from this file: | ||
| {{{ | ||
| atdml speedscope_fmt.atd | ||
| }}} | ||
| "> | ||
|
|
||
| type value_unit | ||
| <doc text="Unit in which all profile values are expressed."> = [ | ||
| | Bytes <json name="bytes"> | ||
| | Microseconds <json name="microseconds"> | ||
| | Milliseconds <json name="milliseconds"> | ||
| | Nanoseconds <json name="nanoseconds"> | ||
| | None_ <json name="none"> | ||
| | Seconds <json name="seconds"> | ||
| ] | ||
|
|
||
| type frame = { | ||
| name : string; | ||
| ?file : string option; | ||
| ?line : int option; | ||
| ?col : int option; | ||
| } | ||
|
|
||
| (* We only export sampled profiles; the Speedscope format also supports | ||
| evented profiles. The 'type' field is the discriminator used by | ||
| Speedscope for the profile union and must always be "sampled". *) | ||
| type sampled_profile = { | ||
| type_ | ||
| <json name="type"> | ||
| <doc text="Type of profile. | ||
| Used as a discriminator in the profile union to future-proof | ||
| the file format. For sampled profiles, always 'sampled'."> | ||
| : string; | ||
|
|
||
| name | ||
| <doc text="Name of the profile. | ||
| Typically a filename for the source of the profile."> | ||
| : string; | ||
|
|
||
| unit | ||
| <json name="unit"> | ||
| <doc text="Unit in which all values in this profile are expressed."> | ||
| : value_unit; | ||
|
|
||
| start_value | ||
| <json name="startValue"> | ||
| <doc text="The starting value of the profile. | ||
| Typically a timestamp. All event values are displayed | ||
| relative to startValue."> | ||
| : float; | ||
|
|
||
| end_value | ||
| <json name="endValue"> | ||
| <doc text="The final value of the profile. | ||
| Must be >= startValue. Useful when the recorded profile | ||
| extends past the last event."> | ||
| : float; | ||
|
|
||
| samples | ||
| <doc text="List of stacks. | ||
| Each stack is a list of indices into the shared frames array."> | ||
| : int list list; | ||
|
|
||
| weights | ||
| <doc text="Weight of the sample at the corresponding index. | ||
| Must have the same length as samples."> | ||
| : float list; | ||
| } | ||
|
|
||
| (* The "shared" section of a Speedscope file. | ||
| "shared" is a reserved word in ATD, hence the name profile_shared here; | ||
| the JSON key is "shared" via the annotation on the file_format field below. *) | ||
| type profile_shared | ||
| <doc text="Data shared between profiles."> | ||
| = { | ||
| frames : frame list; | ||
| } | ||
|
|
||
| (* "$schema" uses a JSON name annotation because "$" is not a valid | ||
| OCaml identifier character. *) | ||
| type file_format = { | ||
| schema | ||
| <json name="$schema"> | ||
| : string; | ||
|
|
||
| ?name | ||
| <doc text="The name of the contained profile group. | ||
| If omitted, the viewer uses the filename."> | ||
| : string option; | ||
|
|
||
| ?exporter | ||
| <doc text="The name of the program that exported this profile. | ||
| Not consumed by speedscope, but useful for debugging. | ||
| Recommended format: {{name@version}}."> | ||
| : string option; | ||
|
|
||
| ?active_profile_index | ||
| <json name="activeProfileIndex"> | ||
| <doc text="Index into the profiles array to display on load. | ||
| Defaults to the first profile if omitted."> | ||
| : int option; | ||
|
|
||
| profiles | ||
| <doc text="List of profile definitions."> | ||
| : sampled_profile list; | ||
|
|
||
| shared | ||
| <json name="shared"> | ||
| <doc text="Data shared between profiles."> | ||
| : profile_shared; | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.