Skip to content

UI Improvement #55

Description

@retry51776

Here is my suggestion:

  • change Block indictor into 12 block buttons; So it's clear to user/viewer that these blocks are serial;
  • change Attention Head switch to Y axis; So it's clear to user/viewer that Attention Head is DIFFERENT dimension than Block; Also highlight active Head.
  • UI should be fixed, Positional Encoding should NOT be part of embedding; Should be part of Attention Block; Because each Attention block applies its own ROPE.
  • more identical Blocks; There should have another similar indicator in beginning of residual stream. So it's clear for user embedding is DONE once.
  • Start & End of residual stream should have same color;
  • Also, modify GPT2 model, so probality section can show different position decode token at the end of residual stream; (This will take more effort)

Thanks for your great work. I love this repo.

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions