Skip to content

Annotate activation functions with Nx blocks#636

Open
seanmor5 wants to merge 2 commits into
mainfrom
sm-nx-block
Open

Annotate activation functions with Nx blocks#636
seanmor5 wants to merge 2 commits into
mainfrom
sm-nx-block

Conversation

@seanmor5

Copy link
Copy Markdown
Contributor

This is related to the introduction of Nx.block and custom calls. To make this transition easier, I added a defblock construct which marks a function as an Nx.block with a generated camel-case module name equivalent to the function name.

The plan would be to use this for all layers as well, but that requires some changes because some layers are deftransform instead of defn.

Axon blocks will be prefixed with Axon.Block. It's confusing a bit because these are different than Axon.block

Comment thread lib/axon/activations.ex

"""
defn softplus(x) do
defblock Axon.Block, softplus(x) do

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@josevalim @seanmor5 wdyt about having a single Axon.Block struct with module, function and arity as the name?

%Axon.Block{module: Axon.Activations, function: :softplus, arity: 1, options: ...}

%Axon.Block{module: Axon.Layers, function: :dense, arity: 2, options: ...}

or something like that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants