Skip to content

初始化问题 #10

@brotherchen

Description

@brotherchen

elif isinstance(module, nn.LayerNorm) or isinstance(module, RMSNorm):
module.bias.data.zero_()

class RMSNorm(nn.Module):
def init(self, hidden_size, eps=1e-5):
super().init()
self.weight = nn.Parameter(torch.ones(hidden_size))
self.eps = eps

文件中的RMSNorm类似乎并不包含bias这个参数,该行函数会导致使用transformers库加载模型时报错?请帮忙确认是代码的问题还是我的运行方式错误?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions