Skip to content

[BUG] nnfusion fail: Unknown op count: 2 #533

@juinshell

Description

@juinshell

🐛 Bug
Hello, I'm trying to use nnfusion to gen cuda code for transformer-based model provided by hugging face. But after I get onnx format model by optimum-cli, I cannot deliver this onnx model to nnfusion correctly. I wonder whether I get the model right or nnfusion cannot smoothly transform these kind of transformer-based model?

To Reproduce
Steps to reproduce the behavior:

  1. use optimum-cli to convert the hugging face checkpoint to onnx format model:
optimum-cli export onnx --model google/vit-base-patch16-224-in21k vit-base-patch16-224-in21k_onnx_32/ --batch_size 32

(--bath_size arg seems not change the onnx output when I visualize both by netron)
2. use nnfusion to convert (ERROR)

nnfusion ./vit-base-patch16-224-in21k_onnx_32/model.onnx -f onnx -p "batch_size:32;num_channels:3;height:224;width:224"

and the error output meg:

...
  name: "main_graph"
  input {
    name: "pixel_values"
    type {
      tensor_type {
        elem_type: 1
        shape {
          dim {
            dim_param: "batch_size"
          }
          dim {
            dim_param: "num_channels"
          }
          dim {
            dim_param: "height"
          }
          dim {
            dim_param: "width"
          }
        }
      }
    }
  }
  output {
    name: "last_hidden_state"
    type {
      tensor_type {
        elem_type: 1
        shape {
          dim {
            dim_param: "batch_size"
          }
          dim {
            dim_param: "Addlast_hidden_state_dim_1"
          }
          dim {
            dim_param: "Addlast_hidden_state_dim_2"
          }
        }
      }
    }
  }
}
opset_import {
  version: 14
}

[INFO] 2024-08-10T12:33:27z thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp 309        Converting Onnx Graph
[ERROR] 2024-08-10T12:33:28z thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp 421       Unknown op type: Where
[ERROR] 2024-08-10T12:33:28z thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp 421       Unknown op type: ConstantOfShape
[ERROR] 2024-08-10T12:33:28z src/nnfusion/util/errors.hpp 169   Failure at /root/nnfusion/thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp:423:
Unknown op count: 2
terminate called after throwing an instance of 'nnfusion::errors::CheckError'
  what():  Failure at /root/nnfusion/thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp:423:
Unknown op count: 2
Aborted (core dumped)

Expected behavior
Successfully generate CUDA code and run well.

Additional context
I use the docker image from your repo, it goes well when I just use the model nnfusion provided in example.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions