currently, FP8 engine can be generated using trtexec command line.
# Not support yet
if data_type == trt.DataType.BF16:
tw.config.set_flag(trt.BuilderFlag.BF16)
if data_type == trt.DataType.FP8:
tw.config.set_flag(trt.BuilderFlag.FP8)
if data_type == trt.DataType.INT4:
tw.config.set_flag(trt.BuilderFlag.INT4)
Is there any plan to support FP8 mode? just setting config.set_flag(trt.BuilderFlag.FP8) like config.set_flag(trt.BuilderFlag.FP16)
currently, FP8 engine can be generated using
trtexeccommand line.Is there any plan to support FP8 mode? just setting
config.set_flag(trt.BuilderFlag.FP8)likeconfig.set_flag(trt.BuilderFlag.FP16)