Evaluate OpenVino Model Server (OVMS)

[goetzrieger]

OVMS is now available for ARM. As it's considerable smaller then Triton, let's explore if we can swap model servers.

https://docs.openvino.ai/2025/get-started/install-openvino/install-openvino-archive-linux.html

[rbo]

The toolkit do not container the model server.

Try to build the following container image on ARM: https://github.com/openvinotoolkit/model_server/blob/main/Dockerfile.redhat

Failed with:

...updated 14937 targets...
Error: building at STEP "RUN wget -nv https://sourceforge.net/projects/boost/files/boost/1.68.0/boost_1_68_0.tar.gz &&  if [ "$VERBOSE_LOGS" == "ON" ] ; then export BVERBOSE="-d+2" ; else export BVERBOSE="" ; fi &&      tar xf boost_1_68_0.tar.gz && cd boost_1_68_0 && ./bootstrap.sh &&      ./b2 -j ${JOBS} cxxstd=17 link=static variant=release               cxxflags="-fPIC $LTO_CXX_FLAGS"                 cflags="-fPIC $LTO_CXX_FLAGS"           linkflags="$LTO_LD_FLAGS"               --with-chrono --with-date_time --with-filesystem            --with-program_options --with-system            --with-random --with-thread --with-atomic --with-regex          --with-log --with-locale "$BVERBOSE"                install": while running runtime: exit status 1