[WIP] nixl connector (very very rough)

[herotai214]

[updates 1217]

============ Serving Benchmark Result ============
Successful requests: 100
Failed requests: 0
Benchmark duration (s): 13.40
Total input tokens: 25524
Total generated tokens: 10656
Request throughput (req/s): 7.46
Output token throughput (tok/s): 795.07
Peak output token throughput (tok/s): 2420.00
Peak concurrent requests: 100.00
Total Token throughput (tok/s): 2699.49
---------------Time to First Token----------------
Mean TTFT (ms): 6366.42
Median TTFT (ms): 6819.96
P99 TTFT (ms): 10187.31
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 45.11
Median TPOT (ms): 42.54
P99 TPOT (ms): 110.58
---------------Inter-token Latency----------------
Mean ITL (ms): 53.41
Median ITL (ms): 25.41
P99 ITL (ms): 602.57
==================================================

---

WORKS; accuracy ~0.8.
haven't edit or test code to work properly without EPD or without nixl.

Purpose

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.