Add vector_indexing to CreateCollection command

[tatu-at-datastax]

Currently "CreateCollection" only takes VectorSearchDesc that contains 2 things to optionally configure:

1. source_model: values like ada002, bert
2. metric (was: "function", in C "similarity_function"): values like dot_product, cosine, euclidean

There are more options that can be added, listed here:

https://github.com/datastax/cassandra/blob/main/src/java/org/apache/cassandra/index/sai/VECTOR.md?plain=1#L18

and some if not all should be added:

• maximum_node_connections (default: 16)
  • valid range: 1-512
• construction_beam_width (default: 100)
  • valid range: 1-3200
• neighborhood_overflow (default: 1.0 in memtable, 1.2 in compaction)
  • valid range: > 0
• alpha (default: dimension > 3 gets 1.2. Otherwise, 2.0 in memtable, 1.4 in compaction)
  • valid range: > 0
• enable_hierarchy (default: false)
  • valid range: true/false

EDIT: looks like DSE-6.9 does NOT support these maximum_node_connections, construction_beam_width
or enable_hierarchy options (and probably not others either), only HCD does?

Not sure how to proceed.

[tatu-at-datastax]

Thinking of adding separate VectorIndexDesc for new options, but keep existing 2 as separate for compatibility reasons.

[tatu-at-datastax]

Not sure we want to take this approach -- more likely we would have "option sets" for usage like:

• Small data, high recall
• Big data, low latency

or something along those lines: named sets where user selects goal, not settings directly.

[tatu-at-datastax]

Decision on approach: Combination of 3 and 1, so that newly added configuration setting can be:

• JSON String -> named profile (option 3)
• JSON Object -> set of raw Indexing options (option 1)
• Anything else -> error