feat: Add support for cds.Vector on SQLite and Postgres#1676
Conversation
vkozyura
changed the title
cds.Vector on SQLite and Postgres
- Remove vectors.cds model (causes HANA table creation to fail) - Use complex.associations.Books for vector function tests - Add describeIf helper to skip vector tests on HANA - Vector functions (COSINE_SIMILARITY, L2DISTANCE, L2NORMALIZE, VECTOR_EMBEDDING) now test with inline vectors without needing a Vector column
| HIERARCHY_ANCESTORS: undefined, | ||
|
|
||
| // Ease of use for stakeholders for auto-filling the remote source | ||
| vector_embedding(text, text_type, model_and_version, remote_source = cds.env.ai?.embeddings?.remoteSource) { |
There was a problem hiding this comment.
What is the source of cds.env.ai?.embeddings?.remoteSource ? If this comes from the previous PR or the AI plugin we should probably communicate this with java as well as they are looking into a global option as well.
I would expect also an global setting for model_and_version so that it doesn't need to be hardcoded into models.
There was a problem hiding this comment.
The cds.env.ai.embeddings.remoteSource configuration is introduced by the @cap-js/ai.
Java stack has no ai property in CdsProperties.java. The Java test hardcodes the model.
The Parameter model_and_version is called embeddingModel in Java.
Suggestion: Sync with Java as follow-up
| try { | ||
| await dbc.query('CREATE EXTENSION IF NOT EXISTS vector') | ||
| await pgvector.registerTypes(dbc) | ||
| } catch (e) { | ||
| const LOG = cds.log('postgres') | ||
| LOG.debug('pgvector extension not available, skipping vector support:', e.message) | ||
| } |
There was a problem hiding this comment.
This should not be ran every create. This might not even need to run at all depending on how the postgres build task is deploying the model.
There was a problem hiding this comment.
commented out with a revisit containing possible options - pg tests are currently disabled
3 participants
Summary
COSINE_SIMILARITY,L2DISTANCE,L2NORMALIZE,VECTOR_EMBEDDINGVECTOR_EMBEDDING(deterministic, suitable for testing/development)Implementation
sqlite/lib/vector_handling/index.jsTest plan