This repository contains a CLIP/SigLIP inspired model for comparing data of two different text modalities (here boolean language queries and natural language queries but adaptable to other types of text). There also is a visualization tool for visualizing the embedding space through Umap.
To start a local server for hosting the embedding visualization website run python app.py
To only calculate the embeddings and cache them use python app.py --precalculate
This repository consists of a CLIP inspired model, which is used to compare Natural Language Queries with Boolean Language Queries. It also contains helper functions for preprocessing data from different sources, and training/evaluating different models.
By running this file a local server which hosts a website showing embeddings of given data in a 2D plot
By adjusting variables in the beginning of this file the input data and used model can be modified.
This notebook contains examples for evaluating models with our evaluation functions
This notebook was used to create plots from data saved to W&B while training models
A notebook for training/fine-tuning models with HuggingFace libraries
