Vector Store

The Vector Store feature in GenLayer allows developers to enhance their Intelligent Contracts by efficiently storing, retrieving, and calculating similarities between texts using vector embeddings. This feature is particularly useful for tasks that require natural language processing (NLP), such as creating context-aware applications or indexing text data for semantic search.

Key Features of Vector Store

The Vector Store provides several powerful features for managing text data:

1. Text Embedding Storage

You can store text data as vector embeddings, which are mathematical representations of the text, allowing for efficient similarity comparisons. Each stored text is associated with a vector and metadata.

2. Similarity Calculation

The Vector Store allows you to calculate the similarity between a given text and stored vectors using cosine similarity. This is useful for finding the most semantically similar texts, enabling applications like recommendation systems or text-based search.

3. Metadata Management

Along with the text and vectors, you can store additional metadata (any data type) associated with each text entry. This allows developers to link additional information (e.g., IDs or tags) to the text for retrieval.

4. CRUD Operations

The Vector Store provides standard CRUD (Create, Read, Update, Delete) operations, allowing developers to add, update, retrieve, and delete text and vector entries efficiently.

How to Use Vector Store in Your Contracts

To use the Vector Store in your Intelligent Contracts, you will interact with its methods to add and retrieve text data, calculate similarities, and manage vector storage. Below are the details of how to use this feature.

Importing Vector Store

First, import the VectorStore class from the standard library in your contract:

from backend.node.genvm.std.vector_store import VectorStore

Creating a Contract with Vector Store

Here’s an example of a contract using the Vector Store for indexing and searching text logs:

from backend.node.genvm.icontract import IContract
from backend.node.genvm.std.vector_store import VectorStore
 
class LogIndexer(IContract):
 
    def __init__(self):
        self.vector_store = VectorStore()
 
    # Find the closest vector to a given text
    def get_closest_vector(self, text: str) -> dict:
        result = self.vector_store.get_closest_vector(text)
        if result is None:
            return None
        return {
            "similarity": result[0],
            "id": result[1],
            "text": result[2],
            "metadata": result[3],
            "vector": result[4],
        }
 
    # Add a new log entry with text and associated metadata
    def add_log(self, log: str, log_id: int) -> None:
        self.vector_store.add_text(log, {"log_id": log_id})
 
    # Update an existing log entry by its ID
    def update_log(self, id: int, log: str, log_id: int) -> None:
        self.vector_store.update_text(id, log, {"log_id": log_id})
 
    # Remove a log entry by its ID
    def remove_log(self, id: int) -> None:
        self.vector_store.delete_vector(id)
 
    # Get metadata associated with a vector by its ID
    def get_vector_metadata(self, id: int) -> None:
        _, metadata, _ = self.vector_store.get_vector(id)
        return metadata

Key Methods in Vector Store

1. Adding Text and Metadata

To store text in the Vector Store along with its metadata, use the add_text() method. This method generates a vector embedding for the text and stores it in the store.

vector_id = self.vector_store.add_text("example text", {"log_id": 123})

Args:

• text (str): The text to store.
• metadata (any): Additional information associated with the text. Returns: vector_id (int) - the unique ID of the stored vector.

2. Retrieving the Closest Vector

To find the vector that is most similar to a given text, use the get_closest_vector() method.

closest = self.vector_store.get_closest_vector("example text")

Args:

• text (str): The query text to compare. Returns: A tuple containing:
• similarity (float): The similarity percentage.
• id (int): The ID of the closest vector.
• text (str): The stored text.
• metadata (any): Associated metadata.
• vector (list[float]): The vector embedding.

3. Updating Text and Metadata

To update the text and metadata associated with a stored vector, use the update_text() method.

self.vector_store.update_text(vector_id, "new text", {"log_id": 456})

Args:

• vector_id (int): The ID of the vector to update.
• new_text (str): The new text to store.
• new_metadata (any): The new metadata.

4. Deleting a Vector

To delete a stored vector and its associated metadata, use the delete_vector() method.

self.vector_store.delete_vector(vector_id)

Args:

• vector_id (int): The ID of the vector to delete.

5. Fetching All Items

To retrieve all stored vectors and their metadata, use the get_all_items() method.

items = self.vector_store.get_all_items()

Returns: A list of tuples containing stored texts and their metadata.