What are Vector Embeddings and Vector Databases?
Why Rememberizer is more than just a database or keyword search engine
Last updated
Why Rememberizer is more than just a database or keyword search engine
Last updated
Rememberizer uses vector embeddings in vector databases to enable searches for semantic similarity within user knowledge sources. This is a fundamentally more advanced and nuanced form of information retrieval than simply looking for keywords in content through a traditional search engine or database.
In their most advanced form (as used by Rememberizer), vector embeddings are created by language models with architectures similar to the AI LLMs (Large Language Models) that underpin OpenAI's GPT models and ChatGPT service, as well as models/services from Google (Gemini), Anthropic (Claude), Meta (LLaMA), and others.
What does a vector embedding look like? Consider a coordinate (x,y) in two dimensions. If it represents a line from the origin to this point, we can think of it as a line with a direction—in other words, a vector in two dimensions.
In the context of Rememberizer, a vector embedding is typically a list of several hundred numbers (often 768, 1024, or 1536) representing a vector in a high-dimensional space. This list of numbers can represent weights in a Transformer model that define the meaning in a phrase such as "A bolt of lightning out of the blue." This is fundamentally the same underlying representation of meaning used in models like GPT-4. As a result, a good vector embedding enables the same sophisticated understanding that we see in modern AI language models.
Vector embeddings can represent more than just text—they can also encode other types of data such as images or sound. With properly trained models, you can compare across media types, allowing a vector embedding of text to be compared to an image, or vice versa.
Currently, Rememberizer enables searches within the text component of user documents and knowledge. Text-to-image and image-to-text search capabilities are on Rememberizer's roadmap for future development.
Major technology companies leverage vector embeddings in their products:
Keyword search finds exact matches or predetermined synonyms. In contrast, Rememberizer's vector search finds content that's conceptually related, even when different terminology is used. For example:
A keyword search for "dog care" might miss a relevant document about "canine health maintenance"
Rememberizer's vector search would recognize these concepts as semantically similar and return both
This capability makes Rememberizer particularly powerful for retrieving relevant information from diverse knowledge sources.
Coming soon: Vector Search Process Visualization
This diagram will illustrate the complete semantic search workflow in Rememberizer:
Document chunking and preprocessing
Vector embedding generation process
Storage in vector database
Search query embedding
Similarity matching calculation
Side-by-side comparison with traditional keyword search
To deeply understand how vector embeddings and vector databases work:
The technologies behind vector embeddings have evolved significantly over time:
This makes vector embeddings a natural choice for discovering relevant knowledge to include in the context of AI model prompts. The technologies are complementary and conceptually related. For this reason, most providers of LLMs as a service also produce vector embeddings as a service (for example: or ).
Google uses vector embeddings to power both their text search (text-to-text) and image search (text-to-image) capabilities ()
Meta (Facebook) has implemented embeddings for their social network search ()
Snapchat utilizes vector embeddings to understand context and serve targeted advertising ()
Start with the
Pinecone (a vector database service) offers a good
Meta's FAISS library: "FAISS: A Library for Efficient Similarity Search and Clustering of Dense Vectors" by Johnson, Douze, and Jégou (2017) provides comprehensive insights into efficient vector similarity search ()
The 2017 paper "Attention Is All You Need" () introduced the Transformer architecture that powers modern LLMs and advanced embedding models
"Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality" (, ) established the theory for efficient similarity search in high-dimensional spaces
BERT (2018, ) demonstrated the power of bidirectional training for language understanding tasks
Earlier methods like GloVe (2014, ) and Word2Vec (2013, ) laid the groundwork for neural word embeddings
For technical implementation details and developer-oriented guidance on using vector stores with Rememberizer, see .
Google researchers were behind the original Transformer architecture described in "Attention Is All You Need" (), though many organizations have since built upon and extended this foundational work.