# Search for documents by semantic similarity

{% openapi src="<https://2952947711-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyNqpTh7Mh66N0RnO0k24%2Fuploads%2Fgit-blob-77b6137eeb641262ec8e531c78123c02b825b865%2Frememberizer_openapi.yml?alt=media&token=cbad765b-1613-4222-b591-9ae17a3b7cfa>" path="/documents/search/" method="get" %}
[rememberizer\_openapi.yml](https://2952947711-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyNqpTh7Mh66N0RnO0k24%2Fuploads%2Fgit-blob-77b6137eeb641262ec8e531c78123c02b825b865%2Frememberizer_openapi.yml?alt=media\&token=cbad765b-1613-4222-b591-9ae17a3b7cfa)
{% endopenapi %}

## Example Requests

{% tabs %}
{% tab title="cURL" %}

```bash
curl -X GET \
  "https://api.rememberizer.ai/api/v1/documents/search/?q=How%20to%20integrate%20Rememberizer%20with%20custom%20applications&n=5&from=2023-01-01T00:00:00Z&to=2023-12-31T23:59:59Z" \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"
```

{% hint style="info" %}
Replace `YOUR_JWT_TOKEN` with your actual JWT token.
{% endhint %}
{% endtab %}

{% tab title="JavaScript" %}

```javascript
const searchDocuments = async (query, numResults = 5, from = null, to = null) => {
  const url = new URL('https://api.rememberizer.ai/api/v1/documents/search/');
  url.searchParams.append('q', query);
  url.searchParams.append('n', numResults);
  
  if (from) {
    url.searchParams.append('from', from);
  }
  
  if (to) {
    url.searchParams.append('to', to);
  }
  
  const response = await fetch(url.toString(), {
    method: 'GET',
    headers: {
      'Authorization': 'Bearer YOUR_JWT_TOKEN'
    }
  });
  
  const data = await response.json();
  console.log(data);
};

searchDocuments('How to integrate Rememberizer with custom applications', 5);
```

{% hint style="info" %}
Replace `YOUR_JWT_TOKEN` with your actual JWT token.
{% endhint %}
{% endtab %}

{% tab title="Python" %}

```python
import requests

def search_documents(query, num_results=5, from_date=None, to_date=None):
    headers = {
        "Authorization": "Bearer YOUR_JWT_TOKEN"
    }
    
    params = {
        "q": query,
        "n": num_results
    }
    
    if from_date:
        params["from"] = from_date
    
    if to_date:
        params["to"] = to_date
    
    response = requests.get(
        "https://api.rememberizer.ai/api/v1/documents/search/",
        headers=headers,
        params=params
    )
    
    data = response.json()
    print(data)

search_documents("How to integrate Rememberizer with custom applications", 5)
```

{% hint style="info" %}
Replace `YOUR_JWT_TOKEN` with your actual JWT token.
{% endhint %}
{% endtab %}

{% tab title="Ruby" %}

```ruby
require 'net/http'
require 'uri'
require 'json'

def search_documents(query, num_results=5, from_date=nil, to_date=nil)
  uri = URI('https://api.rememberizer.ai/api/v1/documents/search/')
  params = {
    q: query,
    n: num_results
  }
  
  params[:from] = from_date if from_date
  params[:to] = to_date if to_date
  
  uri.query = URI.encode_www_form(params)
  
  request = Net::HTTP::Get.new(uri)
  request['Authorization'] = 'Bearer YOUR_JWT_TOKEN'
  
  http = Net::HTTP.new(uri.host, uri.port)
  http.use_ssl = true
  
  response = http.request(request)
  data = JSON.parse(response.body)
  puts data
end

search_documents("How to integrate Rememberizer with custom applications", 5)
```

{% hint style="info" %}
Replace `YOUR_JWT_TOKEN` with your actual JWT token.
{% endhint %}
{% endtab %}
{% endtabs %}

## Query Parameters

| Parameter    | Type    | Description                                                                                           |
| ------------ | ------- | ----------------------------------------------------------------------------------------------------- |
| q            | string  | **Required.** The search query text (up to 400 words).                                                |
| n            | integer | Number of results to return. Default: 3. Use higher values (e.g., 10) for more comprehensive results. |
| from         | string  | Start of the time range for documents to be searched, in ISO 8601 format.                             |
| to           | string  | End of the time range for documents to be searched, in ISO 8601 format.                               |
| prev\_chunks | integer | Number of preceding chunks to include for context. Default: 2.                                        |
| next\_chunks | integer | Number of following chunks to include for context. Default: 2.                                        |

## Response Format

```json
{
  "data_sources": [
    {
      "name": "Google Drive",
      "documents": 3
    },
    {
      "name": "Slack",
      "documents": 2
    }
  ],
  "matched_chunks": [
    {
      "document": {
        "id": 12345,
        "document_id": "1aBcD2efGhIjK3lMnOpQrStUvWxYz",
        "name": "Rememberizer API Documentation.pdf",
        "type": "application/pdf",
        "path": "/Documents/Rememberizer/API Documentation.pdf",
        "url": "https://drive.google.com/file/d/1aBcD2efGhIjK3lMnOpQrStUvWxYz/view",
        "size": 250000,
        "created_time": "2023-05-10T14:30:00Z",
        "modified_time": "2023-06-15T09:45:00Z",
        "indexed_on": "2023-06-15T10:30:00Z",
        "integration": {
          "id": 101,
          "integration_type": "google_drive"
        }
      },
      "matched_content": "To integrate Rememberizer with custom applications, you can use the OAuth2 authentication flow to authorize your application to access a user's Rememberizer data. Once authorized, your application can use the Rememberizer APIs to search for documents, retrieve content, and more.",
      "distance": 0.123
    },
    // ... more matched chunks
  ],
  "message": "Search completed successfully",
  "code": "success"
}
```

## Search Optimization Tips

### For Question Answering

When searching for an answer to a question, try formulating your query as if it were an ideal answer. For example:

Instead of: "What is vector embedding?" Try: "Vector embedding is a technique that converts text into numerical vectors in a high-dimensional space."

{% hint style="info" %}
For a deeper understanding of how vector embeddings work and why this search approach is effective, see [What are Vector Embeddings and Vector Databases?](https://docs.rememberizer.ai/background/what-are-vector-embeddings-and-vector-databases)
{% endhint %}

### Adjusting Result Count

* Start with `n=3` for quick, high-relevance results
* Increase to `n=10` or higher for more comprehensive information
* If search returns insufficient information, try increasing the `n` parameter

### Time-Based Filtering

Use the `from` and `to` parameters to focus on documents from specific time periods:

* Recent documents: Set `from` to a recent date
* Historical analysis: Specify a specific date range
* Excluding outdated information: Set an appropriate `to` date

## Batch Operations

For efficiently handling large volumes of search queries, Rememberizer supports batch operations to optimize performance and reduce API call overhead.

### Batch Search

{% tabs %}
{% tab title="Python" %}

```python
import requests
import time
import json
from concurrent.futures import ThreadPoolExecutor

def batch_search_documents(queries, num_results=5, batch_size=10):
    """
    Perform batch searches with multiple queries
    
    Args:
        queries: List of search query strings
        num_results: Number of results to return per query
        batch_size: Number of queries to process in parallel
    
    Returns:
        List of search results for each query
    """
    headers = {
        "Authorization": "Bearer YOUR_JWT_TOKEN",
        "Content-Type": "application/json"
    }
    
    results = []
    
    # Process queries in batches
    for i in range(0, len(queries), batch_size):
        batch = queries[i:i+batch_size]
        
        # Create a thread pool to send requests in parallel
        with ThreadPoolExecutor(max_workers=batch_size) as executor:
            futures = []
            
            for query in batch:
                params = {
                    "q": query,
                    "n": num_results
                }
                
                future = executor.submit(
                    requests.get,
                    "https://api.rememberizer.ai/api/v1/documents/search/",
                    headers=headers,
                    params=params
                )
                futures.append(future)
            
            # Collect results as they complete
            for future in futures:
                response = future.result()
                results.append(response.json())
        
        # Rate limiting - pause between batches to avoid API throttling
        if i + batch_size < len(queries):
            time.sleep(1)
    
    return results

# Example usage
queries = [
    "How to use OAuth with Rememberizer",
    "Vector database configuration options",
    "Best practices for semantic search",
    # Add more queries as needed
]

results = batch_search_documents(queries, num_results=3, batch_size=5)
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
/**
 * Perform batch searches with multiple queries
 * 
 * @param {string[]} queries - List of search query strings
 * @param {number} numResults - Number of results to return per query
 * @param {number} batchSize - Number of queries to process in parallel
 * @param {number} delayBetweenBatches - Milliseconds to wait between batches
 * @returns {Promise<Array>} - List of search results for each query
 */
async function batchSearchDocuments(queries, numResults = 5, batchSize = 10, delayBetweenBatches = 1000) {
  const results = [];
  
  // Process queries in batches
  for (let i = 0; i < queries.length; i += batchSize) {
    const batch = queries.slice(i, i + batchSize);
    
    // Create an array of promises for concurrent requests
    const batchPromises = batch.map(query => {
      const url = new URL('https://api.rememberizer.ai/api/v1/documents/search/');
      url.searchParams.append('q', query);
      url.searchParams.append('n', numResults);
      
      return fetch(url.toString(), {
        method: 'GET',
        headers: {
          'Authorization': 'Bearer YOUR_JWT_TOKEN'
        }
      }).then(response => response.json());
    });
    
    // Wait for all requests in the batch to complete
    const batchResults = await Promise.all(batchPromises);
    results.push(...batchResults);
    
    // Rate limiting - pause between batches to avoid API throttling
    if (i + batchSize < queries.length) {
      await new Promise(resolve => setTimeout(resolve, delayBetweenBatches));
    }
  }
  
  return results;
}

// Example usage
const queries = [
  "How to use OAuth with Rememberizer",
  "Vector database configuration options",
  "Best practices for semantic search",
  // Add more queries as needed
];

batchSearchDocuments(queries, 3, 5)
  .then(results => console.log(results))
  .catch(error => console.error('Error in batch search:', error));
```

{% endtab %}

{% tab title="Ruby" %}

```ruby
require 'net/http'
require 'uri'
require 'json'
require 'concurrent'

# Perform batch searches with multiple queries
#
# @param queries [Array<String>] List of search query strings
# @param num_results [Integer] Number of results to return per query
# @param batch_size [Integer] Number of queries to process in parallel
# @param delay_between_batches [Float] Seconds to wait between batches
# @return [Array] List of search results for each query
def batch_search_documents(queries, num_results = 5, batch_size = 10, delay_between_batches = 1.0)
  results = []
  
  # Process queries in batches
  queries.each_slice(batch_size).with_index do |batch, batch_index|
    # Create a thread pool for concurrent requests
    pool = Concurrent::FixedThreadPool.new(batch_size)
    futures = []
    
    batch.each do |query|
      futures << Concurrent::Future.execute(executor: pool) do
        uri = URI('https://api.rememberizer.ai/api/v1/documents/search/')
        params = {
          q: query,
          n: num_results
        }
        
        uri.query = URI.encode_www_form(params)
        
        request = Net::HTTP::Get.new(uri)
        request['Authorization'] = 'Bearer YOUR_JWT_TOKEN'
        
        http = Net::HTTP.new(uri.host, uri.port)
        http.use_ssl = true
        
        response = http.request(request)
        JSON.parse(response.body)
      end
    end
    
    # Collect results from all threads
    batch_results = futures.map(&:value)
    results.concat(batch_results)
    
    # Rate limiting - pause between batches to avoid API throttling
    if batch_index < (queries.length / batch_size.to_f).ceil - 1
      sleep(delay_between_batches)
    end
  end
  
  pool.shutdown
  results
end

# Example usage
queries = [
  "How to use OAuth with Rememberizer",
  "Vector database configuration options",
  "Best practices for semantic search",
  # Add more queries as needed
]

results = batch_search_documents(queries, 3, 5)
puts results
```

{% endtab %}
{% endtabs %}

### Performance Considerations

When implementing batch operations, consider these best practices:

1. **Optimal Batch Size**: Start with batch sizes of 5-10 queries and adjust based on your application's performance characteristics.
2. **Rate Limiting**: Include delays between batches to prevent API throttling. A good starting point is 1 second between batches.
3. **Error Handling**: Implement robust error handling to manage failed requests within batches.
4. **Resource Management**: Monitor client-side resource usage, particularly with large batch sizes, to prevent excessive memory consumption.
5. **Response Processing**: Process batch results asynchronously when possible to improve user experience.

For high-volume applications, consider implementing a queue system to manage large numbers of search requests efficiently.

This endpoint provides powerful semantic search capabilities across your entire knowledge base. It uses vector embeddings to find content based on meaning rather than exact keyword matches.