API Reference

Batch Ingest Files

Batch ingest multiple files using the task queue.

Args: files: List of files to ingest metadata: JSON string of metadata (either a single dict or list of dicts) rules: JSON string of rules list. Can be either:

A single list of rules to apply to all files
A list of rule lists, one per file use_colpali: Whether to use ColPali-style embedding folder_name: Optional folder to scope the documents to end_user_id: Optional end-user ID to scope the documents to auth: Authentication context redis: Redis connection pool for background tasks

Returns: BatchIngestResponse containing:

documents: List of created documents with processing status
errors: List of errors that occurred during the batch operation

POST

ingest

files

{
  "documents": [
    {
      "external_id": "<string>",
      "owner": {},
      "content_type": "<string>",
      "filename": "<string>",
      "metadata": {},
      "storage_info": {},
      "storage_files": [
        {
          "bucket": "<string>",
          "key": "<string>",
          "version": 1,
          "filename": "<string>",
          "content_type": "<string>",
          "timestamp": "2023-11-07T05:31:56Z"
        }
      ],
      "system_metadata": {},
      "additional_metadata": {},
      "access_control": {},
      "chunk_ids": [
        "<string>"
      ]
    }
  ],
  "errors": [
    {}
  ]
}

Headers

authorization

string

Body

multipart/form-data

files

file[]

required

metadata

string

default:{}

rules

string

default:[]

use_colpali

boolean | null

parallel

boolean | null

default:true

folder_name

string | null

end_user_id

string | null

Response

200

application/json

Successful Response

Response model for batch ingestion

documents

object[]

required

Represents a document stored in the database documents collection

errors

object[]

required

Was this page helpful?

Suggest edits Raise issue

Ingest File Retrieve Chunks

{
  "documents": [
    {
      "external_id": "<string>",
      "owner": {},
      "content_type": "<string>",
      "filename": "<string>",
      "metadata": {},
      "storage_info": {},
      "storage_files": [
        {
          "bucket": "<string>",
          "key": "<string>",
          "version": 1,
          "filename": "<string>",
          "content_type": "<string>",
          "timestamp": "2023-11-07T05:31:56Z"
        }
      ],
      "system_metadata": {},
      "additional_metadata": {},
      "access_control": {},
      "chunk_ids": [
        "<string>"
      ]
    }
  ],
  "errors": [
    {}
  ]
}