User and Folder Scoping

Organizing data with user and folder scoping in Morphik

Morphik provides powerful mechanisms to organize and isolate your data through user scoping and folder scoping. These features allow you to create logical boundaries for different projects or user groups while maintaining a unified database.

Folder Scoping

Folders in Morphik allow you to organize documents into logical groups, similar to directories in a file system, but for your unstructured data. Operations performed within a folder scope only affect documents within that folder.

When to Use Folder Scoping

• Project Organization: Separate documents by project, department, or purpose
• Data Categorization: Group similar documents together
• Access Control: Create logical boundaries for different document sets

Creating and Using Folders

from morphik import Morphik

with Morphik() as db:
    # Create or get a folder
    folder = db.create_folder("project_x")
    # or
    folder = db.get_folder("project_x")

    # Operations are scoped to this folder
    doc = folder.ingest_text("This document belongs to Project X")
    chunks = folder.retrieve_chunks("project")
    docs = folder.list_documents()  # Only lists documents in this folder

User Scoping

User scoping allows multi-tenant applications to isolate data per end user, ensuring each user only sees their own documents. This is particularly useful for applications where privacy between users is important.

When to Use User Scoping

• Multi-tenant Applications: Keep each user’s data separate
• Privacy Requirements: Ensure users can only access their own documents
• Personal Data Spaces: Create user-specific knowledge bases

Creating and Using User Scopes

from morphik import Morphik

with Morphik() as db:
    # Create a user scope
    user_scope = db.signin("user123")

    # Operations are scoped to this user
    doc = user_scope.ingest_text("This belongs to user123 only")
    docs = user_scope.list_documents()  # Only lists documents for this user
    completion = user_scope.query("What documents do I have?")  # Only searches user123's documents

Combined User and Folder Scoping

For the maximum level of organization, you can combine both scopes to organize documents by both user and folder.

from morphik import Morphik

with Morphik() as db:
    # First get a folder
    folder = db.get_folder("project_x")

    # Then scope to a specific user within that folder
    user_folder_scope = folder.signin("user123")

    # Operations are scoped to both the folder and user
    doc = user_folder_scope.ingest_text("This belongs to user123 in project_x")
    chunks = user_folder_scope.retrieve_chunks("project")
    docs = user_folder_scope.list_documents()  # Only lists user123's documents in project_x

Asynchronous Usage

All scoping features are also available with the asynchronous client:

from morphik import AsyncMorphik

async with AsyncMorphik() as db:
    # Create folder and user scopes
    folder = db.get_folder("project_x")
    user_scope = db.signin("user123")
    combined_scope = folder.signin("user123")

    # Use scoped operations asynchronously
    doc = await combined_scope.ingest_text("Document for user123 in project_x")
    results = await combined_scope.retrieve_docs("project")

Important Considerations

• All methods available on the main Morphik client are also available on folder and user scopes
• Operations performed within a scope are isolated to that scope
• Documents created within a scope are only accessible within that scope, unless explicitly queried with appropriate filters
• Scopes can be used for both reading and writing operations
• User and folder information is stored as metadata with the documents, so you can still filter across scopes with explicit filter parameters if needed

Filtering by Folder Name in Metadata

When you ingest documents in a folder with the folder_name parameter, that value is automatically available in the document’s metadata for filtering. This enables powerful cross-folder queries using Morphik’s metadata filter operators:

from morphik import Morphik

db = Morphik()

# Filter documents from multiple folders
filters = {
    "folder_name": {"$in": ["legal", "hr", "finance"]}
}
docs = db.list_documents(filters=filters)

# Combine folder filtering with other metadata
filters = {
    "$and": [
        {"folder_name": {"$regex": {"pattern": "^project_", "flags": "i"}}},
        {"status": "active"},
        {"priority": {"$gte": 70}}
    ]
}
response = db.query("What are the high-priority project updates?", filters=filters)

# Exclude specific folders
filters = {
    "$and": [
        {"folder_name": {"$nin": ["archived", "drafts"]}},
        {"created_date": {"$gte": "2024-01-01"}}
    ]
}
chunks = db.retrieve_chunks("quarterly report", filters=filters, k=10)

This approach is useful when you need to:

• Query across multiple folders simultaneously
• Use pattern matching on folder names
• Combine folder filters with complex metadata conditions
• Build dynamic queries where folder selection isn’t known at scope creation time

For more filtering examples, see the Complex Metadata Filtering cookbook and Metadata Filtering reference.

Use Cases

Multi-Project Research Team

A research team working on multiple projects can use folder scoping to keep document sets separate:

# Project A research
project_a = db.get_folder("project_a")
doc = project_a.ingest_file("project_a_results.pdf")

# Project B research (completely separate)
project_b = db.get_folder("project_b")
doc = project_b.ingest_file("project_b_results.pdf")

Multi-tenant Application

An application serving multiple end users can use user scoping to keep each user’s data private:

# User 1's personal data
user1 = db.signin("user1")
doc = user1.ingest_text("My private notes")

# User 2's personal data (completely separate)
user2 = db.signin("user2")
doc = user2.ingest_text("My confidential information")

Enterprise Knowledge Management

For complex enterprise setups, combine both scopes to organize by both department and individual:

# Marketing department
marketing = db.get_folder("marketing")

# Individual marketers' workspaces
alice_marketing = marketing.signin("alice")
bob_marketing = marketing.signin("bob")

# Engineering department
engineering = db.get_folder("engineering")

# Individual engineers' workspaces
charlie_engineering = engineering.signin("charlie")
dave_engineering = engineering.signin("dave")