# Orchestrator: Functioning and Working in This Project

This document explains what an **orchestrator** is in AI systems and how it is used in the AskMe backend.

---

## 1. What Is an Orchestrator in AI?

An **orchestrator** is the component that **coordinates** multiple steps, models, or tools to complete a task. It does not do the core work itself; it decides **what** runs **when** and **in what order**, and passes data between steps.

| Orchestrator does | Orchestrator does not do |
|-------------------|---------------------------|
| Decide the flow (step order) | Run LLM calls or DB queries itself |
| Pass state between steps | Implement business logic in each step |
| Choose between different paths (e.g. API vs DB) | Store or process raw data |
| Handle failures and retries at the flow level | Implement individual node logic |

Think of it as a **conductor**: it doesn’t play the instruments, but it tells each section when to play and keeps the overall performance coherent.

---

## 2. How an Orchestrator Works (General)

```
┌─────────────────────────────────────────────────────────────────┐
│                     ORCHESTRATOR                                  │
│                                                                  │
│  1. Receive input (e.g. user question)                            │
│  2. Decide path (e.g. API vs Database)                           │
│  3. Run steps in order, passing state:                           │
│       Step A → state → Step B → state → Step C → result          │
│  4. Handle errors / retries at the flow level                    │
│  5. Return final result                                          │
└─────────────────────────────────────────────────────────────────┘
         │                │                │
         ▼                ▼                ▼
    ┌─────────┐     ┌─────────┐     ┌─────────┐
    │ Step A  │     │ Step B  │     │ Step C  │
    │ (e.g.   │     │ (e.g.   │     │ (e.g.   │
    │ context)│     │ plan)   │     │ generate)│
    └─────────┘     └─────────┘     └─────────┘
```

- **State**: A single object (e.g. `SQLGenerationState`) that is updated by each step and passed to the next.
- **Nodes/Steps**: Units that take state in, do work (LLM, DB, etc.), and return updated state.
- **Flow**: The orchestrator calls the nodes in a fixed or conditional order.

---

## 3. Orchestrators in This Project

There are **three levels** of orchestration:

| Level | Component | Role |
|-------|-----------|------|
| **Request** | `app/routes/askme.py` + `AskmeService` | Chooses datasource (API vs DB), reuse vs generate, then calls the right pipeline or API flow. |
| **SQL generation** | `SQLGenerationPipeline` in `askme_open_agent/askme_base/sql_nodes.py` | Runs: context → plan → generate (with retries). |
| **Documents** | `DocumentService` in `app/services/document_service.py` | Coordinates DB, extraction, images, previews, vector DB, and AI analysis for documents. |

---

## 4. SQL Generation Pipeline (Main Example)

**File:** `askme_open_agent/askme_base/sql_nodes.py`

This is the clearest example of an orchestrator in the project.

### 4.1 What It Orchestrates

The pipeline turns a **natural language question** into **SQL** by running three steps in order:

1. **Context retrieval** – Get similar Q→SQL examples, DDL, and docs (e.g. from ChromaDB).
2. **PlannerNode** – Use the question + context to produce a **structured plan** (tables, columns, joins, filters, aggregations).
3. **GeneratorNode** – Use the plan to **generate SQL**, test it (e.g. with `LIMIT 1`), and retry on failure.

### 4.2 State Object

All steps share one state object that flows through the pipeline:

```python
# Simplified view of SQLGenerationState (sql_nodes.py)
state = {
    "question": "What are the top 10 customers by sales?",
    "context": SQLContext(...),      # filled in Step 1
    "plan": SQLPlan(...),            # filled in Step 2
    "generation_result": {...},      # filled in Step 3
    "retry_count": 0,
    "errors": [],
    "node_statuses": {"PlannerNode": "success", "GeneratorNode": "success"}
}
```

- The **orchestrator** creates the initial state, calls each step with the current state, and replaces state with the result.
- Each **node** only reads and updates this state; it does not call other nodes.

### 4.3 Pipeline Flow (Code-Level)

```text
run(question)
    │
    ├─► Create state = SQLGenerationState(question=question, ...)
    │
    ├─► Step 1: state.context = _retrieve_context(question)
    │       (similar questions, DDL, documentation)
    │
    ├─► Step 2: state = planner.process(state)
    │       (LLM produces plan → state.plan)
    │
    ├─► Step 3: state = generator.process(state)
    │       (LLM produces SQL, validate/execute, retry if needed → state.generation_result)
    │
    └─► return state
```

If the planner fails, the pipeline still continues to the generator (with a warning). Retries and error collection are handled inside the generator; the orchestrator only runs the steps and passes state.

### 4.4 Who Uses This Orchestrator

- **AskmeBase** (`askme_open_agent/askme_base/askme_base.py`) exposes `generate_sql_with_nodes()`.
- That method builds a `SQLGenerationPipeline`, runs `pipeline.run(question)`, then reads `state.generation_result.sql` and returns it (or an error message).

So: **AskmeBase** is the caller; **SQLGenerationPipeline** is the orchestrator for the “question → SQL” workflow.

---

## 5. Document Service Orchestrator

**File:** `app/services/document_service.py`

`DocumentService` is the **main orchestrator for document management**. It does not implement extraction or vector search itself; it **coordinates** several components:

| Component | Responsibility |
|-----------|----------------|
| `DocumentDbService` | Persistence (metadata, file records). |
| `ContentExtractor` | Extract text/content from files. |
| `ImageExtractor` | Extract images from documents. |
| `PreviewGenerator` | Generate preview images. |
| `ChromaVectorDB` | Vector storage and similarity search. |
| `AIAnalyzer` | AI-based analysis over document content. |

For an operation like “upload and index a document”, the orchestrator would typically:

1. Use `ContentExtractor` (and possibly `ImageExtractor`) to get content.
2. Use `DocumentDbService` to store metadata.
3. Use `PreviewGenerator` for previews.
4. Use `ChromaVectorDB` to index for search.
5. Optionally use `AIAnalyzer` for analysis.

So the **flow** and **order** of these steps are decided by `DocumentService`; the actual work is in the sub-components.

---

## 6. Request-Level Orchestration (AskMe Routes + AskmeService)

**Files:** `app/routes/askme.py`, `app/services/askme_service.py`

The **top-level** orchestration happens in the AskMe API and service layer. For each request it:

1. **Chooses datasource**: API vs Database.
2. **If API**: Chooses reuse vs “generate new” (e.g. select API, extract params, then call).
3. **If API**: Chooses SAP vs non-SAP flow (different call and transformation logic).
4. **If Database**: Calls the **SQL generation pipeline** (via `AskmeService` → `AskmeAgent` → `generate_sql_with_nodes`), then runs the SQL, and optionally builds charts or logs.

So the **orchestrator** here is the request handler + service: it doesn’t generate SQL or call external APIs itself; it **decides the path** and **invokes** the right pipeline or service (SQL pipeline, API client, document service, etc.).

---

## 7. Summary Diagram

```text
                    ┌──────────────────────────────────────┐
                    │   REQUEST (askme routes + service)    │
                    │   Orchestrator: API vs DB, reuse vs  │
                    │   generate, SAP vs non-SAP            │
                    └──────────────────────────────────────┘
                         │                      │
            ┌────────────┘                      └────────────┐
            ▼                                               ▼
   ┌─────────────────┐                          ┌─────────────────────────┐
   │ API flow        │                          │ SQL flow                │
   │ (API calls,     │                          │ SQLGenerationPipeline   │
   │  transform)     │                          │ Orchestrator:           │
   └─────────────────┘                          │  context → plan → SQL   │
                                                └─────────────────────────┘
            │
            ▼
   ┌─────────────────────────┐
   │ DocumentService         │
   │ Orchestrator:            │
   │  DB, extract, preview,   │
   │  vector DB, AI           │
   └─────────────────────────┘
```

---

## 8. Takeaways

- **Orchestrator** = component that **coordinates** steps and **passes state**; it usually does not implement the core logic of each step.
- In this project:
  - **SQLGenerationPipeline** orchestrates: context → plan → generate (with state and retries).
  - **DocumentService** orchestrates: document DB, extraction, previews, vectors, AI.
  - **AskMe routes + AskmeService** orchestrate: choice of datasource and high-level flow, then call the right pipeline or service.

Understanding “orchestrator” in this codebase means: find the place that **defines the order of steps** and **calls** the workers (nodes, services, APIs); that place is the orchestrator for that workflow.
