file list

fix gcp location
vertex ai, 2.5 pro
2025-12-16 04:21:28 +01:00 · 2025-12-15 18:58:47 +01:00 · 2025-12-15 02:16:42 +01:00 · 2025-12-14 18:38:07 +01:00 · 2025-12-13 17:35:57 +01:00 · 2025-12-13 17:23:52 +01:00
36 changed files with 1904 additions and 5382 deletions
--- a/README.md
+++ b/README.md
@@ -8,6 +8,7 @@
 ```
                                    +------------------+
                                    |     Auth0        |
+                                    |   (RBAC roles)   |
                                    +--------+---------+
                                             |
                                             v
@@ -18,18 +19,22 @@
 +------------------+              |    - Auth0 SSO      |
                                  +----------+----------+
                                             |
+                                             | AG-UI Protocol
                                             v
                                  +----------+----------+
                                  |   web/agent/        |
-                                  |   (LangGraph)       |
+                                  |   (PydanticAI)      |
                                  |   - Google Gemini   |
+                                  |   - x-user-roles    |
                                  +----------+----------+
                                             |
+                                             | Streamable HTTP
                                             v
                                  +----------+----------+
                                  |      mcp/           |
                                  |   (FastMCP Server)  |
                                  |   - Semantic search |
+                                  |   - Role filtering  |
                                  +----------+----------+
                                             |
                        +--------------------+--------------------+
@@ -65,8 +70,8 @@
 | Component | Description | Tech Stack |
 |-----------|-------------|------------|
 | **web/** | Frontend application with chat UI | Next.js, CopilotKit, Auth0 |
-| **web/agent/** | AI agent for answering cave questions | LangGraph, Google Gemini |
-| **mcp/** | MCP server exposing semantic search tools | FastMCP, Cohere |
+| **web/agent/** | AI agent for answering cave questions | PydanticAI, AG-UI, Google Gemini |
+| **mcp/** | MCP server exposing semantic search tools | FastMCP, Starlette, Cohere |
 | **poller/** | Document ingestion and processing pipeline | Python, Claude API, Cohere |

 ## Data Flow
@@ -80,9 +85,11 @@
   - Stored in PostgreSQL with pgvector

 2. **Search & Chat** (mcp + agent)
+   - User authenticates via Auth0 (roles assigned)
   - User asks question via web UI
-   - Agent calls MCP tools for semantic search
-   - MCP queries pgvector for relevant documents
+   - Web API extracts user roles from session, passes to agent
+   - Agent creates MCP connection with `x-user-roles` header
+   - MCP queries pgvector, filtering by user's roles
   - Agent synthesizes response with citations

 ## Getting Started
@@ -95,6 +102,13 @@ See individual component READMEs:

 Each component requires its own environment variables. See the respective READMEs for details.

+| Component | Key Variables |
+|-----------|---------------|
+| **web/** | `AUTH0_*`, `AGENT_URL` |
+| **web/agent/** | `GOOGLE_API_KEY`, `CAVE_MCP_URL` |
+| **mcp/** | `COHERE_API_KEY`, `DB_*` |
+| **poller/** | `ANTHROPIC_API_KEY`, `COHERE_API_KEY`, `AWS_*`, `DB_*` |
+
 **Never commit `.env` files** - they are gitignored.

 ## CI/CD
@@ -106,6 +120,7 @@ Gitea Actions workflows build and push Docker images on changes to `main`:
 | build-push-web | `web/**` (excluding agent) | `cavepediav2-web:latest` |
 | build-push-agent | `web/agent/**` | `cavepediav2-agent:latest` |
 | build-push-poller | `poller/**` | `cavepediav2-poller:latest` |
+| build-push-mcp | `mcp/**` | `cavepediav2-mcp:latest` |

 ## License

--- a/mcp/Dockerfile
+++ b/mcp/Dockerfile
@@ -0,0 +1,28 @@
+FROM python:3.13-slim
+
+# Install uv
+COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
+
+WORKDIR /app
+
+ENV PYTHONPATH=/app
+
+# Copy dependency files
+COPY pyproject.toml uv.lock ./
+
+# Install dependencies
+RUN uv sync --frozen --no-dev
+
+# Copy application code
+COPY server.py ./
+
+# Create non-root user
+RUN useradd --create-home --shell /bin/bash mcp
+USER mcp
+
+EXPOSE 8021
+
+ENV PORT=8021
+ENV HOST="0.0.0.0"
+
+CMD ["uv", "run", "--frozen", "uvicorn", "server:app", "--host", "0.0.0.0", "--port", "8021"]
--- a/mcp/pyproject.toml
+++ b/mcp/pyproject.toml
@@ -10,4 +10,5 @@ dependencies = [
    "dotenv>=0.9.9",
    "fastmcp>=2.13.3",
    "psycopg[binary]>=3.3.2",
+    "uvicorn>=0.38.0",
 ]
--- a/mcp/search.py
+++ b/mcp/search.py
@@ -1,48 +0,0 @@
-from pgvector.psycopg import register_vector, Bit
-from psycopg.rows import dict_row
-from urllib.parse import unquote
-import anthropic
-import cohere
-import dotenv
-import datetime
-import json
-import minio
-import numpy as np
-import os
-import psycopg
-import time
-
-dotenv.load_dotenv('/home/paul/scripts-private/lech/cavepedia-v2/poller.env')
-
-COHERE_API_KEY = os.getenv('COHERE_API_KEY')
-
-co = cohere.ClientV2(COHERE_API_KEY)
-conn = psycopg.connect(
-    host='127.0.0.1',
-    port=4010,
-    dbname='cavepediav2_db',
-    user='cavepediav2_user',
-    password='cavepediav2_pw',
-    row_factory=dict_row,
-)
-
-def embed(text, input_type):
-    resp = co.embed(
-        texts=[text],
-        model='embed-v4.0',
-        input_type=input_type,
-        embedding_types=['float'],
-    )
-    return resp.embeddings.float[0]
-
-def search():
-    query = 'links trip with not more than 2 people'
-    query_embedding = embed(query, 'search_query')
-
-    rows = conn.execute('SELECT * FROM embeddings ORDER BY embedding <=> %s::vector LIMIT 5', (query_embedding,)).fetchall()
-    for row in rows:
-        print(row['bucket'])
-        print(row['key'])
-
-if __name__ == '__main__':
-    search()
--- a/mcp/server.py
+++ b/mcp/server.py
@@ -35,12 +35,18 @@ mcp = FastMCP("Cavepedia MCP")
 def get_user_roles() -> list[str]:
    """Extract user roles from the X-User-Roles header."""
    headers = get_http_headers()
+    print(f"DEBUG: All headers received: {dict(headers)}")
    roles_header = headers.get("x-user-roles", "")
+    print(f"DEBUG: x-user-roles header value: '{roles_header}'")
    if roles_header:
        try:
-            return json.loads(roles_header)
-        except json.JSONDecodeError:
+            roles = json.loads(roles_header)
+            print(f"DEBUG: Parsed roles: {roles}")
+            return roles
+        except json.JSONDecodeError as e:
+            print(f"DEBUG: JSON decode error: {e}")
            return []
+    print("DEBUG: No roles header found, returning empty list")
    return []

 def embed(text, input_type):
@@ -106,5 +112,14 @@ def get_user_info() -> dict:
        "roles": roles,
    }

+from starlette.responses import JSONResponse
+from starlette.routing import Route
+
+async def health(request):
+    return JSONResponse({"status": "ok"})
+
+app = mcp.http_app()
+app.routes.append(Route("/health", health))
+
 if __name__ == "__main__":
    mcp.run(transport='http', host='::1', port=9031)
--- a/mcp/uv.lock
+++ b/mcp/uv.lock
@@ -82,6 +82,7 @@ dependencies = [
    { name = "dotenv" },
    { name = "fastmcp" },
    { name = "psycopg", extra = ["binary"] },
+    { name = "uvicorn" },
 ]

 [package.metadata]
@@ -91,6 +92,7 @@ requires-dist = [
    { name = "dotenv", specifier = ">=0.9.9" },
    { name = "fastmcp", specifier = ">=2.13.3" },
    { name = "psycopg", extras = ["binary"], specifier = ">=3.3.2" },
+    { name = "uvicorn", specifier = ">=0.38.0" },
 ]

 [[package]]
--- a/poller/main.py
+++ b/poller/main.py
@@ -137,7 +137,7 @@ def split_files():
        key = row["key"]

        with conn.cursor() as cur:
-            logger.info(f"SPLITTING bucket: {bucket}, key: {key}")
+            logger.info(f"Splitting bucket: {bucket}, key: {key}")

            ##### get pdf #####
            s3.download_file(bucket, key, "/tmp/file.pdf")
@@ -146,6 +146,10 @@ def split_files():
            with open("/tmp/file.pdf", "rb") as f:
                reader = PdfReader(f)

+                # Handle PDFs with permission restrictions (no password, but encrypted)
+                if reader.is_encrypted:
+                    reader.decrypt("")
+
                for i in range(len(reader.pages)):
                    writer = PdfWriter()
                    writer.add_page(reader.pages[i])
@@ -351,6 +355,23 @@ def fix_pages():
        i -= 1


+def upload_file_list():
+    """Upload a list of all processed files to S3"""
+    BUCKET_PUBLIC = "cavepediav2-public"
+
+    rows = conn.execute("SELECT key FROM metadata WHERE split = true ORDER BY key")
+    files = [row["key"] for row in rows]
+
+    content = "\n".join(files)
+    s3.put_object(
+        Bucket=BUCKET_PUBLIC,
+        Key="files.txt",
+        Body=content.encode("utf-8"),
+        ContentType="text/plain",
+    )
+    logger.info(f"Uploaded file list with {len(files)} files to s3://{BUCKET_PUBLIC}/files.txt")
+
+
 if __name__ == "__main__":
    create_tables()

@@ -360,6 +381,7 @@ if __name__ == "__main__":
        check_batches()
        ocr_main()
        embeddings_main()
+        upload_file_list()

        logger.info("sleeping 5 minutes")
        time.sleep(5 * 60)
--- a/poller/pyproject.toml
+++ b/poller/pyproject.toml
@@ -8,6 +8,7 @@ dependencies = [
    "anthropic>=0.52.0",
    "boto3>=1.42.4",
    "cohere>=5.15.0",
+    "cryptography>=3.1",
    "pgvector>=0.4.1",
    "psycopg[binary]>=3.2.9",
    "pypdf>=5.5.0",
--- a/poller/uv.lock
+++ b/poller/uv.lock
@@ -1,6 +1,6 @@
 version = 1
 revision = 3
-requires-python = ">=3.13"
+requires-python = "==3.13.*"

 [[package]]
 name = "annotated-types"
@@ -79,6 +79,29 @@ wheels = [
    { url = "https://files.pythonhosted.org/packages/4a/7e/3db2bd1b1f9e95f7cddca6d6e75e2f2bd9f51b1246e546d88addca0106bd/certifi-2025.4.26-py3-none-any.whl", hash = "sha256:30350364dfe371162649852c63336a15c70c6510c2ad5015b21c2345311805f3", size = 159618, upload-time = "2025-04-26T02:12:27.662Z" },
 ]

+[[package]]
+name = "cffi"
+version = "2.0.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "pycparser", marker = "implementation_name != 'PyPy'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/eb/56/b1ba7935a17738ae8453301356628e8147c79dbb825bcbc73dc7401f9846/cffi-2.0.0.tar.gz", hash = "sha256:44d1b5909021139fe36001ae048dbdde8214afa20200eda0f64c068cac5d5529", size = 523588, upload-time = "2025-09-08T23:24:04.541Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/4b/8d/a0a47a0c9e413a658623d014e91e74a50cdd2c423f7ccfd44086ef767f90/cffi-2.0.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb", size = 185230, upload-time = "2025-09-08T23:23:00.879Z" },
+    { url = "https://files.pythonhosted.org/packages/4a/d2/a6c0296814556c68ee32009d9c2ad4f85f2707cdecfd7727951ec228005d/cffi-2.0.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:45d5e886156860dc35862657e1494b9bae8dfa63bf56796f2fb56e1679fc0bca", size = 181043, upload-time = "2025-09-08T23:23:02.231Z" },
+    { url = "https://files.pythonhosted.org/packages/b0/1e/d22cc63332bd59b06481ceaac49d6c507598642e2230f201649058a7e704/cffi-2.0.0-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:07b271772c100085dd28b74fa0cd81c8fb1a3ba18b21e03d7c27f3436a10606b", size = 212446, upload-time = "2025-09-08T23:23:03.472Z" },
+    { url = "https://files.pythonhosted.org/packages/a9/f5/a2c23eb03b61a0b8747f211eb716446c826ad66818ddc7810cc2cc19b3f2/cffi-2.0.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:d48a880098c96020b02d5a1f7d9251308510ce8858940e6fa99ece33f610838b", size = 220101, upload-time = "2025-09-08T23:23:04.792Z" },
+    { url = "https://files.pythonhosted.org/packages/f2/7f/e6647792fc5850d634695bc0e6ab4111ae88e89981d35ac269956605feba/cffi-2.0.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:f93fd8e5c8c0a4aa1f424d6173f14a892044054871c771f8566e4008eaa359d2", size = 207948, upload-time = "2025-09-08T23:23:06.127Z" },
+    { url = "https://files.pythonhosted.org/packages/cb/1e/a5a1bd6f1fb30f22573f76533de12a00bf274abcdc55c8edab639078abb6/cffi-2.0.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:dd4f05f54a52fb558f1ba9f528228066954fee3ebe629fc1660d874d040ae5a3", size = 206422, upload-time = "2025-09-08T23:23:07.753Z" },
+    { url = "https://files.pythonhosted.org/packages/98/df/0a1755e750013a2081e863e7cd37e0cdd02664372c754e5560099eb7aa44/cffi-2.0.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c8d3b5532fc71b7a77c09192b4a5a200ea992702734a2e9279a37f2478236f26", size = 219499, upload-time = "2025-09-08T23:23:09.648Z" },
+    { url = "https://files.pythonhosted.org/packages/50/e1/a969e687fcf9ea58e6e2a928ad5e2dd88cc12f6f0ab477e9971f2309b57c/cffi-2.0.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:d9b29c1f0ae438d5ee9acb31cadee00a58c46cc9c0b2f9038c6b0b3470877a8c", size = 222928, upload-time = "2025-09-08T23:23:10.928Z" },
+    { url = "https://files.pythonhosted.org/packages/36/54/0362578dd2c9e557a28ac77698ed67323ed5b9775ca9d3fe73fe191bb5d8/cffi-2.0.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:6d50360be4546678fc1b79ffe7a66265e28667840010348dd69a314145807a1b", size = 221302, upload-time = "2025-09-08T23:23:12.42Z" },
+    { url = "https://files.pythonhosted.org/packages/eb/6d/bf9bda840d5f1dfdbf0feca87fbdb64a918a69bca42cfa0ba7b137c48cb8/cffi-2.0.0-cp313-cp313-win32.whl", hash = "sha256:74a03b9698e198d47562765773b4a8309919089150a0bb17d829ad7b44b60d27", size = 172909, upload-time = "2025-09-08T23:23:14.32Z" },
+    { url = "https://files.pythonhosted.org/packages/37/18/6519e1ee6f5a1e579e04b9ddb6f1676c17368a7aba48299c3759bbc3c8b3/cffi-2.0.0-cp313-cp313-win_amd64.whl", hash = "sha256:19f705ada2530c1167abacb171925dd886168931e0a7b78f5bffcae5c6b5be75", size = 183402, upload-time = "2025-09-08T23:23:15.535Z" },
+    { url = "https://files.pythonhosted.org/packages/cb/0e/02ceeec9a7d6ee63bb596121c2c8e9b3a9e150936f4fbef6ca1943e6137c/cffi-2.0.0-cp313-cp313-win_arm64.whl", hash = "sha256:256f80b80ca3853f90c21b23ee78cd008713787b1b1e93eae9f3d6a7134abd91", size = 177780, upload-time = "2025-09-08T23:23:16.761Z" },
+]
+
 [[package]]
 name = "charset-normalizer"
 version = "3.4.2"
@@ -130,6 +153,47 @@ wheels = [
    { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
 ]

+[[package]]
+name = "cryptography"
+version = "46.0.3"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "cffi", marker = "platform_python_implementation != 'PyPy'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/9f/33/c00162f49c0e2fe8064a62cb92b93e50c74a72bc370ab92f86112b33ff62/cryptography-46.0.3.tar.gz", hash = "sha256:a8b17438104fed022ce745b362294d9ce35b4c2e45c1d958ad4a4b019285f4a1", size = 749258, upload-time = "2025-10-15T23:18:31.74Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/1d/42/9c391dd801d6cf0d561b5890549d4b27bafcc53b39c31a817e69d87c625b/cryptography-46.0.3-cp311-abi3-macosx_10_9_universal2.whl", hash = "sha256:109d4ddfadf17e8e7779c39f9b18111a09efb969a301a31e987416a0191ed93a", size = 7225004, upload-time = "2025-10-15T23:16:52.239Z" },
+    { url = "https://files.pythonhosted.org/packages/1c/67/38769ca6b65f07461eb200e85fc1639b438bdc667be02cf7f2cd6a64601c/cryptography-46.0.3-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:09859af8466b69bc3c27bdf4f5d84a665e0f7ab5088412e9e2ec49758eca5cbc", size = 4296667, upload-time = "2025-10-15T23:16:54.369Z" },
+    { url = "https://files.pythonhosted.org/packages/5c/49/498c86566a1d80e978b42f0d702795f69887005548c041636df6ae1ca64c/cryptography-46.0.3-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:01ca9ff2885f3acc98c29f1860552e37f6d7c7d013d7334ff2a9de43a449315d", size = 4450807, upload-time = "2025-10-15T23:16:56.414Z" },
+    { url = "https://files.pythonhosted.org/packages/4b/0a/863a3604112174c8624a2ac3c038662d9e59970c7f926acdcfaed8d61142/cryptography-46.0.3-cp311-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:6eae65d4c3d33da080cff9c4ab1f711b15c1d9760809dad6ea763f3812d254cb", size = 4299615, upload-time = "2025-10-15T23:16:58.442Z" },
+    { url = "https://files.pythonhosted.org/packages/64/02/b73a533f6b64a69f3cd3872acb6ebc12aef924d8d103133bb3ea750dc703/cryptography-46.0.3-cp311-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:e5bf0ed4490068a2e72ac03d786693adeb909981cc596425d09032d372bcc849", size = 4016800, upload-time = "2025-10-15T23:17:00.378Z" },
+    { url = "https://files.pythonhosted.org/packages/25/d5/16e41afbfa450cde85a3b7ec599bebefaef16b5c6ba4ec49a3532336ed72/cryptography-46.0.3-cp311-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:5ecfccd2329e37e9b7112a888e76d9feca2347f12f37918facbb893d7bb88ee8", size = 4984707, upload-time = "2025-10-15T23:17:01.98Z" },
+    { url = "https://files.pythonhosted.org/packages/c9/56/e7e69b427c3878352c2fb9b450bd0e19ed552753491d39d7d0a2f5226d41/cryptography-46.0.3-cp311-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:a2c0cd47381a3229c403062f764160d57d4d175e022c1df84e168c6251a22eec", size = 4482541, upload-time = "2025-10-15T23:17:04.078Z" },
+    { url = "https://files.pythonhosted.org/packages/78/f6/50736d40d97e8483172f1bb6e698895b92a223dba513b0ca6f06b2365339/cryptography-46.0.3-cp311-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:549e234ff32571b1f4076ac269fcce7a808d3bf98b76c8dd560e42dbc66d7d91", size = 4299464, upload-time = "2025-10-15T23:17:05.483Z" },
+    { url = "https://files.pythonhosted.org/packages/00/de/d8e26b1a855f19d9994a19c702fa2e93b0456beccbcfe437eda00e0701f2/cryptography-46.0.3-cp311-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:c0a7bb1a68a5d3471880e264621346c48665b3bf1c3759d682fc0864c540bd9e", size = 4950838, upload-time = "2025-10-15T23:17:07.425Z" },
+    { url = "https://files.pythonhosted.org/packages/8f/29/798fc4ec461a1c9e9f735f2fc58741b0daae30688f41b2497dcbc9ed1355/cryptography-46.0.3-cp311-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:10b01676fc208c3e6feeb25a8b83d81767e8059e1fe86e1dc62d10a3018fa926", size = 4481596, upload-time = "2025-10-15T23:17:09.343Z" },
+    { url = "https://files.pythonhosted.org/packages/15/8d/03cd48b20a573adfff7652b76271078e3045b9f49387920e7f1f631d125e/cryptography-46.0.3-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:0abf1ffd6e57c67e92af68330d05760b7b7efb243aab8377e583284dbab72c71", size = 4426782, upload-time = "2025-10-15T23:17:11.22Z" },
+    { url = "https://files.pythonhosted.org/packages/fa/b1/ebacbfe53317d55cf33165bda24c86523497a6881f339f9aae5c2e13e57b/cryptography-46.0.3-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:a04bee9ab6a4da801eb9b51f1b708a1b5b5c9eb48c03f74198464c66f0d344ac", size = 4698381, upload-time = "2025-10-15T23:17:12.829Z" },
+    { url = "https://files.pythonhosted.org/packages/96/92/8a6a9525893325fc057a01f654d7efc2c64b9de90413adcf605a85744ff4/cryptography-46.0.3-cp311-abi3-win32.whl", hash = "sha256:f260d0d41e9b4da1ed1e0f1ce571f97fe370b152ab18778e9e8f67d6af432018", size = 3055988, upload-time = "2025-10-15T23:17:14.65Z" },
+    { url = "https://files.pythonhosted.org/packages/7e/bf/80fbf45253ea585a1e492a6a17efcb93467701fa79e71550a430c5e60df0/cryptography-46.0.3-cp311-abi3-win_amd64.whl", hash = "sha256:a9a3008438615669153eb86b26b61e09993921ebdd75385ddd748702c5adfddb", size = 3514451, upload-time = "2025-10-15T23:17:16.142Z" },
+    { url = "https://files.pythonhosted.org/packages/2e/af/9b302da4c87b0beb9db4e756386a7c6c5b8003cd0e742277888d352ae91d/cryptography-46.0.3-cp311-abi3-win_arm64.whl", hash = "sha256:5d7f93296ee28f68447397bf5198428c9aeeab45705a55d53a6343455dcb2c3c", size = 2928007, upload-time = "2025-10-15T23:17:18.04Z" },
+    { url = "https://files.pythonhosted.org/packages/fd/23/45fe7f376a7df8daf6da3556603b36f53475a99ce4faacb6ba2cf3d82021/cryptography-46.0.3-cp38-abi3-macosx_10_9_universal2.whl", hash = "sha256:cb3d760a6117f621261d662bccc8ef5bc32ca673e037c83fbe565324f5c46936", size = 7218248, upload-time = "2025-10-15T23:17:46.294Z" },
+    { url = "https://files.pythonhosted.org/packages/27/32/b68d27471372737054cbd34c84981f9edbc24fe67ca225d389799614e27f/cryptography-46.0.3-cp38-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:4b7387121ac7d15e550f5cb4a43aef2559ed759c35df7336c402bb8275ac9683", size = 4294089, upload-time = "2025-10-15T23:17:48.269Z" },
+    { url = "https://files.pythonhosted.org/packages/26/42/fa8389d4478368743e24e61eea78846a0006caffaf72ea24a15159215a14/cryptography-46.0.3-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:15ab9b093e8f09daab0f2159bb7e47532596075139dd74365da52ecc9cb46c5d", size = 4440029, upload-time = "2025-10-15T23:17:49.837Z" },
+    { url = "https://files.pythonhosted.org/packages/5f/eb/f483db0ec5ac040824f269e93dd2bd8a21ecd1027e77ad7bdf6914f2fd80/cryptography-46.0.3-cp38-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:46acf53b40ea38f9c6c229599a4a13f0d46a6c3fa9ef19fc1a124d62e338dfa0", size = 4297222, upload-time = "2025-10-15T23:17:51.357Z" },
+    { url = "https://files.pythonhosted.org/packages/fd/cf/da9502c4e1912cb1da3807ea3618a6829bee8207456fbbeebc361ec38ba3/cryptography-46.0.3-cp38-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:10ca84c4668d066a9878890047f03546f3ae0a6b8b39b697457b7757aaf18dbc", size = 4012280, upload-time = "2025-10-15T23:17:52.964Z" },
+    { url = "https://files.pythonhosted.org/packages/6b/8f/9adb86b93330e0df8b3dcf03eae67c33ba89958fc2e03862ef1ac2b42465/cryptography-46.0.3-cp38-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:36e627112085bb3b81b19fed209c05ce2a52ee8b15d161b7c643a7d5a88491f3", size = 4978958, upload-time = "2025-10-15T23:17:54.965Z" },
+    { url = "https://files.pythonhosted.org/packages/d1/a0/5fa77988289c34bdb9f913f5606ecc9ada1adb5ae870bd0d1054a7021cc4/cryptography-46.0.3-cp38-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:1000713389b75c449a6e979ffc7dcc8ac90b437048766cef052d4d30b8220971", size = 4473714, upload-time = "2025-10-15T23:17:56.754Z" },
+    { url = "https://files.pythonhosted.org/packages/14/e5/fc82d72a58d41c393697aa18c9abe5ae1214ff6f2a5c18ac470f92777895/cryptography-46.0.3-cp38-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:b02cf04496f6576afffef5ddd04a0cb7d49cf6be16a9059d793a30b035f6b6ac", size = 4296970, upload-time = "2025-10-15T23:17:58.588Z" },
+    { url = "https://files.pythonhosted.org/packages/78/06/5663ed35438d0b09056973994f1aec467492b33bd31da36e468b01ec1097/cryptography-46.0.3-cp38-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:71e842ec9bc7abf543b47cf86b9a743baa95f4677d22baa4c7d5c69e49e9bc04", size = 4940236, upload-time = "2025-10-15T23:18:00.897Z" },
+    { url = "https://files.pythonhosted.org/packages/fc/59/873633f3f2dcd8a053b8dd1d38f783043b5fce589c0f6988bf55ef57e43e/cryptography-46.0.3-cp38-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:402b58fc32614f00980b66d6e56a5b4118e6cb362ae8f3fda141ba4689bd4506", size = 4472642, upload-time = "2025-10-15T23:18:02.749Z" },
+    { url = "https://files.pythonhosted.org/packages/3d/39/8e71f3930e40f6877737d6f69248cf74d4e34b886a3967d32f919cc50d3b/cryptography-46.0.3-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:ef639cb3372f69ec44915fafcd6698b6cc78fbe0c2ea41be867f6ed612811963", size = 4423126, upload-time = "2025-10-15T23:18:04.85Z" },
+    { url = "https://files.pythonhosted.org/packages/cd/c7/f65027c2810e14c3e7268353b1681932b87e5a48e65505d8cc17c99e36ae/cryptography-46.0.3-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:3b51b8ca4f1c6453d8829e1eb7299499ca7f313900dd4d89a24b8b87c0a780d4", size = 4686573, upload-time = "2025-10-15T23:18:06.908Z" },
+    { url = "https://files.pythonhosted.org/packages/0a/6e/1c8331ddf91ca4730ab3086a0f1be19c65510a33b5a441cb334e7a2d2560/cryptography-46.0.3-cp38-abi3-win32.whl", hash = "sha256:6276eb85ef938dc035d59b87c8a7dc559a232f954962520137529d77b18ff1df", size = 3036695, upload-time = "2025-10-15T23:18:08.672Z" },
+    { url = "https://files.pythonhosted.org/packages/90/45/b0d691df20633eff80955a0fc7695ff9051ffce8b69741444bd9ed7bd0db/cryptography-46.0.3-cp38-abi3-win_amd64.whl", hash = "sha256:416260257577718c05135c55958b674000baef9a1c7d9e8f306ec60d71db850f", size = 3501720, upload-time = "2025-10-15T23:18:10.632Z" },
+    { url = "https://files.pythonhosted.org/packages/e8/cb/2da4cc83f5edb9c3257d09e1e7ab7b23f049c7962cae8d842bbef0a9cec9/cryptography-46.0.3-cp38-abi3-win_arm64.whl", hash = "sha256:d89c3468de4cdc4f08a57e214384d0471911a3830fcdaf7a8cc587e42a866372", size = 2918740, upload-time = "2025-10-15T23:18:12.277Z" },
+]
+
 [[package]]
 name = "distro"
 version = "1.9.0"
@@ -286,19 +350,6 @@ wheels = [
    { url = "https://files.pythonhosted.org/packages/54/46/caa2c1342655f57d8f0f2519774c6d67132205909c65e9aa8255e1d7b4f4/jiter-0.10.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:28ed2a4c05a1f32ef0e1d24c2611330219fed727dae01789f4a335617634b1ca", size = 318225, upload-time = "2025-05-18T19:04:20.583Z" },
    { url = "https://files.pythonhosted.org/packages/43/84/c7d44c75767e18946219ba2d703a5a32ab37b0bc21886a97bc6062e4da42/jiter-0.10.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:14a4c418b1ec86a195f1ca69da8b23e8926c752b685af665ce30777233dfe070", size = 350235, upload-time = "2025-05-18T19:04:22.363Z" },
    { url = "https://files.pythonhosted.org/packages/01/16/f5a0135ccd968b480daad0e6ab34b0c7c5ba3bc447e5088152696140dcb3/jiter-0.10.0-cp313-cp313t-win_amd64.whl", hash = "sha256:d7bfed2fe1fe0e4dda6ef682cee888ba444b21e7a6553e03252e4feb6cf0adca", size = 207278, upload-time = "2025-05-18T19:04:23.627Z" },
-    { url = "https://files.pythonhosted.org/packages/1c/9b/1d646da42c3de6c2188fdaa15bce8ecb22b635904fc68be025e21249ba44/jiter-0.10.0-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:5e9251a5e83fab8d87799d3e1a46cb4b7f2919b895c6f4483629ed2446f66522", size = 310866, upload-time = "2025-05-18T19:04:24.891Z" },
-    { url = "https://files.pythonhosted.org/packages/ad/0e/26538b158e8a7c7987e94e7aeb2999e2e82b1f9d2e1f6e9874ddf71ebda0/jiter-0.10.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:023aa0204126fe5b87ccbcd75c8a0d0261b9abdbbf46d55e7ae9f8e22424eeb8", size = 318772, upload-time = "2025-05-18T19:04:26.161Z" },
-    { url = "https://files.pythonhosted.org/packages/7b/fb/d302893151caa1c2636d6574d213e4b34e31fd077af6050a9c5cbb42f6fb/jiter-0.10.0-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3c189c4f1779c05f75fc17c0c1267594ed918996a231593a21a5ca5438445216", size = 344534, upload-time = "2025-05-18T19:04:27.495Z" },
-    { url = "https://files.pythonhosted.org/packages/01/d8/5780b64a149d74e347c5128d82176eb1e3241b1391ac07935693466d6219/jiter-0.10.0-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:15720084d90d1098ca0229352607cd68256c76991f6b374af96f36920eae13c4", size = 369087, upload-time = "2025-05-18T19:04:28.896Z" },
-    { url = "https://files.pythonhosted.org/packages/e8/5b/f235a1437445160e777544f3ade57544daf96ba7e96c1a5b24a6f7ac7004/jiter-0.10.0-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:e4f2fb68e5f1cfee30e2b2a09549a00683e0fde4c6a2ab88c94072fc33cb7426", size = 490694, upload-time = "2025-05-18T19:04:30.183Z" },
-    { url = "https://files.pythonhosted.org/packages/85/a9/9c3d4617caa2ff89cf61b41e83820c27ebb3f7b5fae8a72901e8cd6ff9be/jiter-0.10.0-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:ce541693355fc6da424c08b7edf39a2895f58d6ea17d92cc2b168d20907dee12", size = 388992, upload-time = "2025-05-18T19:04:32.028Z" },
-    { url = "https://files.pythonhosted.org/packages/68/b1/344fd14049ba5c94526540af7eb661871f9c54d5f5601ff41a959b9a0bbd/jiter-0.10.0-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:31c50c40272e189d50006ad5c73883caabb73d4e9748a688b216e85a9a9ca3b9", size = 351723, upload-time = "2025-05-18T19:04:33.467Z" },
-    { url = "https://files.pythonhosted.org/packages/41/89/4c0e345041186f82a31aee7b9d4219a910df672b9fef26f129f0cda07a29/jiter-0.10.0-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:fa3402a2ff9815960e0372a47b75c76979d74402448509ccd49a275fa983ef8a", size = 392215, upload-time = "2025-05-18T19:04:34.827Z" },
-    { url = "https://files.pythonhosted.org/packages/55/58/ee607863e18d3f895feb802154a2177d7e823a7103f000df182e0f718b38/jiter-0.10.0-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:1956f934dca32d7bb647ea21d06d93ca40868b505c228556d3373cbd255ce853", size = 522762, upload-time = "2025-05-18T19:04:36.19Z" },
-    { url = "https://files.pythonhosted.org/packages/15/d0/9123fb41825490d16929e73c212de9a42913d68324a8ce3c8476cae7ac9d/jiter-0.10.0-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:fcedb049bdfc555e261d6f65a6abe1d5ad68825b7202ccb9692636c70fcced86", size = 513427, upload-time = "2025-05-18T19:04:37.544Z" },
-    { url = "https://files.pythonhosted.org/packages/d8/b3/2bd02071c5a2430d0b70403a34411fc519c2f227da7b03da9ba6a956f931/jiter-0.10.0-cp314-cp314-win32.whl", hash = "sha256:ac509f7eccca54b2a29daeb516fb95b6f0bd0d0d8084efaf8ed5dfc7b9f0b357", size = 210127, upload-time = "2025-05-18T19:04:38.837Z" },
-    { url = "https://files.pythonhosted.org/packages/03/0c/5fe86614ea050c3ecd728ab4035534387cd41e7c1855ef6c031f1ca93e3f/jiter-0.10.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:5ed975b83a2b8639356151cef5c0d597c68376fc4922b45d0eb384ac058cfa00", size = 318527, upload-time = "2025-05-18T19:04:40.612Z" },
-    { url = "https://files.pythonhosted.org/packages/b3/4a/4175a563579e884192ba6e81725fc0448b042024419be8d83aa8a80a3f44/jiter-0.10.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3aa96f2abba33dc77f79b4cf791840230375f9534e5fac927ccceb58c5e604a5", size = 354213, upload-time = "2025-05-18T19:04:41.894Z" },
 ]

 [[package]]
@@ -395,6 +446,7 @@ dependencies = [
    { name = "anthropic" },
    { name = "boto3" },
    { name = "cohere" },
+    { name = "cryptography" },
    { name = "pgvector" },
    { name = "psycopg", extra = ["binary"] },
    { name = "pypdf" },
@@ -414,6 +466,7 @@ requires-dist = [
    { name = "anthropic", specifier = ">=0.52.0" },
    { name = "boto3", specifier = ">=1.42.4" },
    { name = "cohere", specifier = ">=5.15.0" },
+    { name = "cryptography", specifier = ">=3.1" },
    { name = "mypy", marker = "extra == 'dev'", specifier = ">=1.15.0" },
    { name = "pgvector", specifier = ">=0.4.1" },
    { name = "psycopg", extras = ["binary"], specifier = ">=3.2.9" },
@@ -460,6 +513,15 @@ wheels = [
    { url = "https://files.pythonhosted.org/packages/7b/1d/bf54cfec79377929da600c16114f0da77a5f1670f45e0c3af9fcd36879bc/psycopg_binary-3.2.9-cp313-cp313-win_amd64.whl", hash = "sha256:2290bc146a1b6a9730350f695e8b670e1d1feb8446597bed0bbe7c3c30e0abcb", size = 2928009, upload-time = "2025-05-13T16:08:53.67Z" },
 ]

+[[package]]
+name = "pycparser"
+version = "2.23"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/fe/cf/d2d3b9f5699fb1e4615c8e32ff220203e43b248e1dfcc6736ad9057731ca/pycparser-2.23.tar.gz", hash = "sha256:78816d4f24add8f10a06d6f05b4d424ad9e96cfebf68a4ddc99c65c0720d00c2", size = 173734, upload-time = "2025-09-09T13:23:47.91Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/a0/e3/59cd50310fc9b59512193629e1984c1f95e5c8ae6e5d8c69532ccc65a7fe/pycparser-2.23-py3-none-any.whl", hash = "sha256:e5c6e8d3fbad53479cab09ac03729e0a9faf2bee3db8208a550daf5af81a5934", size = 118140, upload-time = "2025-09-09T13:23:46.651Z" },
+]
+
 [[package]]
 name = "pydantic"
 version = "2.11.5"
--- a/web/.dockerignore
+++ b/web/.dockerignore
@@ -1,9 +1,23 @@
-Dockerfile
-.dockerignore
+# Dependencies
 node_modules
+agent/.venv
+
+# Build outputs
 .next
+out
+
+# Environment files
+.env*
+!.env.example
+
+# Git
 .git
 .gitignore
+
+# IDE
+.vscode
+.idea
+
+# Misc
+*.log
 *.md
-.env*.local
-agent/
--- a/web/.env.example
+++ b/web/.env.example
@@ -0,0 +1,9 @@
+# Auth0 Configuration
+AUTH0_SECRET=use-openssl-rand-hex-32-to-generate
+AUTH0_DOMAIN=your-tenant.auth0.com
+AUTH0_CLIENT_ID=your-client-id
+AUTH0_CLIENT_SECRET=your-client-secret
+APP_BASE_URL=https://your-domain.com
+
+# Optional: Agent URL (defaults to http://localhost:8000/)
+# AGENT_URL=http://localhost:8000/
--- a/web/.gitignore
+++ b/web/.gitignore
@@ -30,8 +30,11 @@ yarn-debug.log*
 yarn-error.log*
 .pnpm-debug.log*

-# env files (can opt-in for committing if needed)
-.env*
+# env files
+.env
+.env.local
+.env.*.local
+!.env.example

 # vercel
 .vercel
@@ -40,5 +43,15 @@ yarn-error.log*
 *.tsbuildinfo
 next-env.d.ts

-# LangGraph API
-.langgraph_api
+.mastra/
+
+# lock files
+package-lock.json
+yarn.lock
+pnpm-lock.yaml
+bun.lockb
+
+# python
+venv
+.venv
+__pycache__
--- a/web/Dockerfile
+++ b/web/Dockerfile
@@ -5,30 +5,38 @@ FROM base AS deps
 RUN apk add --no-cache libc6-compat
 WORKDIR /app

+# Install dependencies
 COPY package.json package-lock.json* ./
 RUN npm ci --ignore-scripts

-# Build the application
+# Rebuild the source code only when needed
 FROM base AS builder
 WORKDIR /app
 COPY --from=deps /app/node_modules ./node_modules
 COPY . .

-# Skip agent installation in Docker
-ENV SKIP_AGENT_INSTALL=true
+# Disable telemetry during build
+ENV NEXT_TELEMETRY_DISABLED=1

 RUN npm run build

-# Production image
+# Production image, copy all the files and run next
 FROM base AS runner
 WORKDIR /app

 ENV NODE_ENV=production
+ENV NEXT_TELEMETRY_DISABLED=1

 RUN addgroup --system --gid 1001 nodejs
 RUN adduser --system --uid 1001 nextjs

 COPY --from=builder /app/public ./public
+
+# Set the correct permission for prerender cache
+RUN mkdir .next
+RUN chown nextjs:nodejs .next
+
+# Automatically leverage output traces to reduce image size
 COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
 COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

--- a/web/README.md
+++ b/web/README.md
@@ -1,119 +1,104 @@
-# Cavepedia Web
+# Cavepedia

-Next.js frontend with integrated PydanticAI agent for Cavepedia.
-
-## Project Structure
-
-```
-web/
-├── src/              # Next.js application
-├── agent/            # PydanticAI agent (Python)
-│   ├── main.py       # Agent definition
-│   ├── server.py     # FastAPI server with AG-UI
-│   └── pyproject.toml
-└── ...
-```
+A caving assistant built with [PydanticAI](https://ai.pydantic.dev/), [CopilotKit](https://copilotkit.ai), and Gemini.

 ## Prerequisites

+- Google API Key (for Gemini)
+- Auth0 account with application configured
+- Python 3.13+
+- uv
 - Node.js 24+
- Python 3.13
 - npm
- Google AI API Key (for the PydanticAI agent)
+
+## Environment Variables
+
+### Web App (.env)
+
+```
+AUTH0_SECRET=<random-secret>
+AUTH0_DOMAIN=<your-auth0-domain>
+AUTH0_CLIENT_ID=<your-client-id>
+AUTH0_CLIENT_SECRET=<your-client-secret>
+APP_BASE_URL=https://your-domain.com
+AGENT_URL=http://localhost:8000/
+```
+
+### Agent (agent/.env)
+
+```
+GOOGLE_API_KEY=<your-google-api-key>
+```

 ## Development

-### 1. Install dependencies
-
+1. Install dependencies:
 ```bash
 npm install
 ```

-This also installs the agent's Python dependencies via the `install:agent` script.
-
-### 2. Set up environment variables
-
-```bash
-# Agent environment
-cp agent/.env.example agent/.env
-# Edit agent/.env with your API keys
-```
-
-### 3. Start development servers
-
+2. Start the development server:
 ```bash
 npm run dev
 ```

-This starts both the Next.js UI and PydanticAI agent servers concurrently.
+This starts both the UI and agent servers concurrently.

-## Agent Deployment
+## Production

-The agent can be containerized for production deployment.
-
-### Environment Variables
-
-| Variable | Required | Description |
-|----------|----------|-------------|
-| `GOOGLE_API_KEY` | Yes | Google AI API key for Gemini |
-
-### Running in production
+### Docker Build

+**Web (Next.js):**
 ```bash
-cd agent
-uv run uvicorn server:app --host 0.0.0.0 --port 8000
+docker build -t cavepedia-web .
+docker run -p 3000:3000 \
+  -e AUTH0_SECRET=<secret> \
+  -e AUTH0_DOMAIN=<domain> \
+  -e AUTH0_CLIENT_ID=<client-id> \
+  -e AUTH0_CLIENT_SECRET=<client-secret> \
+  -e APP_BASE_URL=https://your-domain.com \
+  -e AGENT_URL=http://agent:8000/ \
+  cavepedia-web
 ```

-## Web Deployment
+**Agent (PydanticAI):**
+```bash
+cd agent
+docker build -t cavepedia-agent .
+docker run -p 8000:8000 \
+  -e GOOGLE_API_KEY=<api-key> \
+  cavepedia-agent
+```

-### Environment Variables
+### Without Docker

-| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
-| `LANGGRAPH_DEPLOYMENT_URL` | Yes | `http://localhost:8000` | URL to the agent |
-| `AUTH0_SECRET` | Yes | - | Session encryption key (`openssl rand -hex 32`) |
-| `AUTH0_DOMAIN` | Yes | - | Auth0 tenant domain |
-| `AUTH0_CLIENT_ID` | Yes | - | Auth0 application client ID |
-| `AUTH0_CLIENT_SECRET` | Yes | - | Auth0 application client secret |
-| `APP_BASE_URL` | Yes | - | Public URL of the app |
-
-### Docker Compose (Full Stack)
-
-```yaml
-services:
-  web:
-    image: git.seaturtle.pw/cavepedia/cavepediav2-web:latest
-    ports:
-      - "3000:3000"
-    environment:
-      LANGGRAPH_DEPLOYMENT_URL: http://agent:8000
-      AUTH0_SECRET: ${AUTH0_SECRET}
-      AUTH0_DOMAIN: ${AUTH0_DOMAIN}
-      AUTH0_CLIENT_ID: ${AUTH0_CLIENT_ID}
-      AUTH0_CLIENT_SECRET: ${AUTH0_CLIENT_SECRET}
-      APP_BASE_URL: ${APP_BASE_URL}
-    depends_on:
-      - agent
-
-  agent:
-    image: git.seaturtle.pw/cavepedia/cavepediav2-agent:latest
-    environment:
-      GOOGLE_API_KEY: ${GOOGLE_API_KEY}
+```bash
+npm run build
+npm run start:all
 ```

 ## Available Scripts

- `dev` - Start both UI and agent servers
- `dev:ui` - Start only Next.js
- `dev:agent` - Start only PydanticAI agent
- `build` - Build Next.js for production
- `start` - Start production server
- `lint` - Run ESLint
- `install:agent` - Install agent Python dependencies
+- `dev` - Starts both UI and agent servers in development mode
+- `dev:ui` - Starts only the Next.js UI server
+- `dev:agent` - Starts only the PydanticAI agent server
+- `build` - Builds the Next.js application for production
+- `start` - Starts the production Next.js server
+- `start:agent` - Starts the production agent server
+- `start:all` - Starts both servers in production mode
+- `lint` - Runs ESLint

-## References
+## Troubleshooting

- [PydanticAI Documentation](https://ai.pydantic.dev/)
- [CopilotKit Documentation](https://docs.copilotkit.ai)
- [Next.js Documentation](https://nextjs.org/docs)
- [Auth0 Next.js SDK Examples](https://github.com/auth0/nextjs-auth0/blob/main/EXAMPLES.md)
+### Agent Connection Issues
+If you see connection errors:
+1. Ensure the agent is running on port 8000
+2. Check that GOOGLE_API_KEY is set correctly
+3. Verify AGENT_URL has a trailing slash
+
+### Python Dependencies
+```bash
+cd agent
+uv sync
+uv run uvicorn src.main:app --host 127.0.0.1 --port 8000
+```
--- a/web/agent/.dockerignore
+++ b/web/agent/.dockerignore
@@ -1,14 +1,22 @@
-Dockerfile
-.dockerignore
+# Virtual environment
+.venv
+
+# Environment files
+.env
+!.env.example
+
+# Cache
+__pycache__
+*.pyc
+
+# Git
 .git
 .gitignore
-.env
-.env.*
-.venv/
-venv/
-__pycache__/
-*.pyc
-*.pyo
-.langgraph_api/
-.vercel/
+
+# IDE
+.vscode
+.idea
+
+# Misc
+*.log
 *.md
--- a/web/agent/.env.example
+++ b/web/agent/.env.example
@@ -0,0 +1,7 @@
+# Google Gemini API Key (required)
+GOOGLE_API_KEY=your-gemini-api-key
+
+# Optional settings
+# PORT=8000
+# HOST=127.0.0.1
+# DEBUG=false
--- a/web/agent/.gitignore
+++ b/web/agent/.gitignore
@@ -1,9 +0,0 @@
-venv/
-__pycache__/
-*.pyc
-.env
-.vercel
-
-# python
-.venv/
-.langgraph_api/
--- a/web/agent/.python-version
+++ b/web/agent/.python-version
@@ -0,0 +1 @@
+3.13
--- a/web/agent/Dockerfile
+++ b/web/agent/Dockerfile
@@ -1,21 +1,28 @@
-# syntax=docker/dockerfile:1
-
 FROM python:3.13-slim

-WORKDIR /app
-
 # Install uv
 COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/

+WORKDIR /app
+
+ENV PYTHONPATH=/app
+
 # Copy dependency files
 COPY pyproject.toml uv.lock ./

 # Install dependencies
-RUN uv sync --frozen --no-dev --no-install-project
+RUN uv sync --frozen --no-dev

 # Copy application code
-COPY main.py server.py ./
+COPY src ./src
+
+# Create non-root user
+RUN useradd --create-home --shell /bin/bash agent
+USER agent

 EXPOSE 8000

-CMD ["uv", "run", "python", "server.py"]
+ENV PORT=8000
+ENV HOST="0.0.0.0"
+
+CMD ["uv", "run", "--frozen", "uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8000"]
--- a/web/agent/main.py
+++ b/web/agent/main.py
@@ -1,26 +0,0 @@
-"""
-PydanticAI agent with MCP tools from Cavepedia server.
-"""
-
-from pydantic_ai import Agent
-from pydantic_ai.models.google import GoogleModel
-from pydantic_ai.mcp import MCPServerStreamableHTTP
-
-
-# Create MCP server connection to Cavepedia
-mcp_server = MCPServerStreamableHTTP(
-    url="https://mcp.caving.dev/mcp",
-    timeout=30.0,
-)
-
-# Create the agent with Google Gemini model
-agent = Agent(
-    model=GoogleModel("gemini-2.5-pro"),
-    toolsets=[mcp_server],
-    instructions="""You are a helpful assistant with access to cave-related information through the Cavepedia MCP server. You can help users find information about caves, caving techniques, and related topics.
-
-IMPORTANT RULES:
-1. Always cite your sources at the end of each response. List the specific sources/documents you used.
-2. If you cannot find information on a topic, say so clearly. Do NOT make up information or hallucinate facts.
-3. If the MCP tools return no results, acknowledge that you couldn't find the information rather than guessing.""",
-)
--- a/web/agent/pyproject.toml
+++ b/web/agent/pyproject.toml
@@ -1,11 +1,16 @@
 [project]
-name = "vpi-1000"
-version = "1.0.0"
-description = "VPI-1000"
-requires-python = ">=3.13,<3.14"
+name = "cavepedia-agent"
+version = "0.1.0"
+description = "Cavepedia AI Agent with MCP tools"
+requires-python = ">=3.13"
 dependencies = [
-    "pydantic-ai>=0.1.0",
-    "fastapi>=0.115.5,<1.0.0",
-    "uvicorn>=0.29.0,<1.0.0",
-    "python-dotenv>=1.0.0,<2.0.0",
+    "uvicorn",
+    "starlette",
+    "pydantic-ai",
+    "mcp",
+    "google-genai",
+    "google-cloud-aiplatform",
+    "ag-ui-protocol",
+    "python-dotenv",
+    "httpx",
 ]
--- a/web/agent/server.py
+++ b/web/agent/server.py
@@ -1,20 +0,0 @@
-"""
-Self-hosted PydanticAI agent server using AG-UI protocol.
-"""
-
-import os
-import uvicorn
-from dotenv import load_dotenv
-
-from pydantic_ai.ui.ag_ui.app import AGUIApp
-from main import agent
-
-load_dotenv()
-
-# Convert PydanticAI agent to ASGI app with AG-UI protocol
-app = AGUIApp(agent)
-
-
-if __name__ == "__main__":
-    port = int(os.getenv("PORT", "8000"))
-    uvicorn.run(app, host="0.0.0.0", port=port)
--- a/web/agent/src/agent.py
+++ b/web/agent/src/agent.py
@@ -0,0 +1,98 @@
+"""
+PydanticAI agent with MCP tools from Cavepedia server.
+"""
+
+import os
+import logging
+import httpx
+
+from pydantic_ai import Agent
+from pydantic_ai.models.google import GoogleModel
+from pydantic_ai.providers.google import GoogleProvider
+
+# Set up logging based on environment
+log_level = logging.DEBUG if os.getenv("DEBUG") else logging.INFO
+logging.basicConfig(
+    level=log_level,
+    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
+)
+logger = logging.getLogger(__name__)
+
+CAVE_MCP_URL = os.getenv("CAVE_MCP_URL", "https://mcp.caving.dev/mcp")
+
+logger.info("Initializing Cavepedia agent...")
+
+def check_mcp_available(url: str, timeout: float = 5.0) -> bool:
+    """Check if MCP server is reachable via health endpoint."""
+    try:
+        # Use the health endpoint instead of the MCP endpoint
+        health_url = url.rsplit("/", 1)[0] + "/health"
+        response = httpx.get(health_url, timeout=timeout, follow_redirects=True)
+        if response.status_code == 200:
+            return True
+        logger.warning(f"MCP health check returned {response.status_code}")
+        return False
+    except Exception as e:
+        logger.warning(f"MCP server not reachable: {e}")
+        return False
+
+# Check if MCP is available at startup
+MCP_AVAILABLE = check_mcp_available(CAVE_MCP_URL)
+logger.info(f"MCP server available: {MCP_AVAILABLE}")
+
+AGENT_INSTRUCTIONS = """You are a helpful caving assistant. Help users with all aspects of caving including cave exploration, safety, surveying techniques, cave locations, geology, equipment, history, conservation, and any other caving-related topics.
+
+IMPORTANT RULES:
+1. Always cite your sources at the end of each response when possible.
+2. If you're not certain about information, say so clearly. You may infer some information, but NOT make up information or hallucinate facts.
+3. Provide accurate, helpful, and safety-conscious information.
+4. You specialize in creating ascii art diagrams or maps.
+5. Never use sycophantic phrases like "you're absolutely right", "great question", or excessive praise. Be direct and professional."""
+
+
+def create_agent(user_roles: list[str] | None = None):
+    """Create an agent with MCP tools configured for the given user roles."""
+    toolsets = []
+
+    if MCP_AVAILABLE and user_roles:
+        try:
+            import json
+            from pydantic_ai.mcp import MCPServerStreamableHTTP
+
+            roles_header = json.dumps(user_roles)
+            logger.info(f"Creating MCP server with roles: {roles_header}")
+
+            mcp_server = MCPServerStreamableHTTP(
+                url=CAVE_MCP_URL,
+                headers={"x-user-roles": roles_header},
+                timeout=30.0,
+            )
+            toolsets.append(mcp_server)
+            logger.info(f"MCP server configured with roles: {user_roles}")
+        except Exception as e:
+            logger.warning(f"Could not configure MCP server: {e}")
+    elif not user_roles:
+        logger.info("No user roles provided - MCP tools disabled")
+    else:
+        logger.info("MCP server unavailable - running without MCP tools")
+
+    # Use Vertex AI for higher rate limits (requires GOOGLE_APPLICATION_CREDENTIALS)
+    # Note: gemini-3-pro-preview requires location="global"
+    provider = GoogleProvider(
+        vertexai=True,
+        project=os.getenv("GOOGLE_PROJECT_ID"),
+        location=os.getenv("GOOGLE_LOCATION", "global"),
+    )
+    model = GoogleModel("gemini-3-pro-preview", provider=provider)
+
+    return Agent(
+        model=model,
+        toolsets=toolsets if toolsets else None,
+        instructions=AGENT_INSTRUCTIONS,
+    )
+
+
+# Create a default agent for health checks etc
+agent = create_agent()
+
+logger.info("Agent module initialized successfully")
--- a/web/agent/src/main.py
+++ b/web/agent/src/main.py
@@ -0,0 +1,82 @@
+"""
+Self-hosted PydanticAI agent server using AG-UI protocol.
+"""
+
+import os
+import sys
+import json
+import logging
+from dotenv import load_dotenv
+
+# Load environment variables BEFORE importing agent
+load_dotenv()
+
+# Set up logging based on environment
+log_level = logging.DEBUG if os.getenv("DEBUG") else logging.INFO
+logging.basicConfig(
+    level=log_level,
+    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
+)
+logger = logging.getLogger(__name__)
+
+# Validate required environment variables (either API key or service account)
+if not os.getenv("GOOGLE_API_KEY") and not os.getenv("GOOGLE_APPLICATION_CREDENTIALS"):
+    logger.error("Either GOOGLE_API_KEY or GOOGLE_APPLICATION_CREDENTIALS is required")
+    sys.exit(1)
+
+import uvicorn
+from starlette.applications import Starlette
+from starlette.requests import Request
+from starlette.responses import Response, JSONResponse
+from starlette.routing import Route
+
+from pydantic_ai.ui.ag_ui import AGUIAdapter
+
+from src.agent import create_agent
+
+logger.info("Creating AG-UI app...")
+
+
+async def handle_agent_request(request: Request) -> Response:
+    """Handle incoming AG-UI requests with dynamic role-based MCP configuration."""
+    # Debug: log all incoming headers
+    logger.info(f"DEBUG: All request headers: {dict(request.headers)}")
+
+    # Extract user roles from request headers
+    roles_header = request.headers.get("x-user-roles", "")
+    logger.info(f"DEBUG: x-user-roles header value: '{roles_header}'")
+    user_roles = []
+
+    if roles_header:
+        try:
+            user_roles = json.loads(roles_header)
+            logger.info(f"Request received with roles: {user_roles}")
+        except json.JSONDecodeError as e:
+            logger.warning(f"Failed to parse x-user-roles header: {e}")
+
+    # Create agent with the user's roles
+    agent = create_agent(user_roles)
+
+    # Dispatch the request using AGUIAdapter
+    return await AGUIAdapter.dispatch_request(request, agent=agent)
+
+
+async def health(request: Request) -> Response:
+    """Health check endpoint."""
+    return JSONResponse({"status": "ok"})
+
+
+app = Starlette(
+    routes=[
+        Route("/", handle_agent_request, methods=["POST"]),
+        Route("/health", health, methods=["GET"]),
+    ],
+)
+
+logger.info("AG-UI app created successfully")
+
+
+if __name__ == "__main__":
+    port = int(os.getenv("PORT", "8000"))
+    host = os.getenv("HOST", "127.0.0.1")
+    uvicorn.run(app, host=host, port=port)
--- a/web/agent/uv.lock
+++ b/web/agent/uv.lock
--- a/web/eslint.config.mjs
+++ b/web/eslint.config.mjs
@@ -11,7 +11,6 @@ const eslintConfig = [
      "out/**",
      "build/**",
      "next-env.d.ts",
-      "agent",
    ],
  },
 ];
--- a/web/next.config.ts
+++ b/web/next.config.ts
@@ -1,7 +1,57 @@
 import type { NextConfig } from "next";

+const securityHeaders = [
+  {
+    key: "X-DNS-Prefetch-Control",
+    value: "on",
+  },
+  {
+    key: "Strict-Transport-Security",
+    value: "max-age=63072000; includeSubDomains; preload",
+  },
+  {
+    key: "X-Content-Type-Options",
+    value: "nosniff",
+  },
+  {
+    key: "X-Frame-Options",
+    value: "DENY",
+  },
+  {
+    key: "X-XSS-Protection",
+    value: "1; mode=block",
+  },
+  {
+    key: "Referrer-Policy",
+    value: "strict-origin-when-cross-origin",
+  },
+  {
+    key: "Permissions-Policy",
+    value: "camera=(), microphone=(), geolocation=()",
+  },
+];
+
 const nextConfig: NextConfig = {
+  serverExternalPackages: ["@copilotkit/runtime"],
+
+  // Enable standalone output for Docker
  output: "standalone",
+
+  async headers() {
+    return [
+      {
+        source: "/(.*)",
+        headers: securityHeaders,
+      },
+    ];
+  },
+
+  // Production optimizations
+  poweredByHeader: false,
+  reactStrictMode: true,
+
+  // Compress responses
+  compress: true,
 };

 export default nextConfig;
--- a/web/package-lock.json
+++ b/web/package-lock.json
--- a/web/package.json
+++ b/web/package.json
@@ -4,14 +4,16 @@
  "private": true,
  "scripts": {
    "dev": "concurrently \"npm run dev:ui\" \"npm run dev:agent\" --names ui,agent --prefix-colors blue,green --kill-others",
-    "dev:debug": "LOG_LEVEL=debug npm run dev",
-    "dev:agent": "cd agent && uv run uvicorn server:app --host 0.0.0.0 --port 8000 --reload",
+    "dev:agent": "./scripts/run-agent.sh",
    "dev:ui": "next dev --turbopack -H 127.0.0.1",
    "build": "next build --webpack",
-    "start": "next start -H 127.0.0.1",
+    "start": "next start",
+    "start:agent": "./scripts/start-agent.sh",
+    "start:all": "concurrently \"npm run start\" \"npm run start:agent\" --names ui,agent --prefix-colors blue,green",
    "lint": "eslint .",
-    "install:agent": "sh ./scripts/setup-agent.sh || scripts\\setup-agent.bat",
-    "postinstall": "if [ -z \"$SKIP_AGENT_INSTALL\" ]; then npm run install:agent; fi"
+    "typecheck": "tsc --noEmit",
+    "install:agent": "./scripts/setup-agent.sh",
+    "postinstall": "npm run install:agent"
  },
  "dependencies": {
    "@ag-ui/client": "^0.0.42",
--- a/web/scripts/run-agent.sh
+++ b/web/scripts/run-agent.sh
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+# Navigate to the agent directory
+cd "$(dirname "$0")/../agent" || exit 1
+
+# Run the agent using uvicorn with reload for development
+uv run uvicorn src.main:app --host 127.0.0.1 --port 8000 --reload
--- a/web/scripts/setup-agent.bat
+++ b/web/scripts/setup-agent.bat
@@ -1,6 +0,0 @@
-@echo off
-REM Navigate to the agent directory
-cd /d "%~dp0\..\agent" || exit /b 1
-
-REM Install dependencies using uv
-uv sync 
--- a/web/scripts/setup-agent.sh
+++ b/web/scripts/setup-agent.sh
@@ -3,5 +3,5 @@
 # Navigate to the agent directory
 cd "$(dirname "$0")/../agent" || exit 1

-# Install dependencies using uv
+# Install dependencies and create virtual environment using uv
 uv sync
--- a/web/scripts/start-agent.sh
+++ b/web/scripts/start-agent.sh
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+# Navigate to the agent directory
+cd "$(dirname "$0")/../agent" || exit 1
+
+# Run the agent in production mode
+uv run uvicorn src.main:app --host 127.0.0.1 --port 8000 --workers 2
--- a/web/src/app/api/copilotkit/route.ts
+++ b/web/src/app/api/copilotkit/route.ts
@@ -6,18 +6,31 @@ import {

 import { HttpAgent } from "@ag-ui/client";
 import { NextRequest } from "next/server";
+import { auth0 } from "@/lib/auth0";

 const serviceAdapter = new ExperimentalEmptyAdapter();

-const runtime = new CopilotRuntime({
-  agents: {
-    vpi_1000: new HttpAgent({
-      url: process.env.LANGGRAPH_DEPLOYMENT_URL || "http://localhost:8000",
-    }),
-  },
-});
-
 export const POST = async (req: NextRequest) => {
+  // Get user session and roles
+  const session = await auth0.getSession();
+  const userRoles = (session?.user?.roles as string[]) || [];
+
+  console.log("DEBUG: User roles from session:", userRoles);
+
+  // Create HttpAgent with user roles header
+  const agent = new HttpAgent({
+    url: process.env.AGENT_URL || "http://localhost:8000/",
+    headers: {
+      "x-user-roles": JSON.stringify(userRoles),
+    },
+  });
+
+  const runtime = new CopilotRuntime({
+    agents: {
+      vpi_1000: agent,
+    },
+  });
+
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
@@ -25,4 +38,4 @@ export const POST = async (req: NextRequest) => {
  });

  return handleRequest(req);
-};
+};
--- a/web/src/lib/auth0.ts
+++ b/web/src/lib/auth0.ts
@@ -1,6 +1,17 @@
 import { Auth0Client, filterDefaultIdTokenClaims } from '@auth0/nextjs-auth0/server';

+const isProduction = process.env.NODE_ENV === 'production';
+
 export const auth0 = new Auth0Client({
+  session: {
+    rolling: true,
+    absoluteDuration: 60 * 60 * 24 * 7, // 7 days
+    inactivityDuration: 60 * 60 * 24, // 1 day
+    cookie: {
+      secure: isProduction,
+      sameSite: 'lax',
+    },
+  },
  async beforeSessionSaved(session) {
    return {
      ...session,
--- a/web/src/lib/env.ts
+++ b/web/src/lib/env.ts
@@ -0,0 +1,44 @@
+// Environment variable validation for production
+
+const requiredEnvVars = [
+  'AUTH0_SECRET',
+  'AUTH0_DOMAIN',
+  'AUTH0_CLIENT_ID',
+  'AUTH0_CLIENT_SECRET',
+  'APP_BASE_URL',
+] as const;
+
+const optionalEnvVars = [
+  'AGENT_URL',
+] as const;
+
+export function validateEnv() {
+  const missing: string[] = [];
+
+  for (const envVar of requiredEnvVars) {
+    if (!process.env[envVar]) {
+      missing.push(envVar);
+    }
+  }
+
+  if (missing.length > 0) {
+    throw new Error(
+      `Missing required environment variables: ${missing.join(', ')}\n` +
+      'Please check your .env.local file or environment configuration.'
+    );
+  }
+}
+
+export function getEnv() {
+  return {
+    auth0: {
+      secret: process.env.AUTH0_SECRET!,
+      domain: process.env.AUTH0_DOMAIN!,
+      clientId: process.env.AUTH0_CLIENT_ID!,
+      clientSecret: process.env.AUTH0_CLIENT_SECRET!,
+    },
+    appBaseUrl: process.env.APP_BASE_URL!,
+    agentUrl: process.env.AGENT_URL || 'http://localhost:8000/',
+    isProduction: process.env.NODE_ENV === 'production',
+  };
+}
Author	SHA1	Message	Date
Paul Walko	7578c9aab0	file list All checks were successful Build and Push Agent Docker Image / build (push) Successful in 1m29s Details Build and Push Poller Docker Image / lint (push) Successful in 31s Details Build and Push Poller Docker Image / build (push) Successful in 1m0s Details	2025-12-16 04:21:28 +01:00
Paul Walko	00fc4e367f	fix gcp location All checks were successful Build and Push Agent Docker Image / build (push) Successful in 2m28s Details Build and Push Poller Docker Image / lint (push) Successful in 31s Details Build and Push Poller Docker Image / build (push) Successful in 1m1s Details	2025-12-15 18:58:47 +01:00
Paul Walko	cda5496e64	vertex ai, 2.5 pro	2025-12-15 02:16:42 +01:00
Paul Walko	7e8e07c1fd	readme All checks were successful Build and Push Agent Docker Image / build (push) Successful in 2m26s Details	2025-12-14 18:38:07 +01:00
Paul Walko	8b73a7dbd1	use 3 pro All checks were successful Build and Push Agent Docker Image / build (push) Successful in 1m3s Details	2025-12-13 17:35:57 +01:00
Paul Walko	c808f51eb7	pass roles + debugging All checks were successful Build and Push Agent Docker Image / build (push) Successful in 1m3s Details Build and Push Web Docker Image / build (push) Successful in 3m59s Details	2025-12-13 17:23:52 +01:00
Paul Walko	0e24515303	make mcp "prodution ready" All checks were successful Build and Push Agent Docker Image / build (push) Successful in 2m11s Details	2025-12-13 16:55:00 +01:00
Paul Walko	b517c6939f	allow running commands in docker run All checks were successful Build and Push Agent Docker Image / build (push) Successful in 2m22s Details Build and Push Web Docker Image / build (push) Successful in 3m26s Details	2025-12-13 06:36:16 +01:00
Paul Walko	1f313f090a	something is kinda working	2025-12-13 05:41:13 +01:00
Paul Walko	955f992f8e	use pydantic	2025-12-13 04:35:40 +01:00