AI Development Practice

A learning project for building a support assistant with Spring AI, Gemini, retrieval-augmented generation (RAG), conversational memory, tool calling, and token-aware context management. The repository also includes a small React UI for structured support-ticket analysis.

Features

• Free-form conversational chat with in-memory conversation history
• Structured ticket classification by category and priority
• Tool calling for ticket status and priority lookups
• Markdown knowledge-base ingestion and in-memory vector search
• Category-aware retrieval and follow-up query rewriting
• Rolling conversation summaries with failure-safe memory fallback
• Separate token budgets for query rewriting and final chat prompts
• Tool-schema token estimation and budget-aware RAG document selection
• Real vector similarity scores with deterministic rank-based fallback
• Offline JUnit evaluation of retrieval-order, rank-density, similarity-density, and experimental hybrid scoring strategies
• Deterministic answer evaluation for correctness, faithfulness, and relevance

Technology

• Java 17
• Spring Boot 4.1.1
• Spring AI 2.0.1
• Google Gemini chat and embedding models
• Maven
• React 19, TypeScript, and Vite

Project Structure

ai-dev-practice/
|-- backend/                 Spring Boot REST API and AI workflows
|   |-- src/main/java/       Application code
|   |-- src/main/resources/
|   |   |-- knowledge/       Markdown documents indexed for RAG
|   |   `-- prompts/         Externalized AI prompts
|   `-- src/test/java/       Unit, integration, and offline evaluation tests
|-- frontend/                React ticket-analysis UI
`-- pom.xml                  Maven parent project

How the Chat Pipeline Works

User question
     |
     v
Query rewriting (when history exists and its budget permits)
     |
     v
Knowledge category routing and vector search
     |
     v
Conversation summary and recent-message preparation
     |
     v
Required-context token estimate (system + memory + question + tools)
     |
     v
Allocate remaining budget to RAG documents
     |
     v
Value-density selection using real similarity scores
     |
     v
Final prompt budget validation
     |
     v
Gemini response + sources actually supplied to the model
     |
     v
Offline answer evaluation using manually labeled facts and claims

Production document selection uses similarity / estimated tokens. If every retrieved document has a finite score between 0 and 1, the vector-store scores are used. If any score is missing or invalid, the complete result set falls back to 1 / retrieval rank so incompatible scoring scales are never mixed.

Hybrid scoring is currently an offline experiment only. It does not alter the production chat flow.

Prerequisites

• Java 17 or newer
• Maven 3.6.3 or newer
• A current Node.js LTS release and npm
• A Google Gemini API key

Configuration

Set the API key before starting the backend.

PowerShell:

$env:GEMINI_API_KEY = "your-api-key"

Bash or zsh:

export GEMINI_API_KEY="your-api-key"

The backend also supports these environment variables:

Variable	Default	Purpose
GEMINI_API_KEY	none	Gemini chat and embedding authentication
GEMINI_MODEL	gemini-3-flash-preview	Chat model name
SERVER_PORT	8080	Backend HTTP port

Application-level context limits are configured in backend/src/main/resources/application.yml:

ai:
  context:
    query-rewrite-max-input-tokens: 2000
    final-chat-max-input-tokens: 8000

These limits use a deliberately simple three-characters-per-token estimate. They are guardrails for this learning project, not exact provider token counts.

Run the Application

Start the backend from the repository root:

mvn -pl backend spring-boot:run

In a second terminal, start the frontend:

cd frontend
npm install
npm run dev

Open http://localhost:6001. During development, Vite proxies /api requests to the backend at http://localhost:8080.

To target another backend, set VITE_API_BASE_URL before building or starting the frontend.

REST API

Analyze a Support Ticket

POST /api/ai/analyze-ticket

curl -X POST http://localhost:8080/api/ai/analyze-ticket \
  -H "Content-Type: application/json" \
  -d '{"message":"The production checkout API returns 500 errors."}'

Example response:

{
  "category": "TECHNICAL",
  "priority": "HIGH",
  "summary": "Checkout requests are failing in production.",
  "suggestedAction": "Escalate to the on-call engineering team."
}

Start a Conversation

POST /api/ai/chat

curl -X POST http://localhost:8080/api/ai/chat \
  -H "Content-Type: application/json" \
  -d '{"conversationId":null,"message":"What is the password policy?"}'

The response contains a generated conversation ID, the model response, and only the knowledge sources supplied to the final prompt:

{
  "conversationId": "generated-id",
  "message": "...",
  "sources": ["password-policy.md"]
}

Pass the returned ID to continue the conversation:

curl -X POST http://localhost:8080/api/ai/chat \
  -H "Content-Type: application/json" \
  -d '{"conversationId":"generated-id","message":"How often should it change?"}'

Conversation state is held in memory and is lost when the backend restarts. An unknown conversation ID returns HTTP 404. A final prompt whose essential context exceeds the configured budget returns HTTP 422 with code CONTEXT_BUDGET_EXCEEDED without appending a partial conversation turn.

Approval Demo Security

The escalation-proposal controller and its Spring Security filter chain are enabled only under approval-demo. Proposal creation requires TICKET_REVIEW; approval and rejection require TICKET_ESCALATE. The controller also checks the trusted Spring Security context before consuming a proposal. Other existing chat and agent endpoints retain their current access behavior.

The demo accepts OAuth2 JWT bearer tokens. Configure APPROVAL_JWT_ISSUER_URI with your trusted identity provider's HTTPS issuer URL and APPROVAL_JWT_AUDIENCE with this API's audience. Public signing keys are discovered from that issuer; signature, issuer, audience, expiration and not-before claims are validated. Tokens must include an expiration. Verified scope (or scp) values map ticket.review to TICKET_REVIEW and ticket.escalate to TICKET_ESCALATE; unrelated scopes grant no application permissions.

Only the approval API filter chain disables CSRF: it is stateless, does not create or authenticate through sessions, and has no Basic or cookie-based login fallback. Clients must send Authorization: Bearer <access-token> explicitly. No real escalation gateway or approval-service bean is provisioned. Interactive use still requires gateway wiring and frontend identity-provider integration; enabling the profile alone does not supply them.

Run the focused security tests without Gemini:

mvn -pl backend -Dtest=TicketEscalationJwtSecurityTests,ApprovalJwtDecoderTests,SpringSecurityApprovalAuthorizerTests test

MockMvc JWT helpers test permissions using the production scope converter but bypass cryptographic verification. Separate decoder tests generate RSA keys and serve issuer metadata/public keys on loopback, verifying rejection of invalid signatures, issuer, audience, expiry, and unsigned tokens without an external IdP.

Knowledge Base

The backend discovers Markdown files under backend/src/main/resources/knowledge/, parses their YAML front matter, splits them into chunks, and indexes them in an in-memory vector store on the first knowledge search.

Current topics include:

• Password policy
• Refund policy
• Ticket service-level agreements

Additional Markdown files placed directly in that directory are discovered automatically. Each file must contain valid YAML front matter; an optional category value enables category-filtered retrieval.

Tests and Offline RAG Evaluation

Run the complete backend suite from the repository root:

mvn test

Run only the RAG selection evaluation harness:

mvn -pl backend -Dtest=RagDocumentSelectionEvaluationTests test

Run only the RAG answer evaluation harness:

mvn -pl backend -Dtest=RagAnswerEvaluationTests test

The offline harness compares document selection against labeled synthetic cases using relevant-document recall, token utilization, and selected-document count. It evaluates retrieval order and several relevance-scoring policies without calling Gemini. These metrics measure document-selection quality, not final answer correctness; higher token utilization alone does not imply a better result.

The answer evaluation harness separately measures:

• Correctness: expected facts present in the answer divided by all expected facts.
• Faithfulness: answer claims supported by retrieved knowledge divided by all answer claims.
• Relevance: on-topic answer claims divided by all answer claims.

RagAnswerEvaluationTests covers a correct and faithful answer, a correct answer with an unsupported claim, and a supported but irrelevant answer. Fact IDs and claim labels are assigned manually, making the evaluation deterministic and independent of Gemini. The harness is a controlled claim-level baseline; it does not judge arbitrary natural-language answers semantically.

The project requires Maven 3.6.3 or newer for the normal lifecycle. To run only the answer evaluation with Maven 3.6.0, invoke compatible compiler and test plugins directly from the backend directory:

cd backend
mvn org.apache.maven.plugins:maven-compiler-plugin:3.12.1:testCompile `
    org.apache.maven.plugins:maven-surefire-plugin:3.2.5:test `
    -Dtest=RagAnswerEvaluationTests

Build the frontend:

cd frontend
npm run build

Current Limitations

• Conversations, summaries, tickets, and vector data are stored in memory.
• Token estimates are heuristic and provider formatting is not fully modeled.
• RAG selection is greedy and does not account for document redundancy.
• The frontend currently exposes ticket analysis, while conversational chat is available through the REST API.
• The evaluation dataset is synthetic and too small for choosing a production scoring policy.