← All guides AI Release
RU Subscribe to @ai_release1

How to Set Up an Empathetic AI Assistant: System Prompts, RAG and TTS

Updated: 18.10.2026 · AI Release · @ai_release1
How to Set Up an Empathetic AI Assistant: System Prompts, RAG and TTS

Guide passport

Architectural context

Empathy is not a property of the model — it is the result of three independent layers stacked over a single chat-completions call. Three layers, three files, and nothing else:

LayerFileWhat it controlsHow you verify it
Persona`prompts/persona.txt`tone, reply length, empathy rules`python app.py` one-shot reply
Knowledge`index/chunks.json`facts the assistant is allowed to state`python retrieve.py "<question>"`
Voice`out/reply.wav`pacing, pauses, sentence breaks`file out/reply.wav`

Data flow of a single turn:

user utterance
  -> embed(query)                        [embeddings endpoint]
  -> cosine top-4 over index/chunks.json [local, ~1 ms]
  -> persona.txt + "Notes you may cite:" [string concat]
  -> /chat/completions                   [chat model]
  -> answer text
  -> sentence split -> TTS_CMD per sentence
  -> out/part_00.wav, part_01.wav, ...

The practical consequence for debugging: tone breaks in layer 1, facts in layer 2, sound in layer 3. Diagnose from the end of the pipeline, not "the bot as a whole".

Model names, API keys and TTS flags are environment-specific. Everywhere you see <...>, substitute your own value; the exact model identifiers, batch limits and CLI flags must be checked against your provider's documentation.

For microphone capture, wake-word detection and full-duplex playback on top of this pipeline, see How to Create a Voice AI Assistant on Your Computer.

Data flow: utterance → embeddings → top-3 search → persona + notes → chat model → TTS → reply.wav
Data flow: utterance → embeddings → top-3 search → persona + notes → chat model → TTS → reply.wav

Step 1. Write the persona system prompt and a chat wrapper

What to do: create prompts/persona.txt with the behaviour rules, then app.py that loads it and sends it as the system message on every call.

Set up the environment and the four directories first:

cd ~/empathy-assistant
python3 -m venv .venv
source .venv/bin/activate
pip install requests numpy
mkdir -p prompts docs index out
export LLM_BASE_URL="https://<your-endpoint>/v1"
export LLM_API_KEY="<your-key>"
export CHAT_MODEL="<chat-model>"
export EMBED_MODEL="<embedding-model>"
export TTS_CMD="<tts-cmd>"
VariableExample valueCritical because
`LLM_BASE_URL``https://<your-endpoint>/v1``/chat/completions` and `/embeddings` are appended to this root; a duplicated or missing `/v1` returns 404
`CHAT_MODEL``<chat-model>`string and case-sensitive; the valid list exists only in your provider's documentation
`EMBED_MODEL``<embedding-model>`the index and the query must be embedded by the same model, otherwise similarity scores become meaningless
`TTS_CMD``<tts-cmd>`must be on `PATH`; a missing binary raises `FileNotFoundError` on the first sentence
You are a support assistant for a developer tool. You are calm and concrete.

Rules:
1. First sentence: name the feeling you heard, in six words or fewer.
2. Then answer in at most three short sentences. No bullet lists in speech.
3. Ask at most one question per turn.
4. If the answer is not in the provided notes, say "I do not have that note"
   and offer to look it up. Never guess a number, version or date.
5. Never claim to be a human.
# app.py
import json, os, pathlib, requests

BASE = os.environ["LLM_BASE_URL"].rstrip("/")
KEY = os.environ["LLM_API_KEY"]
PERSONA = pathlib.Path("prompts/persona.txt").read_text(encoding="utf-8")

def chat(user_text: str, context: str = "") -> str:
    system = PERSONA
    if context:
        system += "\n\nNotes you may cite:\n" + context
    r = requests.post(
        f"{BASE}/chat/completions",
        headers={"Authorization": f"Bearer {KEY}"},
        json={
            "model": os.environ["CHAT_MODEL"],
            "messages": [
                {"role": "system", "content": system},
                {"role": "user", "content": user_text},
            ],
            "temperature": 0.6,
        },
        timeout=60,
    )
    r.raise_for_status()
    return r.json()["choices"][0]["message"]["content"].strip()

if __name__ == "__main__":
    print(chat("My deploy failed again and I am exhausted."))

Why these values and this order:

How to check:

python app.py

Expected: a three-sentence reply whose first sentence names the emotion, for example That sounds exhausting. A reply that starts with There are several possible reasons: means rules 1 and 2 are too far down the prompt — move them to the top and keep the file under roughly 200 words.

Request breakdown: system message, top-4 retrieved notes, user message, temperature parameter
Request breakdown: system message, top-4 retrieved notes, user message, temperature parameter

Step 2. Build the RAG index and wire in TTS

What to do: chunk your documents, embed each chunk, store the vectors, retrieve the top matches per question, and speak the answer sentence by sentence.

# index.py
import json, pathlib, requests, os

BASE = os.environ["LLM_BASE_URL"].rstrip("/")
KEY  = os.environ["LLM_API_KEY"]
EMB  = os.environ["EMBED_MODEL"]
CHUNK, OVERLAP = 800, 100

def embed(texts):
    r = requests.post(f"{BASE}/embeddings",
        headers={"Authorization": f"Bearer {KEY}"},
        json={"model": EMB, "input": texts}, timeout=60)
    r.raise_for_status()
    return [d["embedding"] for d in r.json()["data"]]

chunks = []
for p in sorted(pathlib.Path("docs").glob("*.md")):
    text = p.read_text(encoding="utf-8")
    for i in range(0, len(text), CHUNK - OVERLAP):
        piece = text[i:i + CHUNK].strip()
        if piece:
            chunks.append({"source": p.name, "text": piece})

for start in range(0, len(chunks), 16):   # batch size: check your provider's limit
    batch = chunks[start:start + 16]
    for c, v in zip(batch, embed([c["text"] for c in batch])):
        c["vector"] = v

pathlib.Path("index/chunks.json").write_text(json.dumps(chunks), encoding="utf-8")
print(f"indexed {len(chunks)} chunks from {len({c['source'] for c in chunks})} files")
# retrieve.py
import json, os, sys, numpy as np, requests

def embed_query(q):
    r = requests.post(f"{os.environ['LLM_BASE_URL'].rstrip('/')}/embeddings",
        headers={"Authorization": f"Bearer {os.environ['LLM_API_KEY']}"},
        json={"model": os.environ["EMBED_MODEL"], "input": [q]}, timeout=60)
    r.raise_for_status()
    return np.array(r.json()["data"][0]["embedding"])

def search(query, k=4):
    chunks = json.loads(open("index/chunks.json", encoding="utf-8").read())
    q = embed_query(query)
    scored = []
    for c in chunks:
        v = np.array(c["vector"])
        scored.append((float(q @ v / (np.linalg.norm(q) * np.linalg.norm(v))), c))
    scored.sort(key=lambda x: x[0], reverse=True)
    return scored[:k]

if __name__ == "__main__":
    for score, c in search(sys.argv[1]):
        print(f"{score:.3f}  {c['source']}")

Why these values:

Example: python retrieve.py "how do I rotate the API key" prints lines such as:

0.842  api-keys.md
0.791  api-keys.md
0.664  onboarding.md
0.512  faq.md

The source column is the citation your assistant can name. If every score is below ~0.5, your question wording and your documents do not share vocabulary — rewrite the document headings before touching the code.

Now add voice. Create speak.py that splits the reply into sentences and synthesizes each one separately. Separate calls give you natural pauses without SSML.

# speak.py
import os, pathlib, subprocess

def speak(text: str, out_dir="out"):
    pathlib.Path(out_dir).mkdir(exist_ok=True)
    plain = text.replace("!", ".").replace("?", ".").replace(";", ".")
    files = []
    for i, s in enumerate(x.strip() for x in plain.split(".") if x.strip()):
        p = f"{out_dir}/part_{i:02d}.wav"
        subprocess.run([os.environ["TTS_CMD"], "--out", p, s], check=True)
        files.append(p)
    return files

The flag that sets the output path differs per engine — check the documentation and replace --out. Sentence-level synthesis is the free quality lever: pauses land on punctuation instead of mid-clause.

How to check: run python index.py. Expected: indexed 42 chunks from 6 files. A count of 0 chunks means docs/ is empty or the files are not .md.

Step 3. Run the end-to-end check and confirm quality

What to do: run the three verification layers in order — knowledge, tone, audio — against one fixed set of questions.

python index.py
python retrieve.py "how do I rotate the API key"
python app.py
python -c "from speak import speak; print(speak('That sounds exhausting. Rotate the key in settings.'))"
file out/part_00.wav

Why: each command isolates one failure class. A wrong answer with good retrieval is a prompt problem; a wrong answer with bad retrieval is a document problem; silence is a TTS problem. Testing them separately stops you from editing the persona prompt to fix a broken index.

Expected screen output on success:

indexed 42 chunks from 6 files
0.842  api-keys.md
...
That sounds exhausting. Rotate the key in Settings → API, then paste the new value into LLM_API_KEY.
['out/part_00.wav', 'out/part_01.wav']
out/part_00.wav: RIFF (little-endian) data, WAVE audio

Finish with the acceptance checklist. Run each item manually once:

Example: ask What is the refund window? when docs/ contains no billing file. Correct behaviour is a refusal plus an offer; a confident number is a hallucination and means the retrieval context was empty but still injected.

Acceptance dashboard: pipeline stage timings and the verification matrix
Acceptance dashboard: pipeline stage timings and the verification matrix

Troubleshooting

Case 1. Every request fails with an authentication error

Case 2. The script starts but crashes on import

Case 3. Retrieval returns chunks but ranking is nonsense

Summary and next steps

You now have a three-layer assistant: a versioned persona prompt, a document index it must cite, and per-sentence TTS output. Next, log every question with its top retrieval score and review the low-score ones weekly — those are the documents to rewrite.

FAQ

What is the difference between the system prompt and RAG?

The system prompt sets behaviour — tone, length, refusal rules — and applies to every turn. RAG supplies facts by inserting retrieved document chunks into the request. Put rules in the prompt and facts in docs/; mixing them makes both untestable.

Do I need a paid TTS engine for natural-sounding speech?

Not necessarily. The quality lever that costs nothing is sentence-level synthesis: separate calls create real pauses at punctuation boundaries. Engine choice and licensing must be checked in each project's documentation.

Can the whole assistant run offline on a phone?

Yes, if the chat model, the embedding model and the TTS engine all run on-device. For an overview of offline chat apps and their limits, see Local AI Companions for Android.

Why does the assistant still sound robotic?

Usually because replies are too long. Cap the answer at three sentences in prompts/persona.txt, synthesize each sentence separately, and drop lists — bullet points read aloud have no natural rhythm.

🤖 AI summary

Related guides

How to Create a Voice AI Assistant on Your Computer: Tools Overview and Setup

Пошаговый гайд по установке и настройке голосовых ИИ-ассистентов для Windows, macOS и Linux, включая open-source решения

Claude Code for Teams: Setting Up Subagents and a Shared AI Superagent

Пошаговый гид по настройке Claude Code для нескольких разработчиков: субагенты, MCP-серверы, общие правила и единый супе

Local AI Companions for Android: Review of Apps for Offline LLM Chat

Сравниваем локальные ИИ-компаньоны для Android, которые работают без интернета через Ollama, LM Studio и другие движки.