Perinatal depression affects a large share of women during pregnancy and after childbirth, yet access to perinatal mental health care in Kazakhstan is limited. This paper presents MamaCare, a multi-agent platform based on open-weight Qwen3.5 large language models (LLMs) that provides conversational support, administers standardized screening instruments, and triages red-flag situations in Kazakh, Russian, and English. Two physician experts rated 286 single- and multi-turn sessions against a rubric adapted from HealthBench and ThReadMed-QA; mean scores for the accuracy, safety, and communication criteria ranged from 4.45 to 4.93 out of 5, and the lowest-scoring criterion was turn-level degradation (3.71). The platform was then compared with five prompt-engineered configurations of its own base model on 120 stratified cases, and every automated judgment was repeated with three judges from three model families: Qwen3.8-27B, Gemma-3-27B, and a Kazakh-adapted Llama-3.1-8B. In red-flag triage, MamaCare had the lowest false-alarm rate of all six configurations under all three judges and against physician urgency labels for the 58 cases on which any judge raised an alarm, and a prompt-only safety configuration escalated routine questions 3.6 to 4.2 times more often; agreement between judges was substantial for urgency labelling (Cohen’s

κ

= 0.67 and 0.79) and for escalation detection (

κ

= 0.78 and 0.47). Pairwise preference did not survive the same test (

κ

= 0.30, with 15.3% to 45.7% of verdicts reversing when the answer order was swapped) and is therefore reported as judge-dependent evidence rather than as a result. Retention of facts from earlier turns was the platform’s weakest property (71.8%); adding a structured patient-state handoff between agents raised it to 97.4% under both judges that measure it reliably, without increasing false alarms and with shorter answers. On a new holdout set of 15 parallel multi-turn dialogues in the three languages, written after the platform was fixed, physician-rated fact retention over the 10 completed dialogues was 0.85 (95% CI 0.74–0.91), with no difference between languages; the main multi-turn errors were information carried over from the user profile and replies truncated by server time-outs.

Publications with shared reference:

0 publications found