From 17ac2ce0ed52874616c79c910667e41e34604826 Mon Sep 17 00:00:00 2001
From: wangzhibo <wangzhibo@shsening.com>
Date: 星期一, 31 八月 2026 13:14:12 +0800
Subject: [PATCH] add translation app
---
xiaozhi-esp32-2.2.6/main/boards/common/board.h | 2
xiaozhi-esp32-server/main/xiaozhi-server/core/handle/textHandler/listenMessageHandler.py | 32
xiaozhi-esp32-server/main/xiaozhi-server/core/providers/tools/unified_tool_manager.py | 7
xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/system.h | 3
xiaozhi-esp32-2.2.6/main/application.h | 1
xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/app_task.h | 24
xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/system.c | 12
xiaozhi-esp32-server/main/manager-web/src/components/ReplacementWordDialog.vue | 7
xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/mqtt.c | 47
phase-1.md | 923 ++++++++
xiaozhi-esp32-server/main/manager-api/src/main/java/xiaozhi/modules/device/service/impl/DeviceServiceImpl.java | 4
xiaozhi-esp32-server/main/xiaozhi-server/core/handle/receiveAudioHandle.py | 92
小智 AI Server-Task、语音分析与用户评估平台架构 Plan.md | 537 ++++
小智 AI 硬件开放应用平台-总体架构与开发 Plan.md | 541 ++++
xiaozhi-esp32-server/main/manager-api/src/main/resources/db/changelog/202504151206.sql | 2
xiaozhi-esp32-server/main/xiaozhi-server/mcp_server_settings.json | 5
xiaozhi-esp32-server/main/xiaozhi-server/plugins_func/functions/translate.py | 61
xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/runtime.py | 374 +++
xiaozhi-esp32-server/main/manager-web/src/components/AddModelDialog.vue | 4
xiaozhi-esp32-server/main/manager-web/src/components/ProviderDialog.vue | 7
xiaozhi-esp32-server/.gitignore | 1
xiaozhi-esp32-server/main/manager-web/.env.development | 4
xiaozhi-esp32-server/main/manager-web/src/components/ModelEditDialog.vue | 4
xiaozhi-esp32-server/main/manager-web/src/apis/api.js | 2
xiaozhi-esp32-server/main/manager-web/src/components/CacheViewer.vue | 14
xiaozhi-esp32-server/main/manager-web/src/components/ParamDialog.vue | 7
xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/registry.py | 77
xiaozhi-esp32-server/main/xiaozhi-server/mcp_servers/translate/server.py | 110 +
xiaozhi-esp32-server/main/manager-web/src/components/ViewPasswordDialog.vue | 17
xiaozhi-esp32-2.2.6/main/application.cc | 20
xiaozhi-esp32-server/main/manager-web/vue.config.js | 37
xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/__init__.py | 15
xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/sening_xiaozhi_s3.cc | 35
xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/mp3_player.cc | 59
xiaozhi-esp32-server/main/xiaozhi-server/core/connection.py | 12
xiaozhi-esp32-server/main/xiaozhi-server/config.yaml | 7
xiaozhi-esp32-server/main/xiaozhi-server/plugins_func/functions/switch_app.py | 41
Xiaozhi AI Architecture Audit.md | 603 +++++
小智 AI 开放应用平台-master Pln.md | 2187 ++++++++++++++++++++
xiaozhi-esp32-server/main/manager-web/src/components/FirmwareDialog.vue | 11
xiaozhi-esp32-server/main/xiaozhi-server/plugins_func/functions/start_repeater.py | 72
xiaozhi-esp32-server/main/manager-api/src/main/java/xiaozhi/common/constant/Constant.java | 2
xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/apps.json | 63
xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/app_task.cc | 371 +++
xiaozhi-esp32-server/main/xiaozhi-server/core/providers/tools/server_mcp/mcp_manager.py | 38
45 files changed, 6,435 insertions(+), 59 deletions(-)
diff --git a/Xiaozhi AI Architecture Audit.md b/Xiaozhi AI Architecture Audit.md
new file mode 100644
index 0000000..2ac72dd
--- /dev/null
+++ b/Xiaozhi AI Architecture Audit.md
@@ -0,0 +1,603 @@
+# Xiaozhi AI Architecture Audit
+
+浣犵幇鍦ㄨ礋璐e Xiaozhi AI 椤圭洰杩涜涓�娆� **Architecture Audit**銆�
+
+鏈浠诲姟鏄細
+
+> **鍙垎鏋愶紝涓嶄慨鏀逛唬鐮併��**
+
+椤圭洰鍖呭惈涓や釜 Repository锛�
+
+1. Xiaozhi Client 鈥斺�� 纭欢绔� / Device Runtime
+2. Xiaozhi Server 鈥斺�� Server / AI / Backend
+
+鍚屾椂鎻愪緵浜嗕竴浠姐�奨iaozhi AI 寮�鏀惧簲鐢ㄥ钩鍙� Master Plan銆嬨��
+
+Master Plan 鏄湭鏉ョ洰鏍囨灦鏋勶紝浣嗗綋鍓嶄唬鐮佸緢鍙兘涓庣洰鏍囨灦鏋勫瓨鍦ㄨ緝澶у樊寮傘��
+
+浣犵殑浠诲姟涓嶆槸绔嬪嵆瀹炵幇 Master Plan锛岃�屾槸锛�
+
+> 鎵惧嚭褰撳墠绯荤粺鏄粈涔堛�佸凡缁忓叿澶囦粈涔堛�佺己灏戜粈涔堬紝浠ュ強搴旇浠ヤ粈涔堟渶灏忎唬浠烽�愭婕旇繘鍒扮洰鏍囨灦鏋勩��
+
+---
+
+## 涓�銆佺姝簨椤�
+
+鏈樁娈电姝細
+
+- 淇敼浠g爜
+- 鍒犻櫎浠g爜
+- 閲嶆瀯浠g爜
+- 鍒涘缓鏂扮殑涓氬姟妯″潡
+- 淇敼鏁版嵁搴�
+- 淇敼 API
+- 淇敼鍗忚
+- 淇敼閰嶇疆
+- 淇敼纭欢绔�昏緫
+
+鍙厑璁革細
+
+- 闃呰浠g爜
+- 鍒嗘瀽浠g爜
+- 寤虹珛鏋舵瀯璁ょ煡
+- 杈撳嚭鍒嗘瀽鎶ュ憡
+- 鎻愬嚭寤鸿
+
+---
+
+# 浜屻�丄udit 鑼冨洿
+
+蹇呴』鍚屾椂鍒嗘瀽 Client 鍜� Server銆�
+
+涓嶈鍙湅鐩綍缁撴瀯銆�
+
+闇�瑕佽拷韪湡瀹炰唬鐮佽皟鐢ㄩ摼銆�
+
+---
+
+# 涓夈�丆lient Audit
+
+閲嶇偣鍒嗘瀽锛�
+
+## 1. 褰撳墠 Runtime
+
+鍥炵瓟锛�
+
+- Client 褰撳墠鍚姩娴佺▼鏄粈涔堬紵
+- 涓诲惊鐜湪鍝噷锛�
+- 褰撳墠 Conversation / Chat Runtime 鍦ㄥ摢閲岋紵
+- 鐢ㄦ埛璇磋瘽 鈫� ASR 鈫� LLM 鈫� TTS 鈫� 鎾斁鐨勫畬鏁磋皟鐢ㄩ摼鏄粈涔堬紵
+- Audio Input / Output 濡備綍瀹炵幇锛�
+- 鏄惁宸茬粡瀛樺湪 Event / Message / Command 鏈哄埗锛�
+- Server 涓� Client 浣跨敤浠�涔堥�氫俊鍗忚锛�
+
+---
+
+## 2. 纭欢鑳藉姏
+
+鏁寸悊褰撳墠宸茬粡瀛樺湪鐨勶細
+
+- 楹﹀厠椋�
+- 鎵0鍣�
+- 闊抽鎾斁
+- 闊抽褰曢煶
+- VAD
+- 鎸夐敭
+- LED
+- 灞忓箷锛堝鏋滄湁锛�
+- SD Card
+- 缃戠粶
+- MQTT / WebSocket / HTTP 绛夐�氫俊鑳藉姏
+
+骞舵寚鍑哄摢浜涜兘鍔涘凡缁忓彲浠ョ洿鎺ヤ綔涓烘湭鏉� Skill銆�
+
+---
+
+## 3. 鎵惧嚭褰撳墠鍙互鎶借薄鎴� Skill 鐨勮兘鍔�
+
+渚嬪锛�
+
+```text
+play_audio
+record_audio
+listen
+speak
+wait
+display
+led
+button
+```
+
+浣嗕笉瑕佸己琛屽垱閫犱笉瀛樺湪鐨勮兘鍔涖��
+
+蹇呴』鍩轰簬鐪熷疄浠g爜鍒嗘瀽銆�
+
+---
+
+## 4. 褰撳墠 Chat Architecture
+
+閲嶇偣鍥炵瓟锛�
+
+```text
+Audio Input
+ 鈫�
+VAD
+ 鈫�
+ASR
+ 鈫�
+Conversation
+ 鈫�
+LLM
+ 鈫�
+TTS
+ 鈫�
+Audio Output
+```
+
+瀹為檯浠g爜涓垎鍒搴斾粈涔堟ā鍧椼�佺被銆佸嚱鏁板拰鏂囦欢锛�
+
+---
+
+## 5. Task Runtime 鎻掑叆鐐�
+
+閲嶇偣鍒嗘瀽锛�
+
+鏈潵澧炲姞锛�
+
+```text
+Task
+ 鈫�
+Step
+ 鈫�
+Skill
+```
+
+搴旇鎻掑叆鐜版湁 Client 鐨勪粈涔堜綅缃紵
+
+閲嶇偣鍥炵瓟锛�
+
+> 濡備綍鍋氬埌鈥滀笉鐮村潖鐜版湁 Chat 鍔熻兘鈥濈殑鎯呭喌涓嬪鍔� Task Runtime銆�
+
+---
+
+# 鍥涖�丼erver Audit
+
+閲嶇偣鍒嗘瀽锛�
+
+## 1. Server 褰撳墠鏋舵瀯
+
+鍒嗘瀽锛�
+
+- Server 鍚姩鍏ュ彛
+- API
+- WebSocket
+- MQTT
+- Session
+- User
+- Device
+- Conversation
+- Audio
+- ASR
+- LLM
+- TTS
+
+涔嬮棿鐨勫叧绯汇��
+
+---
+
+## 2. 褰撳墠瀵硅瘽閾捐矾
+
+瀹屾暣杩借釜锛�
+
+```text
+Device
+ 鈫�
+Server
+ 鈫�
+ASR
+ 鈫�
+Conversation
+ 鈫�
+LLM
+ 鈫�
+TTS
+ 鈫�
+Device
+```
+
+鎸囧嚭鐪熷疄浠g爜瀹炵幇浣嶇疆銆�
+
+---
+
+## 3. 鏁版嵁搴�
+
+鍒楀嚭褰撳墠涓庝互涓嬪璞$浉鍏崇殑鏁版嵁琛細
+
+```text
+User
+Device
+Conversation
+Message
+Audio
+Session
+```
+
+浠ュ強鍏朵粬涓庢湭鏉� Task 绯荤粺鍙兘鐩稿叧鐨勮〃銆�
+
+璇存槑锛�
+
+- 琛ㄧ粨鏋�
+- 涓婚敭
+- 澶栭敭
+- 鍏崇郴
+- 褰撳墠鏁版嵁鐢熷懡鍛ㄦ湡
+
+---
+
+# 浜斻�侀噸鐐瑰鎵炬湭鏉� Domain Model 鐨勮惤鐐�
+
+瀵逛互涓嬫蹇甸�愪竴鍒嗘瀽锛�
+
+```text
+App
+Task
+Step
+Skill
+InteractionMode
+TaskInstance
+StepInstance
+Event
+Audio
+Assessment
+Metric
+Schedule
+Plan
+Subscription
+```
+
+瀵逛簬姣忎竴涓蹇靛洖绛旓細
+
+1. 褰撳墠浠g爜涓槸鍚﹀凡缁忓瓨鍦ㄧ被浼兼蹇碉紵
+2. 濡傛灉瀛樺湪锛屽湪鍝噷锛�
+3. 鏄惁鍙互澶嶇敤锛�
+4. 鏄惁闇�瑕佹墿灞曪紵
+5. 鏄惁闇�瑕佹柊寤猴紵
+6. 鏈�閫傚悎鏀惧湪鍝釜 Repository / Module锛�
+
+---
+
+# 鍏�侀噸鐐瑰垎鏋� InteractionMode
+
+Master Plan 涓細
+
+> InteractionMode 涓嶆槸涓氬姟鍦烘櫙锛岃�屾槸 Xiaozhi 鐨勭‖浠朵氦浜掓柟寮忋��
+
+閲嶇偣纭褰撳墠浠g爜鏄惁瀛樺湪绫讳技锛�
+
+```text
+Chat
+Audio Playback
+Recording
+Listening
+Translation
+Game
+```
+
+鎴栬�呭叾浠栨ā寮忋��
+
+鍒嗘瀽锛�
+
+> InteractionMode 鏈�閫傚悎鏀惧湪 Client銆丼erver锛岃繕鏄袱杈归兘鏈夛紵
+
+骞剁粰鍑烘帹鑽愩��
+
+---
+
+# 涓冦�侀噸鐐瑰垎鏋� Task Runtime
+
+杩欐槸鏈 Audit 鏈�閲嶈鐨勯儴鍒嗐��
+
+鍥炵瓟锛�
+
+褰撳墠绯荤粺搴旇濡備綍浠庯細
+
+```text
+Conversation Runtime
+```
+
+閫愭婕旇繘涓猴細
+
+```text
+Device Runtime
+ 鈹溾攢鈹� Chat Runtime
+ 鈹斺攢鈹� Task Runtime
+ 鈹溾攢鈹� Step
+ 鈹斺攢鈹� Skill
+```
+
+涓嶈鐩存帴璁捐鏂颁唬鐮併��
+
+鍏堝垎鏋愶細
+
+> 褰撳墠浠g爜鍦ㄥ摢閲屾渶閫傚悎鎻掑叆杩欎釜 Runtime銆�
+
+---
+
+# 鍏�侀噸鐐瑰垎鏋� Audio / Recording
+
+鏈潵 Server 闇�瑕佷繚瀛樺効绔� Task 鎵ц杩囩▼涓殑璇煶銆�
+
+鍥犳鍒嗘瀽锛�
+
+- 褰撳墠褰曢煶鍦ㄥ摢閲屼骇鐢燂紵
+- 闊抽鏄惁宸茬粡涓婁紶 Server锛�
+- 闊抽鏍煎紡鏄粈涔堬紵
+- 鏄惁鍙互淇濆瓨鍘熷闊抽锛�
+- 褰撳墠闊抽鐢熷懡鍛ㄦ湡鏄粈涔堬紵
+- 鏄惁瀛樺湪 Audio ID锛�
+- 鏄惁瀛樺湪 Session ID锛�
+- 鏄惁鍙互鍏宠仈鍒� TaskInstance / StepInstance锛�
+
+鎸囧嚭鏈潵鏈�灏忔敼閫犳柟妗堛��
+
+---
+
+# 涔濄�侀噸鐐瑰垎鏋� Event
+
+妫�鏌ュ綋鍓嶆槸鍚﹀凡缁忓瓨鍦細
+
+```text
+Event
+Message
+Command
+State
+Callback
+```
+
+绛夋満鍒躲��
+
+濡傛灉瀛樺湪锛�
+
+> 鍒ゆ柇鑳藉惁婕旇繘鎴愮粺涓� Event Architecture銆�
+
+濡傛灉涓嶅瓨鍦細
+
+> 鎸囧嚭鏈�鍚堥�傜殑鎻掑叆浣嶇疆銆�
+
+---
+
+# 鍗併�佹渶缁堣緭鍑� Audit Report
+
+鏈�缁堜笉瑕佸彧杈撳嚭鎬荤粨銆�
+
+蹇呴』杈撳嚭浠ヤ笅鍐呭锛�
+
+## 1. Current Architecture
+
+鍒嗗埆鐢诲嚭锛�
+
+```text
+Client Architecture
+Server Architecture
+```
+
+鏈�濂戒娇鐢� Mermaid銆�
+
+---
+
+## 2. Current Runtime Call Graph
+
+鐢诲嚭锛�
+
+```text
+鐢ㄦ埛璇磋瘽
+ 鈫�
+...
+```
+
+瀵瑰簲鐪熷疄浠g爜妯″潡銆�
+
+---
+
+## 3. Existing Capability Map
+
+鏁寸悊锛�
+
+```text
+Capability
+Current Implementation
+Repository
+Module
+Reusable?
+Required Change
+```
+
+---
+
+## 4. Domain Mapping
+
+寤虹珛锛�
+
+```text
+Master Plan Domain
+ 鈫�
+Current Code
+ 鈫�
+Gap
+ 鈫�
+Recommended Location
+```
+
+---
+
+## 5. Architecture Gap
+
+鏄庣‘鍒楀嚭锛�
+
+```text
+P0 蹇呴』瑙e喅
+P1 鍚庣画瑙e喅
+P2 鏈潵骞冲彴鍖栬В鍐�
+```
+
+---
+
+## 6. Recommended Evolution Architecture
+
+涓嶆槸瑕佹眰绔嬪嵆瀹炵幇銆�
+
+鑰屾槸璇存槑锛�
+
+```text
+Current
+ 鈫�
+Phase 1
+ 鈫�
+Phase 2
+ 鈫�
+Phase 3
+ 鈫�
+Target Architecture
+```
+
+---
+
+## 7. 鏈�鍏抽敭鐨勪慨鏀圭偣
+
+鏈�缁堟槑纭垪鍑猴細
+
+### Client
+
+闇�瑕佹敼鍝簺妯″潡銆�
+
+### Server
+
+闇�瑕佹敼鍝簺妯″潡銆�
+
+### Database
+
+闇�瑕佹柊澧�/淇敼鍝簺瀹炰綋銆�
+
+### Protocol
+
+闇�瑕佸鍔犲摢浜涙秷鎭��
+
+### API
+
+闇�瑕佸鍔犲摢浜涙帴鍙c��
+
+浣嗘闃舵鍙彁鍑猴紝涓嶅疄鐜般��
+
+---
+
+# 鍗佷竴銆佹渶缁堝繀椤诲洖绛旂殑鏍稿績闂
+
+璇风壒鍒洖绛斾互涓嬮棶棰橈細
+
+### Q1
+
+褰撳墠 Chat Runtime 鑳藉惁鐩存帴浣滀负鏈潵 Task Runtime 鐨勪竴涓瓙 Runtime锛�
+
+---
+
+### Q2
+
+Task Runtime 鏈�閫傚悎鏀惧湪 Client 杩樻槸 Server锛�
+
+璇风粨鍚堝綋鍓嶄唬鐮佸垎鏋愶紝鑰屼笉鏄彧鏍规嵁鐞嗚鍥炵瓟銆�
+
+---
+
+### Q3
+
+Step 鐘舵�佸簲璇ョ敱 Client 淇濆瓨銆丼erver 淇濆瓨锛岃繕鏄袱杈归兘鏈夛紵
+
+---
+
+### Q4
+
+Task 鎵ц杩囩▼涓� Server 涓� Client 鐨勭姸鎬佸悓姝ユ満鍒跺簲璇ュ浣曡璁★紵
+
+---
+
+### Q5
+
+App / Task / Resource 鏄惁鍙互瀹屽叏閫氳繃 JSON + Audio Resource 涓嬪彂锛岃�屼笉闇�瑕� OTA锛�
+
+濡傛灉涓嶈兘锛岃鎸囧嚭鍝簺鑳藉姏蹇呴』鍥哄寲鍒� Firmware銆�
+
+---
+
+### Q6
+
+褰撳墠浠g爜璺濈锛�
+
+```text
+App
+ 鈫�
+Task
+ 鈫�
+Step
+ 鈫�
+Skill
+ 鈫�
+Device Capability
+```
+
+鏈�澶х殑鏋舵瀯闅滅鏄粈涔堬紵
+
+---
+
+### Q7
+
+濡備綍浠ユ渶灏忎慨鏀瑰疄鐜扮涓�涓細
+
+```text
+鑻辫澶嶈 Task
+```
+
+鑰屼笉鐮村潖鐜版湁 Chat锛�
+
+---
+
+### Q8
+
+濡備綍浠ユ渶灏忎慨鏀瑰疄鐜帮細
+
+```text
+鑻辫璺熻
+ 鈫�
+褰曢煶
+ 鈫�
+涓婁紶
+ 鈫�
+ASR
+ 鈫�
+Assessment
+```
+
+---
+
+# 鍗佷簩銆佹渶缁堢粨璁�
+
+鏈�鍚庣粰鍑猴細
+
+```text
+Architecture Audit Conclusion
+```
+
+骞舵槑纭細
+
+1. 褰撳墠鏋舵瀯鍙互淇濈暀浠�涔�
+2. 褰撳墠鏋舵瀯蹇呴』鏀逛粈涔�
+3. 绗竴闃舵搴旇鏀逛粈涔�
+4. 鍝簺浜嬫儏鐜板湪缁濆涓嶈鍋�
+5. 涓嬩竴闃舵搴旇濡備綍鐢熸垚 Phase 1 Coding Plan
+
+鍐嶆寮鸿皟锛�
+
+> 鏈浠诲姟鐨勭洰鏍囦笉鏄啓浠g爜锛岃�屾槸寤虹珛瀵瑰綋鍓嶇郴缁熺殑瀹屾暣鏋舵瀯璁ょ煡锛屽苟鎵惧埌浠� Current Architecture 鍒� Master Plan 鐨勬渶灏忔紨杩涜矾寰勩��
+
+瀹屾垚 Audit 鍚庡仠姝€��
+
+绛夊緟涓嬩竴姝ユ寚浠ゃ��
\ No newline at end of file
diff --git a/phase-1.md b/phase-1.md
new file mode 100644
index 0000000..436a537
--- /dev/null
+++ b/phase-1.md
@@ -0,0 +1,923 @@
+# Phase 1 Coding Plan
+
+> File: `phase-1.md`
+> Date: 2026-08-30
+> Updated: 榛樿 chat.conversation pending锛涢厤缃�/寮�鏈轰富鍔ㄥ彂璧凤紱涓夐�氶亾鎵嶅垏鎹�
+> Based on: Master Plan 路 Architecture Audit 路 Client Plan V1.1 路 Server Plan V1.1 路 Cross-System Review
+> **鏈樁娈典骇鍑鸿璁′笌缂栫爜浠诲姟娓呭崟銆傛寜鏈枃浠跺疄鏂斤紱涓� V1.1 Plan 鍐茬獊鏃讹紝浠ユ湰鏂囦负鍑嗐��**
+
+---
+
+## 0. Cross-System Review 缁撹
+
+Client Plan 涓� Server Plan **鏂瑰悜鍙崗鍚�**锛圫erver 缂栨帓銆丆lient 鎵ц銆丆hat 鐑矾寰勪笉鍔ㄣ�丮CP 浼樺厛锛夛紝浣嗕笉鑳芥寜涓や唤鏂囨。鐨勩�屽師鏂� Phase 1銆嶅悇鑷紑宸ャ�傚繀椤诲厛瀵归綈涓嬮潰涓夋潯锛屽惁鍒欏璇诲拰缈昏瘧閮戒細鍋氫笉瀹屾垨鐮村潖 Chat銆�
+
+### 0.1 宸茬粡瀵归綈锛堜繚鎸侊級
+
+| 濂戠害 | 鍚箟 |
+|------|------|
+| 鏉冨▉鐘舵�佸湪 Server | 璁惧涓嶆寔涔呭寲 App/Task 鍘嗗彶 |
+| 鍥轰欢鏃犱笟鍔″垎鏀� | 绂佹 `if (app == english)` |
+| DeviceState 鈮� InteractionMode | 涓嶆妸澶嶈/缈昏瘧濉炶繘 Listening/Speaking 杩佺Щ琛� |
+| 鍗忚璇箟鍐荤粨 | `hello/listen/abort/tts/stt/mcp` 涓嶆敼瀛楁鍚箟 |
+| 鍙鐢ㄧ‖浠� | Sening `self.mp3.play` / `self.mp3.stop` / `self.power_off`锛沗PauseSdCardMp3` |
+
+### 0.2 鍘熸枃鍐茬獊涓� Phase 1 鍐宠
+
+| 缂哄彛 | 鍘熸枃 | 鍐宠 |
+|------|------|------|
+| App 灞� | Client 璇� Phase 1 涓嶅仛 App锛汼erver 璇� App 琛� Phase 3 | **鍋氳杽 App Runtime锛堜粎 Server JSON + `active_app_id`锛�**銆傝澶囨棤 App 鐩綍銆佹棤 App Store |
+| 澶嶈 N 娆� | 鍙屾柟閮藉亣璁� Server 寰幆绛� `play.completed` | **褰撳墠 `self.mp3.play` 绔嬪嵆杩斿洖 `accepted`**銆侼 娆″惊鐜斁杩�**璁惧 Skill 鍙傛暟** `repeat`锛汼erver 鍙笅鍙� `name/repeat/power_off_after` |
+| 缈昏瘧 | Client 鎶� TRANSLATION Mode 鏀惧埌 Phase 3 | **App 澶嶇敤 ASR+TTS 绠$嚎锛岃烦杩囬棽鑱� LLM锛屽彧鏆撮湶缈昏瘧 MCP** |
+| 杞叧鏈� | 涓や唤 Plan 閮芥湭鍐欏叆澶嶈楠屾敹 | 澶嶇敤鐜版湁 `self.power_off` 鈫� `RequestPowerOffStandby` |
+| 鍙� MQTT | 鍏佽骞跺瓨 | **鑹插畞 snlp MQTT 鏄皬绋嬪簭鍗虫椂鍚姩 / 璁″垝浠诲姟鐨勬帶鍒堕潰锛堜紭鍏堢骇鏈�楂橈級**銆俙START_APP_TASK` 鍙〃杈俱�屽惎鍔ㄥ摢涓� App/Task銆嶏紝**鍏蜂綋璋冪敤鍝簺 Skill 鐢辫 Task 鐨勯厤缃喅瀹�**銆傜姝㈢敤 `PLAY_MEDIA` 鍐掑厖銆屽惎鍔� App銆嶃�� |
+
+### 0.3 鍗忓悓鎬诲浘锛圥hase 1锛�
+
+```text
+杩涘叆閫氶亾锛堜紭鍏堢骇浠庨珮鍒颁綆锛�
+ 鈶� 灏忕▼搴忓嵆鏃跺紑濮� 鈹�鈹�鑹插畞 MQTT鈹�鈹�鈻� 鏈�楂橈紝鎵撴柇涓�鍒�
+ 鈶� 灏忕▼搴� Schedule 鈹�鈹�鍒扮偣/闂归挓鍞ら啋鈹�鈹�鈻� 鍙墦鏂� chat
+ 鈶� 绂荤嚎璇煶鍒囨崲 鈹�鈹�MultiNet 鍙d护鈹�鈹�鈻� 浜や簰閫� App/Task
+ 鈹�
+ 鈻�
+ 缁熶竴 start_app_task(app, task, params, source)
+ 鈹�
+ 鈻�
+ App Runtime (Server + 璁惧鎵ц鍣�)
+ active_app_id 榛樿 = chat
+ 鈹屸攢鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹尖攢鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�
+ 鈹� 鈹� 鈹�
+ chat english_learning english_translation
+ mode=CHAT mode=PLAY_AUDIO mode=TRANSLATION
+ 鈹� 鈹� 鈹�
+ 鈹斺攢鈹�鈹�鈹�鈹�鈹�鈹�鈹� Device Skill锛圡CP / 鏈湴鎾斁鍣級鈹�鈹�鈹�鈹�鈹�鈹�鈹�
+```
+
+鍚屼竴鏃跺埢鍙繍琛屼竴涓� App / 涓�涓� Task銆�
+
+**榛樿锛堟病鏈夋洿楂樹紭鍏堢骇鍏ュ彛鏃讹級锛氭案杩滄槸绗竴涓� App 鐨� default Task = `chat.conversation`锛堝師鏈夊皬鏅哄璇濊兘鍔涳紝浣嗘洿涓诲姩寮�鍙o級銆�**
+
+| 浜嬩欢 | 杩涘叆 |
+|------|------|
+| 棣栨閰嶇綉鎴愬姛 | `chat` / default task `conversation`锛屽皬鏅哄厛鍙戣捣锛堝銆屾垜浠潵鑱婂ぉ鍚с�嶏級 |
+| 涔嬪悗鏅�氬紑鏈� / 杞‖鍏虫満鍐嶅惎鍔� | **缁х画** 杩欎唤 chat pending task锛屼笉瑕佸仠鍦ㄧ函绛夊緟闂瘽鐨� Idle |
+| 鐢ㄦ埛璇淬�屾垜浠潵鑱婂ぉ鍚� / 鑱婁細澶╁惂銆嶆垨鍞ら啋鍥炶亰澶� | 鍒囧洖锛堟垨淇濇寔锛塦chat.conversation` |
+| 灏忕▼搴忓嵆鏃� / 璁″垝鍒扮偣 / 绂荤嚎鍒� App | 鎵嶇寮� chat |
+| RTC 闂归挓涓� pending **涓嶆槸** chat锛堜緥濡傚璇伙級 | 鎵ц璇ヨ鍒� Task锛屼笉瑕佸厛闂茶亰 |
+
+---
+
+## 1. Phase 1 鐩爣
+
+鍋� **鍏ㄩ潰浣嗚杽鐨� App 妗嗘灦**锛屽苟鍦ㄦ鏋朵笂璺戜笁涓� App锛�
+
+1. **`chat`锛坉efault App + default Task锛�** 鈥� 鍘熸湁灏忔櫤瀵硅瘽鑳藉姏锛涢厤缃戝悗涓庢瘡娆¢粯璁ゅ紑鏈洪兘杩涘叆骞� **涓诲姩鍙戣捣鑱婂ぉ**锛屼笉鏄彧绛夌敤鎴锋彁闂�
+2. **`english_learning`** 鈥� 鍚峰璇绘満锛氭湰鍦� SD 鍗� MP3 鎾斁 **N 娆�**锛岀粨鏉熷悗 **鍙�夎蒋鍏虫満**
+3. **`english_translation`** 鈥� 鑻辫缈昏瘧 App锛歋erver 鎻掑叆 **鍙仛缈昏瘧鐨� MCP**锛涜 App 涓嬩笉闂茶亰銆佷笉璋冨叾浠栨彃浠�
+
+### 1.1 闈炵洰鏍囷紙鏄庣‘涓嶅仛锛�
+
+- 璺熻銆佸綍闊冲綊妗c�丄ssessment銆乀askInstance 姝e紡琛�
+- Xiaozhi 鍗忚鏂� `type=task`锛堝皬绋嬪簭璧拌壊瀹� MQTT锛岃 搂2.7锛�
+- 瀹堕暱灏忕▼搴� UI / 瀹屾暣 Java Scheduler 寰湇鍔� / App Store 涓婃灦
+- 鎴愯鎺ラ緳銆佹暟瀛楄儗璇点�佸彛绠楃瓑灏氭湭瀛樺湪鐨� Task 鏈綋锛堣闊宠彍鍗曞彧棰勭暀鎰忓浘妲斤級
+- 鍚堝苟 Xiaozhi MQTT 涓� snlp/v1 涓轰竴鏉$墿鐞嗚繛鎺�
+- 鏀� Opus 鐑矾寰勩�丏eviceState 鍚堟硶杩佺Щ琛�
+- 闈� Sening 鏉跨殑 SD 澶嶈锛堢炕璇� App 涓嶄緷璧� SD锛屽彲鍏ㄦ澘锛�
+- 鐢� `play_music`锛圫erver TTS 閫氶亾锛夊啋鍏呭璇�
+
+**蹇呴』鍐欏叆鏈樁娈垫鏋讹紙鎸傞挬鍙祴锛屽皬绋嬪簭鍙悗鍋氾級锛�** 涓夌杩涘叆閫氶亾銆佷紭鍏堢骇鎶㈠崰銆佸叧鏈哄敜閱掑悗缁窇璁″垝浠诲姟銆佺绾垮彛浠や笉鍐嶆槸绌哄洖璋冦��
+
+---
+
+## 2. 妗嗘灦璁捐锛堟湰闃舵閲嶇偣锛�
+
+### 2.1 姒傚康
+
+| 姒傚康 | Phase 1 钀藉湴 | 涓嶈惤鍦� |
+|------|----------------|--------|
+| App | Server 娉ㄥ唽琛� + 杩炴帴涓� `active_app_id` | 鍥轰欢 App Manager銆丼D `/apps` 瀹夎 |
+| Task | `english_learning` 鍐呬竴涓�屽璇汇�嶅姩浣� | Workflow / StepInstance |
+| InteractionMode | 杩炴帴瀛楁 `CHAT \| PLAY_AUDIO \| TRANSLATION` | 鏂� DeviceState |
+| Skill | 鐜版湁 MCP tools + 鎵╁睍鍙傛暟 | 璁惧渚� ASR/璇勫垎 |
+| Default App / Task | `chat` + `conversation` | 鏃� App 鐨勭┖ Idle |
+
+Chat **灏辨槸** 璁惧涓婄殑绗竴涓� App锛屽叾 **default Task 鍚嶄负 `conversation`**锛氱瓑浜庝粖澶╁皬鏅虹‖浠剁殑鍚�/璇�/LLM 瀵硅瘽锛屽姞涓娿�屽皬鏅烘洿澶氬湴涓诲姩寮�鍙c�嶃��
+
+娌℃湁灏忕▼搴忓嵆鏃躲�佹病鏈夊埌鏈熻鍒掋�佹病鏈夌绾垮垏 App 鏃讹紝**浠讳綍鍚姩閮借繘鍏ユ垨鎭㈠杩欎竴 Task**锛屼笉瑕佸仠鍦ㄣ�岄厤濂界綉鍚庡共绛夌敤鎴峰厛璇磋瘽銆嶃��
+
+### 2.1.1 鍚姩瑙f瀽锛氶粯璁ゆ案杩滄槸 chat.conversation
+
+璁惧鏈湴鎸佷箙鍖栦竴浠� **current/pending**锛�
+
+```text
+pending_task = {
+ app_id, task_id, params, source, updated_at
+}
+```
+
+鍑哄巶 / 棣栨閰嶇綉鎴愬姛鍚庣殑鍒濆�硷細
+
+```text
+app_id = chat
+task_id = conversation
+source = default_chat
+```
+
+姣忔寮�鏈猴紙鍚蒋鍏虫満鍐嶅惎鍔ㄣ�佹柇鐢靛啀涓婄數锛夋寜涓嬮潰瑙f瀽锛�**浠庝笂鍒颁笅绗竴鏉″懡涓�**锛�
+
+```text
+1. 閫氶亾鈶� 缂撳瓨鐨勫嵆鏃� MQTT 鎸囦护灏氭湭鎵ц 鈫� 鎵ц瀹�
+2. 閫氶亾鈶� 闂归挓鍒版湡涓� pending 涓嶆槸 chat.conversation 鈫� 鎵ц璁″垝 Task
+3. 鍚﹀垯 鈫� 鎭㈠ pending锛涜嫢绌烘垨闈炴硶 鈫� chat.conversation
+```
+
+鍥犳锛�
+
+- **绗竴娆¢厤缃戞垚鍔�** 鈫� 杩涘叆绗竴涓� App 鐨� default Task锛歚chat.conversation`
+- **涔嬪悗鍏虫満鍐嶅紑** 鈫� 榛樿浠嶆槸杩欎唤 chat pending锛堢户缁亰澶� Task锛夛紝涓嶆槸绌虹櫧 Idle
+- **鍙湁** 灏忕▼搴忓嵆鏃躲�佸皬绋嬪簭璁″垝銆佺绾胯闊冲垏鎹� 鎵嶄細鏀瑰啓 pending 骞剁寮� chat
+- 璁″垝 Task 璺戝畬鑻ラ厤缃�屽洖鍒� default銆嶁啋 pending 鍐欏洖 `chat.conversation`
+
+`conversation` 涔熸槸涓�涓寮� Task锛屼笉鏄�屾病鏈� Task銆嶃�備笁閫氶亾鍒囨崲鏃讹紝chat 涓庡璇汇�佺炕璇戝悓涓�濂� `start_app_task`銆�
+
+### 2.2 App 娉ㄥ唽琛紙Server锛岄厤缃┍鍔級
+
+寤鸿鏂囦欢锛歚xiaozhi-esp32-server/main/xiaozhi-server/data/apps.json`
+锛堟垨 `config.yaml` 涓� `apps:`銆備紭鍏堢嫭绔� JSON锛岄伩鍏嶅拰妯″瀷閰嶇疆缂犲湪涓�璧枫�傦級
+
+```json
+{
+ "default_app_id": "chat",
+ "apps": {
+ "chat": {
+ "name": "鑱婂ぉ",
+ "mode": "CHAT",
+ "default_task": "conversation",
+ "tool_policy": "inherit_agent",
+ "system_prompt_mode": "agent_proactive",
+ "tasks": {
+ "conversation": {
+ "interaction_mode": "CHAT",
+ "skills": [],
+ "note": "鍘熸湁 Xiaozhi Chat Runtime锛涚敱 Runtime 涓诲姩鍙戣捣锛屼笉鍙︽寕 MCP"
+ }
+ }
+ },
+ "english_learning": {
+ "name": "鑻辫瀛︿範",
+ "mode": "PLAY_AUDIO",
+ "tool_policy": "allowlist",
+ "tools": ["switch_app", "start_repeater", "self.mp3.play", "self.mp3.stop", "self.mp3.get_list", "self.power_off"],
+ "system_prompt_mode": "none",
+ "tasks": {
+ "repeater": {
+ "interaction_mode": "PLAY_AUDIO",
+ "skills": [
+ { "name": "self.mp3.play", "args_from": ["name", "repeat", "power_off_after"] }
+ ]
+ }
+ }
+ },
+ "english_translation": {
+ "name": "鑻辫缈昏瘧",
+ "mode": "TRANSLATION",
+ "tool_policy": "allowlist",
+ "tools": ["switch_app", "translate"],
+ "system_prompt_mode": "translator",
+ "tasks": {
+ "live_translate": {
+ "interaction_mode": "TRANSLATION",
+ "skills": [
+ { "name": "translate", "side": "server_mcp" }
+ ]
+ }
+ }
+ }
+ }
+}
+```
+
+- `inherit_agent`锛氭部鐢ㄥ綋鍓� Agent 宸茬粦瀹氱殑鎻掍欢 + 璁惧 MCP锛�**鐜扮綉琛屼负**锛�
+- `allowlist`锛氬彧鎶婂悕鍗曢噷鐨勫伐鍏蜂氦缁� LLM / 缂栨帓鍣�
+- 鍥轰欢 **涓嶈鍙�** 姝ゆ枃浠�
+
+### 2.3 杩炴帴鎬侊紙Server 鍐呭瓨锛屼笉寤鸿〃锛�
+
+鍦� `ConnectionHandler` 澧炲姞锛�
+
+```text
+active_app_id: str = "chat"
+interaction_mode: str = "CHAT" # 涓庡綋鍓� App.mode 鍚屾
+```
+
+鍒囨崲 App锛�
+
+```text
+switch_app(app_id)
+ 鈫� 鏍¢獙娉ㄥ唽琛�
+ 鈫� 鍋滃綋鍓嶅姩浣滐紙澶嶈鍒� MCP self.mp3.stop锛�
+ 鈫� active_app_id / interaction_mode 鏇存柊
+ 鈫� 鍒锋柊宸ュ叿鍒楄〃锛坕nvalidate ToolManager cache锛�
+ 鈫� 鎸� system_prompt_mode 鍒囨崲鎻愮ず璇�
+```
+
+`switch_app` 鍦� **chat** 涓庡悇 App 鐧藉悕鍗曚腑閮藉彲鐢紝浠ヤ究銆岄��鍑虹炕璇� / 閫�鍑哄璇伙紝鍥炲埌鑱婂ぉ銆嶃��
+
+### 2.4 Mode 瀵瑰疄鏃堕摼璺殑鍒嗘祦
+
+鎻掑叆鐐癸細`startToChat` / `ConnectionHandler.chat` **涔嬪墠**锛坄receiveAudioHandle.py`銆乣connection.py`锛夈��
+
+| mode | ASR | LLM chat() | TTS | 璁惧鎾斁 |
+|------|-----|------------|-----|----------|
+| CHAT | 鏄� | 鏄紙鐜扮綉锛� | 鏄� | 鍚︼紙闄ら潪鎻掍欢锛� |
+| PLAY_AUDIO | 鍚︼紙蹇界暐涓婅褰撳璇濓級 | 鍚� | 浠呭惎鍔ㄦ椂涓�鍙ユ彁绀猴紙鍙�夛級 | SD MP3 |
+| TRANSLATION | 鏄� | **鍚�** | 鏄紙缈昏瘧缁撴灉锛� | 鍚� |
+
+PLAY_AUDIO 鏈熼棿鑻ユ敹鍒�**鐢ㄦ埛鍞ら啋璇�** / 鐢ㄦ埛涓诲姩 `listen`锛氬浣� `chat`锛岃蛋鐜扮綉銆備笉瑕佸湪澶嶈涓妸鐢ㄦ埛璇磋瘽閫佽繘 LLM銆�
+**RTC 闂归挓鍞ら啋**璧伴�氶亾鈶★紝涓嶈蛋杩欐潯澶嶄綅銆�
+
+### 2.5 宸ュ叿杩囨护
+
+鎻掑叆鐐癸細`UnifiedToolHandler.get_functions()` / `ToolManager.get_all_tools()`銆�
+
+```text
+if tool_policy == inherit_agent:
+ 鐜扮綉鍏ㄩ泦
+else:
+ 鍙繚鐣� allowlist 涓瓨鍦ㄧ殑宸ュ叿锛堝惈璁惧 MCP 鍚� sanitize 鍚庯級
+```
+
+缈昏瘧 App 涓� **绂佹** 鍑虹幇 weather / play_music / 璁惧鎷嶇収绛夈��
+杩欐槸銆屽彧鍋氱炕璇戠殑 MCP銆嶈兘鎴愮珛鐨勫墠鎻愶細涓嶆槸 MCP 鑷繁璋﹁櫄锛岃�屾槸 **App 鐧藉悕鍗曟妸鍒殑宸ュ叿鎽樻帀**銆�
+
+### 2.6 Client 妗嗘灦鑱岃矗锛堣杽锛�
+
+璁惧 **涓嶅疄鐜颁笟鍔� App**锛堟棤 `EnglishLearningApp` 绫伙級锛屼絾蹇呴』鏈� **杩涘叆閫氶亾缃戝叧**锛�
+
+1. Chat Runtime 鍗忚涓庣儹璺緞淇濇寔锛坔ello/listen/tts锛�
+2. 閰嶇綉鎴愬姛涓庨粯璁ゅ紑鏈鸿繘鍏� `chat.conversation`锛屽苟璁╁皬鏅� **涓诲姩寮�鍙�**锛堣 搂3.1锛�
+3. Skill 鍙 MCP 璋冪敤
+4. `self.mp3.play` 鏀寔 `repeat` + `power_off_after`锛堜粎褰� Task 閰嶇疆浜嗚 Skill锛�
+5. 闊抽浜掓枼缁х画鐢� `PauseSdCardMp3`
+6. 杞叧鏈虹户缁敤 `RequestPowerOffStandby`
+7. 鑹插畞 MQTT 鑳戒笅鍙� `START_APP_TASK` 骞舵姠鍗犲綋鍓嶄换鍔★紙搂2.7 閫氶亾鈶狅級
+8. 寮�鏈鸿В鏋� pending锛氭棤鏇撮珮浼樺厛绾у垯鎭㈠ chat.conversation锛涢椆閽熼潪 chat 璁″垝鍒欐墽琛岃鍒掞紙搂2.1.1锛�
+9. 绂荤嚎璇煶鍛戒护鍥炶皟杩涘叆鍒囨崲鍏ュ彛锛涖�屾垜浠潵鑱婂ぉ鍚� / 鑱婁細澶╁惂銆嶆槧灏勫洖 `chat.conversation`
+
+---
+
+### 2.7 涓夌杩涘叆 App / Task 鐨勬柟寮忥紙妗嗘灦鏍稿績锛�
+
+灏嗘潵杩涘叆銆屾煇 App 鐨勬煇 Task銆嶅彧鏈変笁鏉¢�氶亾銆侾hase 1 灏辨妸 **浼樺厛绾с�佹姠鍗犮�佽浇鑽枫�佸叧鏈哄敜閱�** 瀹氭锛岄伩鍏嶄互鍚庢瘡涓� App 鑷繁鎶㈣澶囥��
+
+```text
+缁熶竴鍏ュ彛锛�
+ start_app_task(app_id, task_id, params, source)
+
+source:
+ mqtt_immediate 浼樺厛绾� 3 鏈�楂�
+ schedule 浼樺厛绾� 2
+ voice 浼樺厛绾� 1
+ default_chat 浼樺厛绾� 0
+```
+
+鎶㈠崰瑙勫垯锛�
+
+```text
+鏂拌姹�.priority > 褰撳墠.priority 鈫� 涓柇褰撳墠 App/Task锛岀珛鍒诲垏鎹�
+鏂拌姹�.priority = 褰撳墠.priority 鈫� 鍚岄�氶亾鍚庡埌瑕嗙洊锛堝皬绋嬪簭杩炵偣涓ゆ锛氬悗鑰呯敓鏁堬級
+鏂拌姹�.priority < 褰撳墠.priority 鈫� 鎷掔粷鎴栧拷鐣ワ紙渚嬪澶嶈鐢� MQTT 鎷夎捣鏃讹紝璇煶鍒囩炕璇戞棤鏁堬級
+```
+
+銆屼腑鏂�嶇粺涓�鍔ㄤ綔锛堣澶囦晶锛屼笌涓氬姟鏃犲叧锛夛細
+
+```text
+鍋滄褰撳墠姝e湪璺戠殑 Skill锛堟挱銆佸惉銆乀TS銆佹樉绀轰换鍔$瓑锛�
+鈫� 鍐嶆寜銆屾柊 Task 閰嶇疆銆嶆墽琛屽叾 skills[]
+```
+
+鍥轰欢浠嶇劧 **涓嶅啓** `if english`锛汳QTT/璇煶/闂归挓鍙甫 `app_id` + `task_id` + JSON params銆�
+**鎵ц鍝簺 Skill 涓嶇敱閫氶亾鍐冲畾锛屽彧鐢辫 App/Task 鐨勯厤缃喅瀹氥��**
+
+---
+
+#### 鎶�鑳借В鏋愶紙鎵�鏈夐�氶亾鍏辩敤锛屽繀椤绘槑纭級
+
+```text
+START_APP_TASK / start_app_task
+ 鈹�
+ 鈻�
+鏌� App 娉ㄥ唽琛╗app_id].tasks[task_id]
+ 鈹�
+ 鈻�
+璇诲彇璇� Task 鐨勶細
+ interaction_mode
+ skills[] 鈫� 鐪熸瑕佽皟鐢ㄧ殑鑳藉姏
+ params schema
+ 鈹�
+ 鈻�
+鎸� skills[] 渚濇锛堟垨鎸� Step锛夎皟鐢�
+ 璁惧 MCP Skill锛堝 self.mp3.* / self.power_off / display锛�
+ 鍜�/鎴� Server Skill锛堝 translate MCP / ASR / TTS锛�
+```
+
+瑕佺偣锛�
+
+- **閫氶亾鍙礋璐c�屼綍鏃躲�佺敤浠�涔堜紭鍏堢骇鍚姩鍝釜 Task銆�**锛屼笉璐熻矗銆岃繖涓� Task 鎾斁 MP3銆嶃��
+- **`self.mp3.play` 鍙槸 `english_learning.repeater` 杩欎唤閰嶇疆閲岀殑 Skill**锛屼笉鏄墍鏈� App/Task 鐨勯粯璁ゆ妧鑳姐��
+- 缈昏瘧 Task 閰嶇殑鏄� Server MCP `translate`锛涘皢鏉ヨ窡璇讳細閰� `play` + `listen`锛涘彛绠椾細閰� `speak` + `listen` + Server 鍒ら銆傚悓涓�鏉� MQTT `START_APP_TASK` 閮借兘鍚姩銆�
+- `PLAY_MEDIA` 鏄壊瀹佸巻鍙蹭笂鐨勩�屾挱鏈湴鏂囦欢銆嶆寚浠わ紝**涓嶆槸 App 鍗忚**銆傚嵆浣挎煇 Task 鐨� Skill 璇箟涓婁篃鏄挱鏀撅紝涔熷簲璧拌 Task 閰嶇疆鐨� Skill 鍚嶏紙Phase 1 澶嶈鐢� `self.mp3.play`锛夛紝涓嶈璁╁皬绋嬪簭鐩存帴鍙� `PLAY_MEDIA` 鍐掑厖銆屾墦寮�鑻辫 App銆嶃��
+
+Phase 1 宸叉湁 Task 涓� Skill 瀵圭収锛�
+
+| App | Task | 閰嶇疆涓殑 skills锛堢ず渚嬶級 |
+|-----|------|-------------------------|
+| `chat` | `conversation`锛�**default**锛� | 鐜版湁 Chat Runtime锛涘皬鏅轰富鍔ㄥ彂璧凤紝鏃犲己鍒堕澶� MCP |
+| `english_learning` | `repeater` | `self.mp3.play`锛�+ 鍙�� `self.power_off`锛� |
+| `english_translation` | `live_translate` | Server MCP `translate` + 鐜版湁 TTS |
+
+---
+
+#### 閫氶亾鈶� 灏忕▼搴忓嵆鏃跺紑濮嬶紙浼樺厛绾ф渶楂橈級
+
+```text
+鎵嬫満灏忕▼搴�
+ 閫� App 鈫� 閫� Task 鈫� 绔嬪嵆寮�濮�
+ 鈫� 鑹插畞 MQTT 涓嬪彂 START_APP_TASK锛堝彧甯� app_id / task_id / params / source锛�
+ 鈫� 鏃犺褰撳墠鏄� chat銆佺炕璇戣繕鏄埆鐨� Task锛屽叏閮ㄤ腑鏂�
+ 鈫� start_app_task(...)
+ 鈫� 鎸夎 Task 閰嶇疆鐨� skills[] 鎵ц锛堝璇绘墠浼氳皟鍒� self.mp3.play锛涚炕璇戝垯璋� translate锛�
+```
+
+**涓轰粈涔堣蛋鑹插畞 MQTT 鑰屼笉鏄� Xiaozhi WS锛�** 灏忕▼搴忓彲鑳藉湪璁惧 Idle銆佽蒋鍏虫満鎭㈠鍓嶃�佹垨瀵硅瘽閫氶亾鏈墦寮�鏃跺氨瑕佽兘涓嬪彂锛涜壊瀹� `snlp/v1/{device_id}/cmd/ctl|task` 宸插父椹汇�俋iaozhi `type=mcp` / Server MCP 浠嶆槸 **Skill 璋冪敤闈�**銆侻QTT 鐨� `START_APP_TASK` 鏄� **Task 鍚姩闈�**銆備簩鑰呬笉瑕佹贩鎴愪竴鏉°�屾挱鏀炬寚浠ゃ�嶃��
+
+寤鸿鏂� action锛堟寕鍦ㄧ幇鏈� `BoardMqtt_DispatchAction`锛夛細
+
+```json
+{
+ "header": { "device_id": "...", "req_id": "...", "timestamp": 0 },
+ "action": "START_APP_TASK",
+ "payload": {
+ "app_id": "english_learning",
+ "task_id": "repeater",
+ "params": {
+ "name": "lesson001.mp3",
+ "repeat": 10,
+ "power_off_after": false
+ },
+ "source": "mqtt_immediate"
+ }
+}
+```
+
+`payload.params` 鐨勫瓧娈电敱 **璇� Task 鐨� schema** 鍐冲畾锛屼笉鏄叏灞�鍥哄畾涓� MP3 鏂囦欢鍚嶃�傚璇� Task 鎵嶉渶瑕� `name/repeat/power_off_after`锛涚炕璇� Task 鍙互鍑犱箮鏃� params锛涜窡璇讳互鍚庝細鏈� `resource_id` 绛夈��
+
+璁惧 ack 璧扮幇鏈� `rpt/ack`銆傝嫢褰撳墠鏈変綆浼樺厛绾т换鍔★細鍏堝仠鍐嶆寜鏂� Task 鐨� `skills[]` 鍚姩銆�
+
+**Phase 1 缂栫爜锛�** 鍗忚 + 鎶㈠崰 + 鑳藉惎鍔ㄦ湰闃舵宸叉湁 task锛坄repeater` / `translate` / 鍥� `chat`锛夈�傚皬绋嬪簭 UI 鍙悗鍋氾紝鐢� MQTT 瀹㈡埛绔仈璋冨嵆鍙��
+
+---
+
+#### 閫氶亾鈶� 灏忕▼搴� Schedule锛堜紭鍏堢骇绗簩锛夆�斺�斿叧鏈鸿嚜鍔ㄦ墽琛岀殑涓荤洰鏍�
+
+```text
+灏忕▼搴忚缃鍒�
+ 渚嬪锛氭瘡澶� 07:00 鑻辫瀛︿範 / 鍚�-澶嶈 鎴� 璺熻
+ 鈫� Server 淇濆瓨 Schedule锛堟潈濞侊級
+ 鈫� 鍚屾鍒拌澶囷細RTC 闂归挓 SET_ALARM + 鏈湴 pending_task
+ 鈫� 鍒扮偣锛�
+ 鑻ヨ澶囧紑鐫�涓斿綋鍓嶆槸 chat锛堟垨鏇翠綆浼樺厛绾э級鈫� 鍒囨崲骞跺紑濮�
+ 鑻ヨ澶囪蒋鍏虫満 / 纭欢鍏虫満鍚庣敱 RTC 鍞ら啋 鈫� 鑱旂綉鍚庣珛鍒绘墽琛� pending锛屼笉杩� chat
+```
+
+杩欐槸 **灏忔櫤纭欢閲嶅惎 / 闂归挓鍞ら啋鐨勪富瑕佷骇鍝佺洰鏍�**锛氫笉鏄负浜嗗洖鍒板緟鏈鸿亰澶╋紝鑰屾槸鍒扮偣鑷姩鍚銆佽窡璇荤瓑銆�
+
+鐜版湁浠g爜宸茬粡鍏峰鍗婃埅鑳藉姏锛�**灏氭湭鎺ュ埌 App/Task**锛�
+
+| 宸叉湁 | 缂哄彛 |
+|------|------|
+| MQTT `SET_ALARM` / `UNSET_ALARM` + DS3231 | 闂归挓鍙敜閱掞紝涓嶅甫 app/task |
+| 杞叧鏈� `RequestPowerOffStandby`锛岄椆閽� `schedule_power_off_restore_from_alarm` | `RestoreFromPowerOffStandby` 鍙� `ResumePlaylistIfUnfinished` |
+| 杞叧鏈烘湡闂村叧闂敜閱掕瘝涓庣绾垮懡浠� | 绗﹀悎銆屽叧鏈哄悗鍙湁闂归挓鑳芥媺璧锋潵銆� |
+
+**Phase 1 蹇呴』琛ョ殑璁惧璇箟锛�**
+
+```text
+NVS / SD 淇濆瓨 pending_task.json
+ { app_id, task_id, params, source: "schedule", alarm_id }
+
+闂归挓瑙﹀彂涓斾粠 power-off 鎭㈠
+ 鈫� 鑻� pending 鏄潪 chat 鐨勮鍒� Task 鈫� 鎵ц瀹冿紝绂佹鍏堥棽鑱�
+ 鈫� 鑻� pending 鏄� chat.conversation 鎴栨病鏈夐潪 chat 璁″垝 鈫� 杩涘叆 chat.conversation 骞朵富鍔ㄥ紑鍙�
+ 鈫� 璁″垝 Task 鎴愬姛缁撴潫鍚庢寜閰嶇疆鍐欏洖 chat.conversation
+```
+
+寮�鏈轰笖宸插湪 chat锛氬埌鐐瑰悓绛夋姠鍗狅紙priority 2 > 0锛夈��
+姝e湪鎵ц MQTT 鍗虫椂浠诲姟锛坧riority 3锛夋椂锛氭湰娆� Schedule 瑙﹀彂璺宠繃鎴栧欢鍚庯紝涓嶆姠 MQTT銆�
+
+Server 渚у畬鏁� Plan 琛ㄣ�佸闀挎棩鍘� UI锛�**鏈樁娈典笉寤� Java 琛�**锛涘彲鐢� `apps.json` + 璁惧闂归挓 + pending 鏂囦欢鍏堣窇閫氥��7 鐐瑰敜閱掑悗鑷姩澶嶈銆嶃�傝窡璇� Task 鏈綋浠嶉潪鏈樁娈碉紝浣� pending 鐨� `task_id` 瀛楁蹇呴』棰勭暀銆�
+
+---
+
+#### 閫氶亾鈶� 绂荤嚎璇煶浜や簰鍒囨崲锛堜紭鍏堢骇绗笁锛�
+
+纭欢 **宸茬粡瀹炵幇绂荤嚎璇煶璇嗗埆锛圗SP-SR MultiNet锛�**锛岀洰鐨勫氨鏄负灏嗘潵鍒� App / Task / InteractionMode 鍋氬噯澶囥��**褰撳墠鐗堟湰娌℃湁鎺ュ埌鍒囨崲銆�**
+
+鐜扮姸锛圫ening锛夛細
+
+| 浣嶇疆 | 琛屼负 |
+|------|------|
+| `afe_wake_word.cc` / `custom_wake_word.cc` | MultiNet 妫�娴嬬绾垮懡浠� |
+| 鍙d护 | 浠� `鍒囨崲妯″紡`锛堟嫾闊� `qie huan mo shi`锛� |
+| `Application` 鈫� `Board_Handle_Offline_Command` | 鍖归厤鍚庡彧 `ESP_LOGI`锛�**鏃犲垏鎹�** |
+| `CustomWakeWord` 鐨� `index.json` commands | 宸叉湁 `command / text / action` 缁撴瀯锛屽彲鎵╁睍锛屽皻鏈敤浜� App |
+
+鐩爣浣撻獙锛堝皢鏉ワ紝涓嶉檺浜庢湰闃舵涓変釜 App锛夛細
+
+```text
+灏忔櫤锛氱幇鍦ㄦ垜浠潵鍋氫釜娓告垙鍚э紝
+ 鏄兂鍋氭暟瀛楄儗璇点�佹垚璇帴榫欙紝
+ 杩樻槸 10 浠ュ唴鍔犲噺娉曞彛绠楋紝杩樻槸 xx锛�
+鐢ㄦ埛锛氾紙绂荤嚎鍛戒护 / 鐭彞锛�
+绋嬪簭鑳藉垽瀹氭剰鍥� 鈫� start_app_task(瀵瑰簲 app, 瀵瑰簲 task, source=voice)
+鎰忓浘涓嶆竻 鈫� 鍐嶉棶涓�娆� / 鐣欏湪褰撳墠 App
+```
+
+鍒ゅ畾鍘熷垯锛�
+
+- **浼樺厛鏈湴 MultiNet 鍙d护**锛堜綆寤惰繜銆佸彲鍦ㄦ湭寮� Xiaozhi 瀵硅瘽閫氶亾鏃跺伐浣滐級
+- 鍙d护琛ㄦ寜銆屽綋鍓嶄氦浜掓銆嶅姞杞斤紙鑿滃崟姝ュ彧鏈夊嚑涓�欓�夛紝涓嶈涓�娆℃敞鍐屽叏閮ㄤ笟鍔″彞锛�
+- 鏈湴鏃犳硶鍒ゅ畾鏃讹紝鎵嶅厑璁歌蛋浜戠 ASR+鎰忓浘锛堟鏃跺凡鍦� chat 绠$嚎锛宲riority 浠嶇畻 voice锛�
+- 璇煶 **涓嶈兘** 鎵撴柇閫氶亾鈶� MQTT 鍗虫椂浠诲姟锛屼篃 **涓嶈兘** 鎵撴柇姝e湪鎵ц鐨勯�氶亾鈶� 璁″垝浠诲姟
+
+**Phase 1 缂栫爜锛堟寕閽╋紝涓嶆槸鍋氬畬鏁存父鎴忓钩鍙帮級锛�**
+
+1. `Board_Handle_Offline_Command` 涓嶅啀绌鸿浆锛氳繘鍏� `voice_entry`锛堜緥濡傛墦寮�銆岄�� App銆嶈彍鍗曟�侊級
+2. Phase 1 鑿滃崟蹇呴』鍖呭惈鍥炲埌鑱婂ぉ锛氥�屾垜浠潵鑱婂ぉ鍚с�嶃�岃亰浼氬ぉ鍚с�嶁啋 `chat.conversation`锛堟渶甯哥敤锛�
+3. 鍏跺畠鍊欓�夛細`english_learning.repeater` / `english_translation`锛涖�屾暟瀛楄儗璇� / 鎴愯鎺ラ緳 / 鍙g畻銆嶄綔鎰忓浘妲介鐣�
+4. 涓嶅湪鍥轰欢鍐欐娓告垙涓氬姟锛涘彛浠� 鈫� `app_id+task_id` 鏄犲皠琛紙JSON/Assets锛夛紝涓� MQTT 杞借嵎鍚屼竴濂�
+
+涓庡敜閱掕瘝鍒嗗伐锛�
+
+```text
+閰嶇綉鎴愬姛 / 鏅�氬紑鏈� 鈫� chat.conversation锛堜富鍔ㄥ紑鍙o級
+銆屾垜浠潵鑱婂ぉ鍚с�嶇瓑鍙d护 鈫� chat.conversation
+鍞ら啋璇� 鈫� 鍥炲埌鎴栦繚鎸� chat.conversation锛堝彲鎺ョ潃涓诲姩涓�鍙ワ級
+绂荤嚎銆屽垏鎹㈡ā寮忋�� 鈫� 閫氶亾鈶� 閫夊叾瀹� App/Task
+RTC 闂归挓 + 闈� chat 璁″垝 鈫� 閫氶亾鈶� 璁″垝 Task锛屼笉瑕佸厛闂茶亰
+```
+
+杞叧鏈烘湡闂翠袱鑰呴兘鍏抽棴锛堢幇鏈夐�昏緫锛夛紝鍙湁闂归挓鑳芥仮澶嶃��
+
+---
+
+### 2.8 杩涘叆閫氶亾 vs Server App Runtime
+
+| 閫氶亾 | 璁惧鏄惁蹇呴』鍦ㄧ嚎 Xiaozhi WS | 鏉冨▉ |
+|------|------------------------------|------|
+| 鈶� MQTT 鍗虫椂 | 涓嶅繀鍏堝紑瀵硅瘽閫氶亾 | 灏忕▼搴忚姹傦紱璁惧绔嬪嵆鎶㈠崰鎵ц |
+| 鈶� Schedule | 鍒扮偣鏃跺啀鑱旂綉鍗冲彲 | Server 瀛樿鍒掞紱璁惧瀛橀椆閽� + pending |
+| 鈶� 璇煶 | 绾彛浠ゅ彲绂荤嚎鍒ゅ畾锛涙緞娓呭璇濋渶瑕� WS | Server 娉ㄥ唽琛紱鏈湴鍙仛鍙d护鈫抜d |
+
+涓夐�氶亾姹囧悎鍒板悓涓�濂� `start_app_task` 鈫� **鏌� Task 閰嶇疆 鈫� 璋� `skills[]`**銆侻QTT 涓嶈鍐嶄负銆屾挱鏀俱�嶅崟鐙蛋 `PLAY_MEDIA` 涓氬姟鍒嗘敮锛屽惁鍒欏悓涓�澶嶈 Task 浼氬嚭鐜颁袱濂楁墽琛屽櫒銆傚叾瀹� Task 涔熶竴鏍凤細閫氶亾涓嶇粦瀹氭煇绉� Skill銆�
+
+---
+
+## 3. App 瑙勬牸
+
+### 3.1 chat 路 default Task `conversation`
+
+杩欐槸璁惧涓婄殑 **绗竴涓� App銆侀粯璁� Task**锛屽搴斿師鏈夊皬鏅� AI 纭欢鐨勫璇濆姛鑳姐��
+
+**浣曟椂杩涘叆锛堝彧搴旇繖浜涳紝澶栧姞鍚姩瑙f瀽锛夛細**
+
+- 棣栨閰嶇綉鎴愬姛
+- 涔嬪悗姣忔寮�鏈� / 杞‖鍏虫満鍐嶅惎鍔紝涓旀病鏈夋洿楂樹紭鍏堢骇鐨勮鍒� Task
+- 鍙d护鎴栧璇濓細銆屾垜浠潵鑱婂ぉ鍚с�嶃�岃亰浼氬ぉ鍚с�嶃�佸敜閱掑洖鑱�
+- `switch_app("chat")` / MQTT `app_id=chat, task_id=conversation`
+- 鍏跺畠 Task 姝e父缁撴潫涓旈厤缃洖鍒� default
+
+**浣曟椂涓嶈繘 chat锛�** 灏忕▼搴忓嵆鏃跺紑濮嬩簡鍒殑 Task锛涢椆閽熷敜閱掔殑 pending 鏄璇�/缈昏瘧绛夐潪 chat 璁″垝銆�
+
+**鍗忚涓庣姸鎬佹満锛�** Listening 鈫� Speaking銆乭ello/listen/tts銆丄gent 鎻掍欢 **淇濇寔**銆傛鏋跺彧鎶婅繖鏉¢摼璺櫥璁颁负 `chat.conversation`銆�
+
+**涓庡師绋嬪簭鐨勫樊寮傦紙鏈樁娈佃鍋氾級锛氬皬鏅烘洿澶氬湴鍙戣捣锛岃�屼笉鏄畬鍏ㄨ鍔ㄧ瓑鐢ㄦ埛闂��**
+
+鍘熻涓猴細閰嶇綉鍚� Idle 鈫� 绛夊敜閱� 鈫� 绛夌敤鎴疯璇� 鈫� 鍐嶇瓟銆�
+鐩爣琛屼负锛氳繘鍏� `conversation` 鍚庯紝灏忔櫤 **鍏堝紑鍙�** 閭�璇疯亰澶╋紝骞跺湪鍐峰満鏃跺彲浠ュ啀鍙戣捣涓�杞紝鑰屼笉鏄竴鐩存矇榛樸��
+
+Phase 1 涓诲姩鍙戣捣鐨勬渶灏忓疄鐜帮紙涓嶅彟閫� App锛夛細
+
+```text
+杩涘叆 chat.conversation
+ 鈫� 鎵撳紑闊抽閫氶亾锛堣嫢灏氭湭鎵撳紑锛�
+ 鈫� TTS 涓诲姩鍙ワ紙閰嶇疆鎴栬鑹叉彁绀洪┍鍔級
+ 渚嬶細銆屾垜浠潵鑱婂ぉ鍚с�傘�嶃�屾兂鑱婄偣浠�涔堝憖锛熴��
+ 鈫� 杩涘叆 Listening锛岀瓑鐢ㄦ埛
+ 鈫� 涔嬪悗鎸夌幇缃� ASR 鈫� LLM 鈫� TTS
+ 鈫� 鑻ラ暱鏃堕棿鏃犵敤鎴疯璇濓細鍙啀涓诲姩涓�鍙ワ紙娆℃暟/闂撮殧鍙厤锛岄伩鍏嶆寰幆鍌績锛�
+```
+
+瀹炵幇钀界偣寤鸿锛�
+
+- Server锛歚system_prompt_mode = agent_proactive`锛堥紦鍔卞厛闂�佺煭鍙ュ彂璧凤級锛涘彲閫夎繛鎺ュ缓绔嬪悗涓嬪彂涓�鍙� `tts` 鑰屼笉绛夌敤鎴� STT
+- Client锛氶厤缃戞垚鍔� / 榛樿寮�鏈鸿В鏋愬埌 `conversation` 鍚庯紝涓嶈鍙仠鍦� Idle 绛夋寜閿紱搴旇蛋鐜版湁銆屽紑濮嬪惉銆嶈矾寰勫苟鍏佽鍏堟挱 TTS
+- 鍙d护銆屾垜浠潵鑱婂ぉ鍚с�嶄笌涓诲姩鍙ュ叡鐢ㄥ悓涓� Task锛屼笉瑕佸仛鎴愮浜屼釜鑱婂ぉ App
+
+**pending锛�** 鍏虫満鍓嶈嫢姝e湪 chat锛宲ending 淇濇寔 `chat.conversation`銆傚啀寮�鏈虹户缁 Task锛堝彲鍐嶄富鍔ㄦ墦涓嫑鍛硷紝涓嶅繀澶嶈堪鍏虫満鍓嶆渶鍚庝竴鍙ワ紝浣嗚鑹茶蹇嗕粛璧扮幇鏈� Server memory锛夈��
+
+**鍥炲綊绾㈢嚎锛�** 鐢ㄦ埛涓�鏃﹀紑濮嬭璇濓紝鎵撴柇銆乀TS銆佸凡鏈夋彃浠朵笌鐜扮綉涓�鑷淬�備富鍔ㄥ紑鍙� **涓嶅緱** 鏀� Opus 鐑矾寰勬垨 listen 璇箟銆�
+
+### 3.2 english_learning 路 鍚�-澶嶈鏈�
+
+**鐢ㄦ埛浣撻獙**
+
+```text
+杩涘叆鑻辫瀛︿範 / 寮�濮嬪璇�
+ 鈫� 鎾斁 SD 鍗′笂鎸囧畾 MP3
+ 鈫� 瀹屾暣鎾斁 N 娆★紙鏂囦欢宸插湪鏈湴锛屼笉涓嬭浇锛�
+ 鈫� 鑻� power_off_after=true锛氳蒋鍏虫満锛堢幇鏈� power-off standby锛�
+ 鈫� 鍚﹀垯鍋滃湪 Idle锛涘敜閱掑洖鍒� chat
+```
+
+**鍙傛暟锛圫erver 涓嬪彂锛岃澶囨墽琛岋級**
+
+| 鍙傛暟 | 璇存槑 |
+|------|------|
+| `name` | 涓� `self.mp3.get_list` 鐨� `files[].name` 瀹屽叏涓�鑷� |
+| `repeat` | 姝f暣鏁� N锛屾暣棣栧惊鐜鏁帮紙涓嶆槸鏈嶅姟鍣ㄨ繛璋� N 娆� play锛� |
+| `power_off_after` | bool锛岄粯璁� false |
+
+**涓轰粈涔堝惊鐜湪璁惧涓�**
+
+`self.mp3.play` 浠婃棩鍦ㄦ枃浠跺紑濮嬫挱鏀惧悗鍗宠繑鍥� `{"accepted":true}`锛�15s 鍐呭悓鍚嶉噸澶嶈皟鐢ㄨ繕浼� `duplicate_ignored`銆係erver 鏃犳硶鍙潬銆屾挱瀹屽啀璋冧笅涓�娆°�嶃�傛妸 `repeat` 鏀捐繘鎾斁鍣ㄥ惊鐜紝涓嶄緷璧栨柊鍗忚浜嬩欢锛屼篃涓嶆妸銆岃嫳璇�嶅啓杩涘浐浠躲��
+
+**缂栨帓锛圫erver锛�**
+
+鎻掍欢 `start_repeater(name, repeat, power_off_after)`锛�
+
+1. `active_app_id = english_learning`锛宍mode = PLAY_AUDIO`
+2. 鍙�� TTS锛氥�屽紑濮嬪璇� N 娆°�嶏紙鐭�佸彲鍏筹級
+3. Device MCP锛歚self.mp3.play` + 鏂板弬鏁�
+4. `Action.RECORD` 鎴� `NONE`锛�**涓嶅啀** `chat()`
+
+缂烘枃浠讹細鍏� `self.mp3.get_list`锛屾病鏈夊垯 TTS 鎻愮ず骞朵繚鎸� chat銆�
+
+**杞叧鏈�**
+
+璁惧鍦ㄧ N 娆¤嚜鐒剁粨鏉燂紙闈� stop銆侀潪 TTS 鎵撴柇锛夊悗锛岃嫢 `power_off_after`锛氳皟鐢ㄧ幇鏈� `Application::RequestPowerOffStandby()`锛堜笌 `self.power_off` 鍚屼竴璺緞锛夈��
+琚敜閱掕瘝/鎸夐敭/`self.mp3.stop` 鎵撴柇鍒� **涓嶅叧鏈�**锛屽苟搴旇 Server 鍦ㄤ笅娆� `listen` 鏃跺洖鍒� chat銆�
+
+**鐩爣鏉匡細** `sening-xiaozhi-s3`銆傚叾浠栨澘鏃� SD MP3 鏃讹細`start_repeater` 杩斿洖鏄庣‘閿欒锛屼笉寰楁敼 Chat銆�
+
+### 3.3 english_translation
+
+**鐢ㄦ埛浣撻獙**
+
+```text
+杩涘叆鑻辫缈昏瘧
+鐢ㄦ埛璇翠腑鏂� 鈫� 璁惧鎾嫳鏂�
+鐢ㄦ埛璇磋嫳鏂� 鈫� 璁惧鎾腑鏂�
+涓嶉棽鑱娿�佷笉璁茬瑧璇濄�佷笉璋冨ぉ姘�
+銆岄��鍑虹炕璇戙�嶁啋 chat
+```
+
+**Server MCP锛堟湰闃舵鏍稿績鏂板锛�**
+
+鍦� `data/.mcp_server_settings.json` 娉ㄥ唽鐙珛 MCP Server锛屼緥濡傚悕 `english-translate`锛�
+
+```json
+{
+ "mcpServers": {
+ "english-translate": {
+ "command": "python",
+ "args": ["mcp_servers/translate/server.py"]
+ }
+ }
+}
+```
+
+MCP 宸ュ叿锛堝缓璁彧鏆撮湶涓�涓級锛�
+
+```text
+translate(text, source_lang?, target_lang?)
+```
+
+瑙勫垯锛�
+
+- 鏈寚瀹氳瑷�锛氫腑鏂団啋en锛岃嫳鏂団啋zh锛涘叾瀹冭瑷�榛樿璇戞垚涓枃
+- **鍙繑鍥炶瘧鏂�**锛屾棤瑙i噴銆佹棤鎷奸煶璇俱�佹棤 follow-up 鎻愰棶
+- 瀹炵幇锛歁CP 杩涚▼鍐呯敤鐙珛銆岀炕璇戠郴缁熸彁绀恒�嶈皟鐜版湁 LLM HTTP锛堣 xiaozhi 閰嶇疆锛夛紝**涓嶈**璧� `ConnectionHandler.chat`锛堥伩鍏嶅濞冮棽鑱婏級
+- 璇� MCP 涓嶆敞鍐� play_music / 璁惧鎺у埗
+
+**瀹炴椂璺緞**
+
+```text
+ASR 鏂囨湰
+ 鈫� interaction_mode == TRANSLATION
+ 鈫� 鐩存帴 ServerMCPExecutor.execute("translate", {text})
+ 鈫� TTS 鎾� result
+ 鈫� 涓嶈繘鍏� chat()銆佷笉鍐欓暱 Dialogue锛堝彲閫夎涓�鏉� history锛�
+```
+
+涓嶈锛氬厛璁╅棽鑱� LLM銆屽喅瀹氭槸鍚︾炕璇戙�嶃�傚伐鍏风櫧鍚嶅崟 + 鏃佽矾 `chat()` 鍙屼繚闄┿��
+
+---
+
+## 4. 鏃跺簭
+
+### 4.1 澶嶈
+
+```text
+鐢ㄦ埛锛堝湪 chat锛夛細銆屽璇� lesson001 涓夐亶鐒跺悗鍏虫満銆�
+ 鈫� LLM function call start_repeater
+ 鈫� Server: app=english_learning, mode=PLAY_AUDIO
+ 鈫� MCP self.mp3.play(name=lesson001.mp3, repeat=3, power_off_after=true)
+ 鈫� 璁惧鏈湴鎾斁 3 娆�
+ 鈫� RequestPowerOffStandby
+
+鐢ㄦ埛鍞ら啋
+ 鈫� listen/detect
+ 鈫� Server 澶嶄綅 app=chat, mode=CHAT
+ 鈫� 鐜扮綉瀵硅瘽
+```
+
+### 4.2 缈昏瘧
+
+```text
+鐢ㄦ埛锛氥�屾墦寮�缈昏瘧銆�
+ 鈫� switch_app(english_translation)
+ 鈫� 宸ュ叿鍒楄〃浠� switch_app + translate
+鐢ㄦ埛璇磋瘽 鈫� ASR
+ 鈫� translate MCP 鈫� TTS
+鐢ㄦ埛锛氥�岄��鍑虹炕璇戙��
+ 鈫� switch_app(chat)
+```
+
+### 4.3 Default锛氶厤缃戜笌姣忔寮�鏈�
+
+```text
+棣栨閰嶇綉鎴愬姛
+ 鈫� pending = chat.conversation
+ 鈫� start_app_task(chat, conversation, source=default_chat)
+ 鈫� 灏忔櫤涓诲姩锛氥�屾垜浠潵鑱婂ぉ鍚с��
+ 鈫� Listening
+
+涔嬪悗鍏虫満鍐嶅惎鍔紙鏃犲埌鏈熺殑闈� chat 璁″垝锛�
+ 鈫� 璇诲彇 pending = chat.conversation
+ 鈫� 鍐嶆杩涘叆璇� Task锛屽皬鏅哄彲鍐嶄富鍔ㄦ嫑鍛�
+ 鈫� 涓嶆槸绌虹櫧 Idle 骞茬瓑闂瘽
+```
+
+### 4.3b 鐢ㄦ埛瑕佽亰澶�
+
+```text
+绂荤嚎鎴栧璇濓細銆屾垜浠潵鑱婂ぉ鍚� / 鑱婁細澶╁惂銆�
+ 鈫� start_app_task(chat, conversation, source=voice)
+ 鈫� 鎵撴柇褰撳墠鏇翠綆浼樺厛绾э紱鑻ユ鍦� MQTT 鍗虫椂浠诲姟鍒欎笉鎶�
+ 鈫� 灏忔櫤涓诲姩鎺ヨ瘽
+```
+
+### 4.4 灏忕▼搴忓嵆鏃跺紑濮嬶紙閫氶亾鈶狅級
+
+```text
+MQTT START_APP_TASK app=english_learning task=repeater
+ 鈫� 鑻ユ鍦ㄧ炕璇戞垨 chat 鎴栬闊宠彍鍗曪細鍏ㄩ儴涓柇
+ 鈫� 鎾� SD 澶嶈
+```
+
+### 4.5 璁″垝浠诲姟 + 鍏虫満鍞ら啋锛堥�氶亾鈶★級
+
+```text
+SET_ALARM 07:00 + pending_task=repeater
+ 鈫� 澶滈棿杞叧鏈猴紙power_off锛�
+ 鈫� 07:00 DS3231 闂归挓
+ 鈫� RestoreFromPowerOffStandby
+ 鈫� 鎵ц pending_task锛堜笉瑕佸厛 chat锛�
+```
+
+### 4.6 绂荤嚎璇煶鍒囨崲锛堥�氶亾鈶級
+
+```text
+鐢ㄦ埛璇淬�屽垏鎹㈡ā寮忋�嶏紙鐜版湁 MultiNet 鍙d护锛�
+ 鈫� Board_Handle_Offline_Command 杩涘叆 voice_entry
+ 鈫� 锛圥hase 1锛夊彲鍐嶅惉銆屾墦寮�缈昏瘧 / 寮�濮嬪璇� / 鍥炲埌鑱婂ぉ銆�
+ 鈫� start_app_task(source=voice)
+ 鈫� 鑻ュ綋鍓嶆槸 MQTT 鍗虫椂浠诲姟锛氬拷鐣ヨ闊冲垏鎹�
+```
+
+---
+
+## 5. Client 鏀瑰姩娓呭崟
+
+浠撳簱锛歚xiaozhi-esp32-2.2.6`
+鍘熷垯锛氭渶灏忔敼鍔紱涓氬姟鍚嶄笉涓婂浐浠躲��
+
+| ID | 浠诲姟 | 鏂囦欢 | 璇存槑 |
+|----|------|------|------|
+| C1 | `self.mp3.play` 澧炲姞 `repeat`锛坕nt锛岄粯璁� 1锛夈�乣power_off_after`锛坆ool锛岄粯璁� false锛� | `main/boards/sening-xiaozhi-s3/sening_xiaozhi_s3.cc` | 宸ュ叿鎻忚堪鍚屾鏇存柊锛沗repeat=1` 鍏煎鏃ц皟鐢� |
+| C2 | SD 鎾斁鍣ㄦ敮鎸佸惊鐜� N 娆� | `sening` 鐨� `SDCard_Play_MP3*` 瀹炵幇澶勶紙`system.c` / 鎾斁浠诲姟锛� | 鑷劧鎾畬鎵嶈鏁帮紱`self.mp3.stop`銆乀TS Pause 鎵撴柇鍒欏彇娑堝墿浣欐鏁颁笖涓嶅叧鏈� |
+| C3 | N 娆″畬鎴愪笖 `power_off_after` 鈫� `RequestPowerOffStandby` | 鍚屼笂 + `application.cc` 宸叉湁鎺ュ彛 | 绂佹鏂板啓涓�濂楀叧鏈� |
+| C4 | 淇濇寔 `PauseSdCardMp3` 涓� Chat TTS 浜掓枼 | `application.cc` | 涓嶆敼鐑矾寰勶紱澶嶈涓� TTS 浠嶅彲鏆傚仠 MP3 |
+| C5 | 纭 `self.mp3.get_status` 鑳藉尯鍒� playing / 寰幆鍓╀綑锛堝彲閫夛級 | `sening_xiaozhi_s3.cc` | Phase 1 鍙�夊瓧娈� `repeat_left`锛涢潪蹇呴』 |
+| C6 | **绂佹** 鏂板 English App 绫汇�丏eviceState銆乆iaozhi `type=task` | 鈥� | Review 绾㈢嚎 |
+| C7 | MQTT `START_APP_TASK` 鍒嗗彂 + 鎶㈠崰 + **鎸� task 閰嶇疆璋� skills[]** | `mqtt.c` 鍒嗗彂锛涙墽琛岃蛋缁熶竴 start_app_task | 涓嶈鍦� MQTT 閲屽啓姝� play MP3锛沘ck 璧� `rpt/ack` |
+| C8 | pending 鎸佷箙鍖栵紱寮�鏈鸿В鏋� 搂2.1.1 | NVS/SD + `RestoreFromPowerOffStandby` | 榛樿鎭㈠ `chat.conversation`锛涗粎闈� chat 璁″垝闂归挓鎵嶄笉杩涢棽鑱� |
+| C9 | 绂荤嚎鍛戒护鍥炶皟鏀逛负杩涘叆閫氶亾鈶紝涓嶅啀鍙墦鏃ュ織 | `system.c` `Board_Handle_Offline_Command`锛涘彛浠よ〃鍙墿 | 鍏堟敮鎸佸垏鍒板凡鏈� App/Task锛涙父鎴忕被 task_id 棰勭暀 |
+
+闈� Sening 鏉匡細涓嶆敼 SD/MQTT/RTC銆侻CP 鏃� `self.mp3.play` 鏃剁敱 Server 鎶ラ敊銆傜绾� MultiNet 鍦ㄥ叾瀹冩澘鑻ュ瓨鍦紝鍚屾牱鍙仛鎸傞挬銆佷笉鍋氳嫳璇笟鍔°��
+
+---
+
+## 6. Server 鏀瑰姩娓呭崟
+
+浠撳簱锛歚xiaozhi-esp32-server`
+鐗╃悊杈圭晫锛氬彧鍔� **xiaozhi-server (Python)**銆俶anager-api **涓嶆敼琛�**銆�
+
+### 6.1 妗嗘灦
+
+| ID | 浠诲姟 | 寤鸿璺緞 |
+|----|------|----------|
+| S1 | App 娉ㄥ唽琛ㄥ姞杞� | `core/app_runtime/registry.py` + `data/apps.json` |
+| S2 | 杩炴帴鎬侀粯璁� `chat` / `conversation` / CHAT | `core/connection.py` | 涓庤澶� pending 瀵归綈 |
+| S2b | 杩涘叆 conversation 鍚庝富鍔� TTS 鍙戣捣锛堟垜浠潵鑱婂ぉ鍚э級 | 杩炴帴娆㈣繋 / 瑙掕壊 prompt | 涓嶈绛夌敤鎴风涓�鍙ユ墠璇磋瘽锛涘彲閰嶇疆寮�鍏� |
+| S3 | `switch_app` 鎻掍欢 | `plugins_func/functions/switch_app.py` |
+| S4 | 宸ュ叿鐧藉悕鍗曡繃婊� | `unified_tool_handler.py` 鎴� `unified_tool_manager.py`锛涘垏鎹㈠悗 `refresh_tools()` |
+| S5 | `startToChat` / `chat` 鍓嶆寜 mode 鍒嗘祦 | `receiveAudioHandle.py`銆乣connection.py` |
+| S6 | **鐢ㄦ埛**鍞ら啋 / `listen.state=detect` 澶嶄綅 default app | `listenMessageHandler.py` | **涓嶈**鍦ㄩ椆閽� pending 鎵ц鏈熼棿褰撴垚鐢ㄦ埛鍞ら啋娓呬换鍔� |
+| S14 | 缁熶竴 `start_app_task(app, task, params, source)` + 浼樺厛绾� | `core/app_runtime/` | MQTT/璇煶/鎻掍欢閮借蛋杩欓噷 |
+| S15 | 閫氶亾鈶細浜戠浠呬綔鍙d护涓嶅鏃剁殑婢勬竻锛涢粯璁や笉瑕嗙洊 MQTT/Schedule | 鍚屼笂 | 涓庣绾垮懡浠ゆ槧灏勮〃瀵归綈 |
+
+chat 涓� S4 蹇呴』绛変环浜庣幇缃戯紙`inherit_agent`锛夛紝鍚﹀垯 default App 楠屾敹澶辫触銆�
+
+### 6.2 澶嶈
+
+| ID | 浠诲姟 | 寤鸿璺緞 |
+|----|------|----------|
+| S7 | `start_repeater` 鎻掍欢 | `plugins_func/functions/start_repeater.py` |
+| S8 | 閫氳繃 DeviceMCPExecutor 璋� `self.mp3.play` | 澶嶇敤鐜版湁 device MCP锛屼笉瑕佽蛋 `play_music` |
+| S9 | Agent 鎻掍欢缁戝畾 | 绠$悊绔垨閰嶇疆鎶� `switch_app`銆乣start_repeater` 鍔犲埌鐩爣 Agent锛沜hat 闇�瑕佽兘鍒� App |
+
+### 6.3 缈昏瘧 MCP
+
+| ID | 浠诲姟 | 寤鸿璺緞 |
+|----|------|----------|
+| S10 | 鐙珛 MCP Server 杩涚▼锛屽彧娉ㄥ唽 `translate` | `main/xiaozhi-server/mcp_servers/translate/server.py` |
+| S11 | 鍐欏叆 `data/.mcp_server_settings.json` | 鐜版湁 `ServerMCPManager` 浼氬姞杞� |
+| S12 | TRANSLATION 妯″紡锛欰SR 鍚庣洿鎺� `execute_tool(translate)` 鍐� TTS | 涓嶈鍏� `chat()` |
+| S13 | translator 绯荤粺鎻愮ず锛堜粎 switch_app 鏃剁粰 LLM 鐢級 | 鐭細璇存槑褰撳墠鏄炕璇戞満銆侀��鍑洪渶璋冪敤 switch_app |
+
+`translate` 鐨� LLM 璋冪敤搴斾娇鐢ㄧ嫭绔嬬煭 prompt锛屾俯搴︿綆锛岀姝� function call銆�
+
+### 6.4 閰嶇疆绀轰緥锛圫11锛�
+
+```json
+"english-translate": {
+ "command": "python",
+ "args": ["mcp_servers/translate/server.py"]
+}
+```
+
+宸ヤ綔鐩綍涓� `xiaozhi-server`銆備緷璧栵細鐜版湁 `mcp` Python 鍖咃紙Server 宸茬敤锛夈��
+
+---
+
+## 7. 鍗忚涓庢秷鎭�
+
+Phase 1 **涓嶆柊澧�** Xiaozhi WS `type=task`銆傚皬绋嬪簭/璁″垝璧� **鑹插畞 MQTT**銆�
+
+| 鏂瑰悜 | 娑堟伅 | 鐢ㄩ�� |
+|------|------|------|
+| 涓嬭 WS | 鐜版湁 `type=mcp` `tools/call` | play / stop / power_off / 璁惧鍒楄〃 |
+| 涓嬭 WS | 鐜版湁 `type=tts` | 缈昏瘧缁撴灉銆佸璇绘彁绀恒�佽闊宠彍鍗曟彁闂� |
+| 涓婅 WS | 鐜版湁 `listen` / `mcp` result | 鐢ㄦ埛鍞ら啋澶嶄綅銆佸伐鍏疯繑鍥� |
+| 涓嬭 MQTT | **鏂�** `START_APP_TASK` | 閫氶亾鈶犲嵆鏃跺紑濮嬶紱鍙甫 `source=schedule` 浣滆ˉ鍙� |
+| 涓嬭 MQTT | 鐜版湁 `SET_ALARM` | 閫氶亾鈶″埌鐐瑰敜閱� |
+| 涓婅 MQTT | 鐜版湁 `rpt/ack` | 鍚姩/澶辫触鍥炴墽 |
+| 鈥� | 涓嶆柊澧� play.completed | 鐢辫澶囧唴閮� repeat 娑堝寲 |
+
+MCP play 鍙傛暟鍚戝悗鍏煎锛氭棫瀹㈡埛绔棤 `repeat` 鏃舵寜 1 娆°�佷笉鍏虫満銆�
+
+---
+
+## 8. 鏁版嵁搴� / API
+
+| 椤� | Phase 1 |
+|----|---------|
+| MySQL 鏂拌〃 | 鏃� |
+| `/task/instance` | 鏃� |
+| chat_history | 缈昏瘧鍙户缁寜鐜扮綉涓婃姤鏂囨湰锛涗笉寮哄埗鏂� audio 瀛楁 |
+| Redis | 涓嶅瓨 App 鐘舵�侊紙杩炴帴鍐呭瓨鍗冲彲锛涙柇绾垮嵆鍥� default锛� |
+
+Xiaozhi WS 鏂嚎閲嶈繛 = 鏂� `session_id` = 榛樿 chat銆�
+**渚嬪锛�** 璁惧鏈湴 `pending_task`锛堥�氶亾鈶★級鍦ㄩ椆閽熷敜閱掑悗浠嶇劧鎵ц锛屼笉渚濊禆鏃� `session_id`銆備笉鎭㈠銆屽崐鎴鐢ㄦ埛鎵撴柇鐨勫璇汇�嶃��
+
+---
+
+## 9. 瀹炴柦椤哄簭锛堝缓璁� PR 鍒囩墖锛�
+
+```text
+P1-A 妗嗘灦楠ㄦ灦
+ S1鈥揝5銆丼14銆乻witch_app
+ + 杩涘叆閫氶亾锛氫紭鍏堢骇銆丆7 鎶㈠崰銆丆9 绂荤嚎鍙d护鎸傞挬銆丆8 pending 褰㈢姸
+ 楠屾敹锛氶粯璁や粛鏄畬鏁� Chat锛泂witch_app 寰�杩斾笉鐮村潖瀵硅瘽
+ 楠屾敹锛歁QTT START_APP_TASK 鑳芥墦鏂� chat锛涚绾裤�屽垏鎹㈡ā寮忋�嶄笉鍐嶆槸绌� log
+
+P1-B 缈昏瘧 App
+ S10鈥揝13
+ 楠屾敹锛氱炕璇� App 鍐呭彧鍑鸿瘧鏂囷紱鏃犲ぉ姘�/闊充箰宸ュ叿锛涢��鍑哄洖 chat
+
+P1-C 澶嶈鏈�
+ C1鈥揅4 + S7鈥揝9
+ 楠屾敹锛歋D 鏂囦欢鎾� N 娆★紱鍙�夎蒋鍏虫満锛涙墦鏂笉鍏虫満锛汣hat TTS 浠嶈兘鏆傚仠 MP3
+
+P1-D 鍥炲綊
+ 涓嶅垏 App 鐨勯暱瀵硅瘽銆佹墦鏂�佸凡鏈夋彃浠躲�丱TA銆侀厤缃�
+```
+
+鍏� A 鍐� B 鍐� C锛氱炕璇戜笉渚濊禆 SD锛屽彲鍦ㄩ潪 Sening 涓婂厛娴嬫鏋躲��
+
+---
+
+## 10. 娴嬭瘯璁″垝
+
+### 10.1 chat default锛坄conversation`锛�
+
+- [ ] 棣栨閰嶇綉鎴愬姛鍚庤嚜鍔ㄨ繘鍏� chat.conversation锛屼笖灏忔櫤 **鍏堣璇�**
+- [ ] 鍏虫満鍐嶅紑锛堟棤鍏跺畠璁″垝锛変粛杩涘叆 chat.conversation锛屼笉鏄函 Idle
+- [ ] 鐢ㄦ埛璇磋瘽鍚庣殑鎵撴柇銆乀TS銆佸師鎻掍欢涓庣幇缃戜竴鑷�
+- [ ] 銆屾垜浠潵鑱婂ぉ鍚� / 鑱婁細澶╁惂銆嶅垏鍥� conversation
+- [ ] 浠庝笉璋冪敤鍏跺畠閫氶亾鏃讹紝涓嶄細鑷繁璺冲埌澶嶈/缈昏瘧
+
+### 10.2 妗嗘灦
+
+- [ ] `switch_app("english_translation")` 鍚庡伐鍏峰垪琛ㄤ笉鍚� weather
+- [ ] `switch_app("chat")` 鍚庡伐鍏峰垪琛ㄦ仮澶�
+- [ ] 闈炴硶 `app_id` 琚嫆缁濆苟鐣欏湪褰撳墠 App
+- [ ] MQTT `START_APP_TASK` 姝e湪缈昏瘧鏃惰兘涓柇骞舵敼鍘诲璇伙紙閫氶亾鈶� > 鈶�/chat锛�
+- [ ] 澶嶈鐢� MQTT 鎷夎捣鏃讹紝绂荤嚎銆屽垏鎹㈡ā寮忋�嶄笉鑳芥妸瀹冨垏璧�
+- [ ] 杞叧鏈� + `SET_ALARM` + pending 澶嶈锛氶椆閽熷敜閱掑悗鐩存帴鎾斁锛屼笉鍏堣繘鍏ラ棽鑱�
+- [ ] 鐢ㄦ埛鍞ら啋璇嶄粛鍥炲埌 chat锛堜笌闂归挓鍞ら啋鍖哄垎锛�
+- [ ] 銆屽垏鎹㈡ā寮忋�嶈Е鍙� `voice_entry`锛堜笉鍐嶅彧鎵撴棩蹇楋級
+
+### 10.3 缈昏瘧
+
+- [ ] 涓枃鍙� 鈫� 鑻辨枃 TTS
+- [ ] 鑻辨枃鍙� 鈫� 涓枃 TTS
+- [ ] 涓嶅洖绛斻�岃涓瑧璇濄�嶇被闂茶亰锛堝簲缈昏瘧璇ュ彞鎴栨彁绀鸿瑕佽瘧鐨勫唴瀹癸紝鐢� translate prompt 瀹氫竴鏉″苟鍐欒繘娴嬭瘯锛�
+- [ ] 銆岄��鍑虹炕璇戙�嶅洖鍒� chat锛岀瑧璇濈被鎭㈠鍙敤
+
+### 10.4 澶嶈锛圫ening + 鍗′笂宸叉湁 mp3锛�
+
+- [ ] `get_list` 鏈夋枃浠�
+- [ ] N=2 瀹屾暣鍚畬涓ら亶
+- [ ] `power_off_after=false` 缁撴潫鍚庡彲鍞ら啋杩� chat
+- [ ] `power_off_after=true` 缁撴潫鍚庤繘鐜版湁杞叧鏈猴紙鍞ら啋璇嶅叧闂紝涓庝粖鏃� `self.power_off` 涓�鑷达級
+- [ ] 鎾斁涓� `stop` 鎴栧敜閱掞細鍓╀綑閬嶆暟鍙栨秷銆�**涓�**鍏虫満
+- [ ] 澶嶈涓嫢鏈� Chat TTS锛孧P3 pause/resume 琛屼负涓嶅洖褰�
+
+### 10.5 璐熶緥
+
+- [ ] 鏃� `self.mp3.play` 鐨勬澘璋冪敤 `start_repeater` 鈫� 璇煶閿欒锛孋hat 浠嶅彲鐢�
+- [ ] 涓嶅嚭鐜� `play_music` 鎾绋�
+
+---
+
+## 11. 楠屾敹鏍囧噯锛圥hase 1 Done锛�
+
+1. **妗嗘灦锛�** 绗竴涓� App 鐨� default Task 鏄� `chat.conversation`锛涢厤缃戝悗涓庨粯璁ゅ紑鏈洪兘杩涘叆瀹冿紱鍚屾椂鍙湁涓�涓� active銆�
+2. **涓诲姩鑱婂ぉ锛�** 杩涘叆 conversation 鍚庡皬鏅哄厛鍙戣捣锛屼笉鏄畬鍏ㄨ鍔ㄧ瓑闂�傜敤鎴峰簲绛斿悗鐨勫璇濋摼璺笌鐜扮綉涓�鑷淬��
+3. **涓夐�氶亾锛�** 鍙湁灏忕▼搴忓嵆鏃躲�佽鍒掋�佺绾垮垏鎹㈡墠浼氱寮� chat锛涢椆閽熼潪 chat pending 鎵嶈烦杩囬棽鑱娿��
+4. **澶嶈 / 缈昏瘧锛�** 瑙佸師楠屾敹銆�
+5. **杈圭晫锛�** 鏃� Java 鏂拌〃銆佹棤 Xiaozhi `type=task`銆佹棤鍥轰欢鑻辫涓氬姟绫汇�佹棤闊抽鐑矾寰勯噸鍐欍��
+
+---
+
+## 12. 椋庨櫓涓庢妧鏈�猴紙璁板叆 Phase 2锛屾湰闃舵鎺ュ彈锛�
+
+| 鍊� | 璇存槑 |
+|----|------|
+| 澶嶈浠� Sening | 鍏跺畠鏉块渶鍚庣画鎶� play_audio 鎻愪负鏍稿績 Skill |
+| 鏃� play.completed 鍗忚 | Server 涓嶇煡閬撴挱鍒扮鍑犻亶锛涚姸鎬佷互璁惧涓哄噯 |
+| App 鐘舵�佷笉钀藉簱 | WS 鏂嚎鍗冲洖 chat锛�**pending_task 蹇呴』鍦ㄨ澶囨湰鍦�**锛屽惁鍒欓椆閽熷敜閱掍涪澶� |
+| 缈昏瘧 MCP 鍐嶈皟涓�娆� LLM | 姣斻�屾敼绯荤粺鎻愮ず纭瘧銆嶅涓�璺筹紱鎹㈢殑鏄伐鍏烽殧绂讳笌銆屽彧鍋氱炕璇戙�� |
+| 鍙屾帶鍒堕潰 | 灏忕▼搴�/闂归挓璧� snlp锛涘璇�/鎶�鑳借蛋 Xiaozhi WS+MCP銆備笉鍦� Phase 1 鍚堝苟 |
+| 绂荤嚎鍛戒护璇嶅緢灏� | 鐜颁粎銆屽垏鎹㈡ā寮忋�嶏紱鑿滃崟鍙d护涓庢父鎴� Task 鍚庣画鍔� Assets/鍛戒护琛� |
+| 灏忕▼搴忔湭鍋� | 閫氶亾鈶犫憽鐢� MQTT/闂归挓鑱旇皟锛沀I 鍚庣疆 |
+
+---
+
+## 13. 缂栫爜绾︽潫锛堢粰瀹炴柦 Agent锛�
+
+1. 鍏堣鍐嶆敼锛歚application.cc` JSON/MCP銆乣sening_xiaozhi_s3.cc` mp3/power_off銆乣connection.py`銆乣receiveAudioHandle.py`銆乣unified_tool_handler.py`銆乣server_mcp/mcp_manager.py`銆�
+2. 姣忎釜 PR 鍏堣窇 Chat 鍥炲綊鍐嶅悎涓氬姟銆�
+3. 绂佹涓� Demo 鏂板 `EnglishRepeatManager` / `TranslationApp` 鍥轰欢绫汇��
+4. 绂佹淇敼 `hello/listen/tts/stt` 璇箟銆�
+5. `repeat` 榛樿 1锛屼繚璇佹棫 MCP 璋冪敤鑰呭吋瀹广��
+6. 杩涘叆閫氶亾蹇呴』甯� `source` 涓庝紭鍏堢骇锛涚姝笁涓叆鍙e悇鍐欎竴濂� stop/play銆�
+7. 鏅�氬紑鏈� / 閰嶇綉鎴愬姛绂佹鍋氭垚銆屽彧 Idle 绛夌敤鎴枫�嶏紱榛樿 pending 鏄� `chat.conversation` 骞跺厑璁稿厛 TTS銆�
+8. 浠呭綋闂归挓 pending 涓� **闈� chat 璁″垝 Task** 鏃讹紝绂佹鍏堣蛋闂茶亰銆�
+
+瀹炴柦浠� **P1-A** 寮�濮嬨�傛湭瀹屾垚 A锛堝惈杩涘叆閫氶亾鎸傞挬涓庨粯璁� conversation锛変箣鍓嶄笉瑕佹敼鎾斁鍣ㄥ惊鐜��
diff --git a/xiaozhi-esp32-2.2.6/main/application.cc b/xiaozhi-esp32-2.2.6/main/application.cc
index e6e0d7d..643d8db 100644
--- a/xiaozhi-esp32-2.2.6/main/application.cc
+++ b/xiaozhi-esp32-2.2.6/main/application.cc
@@ -393,6 +393,7 @@
Schedule([this]() {
// Play the success sound to indicate the device is ready
audio_service_.PlaySound(Lang::Sounds::OGG_SUCCESS);
+ Board::GetInstance().OnDeviceReady();
});
}
@@ -841,6 +842,21 @@
void Application::StopListening() {
xEventGroupSetBits(event_group_, MAIN_EVENT_STOP_LISTENING);
+}
+
+void Application::SendListenDetect(const std::string& text, bool open_if_idle) {
+ Schedule([this, text, open_if_idle]() {
+ if (!protocol_ || text.empty()) {
+ return;
+ }
+ if (protocol_->IsAudioChannelOpened()) {
+ protocol_->SendWakeWordDetected(text);
+ return;
+ }
+ if (open_if_idle) {
+ WakeWordInvoke(text);
+ }
+ });
}
void Application::HandleToggleChatEvent() {
@@ -1417,7 +1433,9 @@
SetDeviceState(kDeviceStateIdle);
}
audio_service_.EnableWakeWordDetection(true);
- BoardMqtt_ResumePlaylistIfUnfinished();
+ if (!Board::GetInstance().OnRestoredFromPowerOff(network_ready)) {
+ BoardMqtt_ResumePlaylistIfUnfinished();
+ }
});
}
diff --git a/xiaozhi-esp32-2.2.6/main/application.h b/xiaozhi-esp32-2.2.6/main/application.h
index 1f72922..8842bf0 100644
--- a/xiaozhi-esp32-2.2.6/main/application.h
+++ b/xiaozhi-esp32-2.2.6/main/application.h
@@ -104,6 +104,7 @@
* Sends MAIN_EVENT_STOP_LISTENING to be handled in Run()
*/
void StopListening();
+ void SendListenDetect(const std::string& text, bool open_if_idle = true);
void Reboot();
void WakeWordInvoke(const std::string& wake_word);
diff --git a/xiaozhi-esp32-2.2.6/main/boards/common/board.h b/xiaozhi-esp32-2.2.6/main/boards/common/board.h
index c76470d..de2dd14 100644
--- a/xiaozhi-esp32-2.2.6/main/boards/common/board.h
+++ b/xiaozhi-esp32-2.2.6/main/boards/common/board.h
@@ -85,6 +85,8 @@
virtual void SetPowerSaveLevel(PowerSaveLevel level) = 0;
virtual std::string GetBoardJson() = 0;
virtual std::string GetDeviceStatusJson() = 0;
+ virtual void OnDeviceReady() {}
+ virtual bool OnRestoredFromPowerOff(bool network_ready) { (void)network_ready; return false; }
};
#define DECLARE_BOARD(BOARD_CLASS_NAME) \
diff --git a/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/app_task.cc b/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/app_task.cc
new file mode 100644
index 0000000..b9dcc3d
--- /dev/null
+++ b/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/app_task.cc
@@ -0,0 +1,371 @@
+#include "app_task.h"
+
+#include "application.h"
+#include "mqtt.h"
+#include "system.h"
+
+#include <cJSON.h>
+#include <esp_log.h>
+#include <nvs.h>
+#include <strings.h>
+#include <string.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#define TAG "app_task"
+#define APP_TASK_NVS_NS "app_task"
+#define APP_TASK_NVS_KEY "pending"
+#define APP_TASK_JSON_MAX 768
+#define APP_TASK_ID_MAX 32
+#define APP_TASK_SOURCE_MAX 24
+
+static char s_current_app[APP_TASK_ID_MAX] = "chat";
+static char s_current_task[APP_TASK_ID_MAX] = "conversation";
+static char s_current_source[APP_TASK_SOURCE_MAX] = "default_chat";
+static int s_current_priority = 0;
+
+typedef struct {
+ const char *app_id;
+ const char *task_id;
+ const char *skills[4];
+} app_task_skill_map_t;
+
+/* Configuration table: channels only pass app_id/task_id; skills are looked up here. */
+static const app_task_skill_map_t k_task_skills[] = {
+ {"chat", "conversation", {NULL, NULL, NULL, NULL}},
+ {"english_learning", "repeater", {"self.mp3.play", NULL, NULL, NULL}},
+ {"english_translation", "live_translate", {"translate", NULL, NULL, NULL}},
+};
+
+int Board_AppTask_SourcePriority(const char *source)
+{
+ if (!source) {
+ return 1;
+ }
+ if (strcmp(source, "mqtt_immediate") == 0) {
+ return 3;
+ }
+ if (strcmp(source, "schedule") == 0) {
+ return 2;
+ }
+ if (strcmp(source, "voice") == 0) {
+ return 1;
+ }
+ return 0;
+}
+
+int Board_AppTask_CurrentPriority(void)
+{
+ return s_current_priority;
+}
+
+static const app_task_skill_map_t *find_task_spec(const char *app_id, const char *task_id)
+{
+ if (!app_id) {
+ return NULL;
+ }
+ const char *tid = (task_id && task_id[0]) ? task_id : NULL;
+ for (size_t i = 0; i < sizeof(k_task_skills) / sizeof(k_task_skills[0]); ++i) {
+ if (strcmp(k_task_skills[i].app_id, app_id) != 0) {
+ continue;
+ }
+ if (!tid || strcmp(k_task_skills[i].task_id, tid) == 0) {
+ return &k_task_skills[i];
+ }
+ }
+ return NULL;
+}
+
+static bool is_chat_task(const char *app_id, const char *task_id)
+{
+ return app_id && strcmp(app_id, "chat") == 0 &&
+ (!task_id || task_id[0] == '\0' || strcmp(task_id, "conversation") == 0);
+}
+
+static void copy_id(char *dest, size_t dest_size, const char *src, const char *fallback)
+{
+ const char *value = (src && src[0]) ? src : fallback;
+ strlcpy(dest, value, dest_size);
+}
+
+esp_err_t Board_SavePendingTask(const char *app_id, const char *task_id, const char *params_json,
+ const char *source, const char *alarm_id)
+{
+ cJSON *root = cJSON_CreateObject();
+ if (!root) {
+ return ESP_ERR_NO_MEM;
+ }
+ cJSON_AddStringToObject(root, "app_id", app_id && app_id[0] ? app_id : "chat");
+ cJSON_AddStringToObject(root, "task_id", task_id && task_id[0] ? task_id : "conversation");
+ cJSON_AddStringToObject(root, "source", source && source[0] ? source : "default_chat");
+ cJSON_AddStringToObject(root, "alarm_id", alarm_id ? alarm_id : "");
+ if (params_json && params_json[0]) {
+ cJSON *params = cJSON_Parse(params_json);
+ if (params) {
+ cJSON_AddItemToObject(root, "params", params);
+ } else {
+ cJSON_AddRawToObject(root, "params", params_json);
+ }
+ } else {
+ cJSON_AddItemToObject(root, "params", cJSON_CreateObject());
+ }
+
+ char *printed = cJSON_PrintUnformatted(root);
+ cJSON_Delete(root);
+ if (!printed) {
+ return ESP_ERR_NO_MEM;
+ }
+
+ nvs_handle_t handle;
+ esp_err_t err = nvs_open(APP_TASK_NVS_NS, NVS_READWRITE, &handle);
+ if (err == ESP_OK) {
+ err = nvs_set_str(handle, APP_TASK_NVS_KEY, printed);
+ if (err == ESP_OK) {
+ err = nvs_commit(handle);
+ }
+ nvs_close(handle);
+ }
+ free(printed);
+ return err;
+}
+
+static bool load_pending_fields(char *app_id, size_t app_size, char *task_id, size_t task_size,
+ char *source, size_t source_size, char *params_json, size_t params_size)
+{
+ nvs_handle_t handle;
+ if (nvs_open(APP_TASK_NVS_NS, NVS_READONLY, &handle) != ESP_OK) {
+ return false;
+ }
+ char buf[APP_TASK_JSON_MAX];
+ size_t length = sizeof(buf);
+ esp_err_t err = nvs_get_str(handle, APP_TASK_NVS_KEY, buf, &length);
+ nvs_close(handle);
+ if (err != ESP_OK) {
+ return false;
+ }
+
+ cJSON *root = cJSON_Parse(buf);
+ if (!root) {
+ return false;
+ }
+ const cJSON *app = cJSON_GetObjectItem(root, "app_id");
+ const cJSON *task = cJSON_GetObjectItem(root, "task_id");
+ const cJSON *src = cJSON_GetObjectItem(root, "source");
+ const cJSON *params = cJSON_GetObjectItem(root, "params");
+ copy_id(app_id, app_size, cJSON_IsString(app) ? app->valuestring : NULL, "chat");
+ copy_id(task_id, task_size, cJSON_IsString(task) ? task->valuestring : NULL, "conversation");
+ copy_id(source, source_size, cJSON_IsString(src) ? src->valuestring : NULL, "default_chat");
+ if (params_json && params_size > 0) {
+ params_json[0] = '\0';
+ if (params) {
+ char *printed = cJSON_PrintUnformatted(params);
+ if (printed) {
+ strlcpy(params_json, printed, params_size);
+ free(printed);
+ }
+ }
+ }
+ cJSON_Delete(root);
+ return app_id[0] != '\0';
+}
+
+static void preempt_current_skills(void)
+{
+ SDCard_Stop_Play();
+ BoardMqtt_StopPlaylistPlayback();
+ Application::GetInstance().Schedule([]() {
+ auto state = Application::GetInstance().GetDeviceState();
+ if (state == kDeviceStateSpeaking) {
+ Application::GetInstance().AbortSpeaking(kAbortReasonNone);
+ }
+ });
+}
+
+static void notify_server(const char *app_id, const char *task_id, const char *source, bool open_if_idle)
+{
+ char detect[160];
+ snprintf(detect, sizeof(detect), "[app_task]%s|%s|%s",
+ app_id && app_id[0] ? app_id : "chat",
+ task_id && task_id[0] ? task_id : "conversation",
+ source && source[0] ? source : "default_chat");
+ Application::GetInstance().SendListenDetect(detect, open_if_idle);
+}
+
+static void dispatch_skill(const char *skill, const char *app_id, const char *task_id,
+ const char *params_json, const char *source)
+{
+ if (!skill || skill[0] == '\0') {
+ notify_server(app_id, task_id, source, true);
+ return;
+ }
+ if (strcmp(skill, "self.mp3.play") == 0) {
+ char name[SDCARD_PATH_MAX_LEN] = {0};
+ float speed = 1.0f;
+ int repeat = 1;
+ bool power_off_after = false;
+ if (params_json && params_json[0]) {
+ cJSON *params = cJSON_Parse(params_json);
+ if (params) {
+ const cJSON *name_item = cJSON_GetObjectItem(params, "name");
+ const cJSON *speed_item = cJSON_GetObjectItem(params, "speed");
+ const cJSON *repeat_item = cJSON_GetObjectItem(params, "repeat");
+ const cJSON *power_item = cJSON_GetObjectItem(params, "power_off_after");
+ if (cJSON_IsString(name_item) && name_item->valuestring) {
+ strlcpy(name, name_item->valuestring, sizeof(name));
+ }
+ if (cJSON_IsNumber(speed_item)) {
+ speed = static_cast<float>(speed_item->valuedouble);
+ }
+ if (cJSON_IsNumber(repeat_item)) {
+ repeat = repeat_item->valueint;
+ }
+ if (cJSON_IsBool(power_item)) {
+ power_off_after = cJSON_IsTrue(power_item);
+ } else if (cJSON_IsNumber(power_item)) {
+ power_off_after = power_item->valueint != 0;
+ }
+ cJSON_Delete(params);
+ }
+ }
+ if (name[0] == '\0') {
+ ESP_LOGW(TAG, "self.mp3.play missing name");
+ return;
+ }
+ if (repeat < 1) {
+ repeat = 1;
+ }
+ esp_err_t err = SDCard_Play_MP3_With_Speed_Repeat(name, speed, repeat, power_off_after);
+ if (err != ESP_OK) {
+ ESP_LOGW(TAG, "self.mp3.play failed: %s", esp_err_to_name(err));
+ Board_OnSkillPlaybackFinished(true, false);
+ return;
+ }
+ notify_server(app_id, task_id, source, false);
+ return;
+ }
+ if (strcmp(skill, "translate") == 0) {
+ notify_server(app_id, task_id, source, true);
+ return;
+ }
+ ESP_LOGW(TAG, "unsupported skill: %s", skill);
+}
+
+esp_err_t Board_StartAppTask(const char *app_id, const char *task_id, const char *params_json, const char *source)
+{
+ if (!app_id || app_id[0] == '\0') {
+ return ESP_ERR_INVALID_ARG;
+ }
+ const char *src = (source && source[0]) ? source : "mqtt_immediate";
+ const char *tid = (task_id && task_id[0]) ? task_id : "conversation";
+ const app_task_skill_map_t *spec = find_task_spec(app_id, tid);
+ if (!spec) {
+ ESP_LOGW(TAG, "unknown app/task: %s/%s", app_id, tid);
+ return ESP_ERR_NOT_FOUND;
+ }
+
+ int new_priority = Board_AppTask_SourcePriority(src);
+ if (new_priority < s_current_priority) {
+ ESP_LOGW(TAG, "reject %s/%s source=%s prio %d < current %d",
+ app_id, tid, src, new_priority, s_current_priority);
+ return ESP_ERR_INVALID_STATE;
+ }
+
+ preempt_current_skills();
+ copy_id(s_current_app, sizeof(s_current_app), app_id, "chat");
+ copy_id(s_current_task, sizeof(s_current_task), tid, "conversation");
+ copy_id(s_current_source, sizeof(s_current_source), src, "default_chat");
+ s_current_priority = new_priority;
+ Board_SavePendingTask(app_id, tid, params_json, src, NULL);
+
+ ESP_LOGI(TAG, "start app_task %s/%s source=%s", app_id, tid, src);
+ if (!spec->skills[0]) {
+ dispatch_skill(NULL, app_id, tid, params_json, src);
+ return ESP_OK;
+ }
+ for (int i = 0; i < 4 && spec->skills[i]; ++i) {
+ dispatch_skill(spec->skills[i], app_id, tid, params_json, src);
+ }
+ return ESP_OK;
+}
+
+void Board_OnSkillPlaybackFinished(bool stopped, bool power_off_requested)
+{
+ (void)power_off_requested;
+ ESP_LOGI(TAG, "skill playback %s", stopped ? "stopped" : "finished");
+ copy_id(s_current_app, sizeof(s_current_app), "chat", "chat");
+ copy_id(s_current_task, sizeof(s_current_task), "conversation", "conversation");
+ copy_id(s_current_source, sizeof(s_current_source), "default_chat", "default_chat");
+ s_current_priority = 0;
+ Board_SavePendingTask("chat", "conversation", "{}", "default_chat", NULL);
+}
+
+void Board_ResolveBootOrRestore(bool from_alarm_wake)
+{
+ char app_id[APP_TASK_ID_MAX] = "chat";
+ char task_id[APP_TASK_ID_MAX] = "conversation";
+ char source[APP_TASK_SOURCE_MAX] = "default_chat";
+ char params[APP_TASK_JSON_MAX] = "{}";
+ bool has_pending = load_pending_fields(app_id, sizeof(app_id), task_id, sizeof(task_id),
+ source, sizeof(source), params, sizeof(params));
+ if (!has_pending) {
+ Board_SavePendingTask("chat", "conversation", "{}", "default_chat", NULL);
+ Board_StartAppTask("chat", "conversation", "{}", "default_chat");
+ return;
+ }
+
+ if (from_alarm_wake && strcmp(source, "schedule") == 0 && !is_chat_task(app_id, task_id)) {
+ ESP_LOGI(TAG, "alarm restore runs pending %s/%s", app_id, task_id);
+ Board_StartAppTask(app_id, task_id, params, "schedule");
+ return;
+ }
+
+ Board_StartAppTask("chat", "conversation", "{}", "default_chat");
+}
+
+void Board_EnterVoiceEntry(void)
+{
+ if (s_current_priority >= 2) {
+ ESP_LOGI(TAG, "ignore voice_entry, current priority=%d", s_current_priority);
+ return;
+ }
+ Application::GetInstance().SendListenDetect("[voice_entry]", true);
+}
+
+bool Board_Handle_Voice_Command(const char *command)
+{
+ if (!command || command[0] == '\0') {
+ return false;
+ }
+ if (strcmp(command, "鍒囨崲妯″紡") == 0 || strcmp(command, "qie huan mo shi") == 0) {
+ Board_EnterVoiceEntry();
+ return true;
+ }
+ if (strstr(command, "鎴戜滑鏉ヨ亰澶╁惂") || strstr(command, "鑱婁細澶╁惂")) {
+ if (s_current_priority >= 2) {
+ ESP_LOGI(TAG, "ignore voice chat switch, priority=%d", s_current_priority);
+ return true;
+ }
+ Board_StartAppTask("chat", "conversation", "{}", "voice");
+ return true;
+ }
+ if (strstr(command, "鎵撳紑缈昏瘧") || strstr(command, "寮�濮嬬炕璇�")) {
+ if (s_current_priority >= 2) {
+ return true;
+ }
+ Board_StartAppTask("english_translation", "live_translate", "{}", "voice");
+ return true;
+ }
+ if (strstr(command, "寮�濮嬪璇�") || strstr(command, "鎵撳紑澶嶈")) {
+ if (s_current_priority >= 2) {
+ return true;
+ }
+ Board_EnterVoiceEntry();
+ return true;
+ }
+ if (strstr(command, "鏁板瓧鑳岃") || strstr(command, "鎴愯鎺ラ緳") || strstr(command, "鍙g畻")) {
+ ESP_LOGI(TAG, "reserved voice slot: %s", command);
+ Board_EnterVoiceEntry();
+ return true;
+ }
+ return false;
+}
diff --git a/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/app_task.h b/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/app_task.h
new file mode 100644
index 0000000..21deddf
--- /dev/null
+++ b/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/app_task.h
@@ -0,0 +1,24 @@
+#ifndef APP_TASK_H
+#define APP_TASK_H
+
+#include "esp_err.h"
+#include <stdbool.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+int Board_AppTask_SourcePriority(const char *source);
+esp_err_t Board_StartAppTask(const char *app_id, const char *task_id, const char *params_json, const char *source);
+esp_err_t Board_SavePendingTask(const char *app_id, const char *task_id, const char *params_json, const char *source, const char *alarm_id);
+void Board_ResolveBootOrRestore(bool from_alarm_wake);
+void Board_EnterVoiceEntry(void);
+bool Board_Handle_Voice_Command(const char *command);
+void Board_OnSkillPlaybackFinished(bool stopped, bool power_off_requested);
+int Board_AppTask_CurrentPriority(void);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/mp3_player.cc b/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/mp3_player.cc
index 0cf4cf0..cc87034 100644
--- a/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/mp3_player.cc
+++ b/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/mp3_player.cc
@@ -1,7 +1,9 @@
#include "system.h"
+#include "app_task.h"
#include "audio_codec.h"
#include "board.h"
+#include "application.h"
#include <esp_log.h>
#include <freertos/FreeRTOS.h>
@@ -27,11 +29,14 @@
typedef struct {
char path[SDCARD_PATH_MAX_LEN];
float speed;
+ int repeat;
+ bool power_off_after;
} mp3_player_task_arg_t;
static TaskHandle_t mp3_player_task_handle;
static volatile bool mp3_stop_requested;
static volatile bool mp3_pause_requested;
+static volatile int mp3_repeat_left;
static std::vector<int16_t> convert_channels(const int16_t *input, size_t frame_count, uint8_t src_channels, uint8_t dst_channels)
{
@@ -127,6 +132,8 @@
char path[SDCARD_PATH_MAX_LEN];
strlcpy(path, task_arg->path, sizeof(path));
float speed = task_arg->speed;
+ int repeat = task_arg->repeat < 1 ? 1 : task_arg->repeat;
+ bool power_off_after = task_arg->power_off_after;
free(task_arg);
FILE *fp = fopen(path, "rb");
@@ -134,6 +141,7 @@
ESP_LOGE(TAG, "open MP3 failed: %s", path);
mp3_player_task_handle = nullptr;
mp3_stop_requested = false;
+ mp3_repeat_left = 0;
vTaskDelete(nullptr);
return;
}
@@ -144,6 +152,7 @@
fclose(fp);
mp3_player_task_handle = nullptr;
mp3_stop_requested = false;
+ mp3_repeat_left = 0;
vTaskDelete(nullptr);
return;
}
@@ -167,6 +176,7 @@
fclose(fp);
mp3_player_task_handle = nullptr;
mp3_stop_requested = false;
+ mp3_repeat_left = 0;
vTaskDelete(nullptr);
return;
}
@@ -184,11 +194,27 @@
ESP_LOGE(TAG, "no memory for MP3 playback");
ret = ESP_AUDIO_ERR_MEM_LACK;
} else {
- ESP_LOGI(TAG, "play MP3: %s, speed: %.2fx", path, speed);
+ ESP_LOGI(TAG, "play MP3: %s, speed: %.2fx, repeat: %d, power_off_after: %d", path, speed, repeat, power_off_after ? 1 : 0);
if (wait_if_paused()) {
codec->EnableOutput(true);
}
+
+ for (int pass = 0; pass < repeat && ret == ESP_AUDIO_ERR_OK && !mp3_stop_requested; ++pass) {
+ mp3_repeat_left = repeat - pass;
+ if (pass > 0) {
+ if (fseek(fp, 0, SEEK_SET) != 0) {
+ ESP_LOGE(TAG, "rewind MP3 failed: %s", path);
+ break;
+ }
+ esp_audio_simple_dec_close(decoder);
+ decoder = nullptr;
+ ret = esp_audio_simple_dec_open(&dec_cfg, &decoder);
+ if (ret != ESP_AUDIO_ERR_OK || !decoder) {
+ ESP_LOGE(TAG, "reopen MP3 decoder failed: %d", ret);
+ break;
+ }
+ }
while (ret == ESP_AUDIO_ERR_OK && !mp3_stop_requested) {
if (!wait_if_paused()) {
@@ -287,6 +313,7 @@
break;
}
}
+ }
}
if (rate_cvt_handle) {
@@ -297,27 +324,44 @@
}
free(in_buf);
free(out_buf);
- esp_audio_simple_dec_close(decoder);
+ if (decoder) {
+ esp_audio_simple_dec_close(decoder);
+ }
esp_audio_simple_dec_unregister_default();
esp_audio_dec_unregister_default();
fclose(fp);
ESP_LOGI(TAG, "MP3 playback %s: %s", mp3_stop_requested ? "stopped" : "finished", path);
+ bool stopped = mp3_stop_requested;
+ bool should_power_off = power_off_after && !stopped;
mp3_player_task_handle = nullptr;
mp3_stop_requested = false;
mp3_pause_requested = false;
+ mp3_repeat_left = 0;
+ if (should_power_off) {
+ Application::GetInstance().RequestPowerOffStandby();
+ }
+ Board_OnSkillPlaybackFinished(stopped, power_off_after);
vTaskDelete(nullptr);
}
extern "C" esp_err_t SDCard_Play_MP3_File(const char *path)
{
- return SDCard_Play_MP3_File_With_Speed(path, 1.0f);
+ return SDCard_Play_MP3_File_With_Speed_Repeat(path, 1.0f, 1, false);
}
extern "C" esp_err_t SDCard_Play_MP3_File_With_Speed(const char *path, float speed)
{
+ return SDCard_Play_MP3_File_With_Speed_Repeat(path, speed, 1, false);
+}
+
+extern "C" esp_err_t SDCard_Play_MP3_File_With_Speed_Repeat(const char *path, float speed, int repeat, bool power_off_after)
+{
if (!path || path[0] == '\0' || !is_supported_speed(speed)) {
return ESP_ERR_INVALID_ARG;
+ }
+ if (repeat < 1) {
+ repeat = 1;
}
bool start_paused = mp3_pause_requested;
@@ -335,13 +379,17 @@
}
strlcpy(arg->path, path, sizeof(arg->path));
arg->speed = speed;
+ arg->repeat = repeat;
+ arg->power_off_after = power_off_after;
mp3_stop_requested = false;
mp3_pause_requested = start_paused;
+ mp3_repeat_left = repeat;
BaseType_t ret = xTaskCreate(mp3_player_task, "mp3_player", MP3_PLAYER_STACK_SIZE, arg, MP3_PLAYER_PRIORITY, &mp3_player_task_handle);
if (ret != pdPASS) {
free(arg);
mp3_player_task_handle = nullptr;
+ mp3_repeat_left = 0;
return ESP_FAIL;
}
@@ -378,3 +426,8 @@
{
return mp3_player_task_handle != nullptr && mp3_pause_requested;
}
+
+extern "C" int SDCard_Get_Repeat_Left(void)
+{
+ return mp3_repeat_left;
+}
diff --git a/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/mqtt.c b/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/mqtt.c
index fce4e82..219255b 100644
--- a/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/mqtt.c
+++ b/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/mqtt.c
@@ -15,6 +15,7 @@
#include "freertos/task.h"
#include "mqtt_client.h"
#include "system.h"
+#include "app_task.h"
#define BOARD_MQTT_KEEPALIVE_SEC 120
#define BOARD_MQTT_STATE_REPORT_INTERVAL_MS 5000
@@ -1222,8 +1223,20 @@
}
esp_err_t err = DS2312_Set_Alarm_Id_Saved(id->valuestring, hour, minute, second, repeat_mask);
- return err == ESP_OK ? BoardMqtt_ResultOk("alarm updated") :
- BoardMqtt_ResultFail(BOARD_MQTT_ERR_EXEC_FAILED, esp_err_to_name(err));
+ if (err != ESP_OK) {
+ return BoardMqtt_ResultFail(BOARD_MQTT_ERR_EXEC_FAILED, esp_err_to_name(err));
+ }
+
+ cJSON *app_id = cJSON_GetObjectItem(payload, "app_id");
+ cJSON *task_id = cJSON_GetObjectItem(payload, "task_id");
+ if (cJSON_IsString(app_id) && app_id->valuestring && app_id->valuestring[0]) {
+ const char *tid = (cJSON_IsString(task_id) && task_id->valuestring) ? task_id->valuestring : "";
+ cJSON *params = cJSON_GetObjectItem(payload, "params");
+ char *params_json = params ? cJSON_PrintUnformatted(params) : NULL;
+ Board_SavePendingTask(app_id->valuestring, tid, params_json, "schedule", id->valuestring);
+ free(params_json);
+ }
+ return BoardMqtt_ResultOk("alarm updated");
}
static board_mqtt_action_result_t BoardMqtt_HandleUnsetAlarm(cJSON *payload)
@@ -1852,6 +1865,35 @@
return BoardMqtt_ResultOk("manifest accepted");
}
+static board_mqtt_action_result_t BoardMqtt_HandleStartAppTask(cJSON *payload)
+{
+ if (!payload) {
+ return BoardMqtt_ResultFail(BOARD_MQTT_ERR_INVALID_PARAM, "missing payload");
+ }
+ cJSON *app_id = cJSON_GetObjectItem(payload, "app_id");
+ cJSON *task_id = cJSON_GetObjectItem(payload, "task_id");
+ cJSON *source = cJSON_GetObjectItem(payload, "source");
+ cJSON *params = cJSON_GetObjectItem(payload, "params");
+ if (!cJSON_IsString(app_id) || !app_id->valuestring || app_id->valuestring[0] == '\0') {
+ return BoardMqtt_ResultFail(BOARD_MQTT_ERR_INVALID_PARAM, "missing app_id");
+ }
+ const char *tid = (cJSON_IsString(task_id) && task_id->valuestring) ? task_id->valuestring : "";
+ const char *src = (cJSON_IsString(source) && source->valuestring) ? source->valuestring : "mqtt_immediate";
+ char *params_json = params ? cJSON_PrintUnformatted(params) : NULL;
+ esp_err_t err = Board_StartAppTask(app_id->valuestring, tid, params_json, src);
+ free(params_json);
+ if (err == ESP_ERR_NOT_FOUND) {
+ return BoardMqtt_ResultFail(BOARD_MQTT_ERR_NOT_FOUND, "unknown app or task");
+ }
+ if (err == ESP_ERR_INVALID_STATE) {
+ return BoardMqtt_ResultFail(BOARD_MQTT_ERR_BUSY, "preempted by higher priority task");
+ }
+ if (err != ESP_OK) {
+ return BoardMqtt_ResultFail(BOARD_MQTT_ERR_EXEC_FAILED, esp_err_to_name(err));
+ }
+ return BoardMqtt_ResultOk("app task started");
+}
+
static board_mqtt_action_result_t BoardMqtt_DispatchAction(const char *req_id, const char *action, cJSON *payload)
{
if (strcmp(action, "SET_VOLUME") == 0) return BoardMqtt_HandleSetVolume(payload);
@@ -1866,6 +1908,7 @@
if (strcmp(action, "UPDATE_MANIFEST") == 0 || strcmp(action, "SYNC_COURSE_TASK") == 0) return BoardMqtt_HandleUpdateManifest(req_id, payload);
if (strcmp(action, "DOWNLOAD") == 0) return BoardMqtt_HandleDownload(req_id, payload);
if (strcmp(action, "OTA_UPDATE") == 0) return BoardMqtt_HandleOtaUpdate(req_id, payload);
+ if (strcmp(action, "START_APP_TASK") == 0) return BoardMqtt_HandleStartAppTask(payload);
return BoardMqtt_ResultFail(BOARD_MQTT_ERR_UNSUPPORTED, "unsupported action");
}
diff --git a/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/sening_xiaozhi_s3.cc b/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/sening_xiaozhi_s3.cc
index b7a2ba1..7d087c6 100644
--- a/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/sening_xiaozhi_s3.cc
+++ b/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/sening_xiaozhi_s3.cc
@@ -9,6 +9,7 @@
#include "power_save_timer.h"
#include "adc_battery_monitor.h"
#include "wifi_manager.h"
+#include "app_task.h"
#include <esp_log.h>
#include <esp_efuse_table.h>
@@ -722,20 +723,27 @@
static float last_mp3_play_speed = 1.0f;
static int64_t last_mp3_play_time = 0;
mcp_server.AddTool("self.mp3.play",
- "Play one MP3 file from the SD card. The name argument must exactly match a files[].name from self.mp3.get_list. Optional speed supports only 0.5, 0.75, 1.0, 1.25, 1.5, or 2.0; non-1.0 speeds change pitch and affect only SD card MP3 playback. For play, random play, switch song, or named-song requests, call this tool before telling the user playback has started. Do not call this tool repeatedly for the same user request after it has returned accepted=true.",
+ "Play one MP3 file from the SD card. The name argument must exactly match a files[].name from self.mp3.get_list. Optional speed supports only 0.5, 0.75, 1.0, 1.25, 1.5, or 2.0; non-1.0 speeds change pitch and affect only SD card MP3 playback. Optional repeat is the number of full-file loops (default 1). Optional power_off_after requests existing power-off standby after a natural finish, not after stop. For play, random play, switch song, or named-song requests, call this tool before telling the user playback has started. Do not call this tool repeatedly for the same user request after it has returned accepted=true.",
PropertyList({
Property("name", kPropertyTypeString),
- Property("speed", kPropertyTypeNumber, 1.0)
+ Property("speed", kPropertyTypeNumber, 1.0),
+ Property("repeat", kPropertyTypeInteger, 1, 1, 99),
+ Property("power_off_after", kPropertyTypeBoolean, false)
}),
[](const PropertyList& properties) -> ReturnValue {
std::string name = properties["name"].value<std::string>();
float speed = static_cast<float>(properties["speed"].value<double>());
+ int repeat = properties["repeat"].value<int>();
+ bool power_off_after = properties["power_off_after"].value<bool>();
+ if (repeat < 1) {
+ repeat = 1;
+ }
int64_t now = esp_timer_get_time();
if (name == last_mp3_play_name && std::fabs(speed - last_mp3_play_speed) < 0.001f && now - last_mp3_play_time < 15000000) {
return std::string("{\"accepted\":true,\"duplicate_ignored\":true}");
}
- esp_err_t err = SDCard_Play_MP3_With_Speed(name.data(), speed);
+ esp_err_t err = SDCard_Play_MP3_With_Speed_Repeat(name.data(), speed, repeat, power_off_after);
if (err != ESP_OK) {
return std::string("{\"accepted\":false,\"error\":\"") + esp_err_to_name(err) +
"\",\"retry\":\"use exact files[].name from self.mp3.get_list\"}";
@@ -743,7 +751,8 @@
last_mp3_play_name = name;
last_mp3_play_speed = speed;
last_mp3_play_time = now;
- return std::string("{\"accepted\":true}");
+ return std::string("{\"accepted\":true,\"repeat\":") + std::to_string(repeat) +
+ ",\"power_off_after\":" + (power_off_after ? "true" : "false") + "}";
});
mcp_server.AddTool("self.mp3.get_status",
@@ -751,7 +760,8 @@
PropertyList(),
[](const PropertyList& properties) -> ReturnValue {
return std::string("{\"playing\":") + (SDCard_Is_Playing() ? "true" : "false") +
- ",\"tts_paused\":" + (SDCard_Is_Paused() ? "true" : "false") + "}";
+ ",\"tts_paused\":" + (SDCard_Is_Paused() ? "true" : "false") +
+ ",\"repeat_left\":" + std::to_string(SDCard_Get_Repeat_Left()) + "}";
});
mcp_server.AddTool("self.mp3.stop",
@@ -855,6 +865,21 @@
InitializeTools();
}
+ void OnDeviceReady() override {
+ xTaskCreate([](void *arg) {
+ (void)arg;
+ vTaskDelay(pdMS_TO_TICKS(1500));
+ Board_ResolveBootOrRestore(false);
+ vTaskDelete(nullptr);
+ }, "boot_app_task", 4096, nullptr, 3, nullptr);
+ }
+
+ bool OnRestoredFromPowerOff(bool network_ready) override {
+ (void)network_ready;
+ Board_ResolveBootOrRestore(true);
+ return true;
+ }
+
virtual void SetNetworkEventCallback(NetworkEventCallback callback) override {
WifiBoard::SetNetworkEventCallback([this, callback](NetworkEvent event, const std::string& data) {
if (event == NetworkEvent::Connected) {
diff --git a/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/system.c b/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/system.c
index 87b649b..a9c94be 100644
--- a/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/system.c
+++ b/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/system.c
@@ -1,5 +1,6 @@
#include "system.h"
#include "config.h"
+#include "app_task.h"
#include "display/lvgl_display/jpg/jpeg_to_image.h"
#include <esp_heap_caps.h>
@@ -130,11 +131,15 @@
if (!command || Board_Is_Power_Off()) {
return false;
}
+ if (Board_Handle_Voice_Command(command)) {
+ return true;
+ }
if (strcmp(command, OFFLINE_COMMAND_WORD) != 0 && strcmp(command, OFFLINE_COMMAND_WORD_PINYIN) != 0) {
return false;
}
Board_Offline_Command_Callback();
+ Board_EnterVoiceEntry();
return true;
}
@@ -547,6 +552,11 @@
esp_err_t SDCard_Play_MP3_With_Speed(char *mp3_name, float speed)
{
+ return SDCard_Play_MP3_With_Speed_Repeat(mp3_name, speed, 1, false);
+}
+
+esp_err_t SDCard_Play_MP3_With_Speed_Repeat(char *mp3_name, float speed, int repeat, bool power_off_after)
+{
if (!is_supported_mp3_speed(speed)) {
ESP_LOGW(TAG, "invalid MP3 playback speed: %.2f", speed);
SDCard_Resume_Play();
@@ -560,7 +570,7 @@
return ESP_ERR_NOT_FOUND;
}
- esp_err_t err = SDCard_Play_MP3_File_With_Speed(path, speed);
+ esp_err_t err = SDCard_Play_MP3_File_With_Speed_Repeat(path, speed, repeat, power_off_after);
if (err != ESP_OK) {
SDCard_Resume_Play();
ESP_LOGE(TAG, "start MP3 playback failed: %s", esp_err_to_name(err));
diff --git a/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/system.h b/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/system.h
index 930de40..cced675 100644
--- a/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/system.h
+++ b/xiaozhi-esp32-2.2.6/main/boards/sening-xiaozhi-s3/system.h
@@ -202,6 +202,7 @@
* @return ESP_OK 琛ㄧず鎾斁浠诲姟宸叉垚鍔熷惎鍔紱ESP_ERR_INVALID_ARG 琛ㄧず閫熷害鎴栨枃浠跺悕闈炴硶銆�
*/
esp_err_t SDCard_Play_MP3_With_Speed(char *mp3_name, float speed);
+esp_err_t SDCard_Play_MP3_With_Speed_Repeat(char *mp3_name, float speed, int repeat, bool power_off_after);
void SDCard_Show_Matching_JPG_For_MP3_Path(const char *mp3_path);
/**
@@ -272,6 +273,8 @@
* @return ESP_OK 琛ㄧず鎾斁浠诲姟宸叉垚鍔熷惎鍔紱ESP_ERR_INVALID_ARG 琛ㄧず璺緞鎴栭�熷害闈炴硶銆�
*/
esp_err_t SDCard_Play_MP3_File_With_Speed(const char *path, float speed);
+esp_err_t SDCard_Play_MP3_File_With_Speed_Repeat(const char *path, float speed, int repeat, bool power_off_after);
+int SDCard_Get_Repeat_Left(void);
esp_err_t SDCard_Map_Business_Path(const char *business_path, char *sd_path, size_t sd_path_size);
esp_err_t SDCard_Prepare_Business_File(const char *business_path,
char *final_path,
diff --git a/xiaozhi-esp32-server/.gitignore b/xiaozhi-esp32-server/.gitignore
index bbab869..c70c2b4 100644
--- a/xiaozhi-esp32-server/.gitignore
+++ b/xiaozhi-esp32-server/.gitignore
@@ -161,6 +161,7 @@
.secrets.yaml
.private_config.yaml
.env.development
+!main/manager-web/.env.development
# model files
main/xiaozhi-server/models/SenseVoiceSmall/model.pt
diff --git a/xiaozhi-esp32-server/main/manager-api/src/main/java/xiaozhi/common/constant/Constant.java b/xiaozhi-esp32-server/main/manager-api/src/main/java/xiaozhi/common/constant/Constant.java
index 506e676..055b576 100644
--- a/xiaozhi-esp32-server/main/manager-api/src/main/java/xiaozhi/common/constant/Constant.java
+++ b/xiaozhi-esp32-server/main/manager-api/src/main/java/xiaozhi/common/constant/Constant.java
@@ -329,7 +329,7 @@
/**
* 鏃犳晥鍥轰欢URL
*/
- String INVALID_FIRMWARE_URL = "http://xiaozhi.server.com:8002/xiaozhi/otaMag/download/NOT_ACTIVATED_FIRMWARE_THIS_IS_A_INVALID_URL";
+ String INVALID_FIRMWARE_URL = "http://192.168.0.107:8002/xiaozhi/otaMag/download/NOT_ACTIVATED_FIRMWARE_THIS_IS_A_INVALID_URL";
/**
* 瀛楀吀绫诲瀷
diff --git a/xiaozhi-esp32-server/main/manager-api/src/main/java/xiaozhi/modules/device/service/impl/DeviceServiceImpl.java b/xiaozhi-esp32-server/main/manager-api/src/main/java/xiaozhi/modules/device/service/impl/DeviceServiceImpl.java
index 5acde87..d785a7f 100644
--- a/xiaozhi-esp32-server/main/manager-api/src/main/java/xiaozhi/modules/device/service/impl/DeviceServiceImpl.java
+++ b/xiaozhi-esp32-server/main/manager-api/src/main/java/xiaozhi/modules/device/service/impl/DeviceServiceImpl.java
@@ -233,7 +233,7 @@
if (StringUtils.isBlank(wsUrl) || wsUrl.equals("null")) {
log.error("WebSocket鍦板潃鏈厤缃紝璇风櫥褰曟櫤鎺у彴锛屽湪鍙傛暟绠$悊鎵惧埌銆恠erver.websocket銆戦厤缃�");
- wsUrl = "ws://xiaozhi.server.com:8000/xiaozhi/v1/";
+ wsUrl = "ws://192.168.0.107:8000/xiaozhi/v1/";
websocket.setUrl(wsUrl);
} else {
String[] wsUrls = wsUrl.split("\\;");
@@ -242,7 +242,7 @@
websocket.setUrl(wsUrls[RandomUtil.randomInt(0, wsUrls.length)]);
} else {
log.error("WebSocket鍦板潃鏈厤缃紝璇风櫥褰曟櫤鎺у彴锛屽湪鍙傛暟绠$悊鎵惧埌銆恠erver.websocket銆戦厤缃�");
- websocket.setUrl("ws://xiaozhi.server.com:8000/xiaozhi/v1/");
+ websocket.setUrl("ws://192.168.0.107:8000/xiaozhi/v1/");
}
}
diff --git a/xiaozhi-esp32-server/main/manager-api/src/main/resources/db/changelog/202504151206.sql b/xiaozhi-esp32-server/main/manager-api/src/main/resources/db/changelog/202504151206.sql
index cb51cef..4113e24 100644
--- a/xiaozhi-esp32-server/main/manager-api/src/main/resources/db/changelog/202504151206.sql
+++ b/xiaozhi-esp32-server/main/manager-api/src/main/resources/db/changelog/202504151206.sql
@@ -20,7 +20,7 @@
-- 澧炲姞鏄惁鍏佽鐢ㄦ埛娉ㄥ唽鍙傛暟
delete from `sys_params` where id in (103,104);
INSERT INTO `sys_params` (id, param_code, param_value, value_type, param_type, remark) VALUES (103, 'server.allow_user_register', 'false', 'boolean', 1, '鏄惁杩愯绠$悊鍛樹互澶栫殑浜烘敞鍐�');
-INSERT INTO `sys_params` (id, param_code, param_value, value_type, param_type, remark) VALUES (104, 'server.fronted_url', 'http://xiaozhi.server.com', 'string', 1, '涓嬪彂鍏綅楠岃瘉鐮佹椂鏄剧ず鐨勬帶鍒堕潰鏉垮湴鍧�');
+INSERT INTO `sys_params` (id, param_code, param_value, value_type, param_type, remark) VALUES (104, 'server.fronted_url', 'http://192.168.0.107', 'string', 1, '涓嬪彂鍏綅楠岃瘉鐮佹椂鏄剧ず鐨勬帶鍒堕潰鏉垮湴鍧�');
-- 淇CosyVoiceSiliconflow闊宠壊
delete from `ai_tts_voice` where tts_model_id = 'TTS_CosyVoiceSiliconflow';
diff --git a/xiaozhi-esp32-server/main/manager-web/.env.development b/xiaozhi-esp32-server/main/manager-web/.env.development
new file mode 100644
index 0000000..f10c154
--- /dev/null
+++ b/xiaozhi-esp32-server/main/manager-web/.env.development
@@ -0,0 +1,4 @@
+VUE_APP_API_BASE_URL=/xiaozhi
+VUE_APP_PUBLIC_PATH=/
+# 鏄惁寮�鍚疌DN
+VUE_APP_USE_CDN=false
diff --git a/xiaozhi-esp32-server/main/manager-web/src/apis/api.js b/xiaozhi-esp32-server/main/manager-web/src/apis/api.js
index 759ee8a..813fd49 100644
--- a/xiaozhi-esp32-server/main/manager-web/src/apis/api.js
+++ b/xiaozhi-esp32-server/main/manager-web/src/apis/api.js
@@ -20,7 +20,7 @@
* 寮�鍙戞椂鑷姩璇诲彇浣跨敤.env.development鏂囦欢
* 缂栬瘧鏃惰嚜鍔ㄨ鍙栦娇鐢�.env.production鏂囦欢
*/
-const DEV_API_SERVICE = process.env.VUE_APP_API_BASE_URL
+const DEV_API_SERVICE = process.env.VUE_APP_API_BASE_URL || '/xiaozhi'
/**
* 鏍规嵁寮�鍙戠幆澧冭繑鍥炴帴鍙rl
diff --git a/xiaozhi-esp32-server/main/manager-web/src/components/AddModelDialog.vue b/xiaozhi-esp32-server/main/manager-web/src/components/AddModelDialog.vue
index b331480..d249815 100644
--- a/xiaozhi-esp32-server/main/manager-web/src/components/AddModelDialog.vue
+++ b/xiaozhi-esp32-server/main/manager-web/src/components/AddModelDialog.vue
@@ -68,10 +68,10 @@
</el-form-item>
</el-form>
- <teleport v-if="chunkedCallInfoFields.length">
+ <template v-if="chunkedCallInfoFields.length">
<div class="section-title">{{ $t('modelConfigDialog.callInfo') }}</div>
<div class="divider"></div>
- </teleport>
+ </template>
<el-form :model="formData.configJson" label-width="auto" label-position="left" class="custom-form">
<div v-for="(row, rowIndex) in chunkedCallInfoFields" :key="rowIndex" class="form-row">
diff --git a/xiaozhi-esp32-server/main/manager-web/src/components/CacheViewer.vue b/xiaozhi-esp32-server/main/manager-web/src/components/CacheViewer.vue
index 5bd234e..5dfafdb 100644
--- a/xiaozhi-esp32-server/main/manager-web/src/components/CacheViewer.vue
+++ b/xiaozhi-esp32-server/main/manager-web/src/components/CacheViewer.vue
@@ -1,7 +1,7 @@
<template>
<el-dialog
:title="$t('cache.dialogTitle')"
- :visible.sync="visible"
+ :visible.sync="dialogVisible"
width="70%"
:before-close="handleClose"
>
@@ -88,6 +88,7 @@
},
data() {
return {
+ dialogVisible: this.visible,
isLoading: true,
cacheAvailable: false,
cacheData: {
@@ -100,9 +101,13 @@
},
watch: {
visible(newVal) {
+ this.dialogVisible = newVal;
if (newVal) {
this.loadCacheData();
}
+ },
+ dialogVisible(val) {
+ this.$emit('update:visible', val);
}
},
methods: {
@@ -168,8 +173,11 @@
window.location.reload();
},
- handleClose() {
- this.$emit('update:visible', false);
+ handleClose(done) {
+ this.dialogVisible = false;
+ if (typeof done === 'function') {
+ done();
+ }
}
}
};
diff --git a/xiaozhi-esp32-server/main/manager-web/src/components/FirmwareDialog.vue b/xiaozhi-esp32-server/main/manager-web/src/components/FirmwareDialog.vue
index fab0412..a123e60 100644
--- a/xiaozhi-esp32-server/main/manager-web/src/components/FirmwareDialog.vue
+++ b/xiaozhi-esp32-server/main/manager-web/src/components/FirmwareDialog.vue
@@ -1,7 +1,7 @@
<template>
<CustomDialog
:title="title"
- :visible.sync="visible"
+ :visible.sync="dialogVisible"
width="800px"
@confirm="submit"
@close="cancel"
@@ -72,6 +72,7 @@
data() {
return {
+ dialogVisible: this.visible,
uploadProgress: 0,
uploadStatus: '',
isUploading: false,
@@ -93,6 +94,14 @@
}
}
},
+ watch: {
+ visible(val) {
+ this.dialogVisible = val;
+ },
+ dialogVisible(val) {
+ this.$emit('update:visible', val);
+ }
+ },
computed: {
isTypeDisabled() {
// 濡傛灉鏈塱d锛岃鏄庢槸缂栬緫妯″紡锛岀鐢ㄧ被鍨嬮�夋嫨
diff --git a/xiaozhi-esp32-server/main/manager-web/src/components/ModelEditDialog.vue b/xiaozhi-esp32-server/main/manager-web/src/components/ModelEditDialog.vue
index ac6b18e..6c9360c 100644
--- a/xiaozhi-esp32-server/main/manager-web/src/components/ModelEditDialog.vue
+++ b/xiaozhi-esp32-server/main/manager-web/src/components/ModelEditDialog.vue
@@ -57,10 +57,10 @@
</el-form-item>
</el-form>
- <teleport v-if="chunkedCallInfoFields.length">
+ <template v-if="chunkedCallInfoFields.length">
<div class="section-title">{{ $t("modelConfigDialog.callInfo") }}</div>
<div class="section-divider"></div>
- </teleport>
+ </template>
<el-form :model="form.configJson" ref="callInfoForm" label-width="auto" label-position="left">
<template>
diff --git a/xiaozhi-esp32-server/main/manager-web/src/components/ParamDialog.vue b/xiaozhi-esp32-server/main/manager-web/src/components/ParamDialog.vue
index 0f6de8d..d436549 100644
--- a/xiaozhi-esp32-server/main/manager-web/src/components/ParamDialog.vue
+++ b/xiaozhi-esp32-server/main/manager-web/src/components/ParamDialog.vue
@@ -1,7 +1,7 @@
<template>
<CustomDialog
:title="title"
- :visible.sync="visible"
+ :visible.sync="dialogVisible"
width="600px"
class="param-dialog-wrapper"
@confirm="submit"
@@ -84,6 +84,7 @@
},
data() {
return {
+ dialogVisible: this.visible,
dialogKey: Date.now(),
saving: false,
valueTypeOptions: [
@@ -157,6 +158,7 @@
},
watch: {
visible(newVal) {
+ this.dialogVisible = newVal;
if (newVal) {
if (this.form.paramValue) {
// 濡傛灉鏄� json 绫诲瀷锛屾牸寮忓寲鏄剧ず
@@ -178,6 +180,9 @@
// 褰撳璇濇鍏抽棴鏃讹紝閲嶇疆saving鐘舵��
this.saving = false;
}
+ },
+ dialogVisible(val) {
+ this.$emit('update:visible', val);
}
}
};
diff --git a/xiaozhi-esp32-server/main/manager-web/src/components/ProviderDialog.vue b/xiaozhi-esp32-server/main/manager-web/src/components/ProviderDialog.vue
index 713aefd..f03d938 100644
--- a/xiaozhi-esp32-server/main/manager-web/src/components/ProviderDialog.vue
+++ b/xiaozhi-esp32-server/main/manager-web/src/components/ProviderDialog.vue
@@ -1,7 +1,7 @@
<template>
<CustomDialog
:title="title"
- :visible.sync="visible"
+ :visible.sync="dialogVisible"
width="57%"
class="provider-dialog-wrapper"
@confirm="submit"
@@ -138,6 +138,7 @@
},
data() {
return {
+ dialogVisible: this.visible,
saving: false,
isAllFieldsSelected: false,
tableKey: 0
@@ -326,9 +327,13 @@
},
watch: {
visible(val) {
+ this.dialogVisible = val;
if (!val) {
this.resetForm();
}
+ },
+ dialogVisible(val) {
+ this.$emit('update:visible', val);
}
}
};
diff --git a/xiaozhi-esp32-server/main/manager-web/src/components/ReplacementWordDialog.vue b/xiaozhi-esp32-server/main/manager-web/src/components/ReplacementWordDialog.vue
index 5b536f4..a33e260 100644
--- a/xiaozhi-esp32-server/main/manager-web/src/components/ReplacementWordDialog.vue
+++ b/xiaozhi-esp32-server/main/manager-web/src/components/ReplacementWordDialog.vue
@@ -1,7 +1,7 @@
<template>
<CustomDialog
:title="title"
- :visible.sync="visible"
+ :visible.sync="dialogVisible"
width="700px"
class="param-dialog-wrapper"
@confirm="submit"
@@ -126,6 +126,7 @@
};
return {
+ dialogVisible: this.visible,
dialogKey: Date.now(),
saving: false,
localForm: {
@@ -243,6 +244,7 @@
},
watch: {
visible(newVal) {
+ this.dialogVisible = newVal;
if (newVal) {
this.localForm.id = this.form.id;
this.localForm.fileName = this.form.fileName || '';
@@ -256,6 +258,9 @@
} else {
this.saving = false;
}
+ },
+ dialogVisible(val) {
+ this.$emit('update:visible', val);
}
}
};
diff --git a/xiaozhi-esp32-server/main/manager-web/src/components/ViewPasswordDialog.vue b/xiaozhi-esp32-server/main/manager-web/src/components/ViewPasswordDialog.vue
index a5c89af..e0e64a6 100644
--- a/xiaozhi-esp32-server/main/manager-web/src/components/ViewPasswordDialog.vue
+++ b/xiaozhi-esp32-server/main/manager-web/src/components/ViewPasswordDialog.vue
@@ -1,5 +1,5 @@
<template>
- <el-dialog :visible.sync="visible" width="400px" center>
+ <el-dialog :visible.sync="dialogVisible" width="400px" center>
<div
style="margin: 0 10px 10px;display: flex;align-items: center;gap: 10px;font-weight: 700;font-size: 20px;text-align: left;color: #3d4566;">
<div
@@ -38,9 +38,22 @@
visible: { type: Boolean, required: true },
password: { type: String, default: '' }
},
+ data() {
+ return {
+ dialogVisible: this.visible
+ }
+ },
+ watch: {
+ visible(val) {
+ this.dialogVisible = val
+ },
+ dialogVisible(val) {
+ this.$emit('update:visible', val)
+ }
+ },
methods: {
closeDialog() {
- this.$emit('update:visible', false)
+ this.dialogVisible = false
},
copyPassword() {
navigator.clipboard.writeText(this.password)
diff --git a/xiaozhi-esp32-server/main/manager-web/vue.config.js b/xiaozhi-esp32-server/main/manager-web/vue.config.js
index 89b85dd..1501b0d 100644
--- a/xiaozhi-esp32-server/main/manager-web/vue.config.js
+++ b/xiaozhi-esp32-server/main/manager-web/vue.config.js
@@ -16,8 +16,9 @@
return path.join(__dirname, dir)
}
-// 纭繚鍔犺浇 .env 鏂囦欢
+// Vue CLI 浼氭寜 mode 鍔犺浇 .env.*锛涜繖閲屽啀璇讳竴閬嶏紝缁� vue.config.js 鑷韩鐢�
dotenv.config();
+dotenv.config({ path: path.resolve(__dirname, `.env.${process.env.NODE_ENV || 'development'}`) });
// 瀹氫箟CDN璧勬簮鍒楄〃锛岀‘淇漇ervice Worker涔熻兘璁块棶
const cdnResources = {
@@ -87,10 +88,36 @@
}
});
- // 鍚敤浼樺寲璁剧疆
- config.optimization.usedExports(true);
- config.optimization.concatenateModules(true);
- config.optimization.minimize(true);
+ // webpack 5 寮�鍙戞ā寮忛粯璁や篃浼氬紑 usedExports锛屼細鎶� Vue 2 SFC 鐨� style
+ // default export treeshake 鎺夛紙澶ч噺 style0 warning锛夈�俶inimize 杩樹細鐮村潖
+ // Vue 2 娓叉煋鍑芥暟锛屽鑷撮〉闈㈢┖鐧藉拰 setAttribute 鎶ラ敊銆傝繖浜涘彧缁欑敓浜ф瀯寤恒��
+ if (process.env.NODE_ENV === 'production') {
+ config.optimization.usedExports(true);
+ config.optimization.concatenateModules(true);
+ config.optimization.minimize(true);
+ } else {
+ config.optimization.usedExports(false);
+ config.optimization.concatenateModules(false);
+ config.optimization.minimize(false);
+ }
+
+ // Vue 2 + vue-style-loader 闇�瑕� css-loader 鐨� CommonJS 瀵煎嚭
+ const ruleTypes = ['css', 'postcss', 'scss', 'sass', 'less', 'stylus']
+ ruleTypes.forEach(ruleType => {
+ const rule = config.module.rule(ruleType)
+ if (!rule) return
+ const oneOfs = rule.oneOfs && rule.oneOfs.store
+ if (!oneOfs) return
+ oneOfs.forEach(oneOf => {
+ if (oneOf.uses.has('css-loader')) {
+ oneOf.use('css-loader').tap(options => {
+ options = options || {}
+ options.esModule = false
+ return options
+ })
+ }
+ })
+ })
},
configureWebpack: config => {
if (process.env.NODE_ENV === 'production') {
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/config.yaml b/xiaozhi-esp32-server/main/xiaozhi-server/config.yaml
index 85435bb..7ff25d3 100644
--- a/xiaozhi-esp32-server/main/xiaozhi-server/config.yaml
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/config.yaml
@@ -67,6 +67,9 @@
enable_wakeup_words_response_cache: true
# 寮�鍦烘槸鍚﹀洖澶嶅敜閱掕瘝
enable_greeting: true
+# Phase 1锛氳繘鍏� chat.conversation 鍚庝富鍔ㄥ紑鍙�
+enable_proactive_greeting: true
+proactive_chat_greeting: 鎴戜滑鏉ヨ亰澶╁惂銆�
# 璇村畬璇濇槸鍚﹀紑鍚彁绀洪煶
enable_stop_tts_notify: false
# 璇村畬璇濇槸鍚﹀紑鍚彁绀洪煶锛岄煶鏁堝湴鍧�
@@ -270,6 +273,8 @@
- get_weather
- get_news_from_newsnow
- play_music
+ - switch_app
+ - start_repeater
function_call:
# 涓嶉渶瑕佸姩type
type: function_call
@@ -286,6 +291,8 @@
# play_music鏄湇鍔″櫒鑷甫鐨勯煶涔愭挱鏀撅紝hass_play_music鏄�氳繃home assistant鎺у埗鐨勭嫭绔嬪閮ㄧ▼搴忛煶涔愭挱鏀�
# 濡傛灉鐢ㄤ簡hass_play_music锛屽氨涓嶈寮�鍚痯lay_music锛屼袱鑰呭彧鐣欎竴涓�
- play_music
+ - switch_app
+ - start_repeater
#- hass_state
#- hass_play_music
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/__init__.py b/xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/__init__.py
new file mode 100644
index 0000000..8146ddb
--- /dev/null
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/__init__.py
@@ -0,0 +1,15 @@
+"""Phase 1 App Runtime: chat / english_learning / english_translation."""
+
+from .registry import get_app, get_task, get_registry, default_app_id
+from .runtime import start_app_task, switch_app, filter_tools, on_user_wakeup
+
+__all__ = [
+ "get_app",
+ "get_task",
+ "get_registry",
+ "default_app_id",
+ "start_app_task",
+ "switch_app",
+ "filter_tools",
+ "on_user_wakeup",
+]
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/apps.json b/xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/apps.json
new file mode 100644
index 0000000..2ddb63d
--- /dev/null
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/apps.json
@@ -0,0 +1,63 @@
+{
+ "default_app_id": "chat",
+ "apps": {
+ "chat": {
+ "name": "鑱婂ぉ",
+ "mode": "CHAT",
+ "default_task": "conversation",
+ "tool_policy": "inherit_agent",
+ "system_prompt_mode": "agent_proactive",
+ "tasks": {
+ "conversation": {
+ "interaction_mode": "CHAT",
+ "skills": []
+ }
+ }
+ },
+ "english_learning": {
+ "name": "鑻辫瀛︿範",
+ "mode": "PLAY_AUDIO",
+ "default_task": "repeater",
+ "tool_policy": "allowlist",
+ "tools": [
+ "switch_app",
+ "start_repeater",
+ "self_mp3_play",
+ "self_mp3_stop",
+ "self_mp3_get_list",
+ "self_power_off"
+ ],
+ "system_prompt_mode": "none",
+ "tasks": {
+ "repeater": {
+ "interaction_mode": "PLAY_AUDIO",
+ "skills": [
+ {
+ "name": "self.mp3.play",
+ "args_from": ["name", "repeat", "power_off_after"]
+ }
+ ]
+ }
+ }
+ },
+ "english_translation": {
+ "name": "鑻辫缈昏瘧",
+ "mode": "TRANSLATION",
+ "default_task": "live_translate",
+ "tool_policy": "allowlist",
+ "tools": ["switch_app", "translate"],
+ "system_prompt_mode": "translator",
+ "tasks": {
+ "live_translate": {
+ "interaction_mode": "TRANSLATION",
+ "skills": [
+ {
+ "name": "translate",
+ "side": "server_mcp"
+ }
+ ]
+ }
+ }
+ }
+ }
+}
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/registry.py b/xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/registry.py
new file mode 100644
index 0000000..42c4436
--- /dev/null
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/registry.py
@@ -0,0 +1,77 @@
+"""Load the Phase 1 App / Task registry from JSON."""
+
+import json
+import os
+from typing import Any, Dict, Optional, Tuple
+
+from config.config_loader import get_project_dir
+from config.logger import setup_logging
+
+TAG = __name__
+logger = setup_logging()
+
+_REGISTRY: Optional[Dict[str, Any]] = None
+
+_BUNDLED_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), "apps.json")
+
+
+def _load_registry() -> Dict[str, Any]:
+ candidates = [
+ os.path.join(get_project_dir(), "data", "apps.json"),
+ _BUNDLED_PATH,
+ ]
+ for path in candidates:
+ if not os.path.isfile(path):
+ continue
+ try:
+ with open(path, "r", encoding="utf-8") as f:
+ data = json.load(f)
+ if isinstance(data, dict) and data.get("apps"):
+ logger.bind(tag=TAG).info(f"宸插姞杞� App 娉ㄥ唽琛�: {path}")
+ return data
+ except Exception as e:
+ logger.bind(tag=TAG).error(f"鍔犺浇 App 娉ㄥ唽琛ㄥけ璐� {path}: {e}")
+ logger.bind(tag=TAG).warning("鏈壘鍒� apps.json锛屼娇鐢ㄥ唴缃渶灏忔敞鍐岃〃")
+ return {
+ "default_app_id": "chat",
+ "apps": {
+ "chat": {
+ "name": "鑱婂ぉ",
+ "mode": "CHAT",
+ "default_task": "conversation",
+ "tool_policy": "inherit_agent",
+ "system_prompt_mode": "agent_proactive",
+ "tasks": {"conversation": {"interaction_mode": "CHAT", "skills": []}},
+ }
+ },
+ }
+
+
+def get_registry() -> Dict[str, Any]:
+ global _REGISTRY
+ if _REGISTRY is None:
+ _REGISTRY = _load_registry()
+ return _REGISTRY
+
+
+def default_app_id() -> str:
+ return get_registry().get("default_app_id") or "chat"
+
+
+def get_app(app_id: str) -> Optional[Dict[str, Any]]:
+ if not app_id:
+ return None
+ return get_registry().get("apps", {}).get(app_id)
+
+
+def get_task(app_id: str, task_id: str) -> Tuple[Optional[Dict[str, Any]], Optional[Dict[str, Any]]]:
+ app = get_app(app_id)
+ if not app:
+ return None, None
+ tasks = app.get("tasks") or {}
+ if not task_id:
+ task_id = app.get("default_task") or ""
+ task = tasks.get(task_id)
+ if not task:
+ return app, None
+ return app, task
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/runtime.py b/xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/runtime.py
new file mode 100644
index 0000000..e62fdcf
--- /dev/null
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/core/app_runtime/runtime.py
@@ -0,0 +1,374 @@
+"""Unified start_app_task / switch_app / mode helpers."""
+
+from __future__ import annotations
+
+import uuid
+from typing import Any, Dict, Optional, TYPE_CHECKING
+
+from config.logger import setup_logging
+from plugins_func.register import Action, ActionResponse
+
+from .registry import default_app_id, get_app, get_task
+
+if TYPE_CHECKING:
+ from core.connection import ConnectionHandler
+
+TAG = __name__
+logger = setup_logging()
+
+SOURCE_PRIORITY = {
+ "mqtt_immediate": 3,
+ "schedule": 2,
+ "voice": 1,
+ "default_chat": 0,
+}
+
+TRANSLATOR_PROMPT = (
+ "浣犳槸涓�鍙扮炕璇戞満銆傚彧杈撳嚭璇戞枃锛屼笉瑕佽В閲娿�佷笉瑕佹嫾闊宠銆佷笉瑕佽拷闂�佷笉瑕侀棽鑱娿��"
+ "鏈寚瀹氳瑷�鏃讹細涓枃璇戞垚鑻辨枃锛岃嫳鏂囪瘧鎴愪腑鏂囷紱鍏跺畠璇█榛樿璇戞垚涓枃銆�"
+)
+
+CHAT_PROACTIVE_HINT = (
+ "浣犳槸鐢ㄦ埛鐨勮闊充紮浼淬�傝繘鍏ュ璇濆悗鍙互鍏堜富鍔ㄧ煭鍙ラ個璇疯亰澶╋紝渚嬪銆屾垜浠潵鑱婂ぉ鍚с�嶃��"
+ "鐢ㄦ埛寮�鍙e悗鎸夋甯稿璇濆洖绛旓紝涓嶈鍙嶅鍌績銆�"
+)
+
+VOICE_MENU = "鎯宠亰澶┿�佸惉澶嶈锛岃繕鏄墦寮�缈昏瘧锛熶篃鍙互璇存垜浠潵鑱婂ぉ鍚с��"
+
+CHAT_PHRASES = ("鎴戜滑鏉ヨ亰澶╁惂", "鑱婁細澶╁惂", "鍥炲埌鑱婂ぉ", "閫�鍑虹炕璇�", "缁撴潫缈昏瘧", "閫�鍑哄璇�")
+TRANSLATE_PHRASES = ("鎵撳紑缈昏瘧", "寮�濮嬬炕璇�", "鑻辫缈昏瘧", "缈昏瘧妯″紡")
+REPEATER_PHRASES = ("寮�濮嬪璇�", "鎵撳紑澶嶈", "鑻辫瀛︿範", "鍚璇�")
+
+DEVICE_PLAY_TOOL = "self_mp3_play"
+DEVICE_STOP_TOOL = "self_mp3_stop"
+
+
+def source_priority(source: str) -> int:
+ return SOURCE_PRIORITY.get(source or "voice", 1)
+
+
+def _ensure_runtime_fields(conn: "ConnectionHandler") -> None:
+ if not hasattr(conn, "active_app_id"):
+ conn.active_app_id = default_app_id()
+ if not hasattr(conn, "current_task_id"):
+ conn.current_task_id = "conversation"
+ if not hasattr(conn, "interaction_mode"):
+ conn.interaction_mode = "CHAT"
+ if not hasattr(conn, "current_source"):
+ conn.current_source = "default_chat"
+ if not hasattr(conn, "tool_policy"):
+ conn.tool_policy = "inherit_agent"
+ if not hasattr(conn, "tool_allowlist"):
+ conn.tool_allowlist = []
+ if not hasattr(conn, "original_system_prompt"):
+ conn.original_system_prompt = conn.prompt
+
+
+def filter_tools(conn: "ConnectionHandler", tools: Dict[str, Any]) -> Dict[str, Any]:
+ """inherit_agent keeps the live agent toolset; allowlist keeps named tools only."""
+ _ensure_runtime_fields(conn)
+ if getattr(conn, "tool_policy", "inherit_agent") != "allowlist":
+ return tools
+ allow = set(getattr(conn, "tool_allowlist", None) or [])
+ if not allow:
+ return {}
+ filtered = {}
+ for name, definition in tools.items():
+ if name in allow:
+ filtered[name] = definition
+ return filtered
+
+
+def _refresh_tools(conn: "ConnectionHandler") -> None:
+ handler = getattr(conn, "func_handler", None)
+ if handler and hasattr(handler, "upload_functions_desc"):
+ handler.upload_functions_desc()
+
+
+def _apply_prompt(conn: "ConnectionHandler", app: Dict[str, Any]) -> None:
+ mode = app.get("system_prompt_mode") or "none"
+ if conn.original_system_prompt is None:
+ conn.original_system_prompt = conn.prompt
+ if mode == "translator":
+ conn.change_system_prompt(
+ "浣犵幇鍦ㄦ槸鑻辫缈昏瘧鏈恒�傜敤鎴疯璇濆悗搴旇皟鐢� translate锛涚敤鎴疯閫�鍑虹炕璇戙�佸洖鍒拌亰澶╂椂璋冪敤 switch_app锛宎pp_id=chat銆�"
+ "涓嶈璁茬瑧璇濄�佷笉瑕佹煡澶╂皵銆佷笉瑕侀棽鑱娿��"
+ )
+ elif mode == "agent_proactive":
+ base = conn.original_system_prompt or conn.prompt or ""
+ if CHAT_PROACTIVE_HINT not in (base or ""):
+ conn.change_system_prompt((base + "\n" + CHAT_PROACTIVE_HINT).strip())
+ else:
+ conn.change_system_prompt(base)
+ else:
+ if conn.original_system_prompt:
+ conn.change_system_prompt(conn.original_system_prompt)
+
+
+async def _execute_named_tool(conn: "ConnectionHandler", tool_name: str, arguments: Dict[str, Any]):
+ handler = getattr(conn, "func_handler", None)
+ if not handler or not hasattr(handler, "tool_manager"):
+ return ActionResponse(action=Action.ERROR, response="宸ュ叿澶勭悊鍣ㄦ湭灏辩华")
+ try:
+ return await handler.tool_manager.execute_tool(tool_name, arguments or {})
+ except Exception as e:
+ logger.bind(tag=TAG).error(f"鎵ц宸ュ叿 {tool_name} 澶辫触: {e}")
+ return ActionResponse(action=Action.ERROR, response=str(e))
+
+
+async def _stop_current_skills(conn: "ConnectionHandler") -> None:
+ mode = getattr(conn, "interaction_mode", "CHAT")
+ if mode == "PLAY_AUDIO":
+ await _execute_named_tool(conn, DEVICE_STOP_TOOL, {})
+ conn.awaiting_voice_app_choice = False
+
+
+def _speak(conn: "ConnectionHandler", text: str) -> None:
+ if not text:
+ return
+ from core.handle.intentHandler import speak_txt
+
+ if not getattr(conn, "sentence_id", None):
+ conn.sentence_id = uuid.uuid4().hex
+ try:
+ speak_txt(conn, text)
+ except Exception as e:
+ logger.bind(tag=TAG).warning(f"涓诲姩鎾姤澶辫触: {e}")
+
+
+def _greeting_text(conn: "ConnectionHandler") -> str:
+ return (conn.config or {}).get("proactive_chat_greeting") or "鎴戜滑鏉ヨ亰澶╁惂銆�"
+
+
+async def start_app_task(
+ conn: "ConnectionHandler",
+ app_id: str,
+ task_id: str = "",
+ params: Optional[Dict[str, Any]] = None,
+ source: str = "voice",
+ execute_skills: bool = True,
+) -> ActionResponse:
+ """Single entry for MQTT / schedule / voice / default chat."""
+ _ensure_runtime_fields(conn)
+ params = params or {}
+ app, task = get_task(app_id, task_id)
+ if not app:
+ return ActionResponse(action=Action.ERROR, response=f"鏈煡鐨� App: {app_id}")
+ if not task:
+ default_task = app.get("default_task") or ""
+ app, task = get_task(app_id, default_task)
+ if not task:
+ return ActionResponse(action=Action.ERROR, response=f"鏈煡鐨� Task: {app_id}/{task_id}")
+
+ resolved_task_id = task_id or app.get("default_task") or "conversation"
+ new_p = source_priority(source)
+ cur_p = source_priority(getattr(conn, "current_source", "default_chat"))
+ same_task = (
+ getattr(conn, "active_app_id", "") == app_id
+ and getattr(conn, "current_task_id", "") == resolved_task_id
+ )
+ if new_p < cur_p and not same_task:
+ logger.bind(tag=TAG).info(
+ f"鎷掔粷浣庝紭鍏堢骇浠诲姟 {source}({new_p}) < {conn.current_source}({cur_p})"
+ )
+ return ActionResponse(
+ action=Action.NONE,
+ response="褰撳墠鏈夋洿楂樹紭鍏堢骇鐨勪换鍔★紝鍏堝畬鎴愬悗鍐嶅垏鎹�",
+ )
+
+ if not same_task or new_p >= cur_p:
+ await _stop_current_skills(conn)
+
+ conn.active_app_id = app_id
+ conn.current_task_id = resolved_task_id
+ conn.interaction_mode = task.get("interaction_mode") or app.get("mode") or "CHAT"
+ conn.current_source = source
+ conn.tool_policy = app.get("tool_policy") or "inherit_agent"
+ conn.tool_allowlist = list(app.get("tools") or [])
+ _apply_prompt(conn, app)
+ _refresh_tools(conn)
+
+ logger.bind(tag=TAG).info(
+ f"start_app_task app={app_id} task={resolved_task_id} mode={conn.interaction_mode} source={source}"
+ )
+
+ if execute_skills:
+ for skill in task.get("skills") or []:
+ skill_name = (skill or {}).get("name") or ""
+ if skill_name in ("self.mp3.play", DEVICE_PLAY_TOOL):
+ play_args = {
+ "name": params.get("name") or "",
+ "repeat": int(params.get("repeat") or 1),
+ "power_off_after": bool(params.get("power_off_after") or False),
+ }
+ if params.get("speed") is not None:
+ play_args["speed"] = params.get("speed")
+ if not play_args["name"]:
+ return ActionResponse(
+ action=Action.RESPONSE,
+ response="娌℃湁鎸囧畾瑕佸璇荤殑鏂囦欢鍚嶏紝璇峰厛鐢� self.mp3.get_list 纭 files[].name",
+ )
+ result = await _execute_named_tool(conn, DEVICE_PLAY_TOOL, play_args)
+ if result and result.action in (Action.ERROR, Action.NOTFOUND):
+ return ActionResponse(
+ action=Action.RESPONSE,
+ response=result.response or "杩欏彴璁惧娌℃湁鏈湴澶嶈鎾斁鑳藉姏",
+ )
+ elif skill_name == "self.power_off":
+ await _execute_named_tool(conn, "self_power_off", {})
+
+ if conn.interaction_mode == "CHAT":
+ enable = (conn.config or {}).get("enable_proactive_greeting", True)
+ woken = bool(getattr(conn, "just_woken_up", False))
+ if enable and not woken:
+ _speak(conn, _greeting_text(conn))
+
+ app_name = app.get("name") or app_id
+ return ActionResponse(
+ action=Action.NONE,
+ result=f"{app_id}/{resolved_task_id}",
+ response=f"宸叉墦寮�{app_name}",
+ )
+
+
+async def switch_app(conn: "ConnectionHandler", app_id: str, source: str = "voice") -> ActionResponse:
+ app = get_app(app_id)
+ if not app:
+ return ActionResponse(action=Action.ERROR, response=f"涓嶆敮鎸佺殑 App: {app_id}")
+ task_id = app.get("default_task") or ""
+ return await start_app_task(conn, app_id, task_id, {}, source)
+
+
+async def on_user_wakeup(conn: "ConnectionHandler") -> None:
+ """User wake word returns to chat; do not treat [app_task] as wakeup."""
+ _ensure_runtime_fields(conn)
+ if getattr(conn, "interaction_mode", "CHAT") == "CHAT" and getattr(conn, "active_app_id", "chat") == "chat":
+ return
+ await start_app_task(conn, "chat", "conversation", {}, source="voice")
+
+
+def match_voice_switch(text: str) -> Optional[tuple]:
+ """Map a short phrase to (app_id, task_id) or ('__menu__', '')."""
+ if not text:
+ return None
+ _, filtered = _strip(text)
+ if filtered in ("鍒囨崲妯″紡", "qie huan mo shi"):
+ return ("__menu__", "")
+ for p in CHAT_PHRASES:
+ if p in filtered:
+ return ("chat", "conversation")
+ for p in TRANSLATE_PHRASES:
+ if p in filtered:
+ return ("english_translation", "live_translate")
+ for p in REPEATER_PHRASES:
+ if p in filtered:
+ return ("english_learning", "repeater")
+ if any(k in filtered for k in ("鏁板瓧鑳岃", "鎴愯鎺ラ緳", "鍙g畻", "鍔犲噺娉�")):
+ return ("__reserved__", "")
+ return None
+
+
+def _strip(text: str) -> tuple:
+ from core.utils.util import remove_punctuation_and_length
+
+ return remove_punctuation_and_length(text)
+
+
+def is_exit_translation(text: str) -> bool:
+ _, filtered = _strip(text)
+ return any(p in filtered for p in ("閫�鍑虹炕璇�", "缁撴潫缈昏瘧", "鍥炲埌鑱婂ぉ", "鎴戜滑鏉ヨ亰澶╁惂", "鑱婁細澶╁惂"))
+
+
+def _plain_user_text(text: str) -> str:
+ """ASR 鍙兘鎶婂師鏂囧寘鍦� JSON 閲岋紝缈昏瘧鍙 content銆�"""
+ if not text:
+ return ""
+ raw = str(text).strip()
+ if raw.startswith("{") and raw.endswith("}"):
+ try:
+ import json
+
+ data = json.loads(raw)
+ if isinstance(data, dict) and data.get("content"):
+ return str(data["content"]).strip()
+ except Exception:
+ pass
+ return raw
+
+
+def _looks_like_tool_error(text: str) -> bool:
+ if not text:
+ return True
+ s = str(text)
+ if "Error executing tool" in s or "isError=True" in s or "isError=true" in s:
+ return True
+ if "鏈厤缃�" in s and "api_key" in s:
+ return True
+ return False
+
+
+async def run_translation(conn: "ConnectionHandler", text: str) -> str:
+ """Translate with the connection LLM first (鏅烘帶鍙板凡閰嶇殑妯″瀷). MCP is fallback only."""
+ text = _plain_user_text(text)
+ if not text:
+ return ""
+
+ from plugins_func.register import all_function_registry
+
+ item = all_function_registry.get("translate")
+ if item:
+ try:
+ maybe = item.func(conn, text=text)
+ if hasattr(maybe, "__await__"):
+ maybe = await maybe
+ if isinstance(maybe, ActionResponse) and maybe.action not in (
+ Action.ERROR,
+ Action.NOTFOUND,
+ ):
+ out = (maybe.response or maybe.result or "").strip()
+ if out and not _looks_like_tool_error(out):
+ return out
+ except Exception as e:
+ logger.bind(tag=TAG).warning(f"缈昏瘧鎻掍欢澶辫触: {e}")
+
+ handler = getattr(conn, "func_handler", None)
+ if handler and handler.has_tool("translate"):
+ result = await _execute_named_tool(conn, "translate", {"text": text})
+ if result and result.action not in (Action.ERROR, Action.NOTFOUND):
+ out = _extract_translate_text(result.response or result.result or "")
+ if out and not _looks_like_tool_error(out):
+ return out
+
+ llm = getattr(conn, "llm", None)
+ if llm and hasattr(llm, "response_no_stream"):
+ try:
+ return (
+ llm.response_no_stream(TRANSLATOR_PROMPT, text, temperature=0.1) or ""
+ ).strip()
+ except Exception as e:
+ logger.bind(tag=TAG).error(f"缈昏瘧 LLM 澶辫触: {e}")
+ return ""
+
+
+def _extract_translate_text(raw: Any) -> str:
+ if raw is None:
+ return ""
+ if isinstance(raw, str):
+ text = raw.strip()
+ if text.startswith("{") and "text" in text:
+ try:
+ import json
+
+ data = json.loads(text)
+ if isinstance(data, dict):
+ return str(data.get("text") or data.get("content") or text)
+ except Exception:
+ pass
+ return text
+ if hasattr(raw, "content"):
+ content = raw.content
+ if isinstance(content, list) and content:
+ first = content[0]
+ return getattr(first, "text", None) or str(first)
+ return str(content)
+ return str(raw).strip()
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/core/connection.py b/xiaozhi-esp32-server/main/xiaozhi-server/core/connection.py
index 718f390..6e5f417 100644
--- a/xiaozhi-esp32-server/main/xiaozhi-server/core/connection.py
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/core/connection.py
@@ -200,6 +200,16 @@
# 鏍囪褰撳墠鏄惁涓烘潵鐢垫帴鍚ā寮�
self.incoming_call = None
+ # Phase 1 App Runtime锛堣繛鎺ュ唴瀛橈紝涓嶅缓琛級
+ self.active_app_id = "chat"
+ self.current_task_id = "conversation"
+ self.interaction_mode = "CHAT"
+ self.current_source = "default_chat"
+ self.tool_policy = "inherit_agent"
+ self.tool_allowlist = []
+ self.original_system_prompt = None
+ self.awaiting_voice_app_choice = False
+
async def handle_connection(self, ws: websockets.ServerConnection):
try:
# 鑾峰彇杩愯涓殑浜嬩欢寰幆锛堝繀椤诲湪寮傛涓婁笅鏂囦腑锛�
@@ -649,6 +659,8 @@
self._init_prompt_enhancement()
"""娉ㄥ叆宸ュ叿璋冪敤few-shot绀轰緥锛堜粎function_call妯″紡锛�"""
self._inject_tool_call_fewshot()
+ if self.original_system_prompt is None:
+ self.original_system_prompt = self.prompt
except Exception as e:
self.logger.bind(tag=TAG).error(f"瀹炰緥鍖栫粍浠跺け璐�: {e}")
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/core/handle/receiveAudioHandle.py b/xiaozhi-esp32-server/main/xiaozhi-server/core/handle/receiveAudioHandle.py
index 504669b..2985600 100644
--- a/xiaozhi-esp32-server/main/xiaozhi-server/core/handle/receiveAudioHandle.py
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/core/handle/receiveAudioHandle.py
@@ -46,22 +46,24 @@
actual_text = text
try:
- # 灏濊瘯瑙f瀽JSON鏍煎紡鐨勮緭鍏�
+ # FunASR 绛変細鎶婄粨鏋滃寘鎴� {"content","language","emotion"}锛涘0绾瑰満鏅繕鏈� speaker
if text.strip().startswith("{") and text.strip().endswith("}"):
data = json.loads(text)
- if "speaker" in data and "content" in data:
- speaker_name = data["speaker"]
- actual_content = data["content"]
- conn.logger.bind(tag=TAG).info(f"瑙f瀽鍒拌璇濅汉淇℃伅: {speaker_name}")
-
- # 浠呭湪璇ヨ璇濅汉棣栨鍑虹幇鏃朵繚鐣� {"speaker":...} JSON锛岃妯″瀷鑷劧绉板懠涓�娆★紱
- # 鍚庣画杞檷涓虹函鏂囨湰锛岄伩鍏嶆瘡杞噸澶嶅嚭鐜板悕瀛楄瀵兼ā鍨嬪弽澶嶇О鍛�
- if speaker_name not in conn.introduced_speakers:
- conn.introduced_speakers.add(speaker_name)
- actual_text = text
- else:
+ if isinstance(data, dict) and "content" in data:
+ speaker_name = data.get("speaker")
+ actual_content = data.get("content") or ""
+ if speaker_name:
+ conn.logger.bind(tag=TAG).info(f"瑙f瀽鍒拌璇濅汉淇℃伅: {speaker_name}")
+ # 浠呭湪璇ヨ璇濅汉棣栨鍑虹幇鏃朵繚鐣� {"speaker":...} JSON锛岃妯″瀷鑷劧绉板懠涓�娆★紱
+ # 鍚庣画杞檷涓虹函鏂囨湰锛岄伩鍏嶆瘡杞噸澶嶅嚭鐜板悕瀛楄瀵兼ā鍨嬪弽澶嶇О鍛�
+ if speaker_name not in conn.introduced_speakers:
+ conn.introduced_speakers.add(speaker_name)
+ actual_text = text
+ else:
+ actual_text = actual_content
+ elif actual_content:
actual_text = actual_content
- except (json.JSONDecodeError, KeyError):
+ except (json.JSONDecodeError, KeyError, TypeError):
# 濡傛灉瑙f瀽澶辫触锛岀户缁娇鐢ㄥ師濮嬫枃鏈�
pass
@@ -83,6 +85,70 @@
await max_out_size(conn)
return
+ from core.app_runtime.runtime import (
+ is_exit_translation,
+ match_voice_switch,
+ run_translation,
+ start_app_task,
+ VOICE_MENU,
+ )
+ from core.handle.intentHandler import speak_txt
+
+ mode = getattr(conn, "interaction_mode", "CHAT")
+
+ if getattr(conn, "awaiting_voice_app_choice", False):
+ mapped = match_voice_switch(actual_text)
+ conn.awaiting_voice_app_choice = False
+ if mapped and mapped[0] == "__reserved__":
+ speak_txt(conn, "杩欎釜娓告垙杩樻病寮�鏀撅紝鎴戜滑鍏堣亰澶╁惂銆�")
+ await start_app_task(conn, "chat", "conversation", {}, source="voice")
+ return
+ if mapped and mapped[0] == "__menu__":
+ conn.awaiting_voice_app_choice = True
+ speak_txt(conn, VOICE_MENU)
+ return
+ if mapped and mapped[0] != "__menu__":
+ params = {}
+ await start_app_task(conn, mapped[0], mapped[1], params, source="voice")
+ return
+ speak_txt(conn, VOICE_MENU)
+ conn.awaiting_voice_app_choice = True
+ return
+
+ mapped = match_voice_switch(actual_text)
+ if mapped and mapped[0] == "__menu__":
+ conn.awaiting_voice_app_choice = True
+ speak_txt(conn, VOICE_MENU)
+ return
+ if mapped and mapped[0] == "__reserved__":
+ speak_txt(conn, "杩欎釜杩樻病寮�鏀撅紝鍏堢户缁綋鍓嶅唴瀹瑰惂銆�")
+ return
+ if mapped and mapped[0] in ("english_translation", "english_learning", "chat"):
+ if mapped[0] == "english_learning" and mapped[1] == "repeater":
+ speak_txt(conn, "澶嶈闇�瑕佹寚瀹� SD 鍗′笂鐨勬枃浠跺悕锛屾垨鐢ㄥ皬绋嬪簭寮�濮嬨��")
+ return
+ await start_app_task(conn, mapped[0], mapped[1], {}, source="voice")
+ return
+
+ if mode == "PLAY_AUDIO":
+ conn.logger.bind(tag=TAG).info("PLAY_AUDIO 妯″紡蹇界暐涓婅瀵硅瘽")
+ return
+
+ if mode == "TRANSLATION":
+ if is_exit_translation(actual_text):
+ await start_app_task(conn, "chat", "conversation", {}, source="voice")
+ return
+ await send_stt_message(conn, actual_text)
+ conn.client_abort = False
+ translated = await run_translation(conn, actual_text)
+ if not translated:
+ translated = "娌℃湁鍚竻瑕佺炕璇戠殑鍐呭锛岃鍐嶈涓�娆°��"
+ if not conn.sentence_id:
+ import uuid as _uuid
+ conn.sentence_id = _uuid.uuid4().hex
+ speak_txt(conn, translated)
+ return
+
# manual 妯″紡涓嬩笉鎵撴柇姝e湪鎾斁鐨勫唴瀹�
if conn.client_is_speaking and conn.client_listen_mode != "manual":
await handleAbortMessage(conn)
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/core/handle/textHandler/listenMessageHandler.py b/xiaozhi-esp32-server/main/xiaozhi-server/core/handle/textHandler/listenMessageHandler.py
index 545ccec..5952f75 100644
--- a/xiaozhi-esp32-server/main/xiaozhi-server/core/handle/textHandler/listenMessageHandler.py
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/core/handle/textHandler/listenMessageHandler.py
@@ -62,6 +62,35 @@
original_text
)
+ # App Runtime: 璁惧涓婃姤鐨� Task 鍚姩锛堝鐢� listen detect.text锛屼笉鏂板 WS type锛�
+ if original_text.startswith("[app_task]"):
+ payload = original_text[len("[app_task]"):].strip()
+ parts = [p.strip() for p in payload.split("|") if p.strip()]
+ app_id = parts[0] if parts else "chat"
+ task_id = parts[1] if len(parts) > 1 else ""
+ source = parts[2] if len(parts) > 2 else "mqtt_immediate"
+ conn.logger.bind(tag=TAG).info(
+ f"鏀跺埌 app_task: {app_id}/{task_id} source={source}"
+ )
+ from core.app_runtime.runtime import start_app_task
+
+ start_time = time.time()
+ while time.time() - start_time < 3:
+ if conn.tts:
+ break
+ await asyncio.sleep(0.1)
+ await start_app_task(conn, app_id, task_id, {}, source=source, execute_skills=False)
+ return
+
+ if original_text.startswith("[voice_entry]"):
+ from core.app_runtime.runtime import VOICE_MENU
+ from core.handle.intentHandler import speak_txt
+
+ conn.awaiting_voice_app_choice = True
+ conn.sentence_id = uuid.uuid4().hex
+ speak_txt(conn, VOICE_MENU)
+ return
+
# 妫�鏌ユ槸鍚︽槸璁惧鍛煎彨鎸囦护 [device_call]
if original_text.startswith("[device_call]"):
# 鎻愬彇 tag 鍚庣殑鏂囨湰
@@ -105,6 +134,9 @@
conn.client_is_speaking = False
elif is_wakeup_words:
conn.just_woken_up = True
+ from core.app_runtime.runtime import on_user_wakeup
+
+ await on_user_wakeup(conn)
# 涓婃姤绾枃瀛楁暟鎹紙澶嶇敤ASR涓婃姤鍔熻兘锛屼絾涓嶆彁渚涢煶棰戞暟鎹級
enqueue_asr_report(conn, "鍢匡紝浣犲ソ鍛�", [])
await startToChat(conn, "鍢匡紝浣犲ソ鍛�")
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/core/providers/tools/server_mcp/mcp_manager.py b/xiaozhi-esp32-server/main/xiaozhi-server/core/providers/tools/server_mcp/mcp_manager.py
index b1fc0aa..24bf48b 100644
--- a/xiaozhi-esp32-server/main/xiaozhi-server/core/providers/tools/server_mcp/mcp_manager.py
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/core/providers/tools/server_mcp/mcp_manager.py
@@ -2,6 +2,7 @@
import asyncio
import os
+import sys
import json
from typing import Dict, Any, List
@@ -33,18 +34,33 @@
def load_config(self) -> Dict[str, Any]:
"""鍔犺浇MCP鏈嶅姟閰嶇疆"""
- if len(self.config_path) == 0:
- return {}
+ servers: Dict[str, Any] = {}
+ if self.config_path:
+ try:
+ with open(self.config_path, "r", encoding="utf-8") as f:
+ config = json.load(f)
+ servers = config.get("mcpServers", {}) or {}
+ except Exception as e:
+ logger.bind(tag=TAG).error(
+ f"Error loading MCP config from {self.config_path}: {e}"
+ )
+ servers = {}
+ return self._ensure_builtin_servers(servers)
- try:
- with open(self.config_path, "r", encoding="utf-8") as f:
- config = json.load(f)
- return config.get("mcpServers", {})
- except Exception as e:
- logger.bind(tag=TAG).error(
- f"Error loading MCP config from {self.config_path}: {e}"
- )
- return {}
+ def _ensure_builtin_servers(self, servers: Dict[str, Any]) -> Dict[str, Any]:
+ """Register the Phase 1 translate MCP even if data/.mcp_server_settings.json is missing."""
+ if "english-translate" in servers:
+ return servers
+ script = os.path.join(
+ get_project_dir(), "mcp_servers", "translate", "server.py"
+ )
+ if os.path.isfile(script):
+ servers["english-translate"] = {
+ "command": sys.executable or "python",
+ "args": [script],
+ }
+ logger.bind(tag=TAG).info("宸叉敞鍐屽唴缃� english-translate MCP")
+ return servers
async def _init_server(self, name: str, srv_config: Dict[str, Any]):
"""鍒濆鍖栧崟涓狹CP鏈嶅姟"""
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/core/providers/tools/unified_tool_manager.py b/xiaozhi-esp32-server/main/xiaozhi-server/core/providers/tools/unified_tool_manager.py
index f69a6dc..58a8806 100644
--- a/xiaozhi-esp32-server/main/xiaozhi-server/core/providers/tools/unified_tool_manager.py
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/core/providers/tools/unified_tool_manager.py
@@ -43,6 +43,13 @@
except Exception as e:
self.logger.error(f"鑾峰彇{tool_type.value}宸ュ叿鏃跺嚭閿�: {e}")
+ try:
+ from core.app_runtime.runtime import filter_tools
+
+ all_tools = filter_tools(self.conn, all_tools)
+ except Exception as e:
+ self.logger.error(f"鎸� App 杩囨护宸ュ叿澶辫触: {e}")
+
self._cached_tools = all_tools
return all_tools
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/mcp_server_settings.json b/xiaozhi-esp32-server/main/xiaozhi-server/mcp_server_settings.json
index 966741b..4db35e1 100644
--- a/xiaozhi-esp32-server/main/xiaozhi-server/mcp_server_settings.json
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/mcp_server_settings.json
@@ -7,6 +7,11 @@
"褰撳墠鏀寔涓夌浼犺緭妯″紡锛歴tdio(鏍囧噯杈撳叆杈撳嚭), sse(Server-Sent Events), streamable-http(娴佸紡HTTP)銆�"
],
"mcpServers": {
+ "english-translate": {
+ "command": "python",
+ "args": ["mcp_servers/translate/server.py"],
+ "des": "Phase 1 鑻辫缈昏瘧 MCP锛屽彧鏆撮湶 translate"
+ },
"Home Assistant": {
"command": "mcp-proxy",
"args": [
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/mcp_servers/translate/server.py b/xiaozhi-esp32-server/main/xiaozhi-server/mcp_servers/translate/server.py
new file mode 100644
index 0000000..bcd7210
--- /dev/null
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/mcp_servers/translate/server.py
@@ -0,0 +1,110 @@
+#!/usr/bin/env python3
+"""Standalone translation MCP. Only exposes translate. Does not call ConnectionHandler.chat."""
+
+from __future__ import annotations
+
+import os
+from pathlib import Path
+
+import httpx
+import yaml
+from mcp.server.fastmcp import FastMCP
+
+ROOT = Path(__file__).resolve().parents[2]
+TRANSLATOR_PROMPT = (
+ "浣犳槸涓�鍙扮炕璇戞満銆傚彧杈撳嚭璇戞枃锛屼笉瑕佽В閲娿�佷笉瑕佹嫾闊宠銆佷笉瑕佽拷闂�佷笉瑕侀棽鑱娿��"
+ "鏈寚瀹氳瑷�鏃讹細涓枃璇戞垚鑻辨枃锛岃嫳鏂囪瘧鎴愪腑鏂囷紱鍏跺畠璇█榛樿璇戞垚涓枃銆�"
+)
+
+mcp = FastMCP("english-translate")
+
+
+def _read_yaml(path: Path) -> dict:
+ if not path.is_file():
+ return {}
+ with path.open("r", encoding="utf-8") as f:
+ data = yaml.safe_load(f) or {}
+ return data if isinstance(data, dict) else {}
+
+
+def _load_llm_config() -> dict:
+ default_cfg = _read_yaml(ROOT / "config.yaml")
+ custom_cfg = _read_yaml(ROOT / "data" / ".config.yaml")
+
+ def merge(base, overlay):
+ out = dict(base)
+ for k, v in overlay.items():
+ if isinstance(v, dict) and isinstance(out.get(k), dict):
+ out[k] = merge(out[k], v)
+ else:
+ out[k] = v
+ return out
+
+ cfg = merge(default_cfg, custom_cfg)
+ selected = (cfg.get("selected_module") or {}).get("LLM")
+ llm_map = cfg.get("LLM") or {}
+ if selected and selected in llm_map:
+ item = dict(llm_map[selected])
+ item["_name"] = selected
+ return item
+ if llm_map:
+ name, item = next(iter(llm_map.items()))
+ data = dict(item)
+ data["_name"] = name
+ return data
+ return {}
+
+
+def _chat_complete(text: str, source_lang: str, target_lang: str) -> str:
+ llm = _load_llm_config()
+ api_key = llm.get("api_key") or os.environ.get("OPENAI_API_KEY") or ""
+ if api_key and ("浣犵殑" in str(api_key) or "key" == str(api_key).lower()):
+ api_key = os.environ.get("OPENAI_API_KEY") or api_key
+ base_url = llm.get("base_url") or llm.get("url") or "https://api.openai.com/v1"
+ model = llm.get("model_name") or llm.get("model") or "gpt-4o-mini"
+ if not api_key or "浣犵殑" in str(api_key):
+ raise RuntimeError("缈昏瘧 MCP 鏈厤缃彲鐢ㄧ殑 LLM api_key")
+
+ extra = ""
+ if source_lang or target_lang:
+ extra = f"\n婧愯瑷�:{source_lang or 'auto'} 鐩爣璇█:{target_lang or 'auto'}"
+ user = f"缈昏瘧涓嬮潰鍐呭锛屽彧杈撳嚭璇戞枃锛歿extra}\n{text}"
+
+ url = str(base_url).rstrip("/")
+ if not url.endswith("/chat/completions"):
+ if url.endswith("/v1"):
+ url = url + "/chat/completions"
+ else:
+ url = url + "/v1/chat/completions"
+
+ headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
+ payload = {
+ "model": model,
+ "temperature": 0.1,
+ "messages": [
+ {"role": "system", "content": TRANSLATOR_PROMPT},
+ {"role": "user", "content": user},
+ ],
+ }
+ with httpx.Client(timeout=60.0) as client:
+ resp = client.post(url, headers=headers, json=payload)
+ resp.raise_for_status()
+ data = resp.json()
+ choices = data.get("choices") or []
+ if not choices:
+ raise RuntimeError("缈昏瘧鎺ュ彛娌℃湁杩斿洖鍐呭")
+ msg = (choices[0].get("message") or {}).get("content") or ""
+ return str(msg).strip()
+
+
+@mcp.tool()
+def translate(text: str, source_lang: str = "", target_lang: str = "") -> str:
+ """Translate text. Chinese->English, English->Chinese unless languages are set. Return translation only."""
+ if not text or not str(text).strip():
+ return ""
+ return _chat_complete(str(text).strip(), source_lang or "", target_lang or "")
+
+
+if __name__ == "__main__":
+ os.chdir(ROOT)
+ mcp.run(transport="stdio")
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/plugins_func/functions/start_repeater.py b/xiaozhi-esp32-server/main/xiaozhi-server/plugins_func/functions/start_repeater.py
new file mode 100644
index 0000000..58d2a1d
--- /dev/null
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/plugins_func/functions/start_repeater.py
@@ -0,0 +1,72 @@
+from typing import TYPE_CHECKING
+
+from plugins_func.register import register_function, ToolType, ActionResponse, Action
+
+if TYPE_CHECKING:
+ from core.connection import ConnectionHandler
+
+start_repeater_function_desc = {
+ "type": "function",
+ "function": {
+ "name": "start_repeater",
+ "description": "鍦ㄨ澶� SD 鍗′笂澶嶈鎸囧畾 MP3銆俷ame 蹇呴』涓� self.mp3.get_list 杩斿洖鐨� files[].name 瀹屽叏涓�鑷淬�俽epeat 鏄暣棣栧惊鐜鏁般�備笉瑕佺敤 play_music銆�",
+ "parameters": {
+ "type": "object",
+ "properties": {
+ "name": {
+ "type": "string",
+ "description": "SD 鍗� MP3 鏂囦欢鍚嶏紝蹇呴』涓� get_list 鐨� files[].name 涓�鑷�",
+ },
+ "repeat": {
+ "type": "integer",
+ "description": "鏁撮寰幆娆℃暟锛岄粯璁� 1",
+ },
+ "power_off_after": {
+ "type": "boolean",
+ "description": "鎾畬鍚庢槸鍚﹁蒋鍏虫満锛岄粯璁� false",
+ },
+ },
+ "required": ["name"],
+ },
+ },
+}
+
+
+@register_function("start_repeater", start_repeater_function_desc, ToolType.SYSTEM_CTL)
+async def start_repeater(
+ conn: "ConnectionHandler",
+ name: str,
+ repeat: int = 1,
+ power_off_after: bool = False,
+):
+ from core.app_runtime.runtime import start_app_task
+
+ try:
+ repeat_n = int(repeat or 1)
+ except (TypeError, ValueError):
+ repeat_n = 1
+ if repeat_n < 1:
+ repeat_n = 1
+
+ result = await start_app_task(
+ conn,
+ "english_learning",
+ "repeater",
+ {
+ "name": name,
+ "repeat": repeat_n,
+ "power_off_after": bool(power_off_after),
+ },
+ source="voice",
+ )
+ if result.action in (Action.ERROR, Action.RESPONSE):
+ return ActionResponse(
+ action=Action.RESPONSE,
+ result=result.result,
+ response=result.response or "鏃犳硶寮�濮嬪璇�",
+ )
+ return ActionResponse(
+ action=Action.NONE,
+ result=result.result,
+ response=result.response,
+ )
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/plugins_func/functions/switch_app.py b/xiaozhi-esp32-server/main/xiaozhi-server/plugins_func/functions/switch_app.py
new file mode 100644
index 0000000..7282138
--- /dev/null
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/plugins_func/functions/switch_app.py
@@ -0,0 +1,41 @@
+from typing import TYPE_CHECKING
+
+from plugins_func.register import register_function, ToolType, ActionResponse, Action
+
+if TYPE_CHECKING:
+ from core.connection import ConnectionHandler
+
+switch_app_function_desc = {
+ "type": "function",
+ "function": {
+ "name": "switch_app",
+ "description": "鍒囨崲璁惧褰撳墠 App銆傛墦寮�缈昏瘧銆佹墦寮�鑻辫瀛︿範/澶嶈銆佸洖鍒拌亰澶┿�侀��鍑虹炕璇戞椂璋冪敤銆俛pp_id 鍙兘鏄� chat銆乪nglish_learning銆乪nglish_translation銆�",
+ "parameters": {
+ "type": "object",
+ "properties": {
+ "app_id": {
+ "type": "string",
+ "description": "鐩爣 App锛歝hat锛堣亰澶╋級銆乪nglish_learning锛堝璇伙級銆乪nglish_translation锛堢炕璇戯級",
+ }
+ },
+ "required": ["app_id"],
+ },
+ },
+}
+
+
+@register_function("switch_app", switch_app_function_desc, ToolType.SYSTEM_CTL)
+async def switch_app(conn: "ConnectionHandler", app_id: str):
+ from core.app_runtime.runtime import switch_app as runtime_switch
+
+ result = await runtime_switch(conn, app_id, source="voice")
+ if result.action == Action.ERROR:
+ return ActionResponse(
+ action=Action.RESPONSE,
+ result=result.response,
+ response=result.response or "鎵撲笉寮�杩欎釜搴旂敤",
+ )
+ if str(result.result or "").startswith("chat/"):
+ return ActionResponse(action=Action.NONE, result=result.result, response=result.response)
+ name = result.response or "宸插垏鎹�"
+ return ActionResponse(action=Action.RESPONSE, result=result.result, response=name)
diff --git a/xiaozhi-esp32-server/main/xiaozhi-server/plugins_func/functions/translate.py b/xiaozhi-esp32-server/main/xiaozhi-server/plugins_func/functions/translate.py
new file mode 100644
index 0000000..db9222f
--- /dev/null
+++ b/xiaozhi-esp32-server/main/xiaozhi-server/plugins_func/functions/translate.py
@@ -0,0 +1,61 @@
+from typing import TYPE_CHECKING
+
+from plugins_func.register import register_function, ToolType, ActionResponse, Action
+from core.app_runtime.runtime import TRANSLATOR_PROMPT
+
+if TYPE_CHECKING:
+ from core.connection import ConnectionHandler
+
+translate_function_desc = {
+ "type": "function",
+ "function": {
+ "name": "translate",
+ "description": "鎶婄敤鎴疯繖鍙ヨ瘽缈昏瘧鎴愬彟涓�绉嶈瑷�銆傚彧杩斿洖璇戞枃銆備腑鏂囪瘧鑻辨枃锛岃嫳鏂囪瘧涓枃銆備笉瑕佽В閲娿��",
+ "parameters": {
+ "type": "object",
+ "properties": {
+ "text": {"type": "string", "description": "瑕佺炕璇戠殑鍘熸枃"},
+ "source_lang": {
+ "type": "string",
+ "description": "婧愯瑷�浠g爜锛屽彲绌猴紱绌哄垯鑷姩鍒ゆ柇",
+ },
+ "target_lang": {
+ "type": "string",
+ "description": "鐩爣璇█浠g爜锛屽彲绌猴紱绌哄垯涓枃鈫旇嫳鏂�",
+ },
+ },
+ "required": ["text"],
+ },
+ },
+}
+
+
+def _build_user_prompt(text: str, source_lang: str, target_lang: str) -> str:
+ extra = ""
+ if source_lang or target_lang:
+ extra = f"\n婧愯瑷�:{source_lang or 'auto'} 鐩爣璇█:{target_lang or 'auto'}"
+ return f"缈昏瘧涓嬮潰鍐呭锛屽彧杈撳嚭璇戞枃锛歿extra}\n{text}"
+
+
+@register_function("translate", translate_function_desc, ToolType.SYSTEM_CTL)
+def translate(
+ conn: "ConnectionHandler",
+ text: str,
+ source_lang: str = "",
+ target_lang: str = "",
+):
+ llm = getattr(conn, "llm", None)
+ if not llm or not text:
+ return ActionResponse(action=Action.ERROR, response="缈昏瘧鏈嶅姟涓嶅彲鐢�")
+ try:
+ out = llm.response_no_stream(
+ TRANSLATOR_PROMPT,
+ _build_user_prompt(text, source_lang or "", target_lang or ""),
+ temperature=0.1,
+ )
+ out = (out or "").strip()
+ if not out:
+ return ActionResponse(action=Action.ERROR, response="娌℃湁寰楀埌璇戞枃")
+ return ActionResponse(action=Action.RESPONSE, result=out, response=out)
+ except Exception as e:
+ return ActionResponse(action=Action.ERROR, response=str(e))
diff --git "a/\345\260\217\346\231\272 AI Server-Task\343\200\201\350\257\255\351\237\263\345\210\206\346\236\220\344\270\216\347\224\250\346\210\267\350\257\204\344\274\260\345\271\263\345\217\260\346\236\266\346\236\204 Plan.md" "b/\345\260\217\346\231\272 AI Server-Task\343\200\201\350\257\255\351\237\263\345\210\206\346\236\220\344\270\216\347\224\250\346\210\267\350\257\204\344\274\260\345\271\263\345\217\260\346\236\266\346\236\204 Plan.md"
new file mode 100644
index 0000000..fcdcec7
--- /dev/null
+++ "b/\345\260\217\346\231\272 AI Server-Task\343\200\201\350\257\255\351\237\263\345\210\206\346\236\220\344\270\216\347\224\250\346\210\267\350\257\204\344\274\260\345\271\263\345\217\260\346\236\266\346\236\204 Plan.md"
@@ -0,0 +1,537 @@
+# 灏忔櫤 AI Server Architecture Plan
+
+> 鏂囨。锛氬皬鏅� AI Server 鈥斺�� Task銆佽闊冲垎鏋愪笌鐢ㄦ埛璇勪及骞冲彴鏋舵瀯 Plan
+> Version: V1.1
+> Date: 2026-08-30
+> 鏈枃鍦� V1.0 姒傚康绋夸笂锛屼緷鎹�奨iaozhi AI 寮�鏀惧簲鐢ㄥ钩鍙� Master Plan銆嬩笌銆夾rchitecture Audit銆嬩慨璁€��
+> **鍙璁★紝涓嶇紪鐮併��**
+> 涓庛�婂皬鏅� AI 纭欢寮�鏀惧簲鐢ㄥ钩鍙� 鈥斺�� 鎬讳綋鏋舵瀯涓庡紑鍙� Plan銆媀1.1 鎴愬闃呰銆�
+
+---
+
+## 0. 鏈枃瀹氫綅
+
+鏈枃鍙畾涔� **Server**锛堜粨搴� `xiaozhi-esp32-server`锛夊浣曚粠銆屽疄鏃惰闊冲璇濆悗绔�嶆紨杩涗负銆孴ask 缂栨帓 + 璇煶瀵硅薄 + 璇勪及銆嶅钩鍙般��
+
+纭欢鎵ц缁嗚妭瑙� Client Plan銆傛湰鏂囧彧瑙勫畾 **瀵硅澶囩殑濂戠害** 涓� **浜戠妯″潡 / 鏁版嵁 / API**銆�
+
+宸ヤ綔鍖哄彟鏈� `smartlearning/`锛堣绋� JSON + MQTT DOWNLOAD/PLAY_MEDIA锛夈�傚畠鏇存帴杩� App Sync锛屼絾**鏈帴鍏�** Xiaozhi Server銆傛湰鏂囨妸瀹冨綋浣滃彲鍚告敹鐨勬梺璺紝鑰屼笉鏄涓夊闀挎湡涓氬姟浜戙��
+
+**鐩稿 V1.0 鐨勫叧閿慨姝o細**
+
+| V1.0 | V1.1锛堟湰娆★級 |
+|------|----------------|
+| 涓�涓婃潵鎷� Gateway/涓�鍫� Assessment 寰湇鍔� | **閫昏緫妯″潡鍒嗗紑锛岀墿鐞嗕粛鏄� Python xiaozhi-server + Java manager-api** |
+| Phase 1 灏卞仛璺熻+鍊掕儗+鍙g畻涓夊璇勪及 | Phase 1 = 鑻辫澶嶈缂栨帓涓� **涓嶈瘎浼�**锛汸hase 2 = 璺熻 + ASR 鏂囨湰姣斿 |
+| 璁惧涓� Server 鍚勬湁涓�濂楀畬鏁� Task Runtime | **鏉冨▉ Task Runtime 鍙湪 Server**锛涜澶囨槸 Task Executor |
+| 闊抽瑙勫垝涓哄璞″瓨鍌紝浣嗘湭瀵归綈鐜扮姸 | 鎵胯鐜扮姸鏄� `ai_agent_chat_audio` LONGBLOB锛涙紨杩涘啀杩佸璞″瓨鍌� |
+| Agent 鏈尯鍒� | **Agent 鈮� App**锛汚gent 缁х画鍋氫汉璁�/妯″瀷閰嶇疆 |
+| 8锝�10 涓钩鍙伴樁娈垫贩鍦ㄤ竴璧� | 涓� Client 瀵归綈涓� Phase 1/2/3锛屽叾鍚庤瘎浼板钩鍙板啀鎷嗗瓙闃舵 |
+
+---
+
+## 1. 褰撳墠 Server 鏄粈涔堬紙Audit 浜嬪疄锛�
+
+鍙屾爤锛屼笉鏄井鏈嶅姟缃戯細
+
+| 妯″潡 | 鍏ュ彛 | 鑱岃矗 |
+|------|------|------|
+| xiaozhi-server | `main/xiaozhi-server/app.py` | WS `/xiaozhi/v1/`銆丱TA HTTP銆佸疄鏃� ASR/LLM/TTS |
+| manager-api | `AdminApplication.java` | 鐢ㄦ埛/璁惧/Agent/鎻掍欢/鑱婂ぉ钀藉簱 |
+| manager-web / mobile | 鈥� | 绠$悊绔� |
+| digital-human | 鈥� | 娴嬭瘯绔紝闈炰富閾捐矾 |
+
+涓�娆″璇濓細
+
+```text
+Device 鈫� ConnectionHandler (session_id = uuid4)
+ 鈫� VAD 鈫� ASR 鈫� intent / chat()
+ 鈫� UnifiedToolHandler (Plugin / MCP / IoT)
+ 鈫� TTS 鈫� type=tts/stt/llm/mcp + Opus
+ 鈫� 鍙�� report 鈫� POST /agent/chat-history/report
+ 鈫� ai_agent_chat_history + ai_agent_chat_audio (WAV)
+```
+
+鍏抽敭浠g爜锛�
+
+| 闃舵 | 浣嶇疆 |
+|------|------|
+| 鍏ョ珯 | `core/connection.py` `_route_message` |
+| VAD/ASR | `core/handle/receiveAudioHandle.py` |
+| 鎰忓浘 | `core/handle/intentHandler.py` |
+| LLM | `ConnectionHandler.chat` |
+| TTS 鍑虹珯 | `core/handle/sendAudioHandle.py` |
+| 涓婃姤 | `core/handle/reportHandle.py` 鈫� WAV |
+| 宸ュ叿 | `plugins_func/*` + `unified_tool_handler.py` |
+
+**宸叉湁鍙鐢細** Agent銆丳lugin銆丮CP銆乫unction call銆乻ession_id銆佸彲閫� WAV銆乣play_music`锛堜絾璧� TTS 閫氶亾锛�**涓嶉�傚悎**澶嶈寰幆锛夈��
+
+**涓嶅瓨鍦細** App銆乀ask銆丼tep銆乀askInstance銆両nteractionMode銆丄ssessment銆丳lan銆丼ubscription銆�
+`session_id` 鏄� **WS 杩炴帴绾�**锛屼笉鏄� TaskInstance銆�
+
+---
+
+## 2. 鐩爣锛堜粛鏄� V1.0 鎰挎櫙锛屼絾鍒嗗眰钀藉湴锛�
+
+```text
+Xiaozhi Server
+ 鈹溾攢鈹� Device Gateway 鈫� 鐜版湁 WS / MQTT 缃戝叧锛屼繚鐣�
+ 鈹溾攢鈹� Chat Orchestration 鈫� 鐜版湁 ConnectionHandler.chat
+ 鈹溾攢鈹� Task Runtime 鈫� 鏂板锛屾潈濞佺紪鎺�
+ 鈹溾攢鈹� Media / Audio Object 鈫� 浠庛�岃亰澶╅檮浠躲�嶆紨杩涗负鍙寕 Step 鐨勯煶棰戝璞�
+ 鈹斺攢鈹� Assessment 鈫� Phase 2 璧凤紱鍏堟枃鏈瘮瀵癸紝鍚庤闊崇壒寰�
+```
+
+鏈�缁堟暟鎹祦锛堢洰鏍囨�侊紝涓嶆槸 Phase 1锛夛細
+
+```text
+Schedule 鈫� TaskInstance 鈫� Step 鈫� InteractionMode
+ 鈫� Device Skill 鈫� 鍎跨璇煶
+ 鈫� 淇濆瓨 Audio 瀵硅薄 鈫� ASR 鈫� Assessment 鈫� Metric 鈫� 瀹堕暱鎶ュ憡
+```
+
+Phase 1 **鎴柇鍦�**銆孧ode=PLAY_AUDIO 椹卞姩璁惧鎾斁銆嶏紱涓嶈姹傝瘎浼颁笌瀹堕暱鎶ュ憡銆�
+
+---
+
+## 3. 鑱岃矗杈圭晫锛堜笌 Client Plan 瀵归綈锛�
+
+```text
+Server
+ 鍒涘缓 / 鎺ㄨ繘 / 缁撴潫 TaskInstance
+ 鎸囧畾 InteractionMode
+ 璋冪敤璁惧 Skill锛圡CP 鎴� type=task锛�
+ ASR / LLM / TTS / Assessment
+ 闊抽瀵硅薄涓� step 鍏宠仈
+ App JSON 涓庤祫婧� URL 鐨勬潈濞�
+
+Client
+ 鎵ц Skill銆佸垏鎹� Mode銆佸洖鎶� Event
+ 涓嶆寔涔呭寲 Step 鍘嗗彶銆佷笉璇勫垎
+```
+
+**绂佹**鍦� Python 瀵硅瘽鐑矾寰勯噷鍐欐 `if english_repeat` 浣滀负闀挎湡褰㈡�侊紱Phase 1 鍏佽涓�涓� **缂栨帓鎻掍欢**锛孭hase 2 蹇呴』鍗囧埌 Task Runtime + JSON 瀹氫箟銆�
+
+---
+
+## 4. 閫昏緫妯″潡 vs 鐗╃悊閮ㄧ讲
+
+V1.0 鍒楀嚭鐨� Gateway / Task Service / Assessment Service / Emotion 鈥� **鍏ㄩ儴浣滀负閫昏緫杈圭晫**銆�
+
+鐗╃悊閮ㄧ讲淇濇寔锛�
+
+```text
+xiaozhi-server (Python) 瀹炴椂锛氳繛鎺ャ�丄SR銆丩LM銆乀TS銆丮ode 鍒囨崲銆乀ask 姝ヨ繘锛堣繍琛屾椂锛�
+manager-api (Java) 閰嶇疆涓庢寔涔呭寲锛歎ser銆丏evice銆丄gent銆乀ask 瀹氫箟/瀹炰緥銆侀煶棰戝厓鏁版嵁銆佽瘎浼扮粨鏋�
+瀵硅薄瀛樺偍锛圥hase 2+锛� 鍘熷闊抽锛涚涓�鏈熷彲缁х画 BLOB锛屼絾 API 鎸� audio_id 璁块棶
+```
+
+涓嶈涓虹涓�涓璇� Task 鎷嗘柊杩涚▼銆�
+
+妯″潡鏄犲皠锛�
+
+| 閫昏緫妯″潡 | Phase | 钀界偣 |
+|----------|-------|------|
+| Device Gateway | 0 宸叉湁 | `websocket_server.py` / MQTT gateway |
+| Chat Orchestration | 0 宸叉湁 | `ConnectionHandler` |
+| Interaction Mode 寮�鍏� | 1 | `ConnectionHandler` 涓� `interaction_mode`锛涢潪 CHAT 璺宠繃 `chat()` |
+| Repeat Orchestrator | 1 | `plugins_func` 鎴栫嫭绔嬭杽妯″潡锛汳CP `self.mp3.play` |
+| Task Runtime | 2 | Python 杩愯鏃剁姸鎬佹満 + Java 瀹炰緥琛� |
+| Audio Object | 2 | 鎵╁睍涓婃姤锛氭寕 `step_instance_id`锛涢�愭杩佸嚭 LONGBLOB |
+| Assessment v0 | 2 | expected_text vs ASR锛汮ava 鎴� Python 鍧囧彲锛岀粨鏋滃叆搴� |
+| App / Resource | 3 | 鍚告敹 smartlearning 娓呭崟 + 涓嬭浇 URL锛汮ava 涓烘潈濞� |
+| Scheduler / Plan | 3 | manager-api + 瀹氭椂锛涗笉瑕佺敤璁惧 RTC 闂归挓鍐掑厖 |
+| Voice Feature / Baseline / Emotion | 3+ | 寮傛闃熷垪锛涚姝㈣繘瀹炴椂 TTS 璺緞 |
+| Subscription / Developer API | 鏇村悗 | 涓嶅仛 Phase 1鈥�2 |
+
+---
+
+## 5. 鐜版湁瀵硅瘽閾捐矾锛氬繀椤讳繚鐣�
+
+鏃� Task 鏃惰涓轰笌鐜扮綉涓�鑷达細
+
+```text
+listen 鈫� ASR 鈫� intent/LLM/tools 鈫� TTS
+```
+
+鍗忚 `hello/listen/abort/iot/mcp/ping` 涓庝笅琛� `stt/tts/llm/mcp` **绂佹鏀硅涔�**銆�
+
+鎻掑叆鏂瑰紡锛�
+
+```text
+_route_message / startToChat / chat()
+ 涔嬪墠鍒ゆ柇 interaction_mode
+ CHAT 鈫� 鍘熻矾寰�
+ PLAY_AUDIO / FOLLOW_READ 鈫� 涓嶈皟鐢� chat()
+ FOLLOW_READ 鐨勮闊充粛璧� ASR.receive_audio锛岀粨鏋滃啓鍏� Step 鑰岄潪 Dialogue
+```
+
+`play_music` 缁х画鏈嶅姟銆岀敤鎴疯鎾斁姝屾洸銆嶃��**涓嶈**鐢ㄥ畠瀹炵幇鑻辫澶嶈锛堝畠鍗犵敤 TTS 闃熷垪锛屽拰璇剧▼ SD 鎾斁鍐茬獊锛夈��
+
+---
+
+## 6. Agent vs App
+
+| | Agent锛堝凡鏈� `ai_agent`锛� | App锛堟柊寤猴紝Phase 3锛� |
+|--|--------------------------|----------------------|
+| 鏄粈涔� | 浜鸿 + ASR/LLM/TTS/鎻掍欢閰嶇疆 | 涓氬姟瀹瑰櫒锛堣嫳璇�佸笀銆佸彛绠椻�︼級 |
+| 璋佺敤 | 姣忓彴璁惧褰撳墠缁戝畾涓�涓� | 鍎跨鍙闃呭涓紝鍚屾椂鍙� Active 涓�涓� |
+| Phase 1 | **缁х画鐢� Agent 璺� Chat** | 涓嶅缓 App 琛ㄤ篃鍙互鍋氬璇� Demo |
+
+Phase 1 澶嶈鍙互鎸傚湪銆屽綋鍓� Agent + 涓�涓彃浠躲�嶏紝涓嶅繀鍏堟湁 App 瀹炰綋銆侾hase 3 鍐嶆妸 Task Definition 褰掑埌 App 涓嬨��
+
+涓嶈鎶� `ai_agent` 鏀瑰悕涓� App锛屼篃涓嶈鍦� Agent 涓婂爢 Workflow JSON 浣滀负闀挎湡妯″瀷锛堜复鏃跺彲浠ワ紝姝e紡妯″瀷瑙佺 10 鑺傦級銆�
+
+---
+
+## 7. Task Runtime锛圫erver 鏉冨▉锛�
+
+### 7.1 鑱岃矗
+
+```text
+鍔犺浇 Task Definition
+鍒涘缓 TaskInstance
+鎸� Step 鎺ㄨ繘
+璁剧疆 Connection.interaction_mode
+涓嬪彂 skill.execute / MCP
+绛夊緟 Device Event 鎴� ASR 瀹屾垚
+鍐� StepInstance
+鍐冲畾涓嬩竴姝� / 缁撴潫 / 澶辫触
+```
+
+杩愯鏃跺彲娲诲湪 `ConnectionHandler` 鏃佺殑 `TaskRuntime` 瀵硅薄锛堟瘡杩炴帴涓�涓綋鍓� Instance锛夈�傛寔涔呭寲鍦� manager-api銆�
+
+### 7.2 鐘舵��
+
+```text
+CREATED 鈫� READY 鈫� RUNNING
+ 鈫� WAITING_DEVICE | WAITING_USER | WAITING_AI
+ 鈫� COMPLETED | FAILED | CANCELLED | TIMEOUT
+```
+
+璁惧鍙紦瀛� RUNNING 鐨勫綋鍓� step锛涢噸杩炲悗 Server 閲嶆斁鎴栫粨鏉熴��
+
+### 7.3 Definition 涓� Instance 鍒嗙锛堜繚鐣� V1.0锛�
+
+Definition锛氭ā鏉匡紙鑻辫澶嶈 10 閬嶏級銆�
+Instance锛氭煇鏃ユ煇璁惧鐨勪竴娆℃墽琛岋紙杩涘害銆佺姸鎬併�佺粨鏋滐級銆�
+
+### 7.4 Phase 1 鐗逛緥
+
+娌℃湁姝e紡 Runtime 涔熷彲浠ワ細
+
+```text
+鎻掍欢 english_repeat(n, resource)
+ loop: MCP play 鈫� wait play.completed 鎴栬秴鏃�
+ mode=PLAY_AUDIO 鏈熼棿绂佹 chat()
+```
+
+杩欐槸涓轰簡楠岃瘉閾捐矾锛汸hase 2 鐢� JSON Step 鏇挎崲璇ュ惊鐜��
+
+---
+
+## 8. InteractionMode 鍦� Server 鐨勫惈涔�
+
+Server 鏄� Mode 鐨� **鎸囧畾鑰�**锛�
+
+```text
+Step.mode = PLAY_AUDIO | FOLLOW_READ | CHAT | LISTEN | 鈥�
+ 鈫� conn.interaction_mode = 鈥�
+ 鈫� 鍙�変笅琛� mode.set锛圥hase 2锛�
+```
+
+| Mode | Server 琛屼负 |
+|------|-------------|
+| CHAT | 鐜版湁 `startToChat` / `chat()` |
+| PLAY_AUDIO | 涓� ASR 瀵硅瘽銆佷笉 LLM锛涢┍鍔� play锛涚瓑瀹屾垚浜嬩欢 |
+| FOLLOW_READ | 鍏� play锛屽啀鏀跺惉锛汚SR 鏂囨湰杩� Step锛�**涓�**杩� Dialogue/LLM |
+| LISTEN | 鍙� ASR锛岀粨鏋滅粰褰撳墠 Step |
+| QUIZ/GAME | Phase 3锛氬嚭棰樺湪 Server锛岃澶� CHAT 鎴� TTS+LISTEN |
+
+V1.0 鐨勩�孖nteraction Runtime 浣滀负鐙珛鏈嶅姟銆嶉檷绾т负 **ConnectionHandler 涓婄殑 mode 瀛楁 + TaskRuntime 绛栫暐琛�**銆�
+
+---
+
+## 9. 闊抽鏋舵瀯
+
+### 9.1 鐜扮姸
+
+- 鍏ワ細Opus 娴侊紱鍐呴儴 ASR PCM 16k
+- 鍑猴細Opus锛坔ello 閲屽父瑙� 24k / 60ms锛�
+- 钀藉簱锛歚chat_history_conf==2` 鏃� PCM/Opus **杞� WAV** 杩� `ai_agent_chat_audio.audio`锛堟敞閲婂啓 opus锛屽疄闄� WAV锛�
+- `audio_id` 浠呭湪涓婃姤鍚庡瓨鍦紱history 鏈� `session_id`锛�**鏃�** task/step
+
+### 9.2 鍘熷垯锛堜繚鐣� V1.0锛�
+
+> 璺熻/璇勪及涓嶈鍙瓨 ASR 鏂囨湰銆傚師濮嬮煶棰戞槸鍙噸鍒嗘瀽璧勪骇銆�
+
+浣� Phase 1 澶嶈 **鍙互涓嶅瓨鐢ㄦ埛闊抽**锛堢敤鎴峰彲鑳芥病璇磋瘽锛夈��
+
+### 9.3 婕旇繘
+
+**Phase 2 鏈�灏忥細**
+
+```text
+娌跨敤 reportHandle WAV 涓婃姤
+瀛楁澧炲姞 task_instance_id, step_instance_id, purpose=follow_read
+audio_id 鍐欏叆 step_instance
+浠嶅彲鏆傚瓨 BLOB
+```
+
+**Phase 2+ 鐩爣锛圴1.0 瀵硅薄瀛樺偍锛夛細**
+
+```text
+DB: audio_id, user/device, task/step, storage_key, duration, format, sample_rate
+Blob: 瀵硅薄瀛樺偍 raw wav/opus
+涓嶈鎶婇暱鏈熻绋嬮煶棰戠户缁爢鍦� MySQL LONGBLOB
+```
+
+**Segment锛�** 璺熻涓�姝ヤ竴娈电敤鎴烽煶棰戝嵆鍙紱鍗曡瘝绾у垏鍒嗘斁鍒拌瘎浼版ā鍨嬮渶瑕佹椂鍐嶅仛銆備笉瑕� Phase 2 灏变笂瀹屾暣 AudioSession 鏍戙��
+
+**璁惧鏂囦欢褰曢煶 / pending 涓婁紶锛�** Client Phase 3锛汼erver 棰勭暀 upload API锛屼笉闃诲璺熻 v0銆�
+
+鏃堕棿鎴筹細杩炴帴绾ц冻澶熷仛 v0锛汵TP 涓庣簿纭� latency 鏀捐瘎浼� Phase 3銆係ening 宸叉湁鏃堕挓鍚屾鎸囦护锛屽彲澶嶇敤锛屼笉绠� Server 鏂版湇鍔°��
+
+---
+
+## 10. 鏁版嵁妯″瀷
+
+### 10.1 鐜版湁琛紙淇濈暀锛孋hat 缁х画鐢級
+
+| 琛� | 浣滅敤 |
+|----|------|
+| `sys_user` | 璐﹀彿 |
+| `ai_device` | 璁惧锛沗user_id` `agent_id` `mac_address` |
+| `ai_agent` | 浜鸿涓庢ā鍨嬶紱`chat_history_conf` |
+| `ai_agent_chat_history` | 浼氳瘽鏂囨湰锛沗session_id` `audio_id` `chat_type` |
+| `ai_agent_chat_audio` | 闊抽 BLOB |
+| `ai_agent_plugin_mapping` | Agent 鈫� 鎻掍欢 |
+| `ai_ota` / 妯″瀷 / RAG 绛� | 涓� Task 鏃犲叧锛屼笉鍔� |
+
+Chat 鍘嗗彶 **涓嶈**鏀硅涔夊幓瀛� Task锛涚敤鏂拌〃鎴栧彲绌哄閿墿灞曘��
+
+### 10.2 Phase 2 鏈�灏忔柊琛�
+
+```text
+task_definition id, app_id(鍙┖), name, version, json_body, interaction 榛樿 mode
+task_instance id, device_id, task_definition_id, session_id,
+ status, current_step, started_at, ended_at
+step_instance id, task_instance_id, step_index, mode, status,
+ audio_id, asr_text, result_json, started_at, ended_at
+```
+
+绗竴鏈� `json_body` 鍙唴宓� steps锛屼笉蹇呭厛鎷� `task_step` 瀹氫箟琛ㄣ��
+
+### 10.3 Phase 3+锛圴1.0 娓呭崟鐨勫瓙闆嗭紝鎸夐渶锛�
+
+```text
+app / app_version / app_subscription
+plan / schedule
+audio_segment # 浠� chat_audio 杩佸嚭
+assessment / assessment_metric
+user_metric_baseline
+daily_summary
+```
+
+V1.0 鐨� emotion_signal銆乨eveloper 瀹借〃 **涓嶈**鍒楀叆 Phase 2 DDL銆�
+
+Child 瀹炰綋锛氱幇缃戞槸 User+Device銆侾hase 3 鑻ラ渶瑕併�屼竴涓闀垮涓瀛愩�嶅啀鎷� `child`锛汸hase 1鈥�2 鐢� `user_id + device_id`銆�
+
+---
+
+## 11. Assessment
+
+淇濈暀 V1.0 鍘熷垯锛屾敹绱ц惤鍦般��
+
+### 11.1 涓夊眰锛堟蹇典繚鐣欙級
+
+```text
+L1 浠诲姟瀵归敊 / 鏂囨湰鍖归厤 鈫� Phase 2 v0
+L2 璇煶鐗瑰緛锛堣閫熴�佸仠椤匡級 鈫� 寮傛锛孭hase 3
+L3 瓒嬪娍 / 鍩虹嚎 / 鍙備笌搴� 鈫� 瑕佹湁瓒冲 Instance
+```
+
+鎯呯华鍙仛 **淇″彿** 涓嶆槸璇婃柇锛圴1.0 搂20 淇濈暀锛夛紝涓� **鏅氫簬** L1銆�
+
+### 11.2 鑻辫璺熻 v0锛圥hase 2锛�
+
+```text
+expected_text vs ASR text
+鈫� 鍑嗙‘鐜� / 鏄惁璇村畬
+鈫� 鍗虫椂 TTS 鍙嶉锛堣蛋鐜版湁 TTS锛�
+鈫� 缁撴灉鍐欏叆 step_instance.result_json
+```
+
+涓嶅湪 Phase 2 涓婂彂闊虫ā鍨嬨�侀煶绱犲榻愩�亀ord timing锛堟湁鍒欏瓨 ASR 鍘熷 JSON锛岃瘎浼板櫒鍚庤ˉ锛夈��
+
+### 11.3 鍙g畻 / 鍊掕儗 / 鎴愯
+
+褰� Phase 3+ Task Definition锛岀敤 QUIZ/GAME + ASR 瑙f瀽銆�**涓嶈**涓哄畠浠厛鏀� Client銆�
+
+### 11.4 璇勪及鐗堟湰鍖栵紙淇濈暀 V1.0锛�
+
+```text
+assessment.model_version
+鍚屼竴 step 鍙拷鍔犳柊璇勪及琛岋紝绂佹瑕嗙洊鏃х粨鏋�
+```
+
+瀹炴椂璺緞鍙仛 L1锛汱2+ 璧伴槦鍒楋紝閬垮厤鍗′綇 TTS銆�
+
+---
+
+## 12. 鍗忚涓� API
+
+### 12.1 璁惧鍗忚
+
+Phase 1锛氬彧澧炲姞 **MCP 璋冪敤**锛堝凡鏈� `type=mcp`锛夈�係erver 涓诲姩 `tools/call` 鎾斁銆�
+
+Phase 2锛氫笌 Client Plan 鐩稿悓鐨� `task.*` / `mode.set` / 瀹屾垚浜嬩欢銆侾ython `TextMessageType` 澧炲姞鏋氫妇锛孒andler 涓� `hello/listen/mcp` 骞跺垪銆�
+
+浜岃繘鍒� Opus **涓嶆敼**銆侻QTT 缃戝叧 16 瀛楄妭澶撮�昏緫淇濇寔銆�
+
+### 12.2 HTTP API锛坢anager-api锛�
+
+Phase 1锛氬彲鏃犳柊 API锛堟彃浠� + 绠$悊绔幇鏈夐厤缃冻澶熻Е鍙� Demo锛夈��
+
+Phase 2锛�
+
+```text
+POST /task/instance 鍚姩锛坉eviceId, taskId, params锛�
+GET /task/instance/{id} 鏌ヨ鐘舵�佷笌 steps
+POST /agent/chat-history/report 鎵╁睍鍙�� task/step 瀛楁锛堝吋瀹规棫瀹㈡埛绔級
+```
+
+Phase 3锛�
+
+```text
+/apps /resources /schedule /assessment /reports
+```
+
+V1.0 涓�娆¢摵寮� `/analytics` `/developer` 鈥斺�� 鍚庣疆銆�
+
+涓婃姤浠嶈蛋鐜版湁 `manage_api_client.report`锛岄伩鍏嶅啀閫犱竴鏉¤亰澶╃閬撱��
+
+---
+
+## 13. Plugin / MCP 澶嶇敤
+
+```text
+Server Plugin 缂栨帓銆佷笟鍔″垽鏂�佽皟 ASR 缁撴灉
+Device MCP 鎾斁銆侀煶閲忋�佸睆銆佷笅杞斤紙璁惧 Skill锛�
+Server MCP 澶栭儴宸ュ叿锛屼笌 Task 鏃犲己缁戝畾
+```
+
+鑻辫澶嶈 Phase 1锛歋erver Plugin 璋� Device MCP銆�
+涓嶈鎶婂璇诲啓杩涘浐浠讹紝涔熶笉瑕佹柊寤轰笌 `UnifiedToolHandler` 骞宠鐨勭浜屽宸ュ叿鎬荤嚎銆�
+
+---
+
+## 14. smartlearning 鍚告敹绛栫暐
+
+| 宸叉湁鑳藉姏 | 褰掑睘 |
+|----------|------|
+| 璇剧▼ JSON / manifest | Phase 3 App Resource锛涙潈濞佽縼鍏� manager-api |
+| MQTT DOWNLOAD / PLAY_MEDIA | Client 鏉跨骇宸插疄鐜帮紱Server 鏈�缁堢粡 Xiaozhi 杩炴帴涓嬪彂鍚岀瓑璇箟 |
+| 鍦烘櫙璇嗗埆 HTTP | 涓嶈繘鍏� Xiaozhi 涓婚摼璺紝闄ら潪浜у搧鏄庣‘瑕� |
+
+**绂佹**璁� Xiaozhi Server 涓� smartlearning 闀挎湡鍙屽啓鍚屼竴璁惧鐘舵�併�侾hase 1 Demo 鑻ュ彧鑳借窇 Sening锛屽厑璁告殏鏃舵墦 snlp锛屼絾 Coding Plan 蹇呴』鏍囨槑鎶�鏈�恒��
+
+---
+
+## 15. 鍒嗛樁娈碉紙Server锛�
+
+涓� Client Plan 缂栧彿涓�鑷淬��
+
+### Phase 1 鈥� 澶嶈缂栨帓锛屼笉鐮村潖 Chat
+
+- `ConnectionHandler.interaction_mode`锛歚CHAT` | `PLAY_AUDIO`
+- PLAY_AUDIO 鏃讹細涓� `chat()`銆佷笉鎶婅绋嬪綋鐢ㄦ埛鎰忓浘
+- 鎻掍欢锛氬惊鐜� N 娆� MCP/`PLAY_MEDIA`锛堟垨绛変环锛�
+- **涓嶅缓** Task 琛ㄣ�佷笉璇勪及銆佷笉鏀� history 璇箟銆佷笉鎷嗗井鏈嶅姟
+- Chat 鍥炲綊锛歮ode 鍥� CHAT 鎴栬繛鎺ヤ粛鎸夌幇缃� listen
+
+楠屾敹锛氱鐞嗙鎴栦竴鍙ヨЕ鍙戝彲璁╄澶囧璇伙紱鍚屾椂鍞ら啋鑱婂ぉ涓庣幇缃戜竴鑷淬��
+
+### Phase 2 鈥� Task Runtime 鍒囩墖 + 璺熻 v0
+
+- `task_definition` / `task_instance` / `step_instance`
+- 鍗忚 `task.*` + `mode.set`锛堣嫢 MCP 涓嶅琛ㄨ揪姝ヨ繘锛�
+- FOLLOW_READ锛欰SR 鍐欏叆 step锛涙墿灞� report 鍏宠仈 audio_id
+- Assessment v0锛氭枃鏈瘮瀵� + 缁撴灉 JSON
+- 浠嶄笉瑕佹眰瀵硅薄瀛樺偍銆佸熀绾裤�佹儏缁�佸闀垮皬绋嬪簭
+
+楠屾敹锛氫竴娆¤窡璇� Instance 鍙煡璇㈠埌鍚� step 鐨勯煶棰戜笌鏂囨湰涓庡垎鏁般��
+
+### Phase 3 鈥� 骞冲彴涓庤瘎浼版繁鍖�
+
+- App / Resource Sync锛堝惛鏀惰绋嬫竻鍗曪級
+- Schedule / Plan锛堝闀裤�屾瘡澶� 19:30銆嶏級
+- 闊抽杩佸璞″瓨鍌�
+- 寮傛 Voice Feature锛涘彲閫� Baseline
+- QUIZ 绛夋柊 Task 鍙姞 Definition锛屼笉鍔犲浐浠朵笟鍔�
+- 鍐嶅線鍚庯細Emotion Signal銆丼ubscription銆丏eveloper API锛圴1.0 鍚庡崐娈碉級
+
+---
+
+## 16. 绂佹浜嬮」锛圫erver锛�
+
+1. 涓� Task 閲嶅啓 ASR/TTS Provider 鎴� WS 甯ф牸寮�
+2. 鐢� `play_music` 鍐掑厖璇剧▼澶嶈
+3. 鎶� Task 琛屽杩� `ai_agent_chat_history` 褰撴寮忔ā鍨�
+4. Phase 1 涓婃儏缁�/蹇冪悊/鍙備笌搴﹀ぇ鐩�
+5. Phase 1 鎷嗗井鏈嶅姟 / 涓婄嫭绔� MQ 璇勪及闆嗙兢
+6. 鏀� hello/listen/tts 瀛楁鍚箟
+7. 璁╄澶囨垚涓� Step 鏉冨▉
+8. 鏈 `ConnectionHandler` 灏辨柊鍐欎竴濂楀璇濇湇鍔″櫒
+9. 涓� smartlearning 姘镐箙鍙屾爤璋冨害
+
+---
+
+## 17. AI Coding 绾︽潫
+
+1. 鍏堣 `connection.py`銆乣receiveAudioHandle.py`銆乣reportHandle.py`銆丄gent 涓婃姤 API銆佺幇鏈夎〃缁撴瀯銆�
+2. 榛樿 Chat 蹇呴』鍙叧 Task 鐙珛杩愯銆�
+3. 瀹炴椂璺緞鍙仛 Mode 鍒嗘祦涓� L1 璇勪及锛涢噸鍒嗘瀽寮傛銆�
+4. 鏂拌〃鏂板崗璁兘瑕佸悜鍚庡吋瀹规棫璁惧锛堝拷鐣ユ湭鐭� type / 鍙┖ task 瀛楁锛夈��
+5. Phase 1 鑳界敤鎻掍欢瑙e喅鍒欎笉鏀� Java DDL銆�
+
+---
+
+## 18. 涓� Client Plan 鐨勫绾�
+
+Server 淇濊瘉锛�
+
+- CHAT 杩炴帴琛屼负鍚戝悗鍏煎
+- 闈� CHAT mode 涓嶈瑙﹀彂 LLM
+- Instance 鏉冨▉鐘舵�佸彲鏌ヨ
+- Phase 2 闊抽鑳芥寕鍒� step
+
+Server 瑕佹眰 Client锛�
+
+- 鎻愪緵鍙皟鐢ㄧ殑 play_audio锛堣嚦灏戜竴鍧楃洰鏍囨澘锛�
+- FOLLOW_READ 鏃剁户缁笂浼� Opus
+- 瀹屾垚浜嬩欢鍙叧鑱� session / instance
+- 涓嶅湪鍥轰欢鍐欒瘎浼�
+
+缁熶竴楠屾敹閾撅紙鐩爣鎬侊級锛�
+
+```text
+Server 鍒涘缓 Instance 鈫� mode + skill 鈫� Device 鎵ц 鈫� Event/Audio 鈫� ASR 鈫� Assessment 鈫� 鏌ヨ Instance
+```
+
+Phase 1 鍙獙璇佸墠鍗婏細鍒涘缓鎰忓浘 鈫� play 寰幆 鈫� Chat 浠嶅彲鐢ㄣ��
+
+---
+
+## 19. 涓嬩竴闃舵
+
+涓嬩竴姝ヤ笉鏄疄鐜拌瘎浼板钩鍙帮紝鑰屾槸浜у嚭锛�
+
+```text
+Phase 1 Coding Plan
+ 鈹溾攢鈹� Server锛歮ode 鏃佽矾 + 澶嶈鎻掍欢
+ 鈹溾攢鈹� Client锛氬鐢� MCP/PLAY_MEDIA + 闊抽浜掓枼锛堝凡鏈� PauseSdCardMp3锛�
+ 鈹斺攢鈹� 鍥炲綊锛氱幇鏈� Chat E2E
+```
+
+V1.0 涓殑瀹堕暱鎶ュ憡鍒嗙骇銆佽法浠诲姟鍒嗘瀽銆佸紑鍙戣�呭垎鎴愮瓑锛屼繚鐣欎负 Phase 3 浠ュ悗鐨勪骇鍝侀檮褰曪紝涓嶈繘鍏ヨ繎鏈熸灦鏋勬柦宸ュ浘銆�
diff --git "a/\345\260\217\346\231\272 AI \345\274\200\346\224\276\345\272\224\347\224\250\345\271\263\345\217\260-master Pln.md" "b/\345\260\217\346\231\272 AI \345\274\200\346\224\276\345\272\224\347\224\250\345\271\263\345\217\260-master Pln.md"
new file mode 100644
index 0000000..9d82397
--- /dev/null
+++ "b/\345\260\217\346\231\272 AI \345\274\200\346\224\276\345\272\224\347\224\250\345\271\263\345\217\260-master Pln.md"
@@ -0,0 +1,2187 @@
+# Xiaozhi AI 寮�鏀惧簲鐢ㄥ钩鍙�
+# 椤跺眰鏋舵瀯涓� AI Coding Master Plan
+
+> Version: V1.0
+> Date: 2026-08-30
+> Purpose: 浣滀负 Xiaozhi AI 纭欢绔�丼erver 绔�佸皬绋嬪簭绔互鍙婃湭鏉ュ紑鍙戣�呭紑鏀惧钩鍙扮殑鏈�楂樺眰鏋舵瀯璁捐涓� AI Coding 鎬讳綋鎸囧鏂囨。銆�
+
+---
+
+# 1. 椤圭洰鏈�缁堢洰鏍�
+
+灏嗙幇鏈� Xiaozhi AI 浠庯細
+
+```text
+AI 瀵硅瘽纭欢
+```
+
+閫愭鍗囩骇涓猴細
+
+```text
+AI Application Terminal
+```
+
+鏈�缁堝舰鎬佺被浼硷細
+
+```text
+鎵嬫満
+ 鈹斺攢鈹� 鍙互瀹夎澶氫釜 App
+ 鈹斺攢鈹� 姣忎釜 App 鍖呭惈澶氫釜 Task
+ 鈹斺攢鈹� 姣忎釜 Task 鍖呭惈澶氫釜 Step
+ 鈹斺攢鈹� Step 璋冪敤搴曞眰鑳藉姏
+```
+
+浣嗘槸 Xiaozhi 涓庢墜鏈虹殑鏍稿績鍖哄埆鏄細
+
+```text
+鎵嬫満锛�
+鐢ㄦ埛涓诲姩閫夋嫨 App 鈫� 涓诲姩鎿嶄綔
+
+Xiaozhi锛�
+瀹堕暱閰嶇疆 鈫� 绯荤粺璋冨害 鈫� Xiaozhi 涓诲姩鎵ц 鈫� 鍎跨鍙備笌
+```
+
+鍥犳 Xiaozhi 鏈�缁堜笉鏄崟绾殑鑱婂ぉ鏈哄櫒浜猴紝鑰屾槸锛�
+
+> **涓�涓彲浠ユ壙杞藉涓� AI App锛屽苟鏍规嵁瀹堕暱閰嶇疆涓诲姩鎵ц Task锛屽悓鏃舵寔缁褰曘�佽瘎浼板拰鍒嗘瀽鍎跨琛屼负鐨� AI 缁堢骞冲彴銆�**
+
+---
+
+# 2. 鏈�缁堜骇鍝佸舰鎬�
+
+鏈�缁堟暣涓郴缁熺敱锛�
+
+```text
+Xiaozhi Device
+ 鈹�
+ 鈹�
+Xiaozhi Server
+ 鈹�
+ 鈹溾攢鈹� App Platform
+ 鈹溾攢鈹� Task Platform
+ 鈹溾攢鈹� Assessment Platform
+ 鈹溾攢鈹� User Data Platform
+ 鈹溾攢鈹� Developer Platform
+ 鈹溾攢鈹� Subscription Platform
+ 鈹斺攢鈹� Parent Platform
+```
+
+缁勬垚銆�
+
+鐢ㄦ埛鍏崇郴锛�
+
+```text
+骞冲彴
+ 鈹�
+ 鈹溾攢鈹� App A
+ 鈹� 鈹溾攢鈹� Task 1
+ 鈹� 鈹溾攢鈹� Task 2
+ 鈹� 鈹斺攢鈹� Task 3
+ 鈹�
+ 鈹溾攢鈹� App B
+ 鈹� 鈹溾攢鈹� Task 1
+ 鈹� 鈹斺攢鈹� Task 2
+ 鈹�
+ 鈹斺攢鈹� App C
+ 鈹斺攢鈹� ...
+```
+
+涓�涓瀛愬彲浠ヨ闃呭涓� App銆�
+
+浣嗘槸锛�
+
+```text
+鍚屼竴鏃堕棿
+鍙兘杩愯涓�涓� App / Task Runtime
+```
+
+鍥犳涓嶈姹� Xiaozhi 纭欢鍚屾椂杩愯澶氫釜澶嶆潅搴旂敤銆�
+
+---
+
+# 3. 鏍稿績鏋舵瀯鍘熷垯
+
+鏁翠釜绯荤粺蹇呴』鍧氭寔浠ヤ笅鍘熷垯銆�
+
+## 3.1 App 鏄笟鍔¤兘鍔涚殑瀹瑰櫒
+
+App 涓嶅簲璇ョ瓑鍚屼簬纭欢鍔熻兘銆�
+
+渚嬪锛�
+
+```text
+鑻辫瀛︿範 App
+鍥炲繂褰� App
+鐩婃櫤娓告垙 App
+AI 涓绘寔浜� App
+```
+
+閮芥槸閫昏緫搴旂敤銆�
+
+---
+
+## 3.2 Task 鏄郴缁熺湡姝f墽琛岀殑鍩烘湰涓氬姟鍗曚綅
+
+绯荤粺鏈�缁堟墽琛岀殑鏄細
+
+```text
+Task
+```
+
+鑰屼笉鏄細
+
+```text
+Scenario
+```
+
+渚嬪锛�
+
+```text
+鑻辫 App
+ 鈹溾攢鈹� 鑻辫璺熻 10 娆�
+ 鈹溾攢鈹� 鍗曡瘝澶嶄範
+ 鈹斺攢鈹� 鑻辫鍚姏
+```
+
+鍏朵腑锛�
+
+```text
+鑻辫璺熻 10 娆�
+```
+
+灏辨槸涓�涓� Task銆�
+
+---
+
+# 4. Scenario / Interaction Mode 鐨勬渶缁堝畾涔�
+
+鏈」鐩腑涓嶅啀鎶� Scenario 鐞嗚В涓猴細
+
+```text
+涓氬姟鍦烘櫙
+```
+
+鑰屽畾涔変负锛�
+
+> **Device Interaction Mode**
+
+鍗筹細
+
+> Xiaozhi 鍦ㄦ墽琛� Task 鏃讹紝搴旇浠ヤ粈涔堟柟寮忎笌鐢ㄦ埛浜や簰銆�
+
+渚嬪锛�
+
+```text
+CHAT
+PLAY_AUDIO
+REPEAT
+FOLLOW_READ
+QUESTION_ANSWER
+TRANSLATION
+CALCULATION
+MEMORY
+GAME
+SILENT
+```
+
+鍥犳锛�
+
+```text
+Task
+ 鈫�
+鎸囧畾 Interaction Mode
+ 鈫�
+Device Runtime
+ 鈫�
+浠ュ搴旀柟寮忔墽琛�
+```
+
+渚嬪锛�
+
+```text
+Task锛�
+鍚嫳璇璇绘満 10 閬�
+
+interactionMode锛�
+PLAY_AUDIO
+
+steps锛�
+鎾斁 鈫� 绛夊緟 鈫� 鎾斁 鈫� 绛夊緟 ...
+```
+
+涓嶉渶瑕侀澶栧缓绔嬶細
+
+```text
+Scenario
+ 鈫�
+Task
+```
+
+杩欐牱鐨勪笟鍔″眰绾с��
+
+---
+
+# 5. Task 鏄牳蹇冩墽琛屾娊璞�
+
+缁熶竴妯″瀷锛�
+
+```text
+App
+ 鈹斺攢鈹� Task
+ 鈹斺攢鈹� Step
+ 鈹斺攢鈹� Action / Skill
+```
+
+渚嬪锛�
+
+```text
+English App
+ 鈹斺攢鈹� English Repeat 10 Times
+ 鈹溾攢鈹� Play Audio
+ 鈹溾攢鈹� Wait
+ 鈹溾攢鈹� Record
+ 鈹溾攢鈹� Evaluate
+ 鈹斺攢鈹� Repeat
+```
+
+---
+
+# 6. Step 鏄湡姝g殑鎵ц鍗曞厓
+
+Task 璐熻矗锛�
+
+```text
+鍋氫粈涔�
+```
+
+Step 璐熻矗锛�
+
+```text
+鍏蜂綋鎬庝箞涓�姝ヤ竴姝ュ仛
+```
+
+渚嬪锛�
+
+```text
+Task锛�
+鑻辫璺熻
+
+Step 1锛�
+鎾斁鏍囧噯闊抽
+
+Step 2锛�
+绛夊緟鐢ㄦ埛璇磋瘽
+
+Step 3锛�
+褰曢煶
+
+Step 4锛�
+ASR
+
+Step 5锛�
+姣旇緝
+
+Step 6锛�
+璇勫垎
+
+Step 7锛�
+鍙嶉
+
+Step 8锛�
+杩涘叆涓嬩竴鍙�
+```
+
+---
+
+# 7. Skill / Atomic Capability
+
+Step 涓嶇洿鎺ユ搷浣滅‖浠躲��
+
+Step 璋冪敤锛�
+
+```text
+Skill
+```
+
+渚嬪锛�
+
+```text
+play_audio
+record_audio
+listen
+speak
+wait
+count
+asr
+evaluate
+display
+led
+button
+```
+
+鏈�缁堬細
+
+```text
+Task
+ 鈫�
+Step
+ 鈫�
+Skill
+ 鈫�
+Device Capability
+```
+
+---
+
+# 8. Device 涓� Server 鐨勮亴璐h竟鐣�
+
+## Device
+
+璐熻矗锛�
+
+```text
+鎵ц
+鎾斁
+褰曢煶
+閲囬泦
+缂撳瓨
+閫氫俊
+鍩虹鐘舵��
+```
+
+---
+
+## Server
+
+璐熻矗锛�
+
+```text
+璋冨害
+Task
+App
+鐢ㄦ埛
+鏁版嵁
+ASR
+AI
+璇勪及
+鍒嗘瀽
+闀挎湡瓒嬪娍
+```
+
+---
+
+# 9. App 涓嶅簲璇ユ垚涓虹‖浠跺浐浠�
+
+鏈潵鏂板 App锛�
+
+```text
+涓嬭浇閰嶇疆
+涓嬭浇璧勬簮
+```
+
+鑰屼笉鏄細
+
+```text
+淇敼 C++ 鍥轰欢
+閲嶆柊 OTA
+```
+
+鐞嗘兂鎯呭喌涓嬶細
+
+```text
+鏂板 App
+ 鈫�
+App Manifest
+Task Definition
+Step Definition
+Resource
+Assessment Definition
+ 鈫�
+Server
+ 鈫�
+Device Sync
+```
+
+鍗冲彲杩愯銆�
+
+---
+
+# 10. App Manifest
+
+姣忎竴涓� App 閮芥湁锛�
+
+```text
+app.json
+```
+
+瀹氫箟锛�
+
+```text
+appId
+name
+version
+developerId
+description
+icon
+tasks
+resources
+permissions
+assessment
+```
+
+---
+
+# 11. Task Definition
+
+Task 瀹氫箟锛�
+
+```text
+taskId
+name
+description
+version
+steps
+parameters
+interactionMode
+assessment
+resources
+```
+
+渚嬪锛�
+
+```json
+{
+ "taskId": "english_repeat_10",
+ "interactionMode": "PLAY_AUDIO",
+ "parameters": {
+ "repeatCount": 10
+ }
+}
+```
+
+---
+
+# 12. Resource
+
+App 鍙互鎼哄甫锛�
+
+```text
+MP3
+WAV
+鍥剧墖
+JSON
+鏂囨湰
+閰嶇疆
+妯″瀷鍙傛暟
+```
+
+杩欎簺灞炰簬锛�
+
+```text
+Application Resource
+```
+
+鑰屼笉鏄浐浠躲��
+
+---
+
+# 13. OTA 涓� App 鏇存柊蹇呴』褰诲簳鍖哄垎
+
+## OTA
+
+鐢ㄤ簬锛�
+
+```text
+Device Firmware
+```
+
+渚嬪锛�
+
+```text
+搴曞眰椹卞姩
+閫氫俊鍗忚
+闊抽椹卞姩
+绯荤粺 Runtime
+```
+
+---
+
+## App Sync
+
+鐢ㄤ簬锛�
+
+```text
+App
+Task
+Config
+Resource
+Audio
+```
+
+渚嬪锛�
+
+```text
+鏂板鑻辫 App
+```
+
+鍙渶瑕侊細
+
+```text
+Sync App
+```
+
+鏃犻渶 OTA銆�
+
+---
+
+# 14. SD Card 鐨勫畾浣�
+
+16GB SD Card 涓昏鐢ㄤ簬锛�
+
+```text
+App Resources
+Audio Resources
+Cache
+Offline Tasks
+Pending Upload Audio
+Temporary Files
+```
+
+渚嬪锛�
+
+```text
+/sdcard/
+ apps/
+ resources/
+ audio/
+ cache/
+ recordings/
+ pending/
+```
+
+---
+
+# 15. Server Task Runtime
+
+Server 蹇呴』寤虹珛缁熶竴锛�
+
+```text
+Task Runtime
+```
+
+璐熻矗锛�
+
+```text
+鍒涘缓 TaskInstance
+鍚姩 Task
+鎺ㄨ繘 Step
+绛夊緟 Device
+澶勭悊 Event
+鍒ゆ柇 Step 瀹屾垚
+杩涘叆涓嬩竴 Step
+缁撴潫 Task
+```
+
+---
+
+# 16. Task Instance
+
+涓嶈鐩存帴淇敼 Task Definition銆�
+
+姣忎竴娆″疄闄呮墽琛岋細
+
+```text
+Task
+ 鈫�
+TaskInstance
+```
+
+渚嬪锛�
+
+```text
+Task锛�
+鑻辫璺熻 10 娆�
+
+TaskInstance锛�
+2026-08-30 19:30
+```
+
+淇濆瓨锛�
+
+```text
+startTime
+endTime
+status
+currentStep
+result
+```
+
+---
+
+# 17. Scheduler
+
+瀹堕暱閫氳繃灏忕▼搴忛厤缃細
+
+```text
+姣忓ぉ 19:30
+鑻辫瀛︿範
+```
+
+Server锛�
+
+```text
+Schedule
+ 鈫�
+Scheduler
+ 鈫�
+TaskInstance
+ 鈫�
+Device
+```
+
+鍥犳锛�
+
+> 瀹堕暱閰嶇疆鐨勬槸鈥滀粈涔堟椂鍊欏仛浠�涔� Task鈥濓紝涓嶆槸鐩存帴鎺у埗纭欢銆�
+
+---
+
+# 18. Plan
+
+杩涗竴姝ュ厑璁革細
+
+```text
+Plan
+```
+
+渚嬪锛�
+
+```text
+姣忔棩瀛︿範璁″垝
+```
+
+鍖呭惈锛�
+
+```text
+19:30 鑻辫璺熻
+19:45 鍙g畻
+19:55 璁板繂娓告垙
+```
+
+鏈�缁堬細
+
+```text
+Plan
+ 鈫�
+Schedule
+ 鈫�
+TaskInstance
+```
+
+---
+
+# 19. Device Runtime
+
+Device 鍙渶瑕佺悊瑙f爣鍑嗗崗璁細
+
+```text
+APP_SYNC
+TASK_START
+STEP_START
+SKILL_EXECUTE
+TASK_PAUSE
+TASK_RESUME
+TASK_STOP
+```
+
+浠ュ強鍚� Server 涓婃姤锛�
+
+```text
+TASK_STARTED
+STEP_STARTED
+STEP_COMPLETED
+AUDIO_PLAYED
+SPEECH_STARTED
+SPEECH_STOPPED
+TASK_COMPLETED
+```
+
+---
+
+# 20. Interaction Runtime
+
+Device Runtime 鍐呴儴澧炲姞锛�
+
+```text
+Interaction Runtime
+```
+
+鏍规嵁锛�
+
+```text
+interactionMode
+```
+
+閫夋嫨浜や簰鏂瑰紡銆�
+
+渚嬪锛�
+
+```text
+PLAY_AUDIO
+ 鈫� Audio Player
+
+FOLLOW_READ
+ 鈫� Audio Player
+ 鈫� Recorder
+ 鈫� VAD
+
+CHAT
+ 鈫� ASR
+ 鈫� LLM
+ 鈫� TTS
+
+TRANSLATION
+ 鈫� ASR
+ 鈫� Translation
+ 鈫� TTS
+```
+
+---
+
+# 21. Chat 涓嶅啀鏄敮涓� Runtime
+
+鐜版湁 Xiaozhi 寰堝彲鑳斤細
+
+```text
+Audio
+ 鈫�
+ASR
+ 鈫�
+LLM
+ 鈫�
+TTS
+```
+
+杩欑鏋舵瀯銆�
+
+蹇呴』鍗囩骇鎴愶細
+
+```text
+Task Runtime
+ 鈫�
+Interaction Runtime
+ 鈫�
+鈹屸攢鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹攢鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹攢鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹攢鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�
+Chat Audio Repeat Game ...
+```
+
+Chat 鍙槸鍏朵腑涓�绉嶈兘鍔涖��
+
+---
+
+# 22. Server Audio Architecture
+
+Server 淇濆瓨锛�
+
+```text
+Original Audio
+```
+
+鑰屼笉鏄彧淇濆瓨锛�
+
+```text
+ASR Text
+```
+
+缁撴瀯锛�
+
+```text
+AudioSession
+ 鈹斺攢鈹� AudioSegment
+ 鈹溾攢鈹� Original Audio
+ 鈹溾攢鈹� ASR
+ 鈹溾攢鈹� Voice Feature
+ 鈹斺攢鈹� Assessment
+```
+
+---
+
+# 23. ASR
+
+缁熶竴 ASR Service锛�
+
+```text
+Audio
+ 鈫�
+ASR
+ 鈫�
+Transcript
+```
+
+淇濆瓨锛�
+
+```text
+text
+confidence
+word timing
+word confidence
+```
+
+---
+
+# 24. Voice Analysis
+
+缁熶竴鎻愬彇锛�
+
+```text
+speechDuration
+pauseCount
+pauseDuration
+speechRate
+responseLatency
+volume
+pitch
+energy
+prosody
+```
+
+---
+
+# 25. Task Assessment
+
+涓嶅悓 Task 浣跨敤涓嶅悓璇勪及绛栫暐銆�
+
+渚嬪锛�
+
+```text
+鑻辫璺熻
+ 鈫� pronunciation
+ 鈫� fluency
+ 鈫� accuracy
+
+鏁板瓧鍊掕儗
+ 鈫� sequenceAccuracy
+ 鈫� memoryAccuracy
+ 鈫� responseTime
+
+鍙g畻
+ 鈫� accuracy
+ 鈫� responseTime
+ 鈫� difficulty
+
+鎴愯鎺ラ緳
+ 鈫� correctness
+ 鈫� responseTime
+ 鈫� continuity
+```
+
+---
+
+# 26. Metric System
+
+鎵�鏈夌粨鏋滄渶缁堢粺涓�涓猴細
+
+```text
+Metric
+```
+
+渚嬪锛�
+
+```text
+accuracy
+response_time
+pause_count
+speech_rate
+pronunciation
+fluency
+completion
+participation
+engagement
+```
+
+鏂� App 鍙互娉ㄥ唽鏂扮殑 Metric銆�
+
+---
+
+# 27. Behavioral Analysis
+
+Server 寤虹珛锛�
+
+```text
+User Baseline
+```
+
+閫氳繃锛�
+
+```text
+7 Days
+30 Days
+90 Days
+```
+
+褰㈡垚鍔ㄦ�佸熀绾裤��
+
+鐒跺悗姣旇緝锛�
+
+```text
+Today
+vs
+Baseline
+```
+
+---
+
+# 28. Emotion Analysis
+
+绯荤粺鍙互鎻愪緵锛�
+
+```text
+Emotion Signal
+```
+
+浣嗗繀椤诲畾涔変负锛�
+
+> **鍩轰簬璇煶銆佽瑷�鍜岃涓烘暟鎹殑鎯呯华琛ㄧ幇淇″彿銆�**
+
+涓嶈兘鐩存帴瀹氫箟涓猴細
+
+```text
+蹇冪悊鐤剧梾璇婃柇
+```
+
+杈撳嚭锛�
+
+```text
+state
+confidence
+trend
+```
+
+---
+
+# 29. Engagement
+
+寤虹珛锛�
+
+```text
+Engagement Score
+```
+
+缁煎悎锛�
+
+```text
+鍝嶅簲
+瀹屾垚
+涓诲姩鎬�
+鎸佺画鎬�
+浜掑姩
+```
+
+褰㈡垚锛�
+
+```text
+0 ~ 100
+```
+
+---
+
+# 30. Parent Dashboard
+
+瀹堕暱鏈�缁堢湅鍒帮細
+
+```text
+浠婃棩浠诲姟
+瀹屾垚鎯呭喌
+Task Score
+鑳藉姏鎸囨爣
+鍙備笌搴�
+瓒嬪娍
+```
+
+鑰屼笉鏄鏉傛妧鏈寚鏍囥��
+
+---
+
+# 31. Long-term Profile
+
+绯荤粺鎸佺画褰㈡垚锛�
+
+```text
+User Profile
+```
+
+鍖呮嫭锛�
+
+```text
+Learning
+Memory
+Speech
+Task Performance
+Participation
+Behavior Trend
+Preference
+```
+
+鎵�鏈� Profile 閮藉簲璇ユ潵鑷細
+
+```text
+Event
+ 鈫�
+Metric
+ 鈫�
+Aggregation
+```
+
+鑰屼笉鏄墜宸ョ淮鎶ゃ��
+
+---
+
+# 32. Developer Platform
+
+鏈�缁堝紑鏀撅細
+
+```text
+Developer
+ 鈫�
+App
+ 鈫�
+Task
+ 鈫�
+Step
+ 鈫�
+Skill
+ 鈫�
+Assessment
+```
+
+寮�鍙戣�呬笉搴旇鐩存帴淇敼 Device Firmware銆�
+
+---
+
+# 33. Developer Sandbox
+
+鎻愪緵锛�
+
+```text
+App SDK
+Task SDK
+Assessment SDK
+Resource SDK
+API
+Simulator
+```
+
+寮�鍙戣�呭彲浠ュ湪 PC 涓婃ā鎷燂細
+
+```text
+Xiaozhi Device
+```
+
+娴嬭瘯 Task銆�
+
+---
+
+# 34. App Store
+
+鏈�缁堝钩鍙板舰鎴愶細
+
+```text
+App Store
+```
+
+鐢ㄦ埛鍙互锛�
+
+```text
+娴忚 App
+鏌ョ湅浠嬬粛
+璇曠敤
+璁㈤槄
+瀹夎
+璇勪环
+```
+
+渚嬪锛�
+
+```text
+鑻辫瀛︿範
+鏁板璁粌
+璁板繂璁粌
+鎴愯娓告垙
+AI 涓绘寔浜�
+鍥炲繂褰�
+鏁呬簨
+```
+
+---
+
+# 35. Subscription
+
+璁㈤槄鍏崇郴锛�
+
+```text
+User
+ 鈫�
+Subscription
+ 鈫�
+App
+```
+
+鑰屼笉鏄細
+
+```text
+User
+ 鈫�
+Firmware
+```
+
+---
+
+# 36. Developer Revenue
+
+骞冲彴璐熻矗锛�
+
+```text
+璁㈠崟
+鏀粯
+璁㈤槄
+閫�娆�
+缁撶畻
+鍒嗘垚
+```
+
+寮�鍙戣�呰幏寰楋細
+
+```text
+Revenue Share
+```
+
+---
+
+# 37. Permission Architecture
+
+蹇呴』寤虹珛锛�
+
+```text
+User
+Child
+Parent
+Developer
+App
+```
+
+涔嬮棿鐨勬潈闄愪綋绯汇��
+
+灏ゅ叾鏄効绔ユ暟鎹��
+
+寮�鍙戣�咃細
+
+```text
+榛樿涓嶈兘璇诲彇鍏朵粬 App 鏁版嵁
+```
+
+瀹堕暱锛�
+
+```text
+鍙互鏌ョ湅瀛╁瓙鏁版嵁
+```
+
+骞冲彴锛�
+
+```text
+璐熻矗鏉冮檺鎺у埗
+```
+
+---
+
+# 38. Privacy
+
+鍎跨璇煶蹇呴』閲嶇偣淇濇姢銆�
+
+寤虹珛锛�
+
+```text
+Consent
+Encryption
+Access Control
+Audit Log
+Retention Policy
+Data Deletion
+```
+
+---
+
+# 39. 鏁版嵁鏍稿績閾捐矾
+
+鏈�缁堢粺涓�鎴愶細
+
+```text
+App
+ 鈫�
+Task
+ 鈫�
+TaskInstance
+ 鈫�
+StepInstance
+ 鈫�
+Device Event
+ 鈫�
+Audio
+ 鈫�
+ASR
+ 鈫�
+Voice Feature
+ 鈫�
+Assessment
+ 鈫�
+Metric
+ 鈫�
+Aggregation
+ 鈫�
+Profile
+ 鈫�
+Report
+```
+
+杩欐槸鏁翠釜绯荤粺鏈�閲嶈鐨勬暟鎹摼璺��
+
+---
+
+# 40. Event-Driven Architecture
+
+鏍稿績鐘舵�佸敖閲忕敱 Event 椹卞姩銆�
+
+渚嬪锛�
+
+```text
+TaskStarted
+StepStarted
+AudioPlayed
+SpeechStarted
+SpeechStopped
+ASRCompleted
+AssessmentCompleted
+StepCompleted
+TaskCompleted
+```
+
+Event 鎴愪负绯荤粺鐨勯噸瑕佷簨瀹炴潵婧愩��
+
+---
+
+# 41. 寮傛鍒嗘瀽
+
+瀹炴椂閾捐矾锛�
+
+```text
+Device
+ 鈫�
+Server
+ 鈫�
+Task Runtime
+ 鈫�
+瀹炴椂鍙嶉
+```
+
+寮傛閾捐矾锛�
+
+```text
+Audio
+ 鈫�
+Queue
+ 鈫�
+ASR
+ 鈫�
+Voice Analysis
+ 鈫�
+Assessment
+ 鈫�
+Aggregation
+ 鈫�
+Report
+```
+
+澶嶆潅 AI 鍒嗘瀽涓嶈兘闃诲 Task Runtime銆�
+
+---
+
+# 42. 绂荤嚎鑳藉姏
+
+Device 缃戠粶鏂紑鏃讹細
+
+```text
+Task
+ 鈫�
+Local Cache
+ 鈫�
+鎵ц
+ 鈫�
+Recording
+ 鈫�
+SD Card
+ 鈫�
+缃戠粶鎭㈠
+ 鈫�
+Sync
+```
+
+鍥犳锛�
+
+> Server 鏄腑蹇冩帶鍒跺櫒锛屼絾 Device 蹇呴』鍏峰涓�瀹氱▼搴︾殑 Offline Runtime銆�
+
+---
+
+# 43. 鐗堟湰绠$悊
+
+鎵�鏈夎繖浜涘璞″繀椤荤増鏈寲锛�
+
+```text
+Firmware
+App
+Task
+Step
+Assessment
+Model
+Resource
+```
+
+渚嬪锛�
+
+```text
+App v1.2
+Task v3
+Assessment v2
+Model v5
+```
+
+鍘嗗彶鏁版嵁蹇呴』鑳藉杩芥函鍒板叿浣撶増鏈��
+
+---
+
+# 44. 涓嶈鐩栧巻鍙茬粨鏋�
+
+渚嬪锛�
+
+```text
+Pronunciation Model v1
+Score = 82
+```
+
+浠ュ悗锛�
+
+```text
+Pronunciation Model v2
+Score = 87
+```
+
+涓や釜缁撴灉閮戒繚鐣欍��
+
+---
+
+# 45. 鎺ㄨ崘鐨勬暟鎹簱鍏崇郴
+
+鏍稿績锛�
+
+```text
+User
+ 鈹斺攢鈹� Child
+
+Child
+ 鈹溾攢鈹� Subscription
+ 鈹溾攢鈹� Plan
+ 鈹溾攢鈹� Schedule
+ 鈹斺攢鈹� TaskInstance
+
+App
+ 鈹斺攢鈹� Task
+ 鈹斺攢鈹� TaskStep
+
+TaskInstance
+ 鈹斺攢鈹� TaskStepInstance
+ 鈹溾攢鈹� DeviceEvent
+ 鈹溾攢鈹� AudioSegment
+ 鈹� 鈹溾攢鈹� ASR
+ 鈹� 鈹斺攢鈹� VoiceFeature
+ 鈹斺攢鈹� Assessment
+ 鈹斺攢鈹� Metric
+```
+
+---
+
+# 46. 绗竴闃舵寮�鍙戠洰鏍�
+
+涓嶈涓�寮�濮嬪紑鍙戝畬鏁村钩鍙般��
+
+棣栧厛寤虹珛鏈�灏忛棴鐜細
+
+```text
+瀹堕暱
+ 鈫�
+閰嶇疆 Task
+ 鈫�
+Server Scheduler
+ 鈫�
+TaskInstance
+ 鈫�
+Xiaozhi
+ 鈫�
+鎵ц
+ 鈫�
+褰曢煶
+ 鈫�
+Server
+ 鈫�
+ASR
+ 鈫�
+Assessment
+ 鈫�
+瀹堕暱鏌ョ湅缁撴灉
+```
+
+蹇呴』瀹屾暣璺戦�氥��
+
+---
+
+# 47. MVP Task
+
+绗竴闃舵鍙疄鐜颁笁涓� Task锛�
+
+## Task A锛氳嫳璇璇�
+
+```text
+鎾斁
+ 鈫�
+绛夊緟
+ 鈫�
+鎾斁
+ 鈫�
+...
+```
+
+---
+
+## Task B锛氳嫳璇窡璇�
+
+```text
+鎾斁鏍囧噯闊抽
+ 鈫�
+褰曢煶
+ 鈫�
+ASR
+ 鈫�
+鍑嗙‘鐜�
+```
+
+---
+
+## Task C锛氬彛绠�
+
+```text
+鎾斁闂
+ 鈫�
+绛夊緟鍥炵瓟
+ 鈫�
+ASR
+ 鈫�
+绛旀瑙f瀽
+ 鈫�
+姝g‘鐜�
+```
+
+---
+
+# 48. 绗簩闃舵
+
+澧炲姞锛�
+
+```text
+鏁板瓧鍊掕儗
+鎴愯鎺ラ緳
+绠�鍗曡蹇嗘父鎴�
+```
+
+骞剁粺涓�锛�
+
+```text
+Task Engine
+Step Engine
+Assessment Engine
+```
+
+---
+
+# 49. 绗笁闃舵
+
+鍔犲叆锛�
+
+```text
+Voice Feature
+```
+
+瀹炵幇锛�
+
+```text
+Pause
+Speech Rate
+Response Latency
+Volume
+Pitch
+```
+
+---
+
+# 50. 绗洓闃舵
+
+鍔犲叆锛�
+
+```text
+Baseline
+Behavior
+Engagement
+Participation
+```
+
+瀹炵幇锛�
+
+```text
+Today
+7 Days
+30 Days
+```
+
+瓒嬪娍銆�
+
+---
+
+# 51. 绗簲闃舵
+
+寤虹珛锛�
+
+```text
+Emotion Signal
+```
+
+杩涜瀹為獙鎬у垎鏋愩��
+
+瑕佹眰锛�
+
+```text
+妯″瀷鐗堟湰鍖�
+缃俊搴�
+瓒嬪娍
+浜哄伐楠岃瘉
+```
+
+涓嶅厑璁哥洿鎺ュ仛鍖荤枟璇婃柇銆�
+
+---
+
+# 52. 绗叚闃舵
+
+寮�鏀撅細
+
+```text
+App Manifest
+Task DSL
+Assessment SDK
+Developer API
+```
+
+瀹炵幇绗笁鏂� App銆�
+
+---
+
+# 53. 绗竷闃舵
+
+寤虹珛锛�
+
+```text
+App Store
+Subscription
+Payment
+Revenue Share
+Developer Console
+```
+
+---
+
+# 54. 绗叓闃舵
+
+寤虹珛锛�
+
+```text
+Parent AI Assistant
+```
+
+瀹堕暱鍙互璇㈤棶锛�
+
+```text
+瀛╁瓙鏈�杩戝涔犳�庝箞鏍凤紵
+
+鍝釜 App 鏈�鍠滄锛�
+
+鑻辫鏈�杩戞湁娌℃湁杩涙锛�
+
+鏈�杩戝弬涓庡害鏈夋病鏈変笅闄嶏紵
+
+杩囧幓涓�涓湀鏈変粈涔堟槑鏄惧彉鍖栵紵
+```
+
+AI 浠庯細
+
+```text
+Metrics
+Assessment
+Trend
+Profile
+```
+
+涓敓鎴愯В閲娿��
+
+---
+
+# 55. AI Coding 鎬讳綋瀹炴柦椤哄簭
+
+AI Coding 涓嶅厑璁镐竴娆℃�ч噸鍐欐暣涓」鐩��
+
+蹇呴』锛�
+
+```text
+Phase 0
+浠g爜瀹¤
+
+Phase 1
+寤虹珛鏍稿績 Domain Model
+
+Phase 2
+Task Runtime
+
+Phase 3
+Device Runtime
+
+Phase 4
+App / Resource Sync
+
+Phase 5
+Audio / ASR
+
+Phase 6
+Assessment
+
+Phase 7
+Parent / Schedule
+
+Phase 8
+Behavior / Metrics
+
+Phase 9
+Developer Platform
+
+Phase 10
+Subscription / Marketplace
+```
+
+姣忎釜 Phase锛�
+
+```text
+鍒嗘瀽
+ 鈫�
+璁捐
+ 鈫�
+瀹炵幇
+ 鈫�
+娴嬭瘯
+ 鈫�
+鍥炲綊
+ 鈫�
+鎻愪氦
+```
+
+---
+
+# 56. AI Coding 绗竴鍘熷垯
+
+AI 缂栫▼ Agent 蹇呴』锛�
+
+> **浼樺厛鐞嗚В鐜版湁浠g爜锛屽啀淇敼浠g爜銆�**
+
+绂佹锛�
+
+```text
+娌℃湁闃呰浠g爜
+ 鈫�
+鐩存帴閲嶆瀯
+```
+
+蹇呴』锛�
+
+```text
+Repository Audit
+ 鈫�
+Architecture Mapping
+ 鈫�
+Dependency Analysis
+ 鈫�
+Minimal Change
+```
+
+---
+
+# 57. 涓嶅厑璁告棤鐞嗙敱閲嶅啓鐜版湁鍔熻兘
+
+鐜版湁 Xiaozhi 鐨勶細
+
+```text
+Audio
+ASR
+TTS
+LLM
+MQTT / WebSocket
+Device Communication
+```
+
+鑳藉澶嶇敤鐨勫繀椤诲鐢ㄣ��
+
+鏂版灦鏋勫簲璇ワ細
+
+```text
+鍖呰9鐜版湁鑳藉姏
+```
+
+鑰屼笉鏄細
+
+```text
+鍏ㄩ儴鎺ㄥ�掗噸鏉�
+```
+
+---
+
+# 58. 鏂版棫绯荤粺杩佺Щ绛栫暐
+
+鏃х郴缁燂細
+
+```text
+Conversation
+```
+
+淇濈暀銆�
+
+澧炲姞锛�
+
+```text
+Task Runtime
+```
+
+鏈�缁堬細
+
+```text
+Task Runtime
+ 鈹溾攢鈹� Chat Task
+ 鈹溾攢鈹� Repeat Task
+ 鈹溾攢鈹� Follow Read Task
+ 鈹溾攢鈹� Translation Task
+ 鈹斺攢鈹� Game Task
+```
+
+鍥犳锛�
+
+> Chat 鏈�缁堝彧鏄� Task Runtime 鐨勪竴绉� Task锛岃�屼笉鏄暣涓郴缁熺殑鏍稿績銆�
+
+---
+
+# 59. API 璁捐鍘熷垯
+
+鎵�鏈� API 蹇呴』鍥寸粫锛�
+
+```text
+App
+Task
+TaskInstance
+StepInstance
+Event
+Audio
+Assessment
+Metric
+```
+
+璁捐銆�
+
+閬垮厤澶ч噺锛�
+
+```text
+/english/xxx
+/math/xxx
+/game/xxx
+```
+
+杩欑 App-specific API銆�
+
+---
+
+# 60. Task DSL 鍘熷垯
+
+鏈潵寮�鍙戣�呭簲璇ヨ兘澶熼�氳繃锛�
+
+```text
+JSON / YAML / SDK
+```
+
+瀹氫箟 Task銆�
+
+渚嬪锛�
+
+```text
+Task
+ 鈹溾攢鈹� parameters
+ 鈹溾攢鈹� steps
+ 鈹� 鈹溾攢鈹� action
+ 鈹� 鈹溾攢鈹� condition
+ 鈹� 鈹溾攢鈹� wait
+ 鈹� 鈹溾攢鈹� loop
+ 鈹� 鈹斺攢鈹� branch
+ 鈹斺攢鈹� assessment
+```
+
+杩欐牱锛�
+
+> 鏂板 Task 灏介噺涓嶉渶瑕佷慨鏀规牳蹇� Runtime銆�
+
+---
+
+# 61. Runtime 鐨勭ǔ瀹氳竟鐣�
+
+蹇呴』寤虹珛绋冲畾鐨勶細
+
+```text
+Task Runtime API
+Skill API
+Device API
+Assessment API
+```
+
+鏈潵 App 瓒婂锛�
+
+```text
+Runtime 涓嶅彉
+App 澧炲姞
+Task 澧炲姞
+```
+
+杩欐槸寮�鏀惧钩鍙版垚绔嬬殑鍏抽敭銆�
+
+---
+
+# 62. 鏈�缁堟灦鏋勫浘
+
+```text
+ Xiaozhi Platform
+ 鈹�
+ 鈹屸攢鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹尖攢鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�
+ 鈹� 鈹� 鈹�
+ Parent Developer Device
+ 鈹� 鈹� 鈹�
+ 鈫� 鈫� 鈫�
+ Plan App Store Device Runtime
+ 鈹� 鈹� 鈹�
+ Schedule App Interaction
+ 鈹� 鈹� 鈹�
+ 鈹斺攢鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈫� Task 鈫愨攢鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�
+ 鈹�
+ Step
+ 鈹�
+ Skill
+ 鈹�
+ Device Capability
+ 鈹�
+ 鈫�
+ Audio / Event
+ 鈹�
+ 鈫�
+ Server Pipeline
+ 鈹�
+ 鈹屸攢鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹尖攢鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�
+ 鈫� 鈫� 鈫�
+ ASR Voice Analysis Assessment
+ 鈹� 鈹� 鈹�
+ 鈹斺攢鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹尖攢鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�鈹�
+ 鈫�
+ Metric
+ 鈫�
+ Aggregation
+ 鈫�
+ User Profile
+ 鈫�
+ Parent Report
+```
+
+---
+
+# 63. 鏈�缁堝钩鍙扮殑鏍稿績鎶借薄
+
+鏁翠釜椤圭洰鏈�缁堝彧闇�瑕佺墷鐗㈡姄浣忎互涓� 8 涓牳蹇冩蹇碉細
+
+```text
+App
+Task
+Step
+Skill
+Interaction Mode
+Event
+Assessment
+Metric
+```
+
+瀹冧滑涔嬮棿锛�
+
+```text
+App
+ 鈫�
+Task
+ 鈫�
+Step
+ 鈫�
+Skill
+ 鈫�
+Device
+```
+
+鎵ц杩囩▼涓骇鐢燂細
+
+```text
+Event
+ 鈫�
+Audio
+ 鈫�
+Assessment
+ 鈫�
+Metric
+```
+
+---
+
+# 64. 鏈�缁堜骇鍝侀�昏緫
+
+鍥犳鏈�缁� Xiaozhi 鐨勯�昏緫涓嶆槸锛�
+
+```text
+鐢ㄦ埛
+ 鈫�
+璇磋瘽
+ 鈫�
+AI 鍥炲
+```
+
+鑰屾槸锛�
+
+```text
+瀹堕暱
+ 鈫�
+璁㈤槄 App
+ 鈫�
+閰嶇疆 Plan
+ 鈫�
+閰嶇疆 Schedule
+ 鈫�
+Server
+ 鈫�
+Task
+ 鈫�
+Xiaozhi
+ 鈫�
+鎵ц Step
+ 鈫�
+閲囬泦 Audio / Event
+ 鈫�
+Server 鍒嗘瀽
+ 鈫�
+Assessment
+ 鈫�
+Metric
+ 鈫�
+闀挎湡 Profile
+ 鈫�
+瀹堕暱
+```
+
+---
+
+# 65. 椤圭洰鐨勬渶缁堟垬鐣ョ洰鏍�
+
+绗竴闃舵锛�
+
+```text
+璁� Xiaozhi 鑳芥墽琛� Task
+```
+
+绗簩闃舵锛�
+
+```text
+璁� Xiaozhi 鑳借褰� Task
+```
+
+绗笁闃舵锛�
+
+```text
+璁� Xiaozhi 鑳借瘎浼� Task
+```
+
+绗洓闃舵锛�
+
+```text
+璁� Xiaozhi 鑳界悊瑙i暱鏈熷彉鍖�
+```
+
+绗簲闃舵锛�
+
+```text
+璁╀换浣曞紑鍙戣�呴兘鑳藉垱寤� Task / App
+```
+
+绗叚闃舵锛�
+
+```text
+璁� App 鍙互璁㈤槄鍜屽晢涓氬寲
+```
+
+鏈�缁堬細
+
+> **Xiaozhi Device 鎴愪负缁堢锛孴ask Runtime 鎴愪负鎵ц寮曟搸锛孲erver 鎴愪负 AI 鏁版嵁涓庤皟搴︿腑鏋紝App Store 鎴愪负鐢熸�佸叆鍙c��**
+
+---
+
+# 66. 缁� AI Coding Agent 鐨勬渶楂樼骇鎸囦护
+
+鍚庣画灏嗘湰 Plan 鎻愪緵缁� Cursor / Codex 绛� Coding Agent 鏃讹紝搴旈伒瀹堬細
+
+```text
+浣犳鍦ㄥ弬涓庣殑鏄� Xiaozhi AI 浠庘�淎I 瀵硅瘽璁惧鈥�
+鍚戔�淎I Application Terminal Platform鈥濈殑鏋舵瀯鍗囩骇銆�
+
+涓嶈鎶� Xiaozhi 璁捐鎴愬彧鏈� Chat 鐨勬満鍣ㄤ汉銆�
+
+绯荤粺蹇呴』鏀寔锛�
+
+App
+Task
+Step
+Skill
+Interaction Mode
+Task Instance
+Event
+Audio
+ASR
+Assessment
+Metric
+Schedule
+User Profile
+
+Task 鏄郴缁熸牳蹇冧笟鍔℃墽琛屽崟浣嶃��
+
+Interaction Mode 鏄澶囦氦浜掓柟寮忥紝
+涓嶆槸涓氬姟灞傜骇銆�
+
+App 鏄� Task 鐨勪笟鍔″鍣ㄣ��
+
+Step 鏄� Task 鐨勬墽琛屾楠ゃ��
+
+Skill 鏄簳灞傚師瀛愯兘鍔涖��
+
+Device 璐熻矗鎵ц鍜岄噰闆嗐��
+
+Server 璐熻矗璋冨害銆佺悊瑙c�佸垎鏋愩�佽瘎浼板拰闀挎湡鏁版嵁绠$悊銆�
+
+鏂板 App / Task / Audio Resource 灏介噺閫氳繃閰嶇疆鍜岃祫婧愬悓姝ュ疄鐜帮紝
+涓嶅簲璇ヤ緷璧� Firmware OTA銆�
+
+蹇呴』淇濇寔鏃� Chat 鑳藉姏缁х画宸ヤ綔銆�
+
+鎵�鏈夋柊鏋舵瀯蹇呴』灏藉彲鑳藉吋瀹瑰拰澶嶇敤鐜版湁浠g爜銆�
+
+绂佹鍦ㄦ湭瀹屾垚浠g爜瀹¤涔嬪墠澶ц妯¢噸鏋勩��
+
+姣忎釜 Phase 蹇呴』锛�
+
+1. 鍒嗘瀽鐜版湁浠g爜
+2. 鏄庣‘淇敼鑼冨洿
+3. 缁欏嚭鏋舵瀯鏂规
+4. 瀹炴柦鏈�灏忓繀瑕佷慨鏀�
+5. 缂栧啓娴嬭瘯
+6. 鎵ц娴嬭瘯
+7. 楠岃瘉鏃у姛鑳芥病鏈夊洖褰�
+8. 杈撳嚭淇敼鎬荤粨
+
+涓嶈涓�娆℃�у疄鐜版暣涓郴缁熴��
+
+蹇呴』鎸夌収 Phase 閫愭婕旇繘銆�
+
+浼樺厛寤虹珛绋冲畾鐨� Domain Model 鍜� Runtime Boundary銆�
+
+鏈潵绗笁鏂瑰紑鍙戣�呭簲璇ヨ兘澶熷湪涓嶄慨鏀� Xiaozhi 鏍稿績 Runtime 鐨勬儏鍐典笅鍒涘缓鏂扮殑 App 鍜� Task銆�
+
+鏈�缁堢洰鏍囨槸锛�
+
+鈥滄牳蹇� Runtime 绋冲畾锛孉pp / Task / Resource / Assessment 鍙棤闄愭墿灞曘�傗��
+```
+
+---
+
+# 67. Master Plan 鐨勬墽琛屾柟寮�
+
+浠ュ悗鎵�鏈� Coding Plan 閮藉簲璇ヤ粠鏈� Master Plan 娲剧敓銆�
+
+缁撴瀯锛�
+
+```text
+Master Plan
+ 鈹�
+ 鈹溾攢鈹� Device Plan
+ 鈹�
+ 鈹溾攢鈹� Server Plan
+ 鈹�
+ 鈹溾攢鈹� Mini Program Plan
+ 鈹�
+ 鈹溾攢鈹� Developer Platform Plan
+ 鈹�
+ 鈹溾攢鈹� Assessment Plan
+ 鈹�
+ 鈹斺攢鈹� Marketplace Plan
+```
+
+鐒跺悗鍐嶈繘涓�姝ワ細
+
+```text
+Architecture Plan
+ 鈫�
+Module Plan
+ 鈫�
+Implementation Plan
+ 鈫�
+Coding Task
+ 鈫�
+Code
+ 鈫�
+Test
+```
+
+---
+
+# 68. 鏈�缁堝垽鏂爣鍑�
+
+浠ュ悗浠讳綍涓�涓柊闇�姹傦紝閮藉厛闂細
+
+```text
+杩欐槸 App锛�
+鏄� Task锛�
+鏄� Step锛�
+鏄� Skill锛�
+鏄� Interaction Mode锛�
+鏄� Event锛�
+鏄� Assessment锛�
+杩樻槸 Metric锛�
+```
+
+濡傛灉涓�涓柊闇�姹傛棤娉曞綊鍏ヨ繖浜涙牳蹇冩娊璞★細
+
+> 涓嶈绔嬪嵆鍐欎唬鐮併��
+
+鍏堥噸鏂板瑙嗘灦鏋勩��
+
+杩欏鍘熷垯灏嗗喅瀹� Xiaozhi 鏈�缁堣兘涓嶈兘浠庝竴涓骇鍝侊紝鐪熸婕斿寲鎴愪竴涓紑鏀惧钩鍙般��
\ No newline at end of file
diff --git "a/\345\260\217\346\231\272 AI \347\241\254\344\273\266\345\274\200\346\224\276\345\272\224\347\224\250\345\271\263\345\217\260-\346\200\273\344\275\223\346\236\266\346\236\204\344\270\216\345\274\200\345\217\221 Plan.md" "b/\345\260\217\346\231\272 AI \347\241\254\344\273\266\345\274\200\346\224\276\345\272\224\347\224\250\345\271\263\345\217\260-\346\200\273\344\275\223\346\236\266\346\236\204\344\270\216\345\274\200\345\217\221 Plan.md"
new file mode 100644
index 0000000..cd70375
--- /dev/null
+++ "b/\345\260\217\346\231\272 AI \347\241\254\344\273\266\345\274\200\346\224\276\345\272\224\347\224\250\345\271\263\345\217\260-\346\200\273\344\275\223\346\236\266\346\236\204\344\270\216\345\274\200\345\217\221 Plan.md"
@@ -0,0 +1,541 @@
+# 灏忔櫤 AI Client Architecture Plan
+
+> 鏂囨。锛氬皬鏅� AI 纭欢寮�鏀惧簲鐢ㄥ钩鍙� 鈥斺�� 鎬讳綋鏋舵瀯涓庡紑鍙� Plan
+> Version: V1.1
+> Date: 2026-08-30
+> 鏈枃鍦� V1.0 姒傚康绋夸笂锛屼緷鎹�奨iaozhi AI 寮�鏀惧簲鐢ㄥ钩鍙� Master Plan銆嬩笌銆夾rchitecture Audit銆嬩慨璁€��
+> **鍙璁★紝涓嶇紪鐮併��**
+
+---
+
+## 0. 鏈枃瀹氫綅
+
+鏈枃鍙畾涔� **Device / Client**锛堜粨搴� `xiaozhi-esp32-2.2.6`锛夊浣曚粠銆岃闊冲璇濆浐浠躲�嶆紨杩涗负銆屽彲鎵ц澶氱 InteractionMode 鐨� Device Runtime銆嶃��
+
+涓嶅湪鏈枃灞曞紑銆佺敱 Server Plan 璐熻矗鐨勫唴瀹癸細
+
+- App / Task Definition 鐨勬潈濞佸瓨鍌�
+- TaskInstance / StepInstance 鎸佷箙鍖�
+- Scheduler / Plan / Subscription
+- ASR / LLM / Assessment / 瀹堕暱鎶ュ憡
+- 绠$悊鍚庡彴涓庡紑鍙戣�呭钩鍙�
+
+**鐩稿 V1.0 鐨勫叧閿慨姝o細**
+
+| V1.0 | V1.1锛堟湰娆★級 |
+|------|----------------|
+| 璁惧涓婃湁瀹屾暣 Task Runtime | 鏉冨▉ Task Runtime 鍦� Server锛涜澶囧彧鏈� **Task Executor** |
+| Phase 1 鍋氬璇� + 璺熻 + 鍙g畻 + 鎴愯鎺ラ緳 | Phase 1 鍙仛 `CHAT` + `PLAY_AUDIO`锛堣嫳璇璇伙級锛屼笉鐮村潖鐜版湁 Chat |
+| 鏂伴�犵粺涓� Command Protocol | **鎵╁睍鐜版湁 JSON `type` + MCP**锛屼笉鏂板紑鎬荤嚎 |
+| Skill 鍒楄〃鍚� `math.calculate` / `llm.evaluate` | Skill 蹇呴』鏄犲皠鍒�**宸叉湁浠g爜**锛涙病鏈夌殑鏍囦负鏂板缓 |
+| 璁惧缁存姢 App Manager 鍏ㄥ鐢熷懡鍛ㄦ湡 | Phase 1鈥�2 涓嶅湪鍥轰欢瀹炵幇 App 鍟嗗簵锛涜祫婧愬悓姝ュ彲澶嶇敤鏉跨骇涓嬭浇 |
+
+---
+
+## 1. 褰撳墠 Client 鏄粈涔堬紙Audit 浜嬪疄锛�
+
+褰撳墠鍥轰欢**涓嶆槸** App 骞冲彴锛岃�屾槸锛�
+
+```text
+Application 鍗曚緥
+ + DeviceState 鐘舵�佹満
+ + Opus 瀹炴椂鍙屽伐
+ + MCP 宸ュ叿闈�
+```
+
+| 椤� | 鐪熷疄浣嶇疆 |
+|----|----------|
+| 鍏ュ彛 | `main/main.cc` 鈫� `Application::Initialize` 鈫� `Application::Run` |
+| 涓诲惊鐜� | `application.cc`锛欶reeRTOS EventGroup锛屾案涓嶈繑鍥� |
+| 瀵硅瘽鎬� | `device_state.h`锛欼dle / Connecting / Listening / Speaking 鈥� |
+| 闊抽绠¢亾 | `audio/audio_service.{h,cc}`锛歁ic鈫扥pus鈫扴erver锛汼erver鈫扥pus鈫扴peaker |
+| 鍗忚 | `protocols/protocol.{h,cc}` + `websocket_protocol` / `mqtt_protocol` |
+| Skill 闈� | `mcp_server.cc` + 鍚� board `InitializeTools()` |
+| 璧勬簮 OTA | `ota.{h,cc}`锛堝浐浠�/閰嶇疆锛夈�乣assets.{h,cc}`锛堣闊虫ā鍨嬬瓑锛� |
+| 鏉跨骇鏃佽矾 | Sening锛歚self.mp3.*`銆乣BoardMqtt`锛坄snlp/v1` DOWNLOAD / PLAY_MEDIA锛� |
+
+瀹屾暣瀵硅瘽閾撅細
+
+```text
+Mic 鈫� AudioService 鈫� Opus 16k/60ms 鈫� Protocol.SendAudio
+ 鈫� Server ASR/LLM/TTS
+ 鈫� type=tts/stt + binary Opus 鈫� AudioService decode 鈫� Speaker
+```
+
+**娌℃湁锛�** InteractionMode 鏋氫妇銆乀ask/Step銆佸彲褰掓。褰曢煶鏂囦欢銆佹牳蹇� `play_audio`锛堜粎 Sening 鏉跨骇锛夈�佺粺涓� Event Bus銆�
+
+`ListeningMode`锛坅uto/manual/realtime锛夊彧鏄嬀闊崇瓥鐣ワ紝**涓嶆槸** Master Plan 鐨� InteractionMode銆�
+
+---
+
+## 2. 鐩爣 Device Runtime
+
+鐩爣涓嶆槸鎶婃墜鏈� OS 鎼埌 ESP32锛岃�屾槸锛�
+
+```text
+Device Runtime 鈫� 鐜版湁 Application 浜嬩欢鐜崌绾э紝涓嶆槸鎺ㄧ炕
+ 鈹溾攢鈹� Chat Runtime 鈫� 鐜版湁 Listening 鈫� Speaking锛岄檷涓� Mode=CHAT
+ 鈹斺攢鈹� Task Executor 鈫� 钖勬墽琛屽櫒锛氭敹鎸囦护銆佽窇 Skill銆佸洖 Event
+ 鈹溾攢鈹� InteractionMode 鍒囨崲
+ 鈹斺攢鈹� Skill 鈫� Device Capability
+```
+
+鍚屼竴鏃堕棿鍙繍琛屼竴涓� InteractionMode锛圕hat 涓� Task 闊抽浜掓枼锛夈��
+
+璁惧**涓嶇悊瑙�**涓氬姟锛�
+
+```text
+绂佹鍥轰欢鍑虹幇锛歩f (app == english) / if (task == idiom)
+```
+
+璁惧鍙悊瑙o細
+
+```text
+mode + skill + resource_ref + 瀹屾垚鏉′欢锛堣秴鏃� / 鎾斁缁撴潫 / VAD 闈欓煶锛�
+```
+
+---
+
+## 3. 鑱岃矗杈圭晫
+
+```text
+Server锛堟潈濞侊級
+ 鍒涘缓 TaskInstance
+ 鎺ㄨ繘 Step
+ 鎸囧畾 InteractionMode
+ ASR / LLM / Assessment
+ 鎸佷箙鍖栫姸鎬佷笌闊抽瀵硅薄
+
+Client锛堟墽琛岋級
+ 淇濇寔 Chat 鐑矾寰�
+ 鎸� mode 浣跨敤楹�/鎵0鍣�/灞�
+ 鎵ц Skill锛堟挱銆佸惉銆佹樉绀恒�佷笅杞斤級
+ 涓婃姤 Event锛堝紑濮�/缁撴潫/澶辫触锛�
+ 缂撳瓨褰撳墠 step锛堟柇绾� UI锛夛紝涓嶄互鏈湴涓烘潈濞�
+```
+
+瀵瑰簲 Master Plan锛欴evice 璐熻矗鎵ц銆佹挱鏀俱�侀噰闆嗐�佺紦瀛樸�侀�氫俊锛汼erver 璐熻矗璋冨害涓庤瘎浼般��
+
+---
+
+## 4. 鍒嗗眰锛堜繚鐣� V1.0锛岃惤鍒扮幇鏈変唬鐮侊級
+
+```text
+涓氬姟灞� App / Task 鈫� 涓嶄笅鍙戝埌鍥轰欢涓氬姟浠g爜锛涘彧涓嬪彂 JSON + 璧勬簮
+浜や簰灞� Interaction Mode 鈫� 鏂板锛涙寕鍦� Application锛屼笌 DeviceState 姝d氦
+鑳藉姏灞� Skill 鈫� 鐜版湁 MCP tools 鍗囩骇涓烘寮� Skill 鐩綍
+璁惧灞� Device Capability 鈫� AudioCodec / Display / Button / SD / LED
+```
+
+**DeviceState 涓� InteractionMode 蹇呴』鍒嗗紑锛�**
+
+| | DeviceState锛堝凡鏈夛級 | InteractionMode锛堟柊澧烇級 |
+|--|---------------------|-------------------------|
+| 鍥炵瓟 | 璁惧鐜板湪鑳戒笉鑳藉惉/璇�/鍗囩骇 | 杩欎竴姝ョ敤浠�涔堜氦浜掓柟寮� |
+| 渚嬪瓙 | Idle, Listening, Speaking | CHAT, PLAY_AUDIO, FOLLOW_READ |
+| 璋佹敼 | 闊抽閫氶亾鍗犵敤 | Server 鎸囧畾 / Client 鎵ц |
+
+涓嶈鎶� `PLAY_AUDIO` 鍋氭垚涓�涓柊鐨� `kDeviceStatePlayingEnglishLesson`銆傛挱鏀炬椂鍙繚鎸� Idle 鎴栧鍔�**闈炲璇�**鍗犵敤鏍囪锛屼絾 **绂佹**鎶� Task 濉炶繘 Listening/Speaking 杩佺Щ琛ㄣ��
+
+---
+
+## 5. Chat Runtime锛氫繚鐣欏苟闄嶇骇
+
+鐜版湁 Chat **蹇呴』缁х画浣滀负榛樿鑳藉姏**銆傛棤 App銆佹棤 Task 鏃讹細
+
+```text
+鍞ら啋 / 鎸夐敭 鈫� Connecting 鈫� Listening 鈫� Speaking 鈫� Idle
+```
+
+璺緞銆佸崗璁� `hello/listen/abort/tts/stt/mcp`銆丱pus 鐑矾寰� **Phase 1鈥�2 绂佹鏀硅涔�**銆�
+
+闄嶇骇鏂瑰紡锛�
+
+```text
+InteractionMode = CHAT
+ 鈫� 璧扮幇鏈� Application 瀵硅瘽鐘舵�佹満
+
+InteractionMode = PLAY_AUDIO
+ 鈫� 涓嶈繘鍏� LLM
+ 鈫� 鎵0鍣ㄧ粰 play_audio
+ 鈫� 楹﹀彲鍏虫垨鍙敤浜庢墦鏂�/鍞ら啋閫�鍑�
+
+浠绘剰 Mode 涓嬪敜閱掕瘝 / 閫�鍑�
+ 鈫� 鍥炲埌 CHAT 鎴� Idle锛堝彲閰嶇疆锛�
+```
+
+Chat 涓嶆槸 Task Runtime 鐨勭埗瀹瑰櫒锛岃�屾槸 Device Runtime 涓嬩笌 Task Executor **骞跺垪**鐨勫瓙 Runtime銆�
+
+---
+
+## 6. Task Executor锛堜笉鏄畬鏁� Task Runtime锛�
+
+V1.0 鎶� Task Runtime 鐢诲湪璁惧涓娿�侫udit 缁撹锛氱紪鎺掑凡鍦� Server `ConnectionHandler`銆傝澶囦晶鍙鍔狅細
+
+```text
+鏀跺埌 skill.execute / 鏉跨骇 PLAY_MEDIA / MCP tools/call
+ 鈫� 鎵ц
+ 鈫� 涓婃姤 completed / failed
+ 鈫� 绛夊緟涓嬩竴姝�
+```
+
+璁惧 **涓�** 璐熻矗锛�
+
+- 鍒涘缓 TaskInstance
+- 鍒ゆ柇銆岃窡璇绘槸鍚﹀強鏍煎悗鏄惁閲嶈瘯銆嶏紙鍙墽琛� Server 涓嬪彂鐨勩�屽啀鎾竴娆°�嶏級
+- 璇勫垎銆丄SR銆丩LM
+
+璁惧 **鍙互** 鏈湴鍒ゆ柇鐨勫畬鎴愭潯浠朵粎闄愮‖浠跺彲瑙傛祴锛�
+
+```text
+鎾斁缁撴潫
+VAD 闈欓煶瓒呮椂
+鐢ㄦ埛鎸夐敭鍙栨秷
+涓嬭浇瀹屾垚 / 澶辫触
+```
+
+鐘舵�佹満锛堟墽琛屽櫒绾э紝缂撳瓨鐢級锛�
+
+```text
+IDLE 鈫� EXECUTING 鈫� WAITING_USER? 鈫� COMPLETED | FAILED | CANCELLED
+```
+
+鏉冨▉鐘舵�佷粛鍦� Server 鐨� TaskInstance / StepInstance銆�
+
+---
+
+## 7. Interaction Mode
+
+> Interaction Mode 涓嶆槸涓氬姟鍦烘櫙锛堜笉鏄�岃嫳璇銆嶏級锛屽彧鎻忚堪纭欢鎬庝箞涓庝汉浜や簰銆�
+
+### 7.1 鍒嗛樁娈垫敮鎸�
+
+| Mode | Phase | 鐜版湁浠g爜鍩虹 | 璁惧瑕佸仛鐨� |
+|------|-------|--------------|------------|
+| `CHAT` | 0锛堝凡鏈夛級 | Listening/Speaking | 淇濇寔 |
+| `PLAY_AUDIO` | 1 | Sening `self.mp3.play` / `PLAY_MEDIA`锛沗PauseSdCardMp3` | 鎶芥垚鍙皟鐢� Skill锛涗笌 TTS 浜掓枼 |
+| `FOLLOW_READ` | 2 | listen 娴� + 鎾斁 | 鍏堟挱鍚庡惉锛涘惉瀹屼笂鎶ワ紝涓嶈繘 LLM |
+| `LISTEN` | 2 | `SendStartListening` | 鍙崟鐙惉锛屼笉鑷姩 Chat |
+| `QUIZ` / `TRANSLATION` / `GAME` | 3 | 鏃犵嫭绔� Mode | 鍏堢敤 CHAT+Skill 缁勫悎锛屽啀鍥哄寲 Mode |
+
+V1.0 瑕佹眰绗竴闃舵灏辨敮鎸� QUIZ/TRANSLATION/GAME 鈥斺�� **鍙栨秷**銆傞偅浜涙槸 Server 缂栨帓 + CHAT/LISTEN 鍗冲彲鍏堝仛锛屼笉鍊煎緱鍏堟敼鍥轰欢鐘舵�佹満銆�
+
+### 7.2 PLAY_AUDIO锛堣嫳璇璇伙級
+
+```text
+Server锛氶噸澶� N 娆$殑缂栨帓锛堟潈濞侊級
+Client锛歱lay_audio(resource) 鈫� 鎾畬 event 鈫� 绛変笅涓�娆� play 鎴� stop
+LLM = false锛孉SR = false
+```
+
+TTS 鏉ユ椂娌跨敤鐜版湁 `PauseSdCardMp3` / `resume_sdcard_mp3_after_tts_`锛�**涓嶈**鍙﹀啓涓�濂楁贩闊炽��
+
+### 7.3 FOLLOW_READ锛堣嫳璇窡璇伙級
+
+```text
+play 鏍囧噯闊抽
+ 鈫� play.completed
+listen锛堝鐢ㄤ笂琛� Opus锛�
+ 鈫� speech.stopped / listen.completed
+Server ASR + v0 鏂囨湰璇勪及
+ 鈫� speak 鍙嶉锛堝彲璧扮幇鏈� TTS锛�
+```
+
+璁惧涓嶈瘎鍒嗐��
+
+---
+
+## 8. Skill锛氬彧鍩轰簬鐪熷疄鑳藉姏
+
+Skill = 瀵� Device Capability 鐨勭ǔ瀹氬皝瑁呫�傜涓夋柟 App 绂佹鐩存帴 GPIO/I2S銆�
+
+### 8.1 宸叉湁锛屽彲鐩存帴褰撲綔 Skill
+
+| Skill | 褰撳墠瀹炵幇 | 鑼冨洿 |
+|-------|----------|------|
+| `device.get_status` | `self.get_device_status` | 鍏ㄥ眬 MCP |
+| `audio.set_volume` | `self.audio_speaker.set_volume` | 鍏ㄥ眬 |
+| `screen.set_brightness` / `set_theme` | `self.screen.*` | 鏈夊睆 |
+| `camera.take_photo` | `self.camera.take_photo` | 鏈夋憚鍍忓ご |
+| `audio.play_sound` | `AudioService::PlaySound`锛圤GG 鎻愮ず闊筹級 | 鍏ㄥ眬锛涗笉鑳藉綋璇剧▼闊抽 |
+| `mp3.play/stop/list` | `self.mp3.*` | **Sening** |
+| `media.play` | `BoardMqtt` `PLAY_MEDIA` | **Sening snlp** |
+| `resource.download` | `DOWNLOAD` / Assets | Sening / Assets |
+| `alarm.*` | `self.alarm.*` | Sening锛涒墵 瀹堕暱 Schedule |
+| `led` / 杩愬姩绫� | 鍚� board MCP | 鏉跨浉鍏� |
+
+### 8.2 Phase 1 闇�瑕併�屾彁鍗囥�嶇殑
+
+```text
+play_audio
+ 杈撳叆锛歋D 璺緞鎴栧凡涓嬭浇 resource_id
+ 瀹炵幇锛氫紭鍏堝鐢� self.mp3.play / PLAY_MEDIA
+ 鐩爣锛氫粠 board 绉佹湁宸ュ叿鎻愬崌涓� Runtime Skill锛堝彲浠� ifdef 鏉垮瀷锛�
+```
+
+涓嶈姹傛墍鏈夋澘绗竴澶╅兘鑳� play MP3銆傛棤 SD 鐨勬澘 Phase 1 鍙敤 Server 鎺ㄦ祦锛堜笉鎺ㄨ崘鍗� TTS 閫氶亾鍋氬璇诲惊鐜級銆�
+
+### 8.3 Phase 2 闇�瑕佹柊寤虹殑
+
+| Skill | 鐜扮姸 | 璁捐 |
+|-------|------|------|
+| `record_audio`锛堝彲褰掓。锛� | **涓嶅瓨鍦�**锛涘彧鏈夋祦寮� Opus | 浼樺厛澶嶇敤 listen 娴侊紝鐢� Server 鍙ョ骇钀� WAV锛涜澶囩鏂囦欢褰曢煶涓哄寮猴紙SD `/recordings/pending`锛� |
+| `listen`锛堜笉杩� Chat锛� | listen 娑堟伅浼氳繘鍏ュ璇� | Mode=FOLLOW_READ/LISTEN 鏃� Client 浠嶅彂 Opus锛孲erver **璺宠繃 LLM** |
+| `wait` | 鏃� | 鍙敤 Server 璁℃椂锛涜澶囧彧闇�鎾畬/瓒呮椂浜嬩欢 |
+| `display.show` | 鏈夊睆 MCP / `SetChatMessage` | 鍙墿灞曟枃鏈�/鍥剧墖璧勬簮 |
+
+### 8.4 涓嶈鍋氭垚璁惧 Skill 鐨�
+
+```text
+asr.transcribe 鈫� Server
+llm.chat/evaluate 鈫� Server
+score.evaluate 鈫� Server
+translation.translate 鈫� Server
+math.calculate 鈫� Server
+```
+
+V1.0 Skill 琛ㄩ噷鐨勮繖浜涜兘鍔涙敼鎸� Server Plan銆�
+
+---
+
+## 9. 闊抽銆佸綍闊炽�佷簰鏂�
+
+### 9.1 鐜版湁闊抽鐢熷懡鍛ㄦ湡锛堝繀椤讳繚鐣欙級
+
+```text
+瀵硅瘽锛歅CM 16k 鈫� Opus 60ms 娴� 鈫� 涓嶅啓鐩� 鈫� Server 鍙�夎浆 WAV 鍏ュ簱
+鎻愮ず闊筹細OGG PlaySound
+Sening 璇剧▼锛歋D MP3 鐙珛鎾斁锛孴TS 鏃� Pause
+```
+
+鐑矾寰勭姝㈡敼锛歚MAIN_EVENT_SEND_AUDIO`銆乣AudioInputTask`銆乣OpusCodecTask`銆乣FinishTtsResponse`銆�
+
+### 9.2 浜掓枼瑙勫垯
+
+```text
+鍚屼竴鏃跺埢鎵0鍣ㄥ彧鏈嶅姟涓�涓簮锛歍TS | SD/MP3 | PlaySound
+CHAT 鐨� Listening/Speaking 浼樺厛浜� PLAY_AUDIO锛堝彲鏆傚仠璇剧▼闊抽锛�
+Task 闇�瑕侀害鏃讹細瑕佷箞 Mode 鐙崰 listen锛岃涔堟槑纭笌 Chat 浜掓枼
+```
+
+### 9.3 璺熻闊抽鏈�灏忔柟妗堬紙Phase 2锛�
+
+**涓嶅厛鍋�**璁惧瀹屾暣 wav 鏂囦欢涓婁紶銆�
+
+```text
+澶嶇敤鐜版湁涓婅 Opus
+Server 鎸夊彞 PCM鈫扺AV锛堝凡鏈� chat_history_conf=2锛�
+鏂板鍏宠仈锛歛udio_id 鈫� step_instance_id
+```
+
+绂荤嚎缂撳瓨 `/recordings/pending` 鏀惧埌 Phase 3锛岄櫎闈炶窡璇诲湪寮辩綉鎴愪负闃诲銆�
+
+璁惧渚� **娌℃湁** audio_id锛涘彧鏈� `session_id`锛堣繛鎺ョ骇锛夈�俛udio_id 鐢� Server 鐢熸垚鍚庡洖浼狅紙鍙�夛級銆�
+
+---
+
+## 10. App 璧勬簮 vs Firmware OTA
+
+鍘熷垯淇濈暀 V1.0 / Master Plan锛�
+
+```text
+OTA = 椹卞姩銆佸崗璁�丄udio Engine銆丷untime銆丼kill 瀹夸富
+App Sync = manifest銆乼ask JSON銆丮P3銆佸浘鐗囥�侀厤缃�
+```
+
+App **鍘熷垯涓婁笉 OTA**銆備絾涓嬪垪鑳藉姏蹇呴』鍦� Firmware 閲岋紙鍙竴娆� OTA 琛ラ綈锛屼箣鍚庢柊 App 涓嶅啀鏀� C++锛夛細
+
+```text
+Codec / VAD / 鍞ら啋
+WS 鎴� MQTT+UDP 鍗忚鏍�
+MCP / Skill 瀹夸富
+SD 鏂囦欢绯荤粺
+HTTP 涓嬭浇锛堟垨绛変环锛�
+闊抽浜掓枼
+InteractionMode 寮�鍏�
+锛圥hase 2锛夐潪 Chat 鐨� listen 璇箟
+```
+
+### 10.1 SD 甯冨眬锛堢洰鏍囷紝Phase 2+ 钀藉湴锛�
+
+```text
+/sdcard/
+ apps/<appId>/ # manifest + tasks JSON
+ resources/<appId>/ # mp3 / 鍥�
+ recordings/pending/ # Phase 3
+ cache/
+```
+
+Sening 褰撳墠涓嬭浇璺緞鍋� `/data/courses/...`锛坰martlearning锛夈�傛紨杩涙椂 **鏄犲皠鍒扮粺涓�甯冨眬**锛屼笉瑕佸己杩珛鍗虫敼缁堢宸叉湁鏂囦欢銆�
+
+### 10.2 鍙� MQTT 鏀舵暃
+
+褰撳墠瀛樺湪涓ゅ锛�
+
+| 閫氶亾 | 鐢ㄩ�� |
+|------|------|
+| Xiaozhi `Protocol` | 瀵硅瘽 JSON + Opus |
+| Sening `snlp/v1` | 蹇冭烦銆丏OWNLOAD銆丳LAY_MEDIA銆乤ck/progress |
+
+**Phase 1锛�** 鍏佽骞跺瓨锛涘璇讳紭鍏堣蛋宸茶兘宸ヤ綔鐨� MCP/`PLAY_MEDIA`銆�
+**Phase 2鈥�3锛�** 璧勬簮鍚屾涓庢挱鏀炬帶鍒堕�愭鏀跺埌 Xiaozhi JSON `type` 鎴� MCP锛宻nlp 闄嶄负鏉跨骇瀹炵幇缁嗚妭锛岄伩鍏嶄笟鍔′簯闀挎湡鍒嗗弶銆�
+**绂佹 Phase 1 鍚堝苟涓ゅ MQTT銆�**
+
+---
+
+## 11. 鍗忚锛氭墿灞曪紝涓嶆浛鎹�
+
+鐜版湁璁惧鈫掍簯锛歚hello / listen / abort / iot / mcp / ping / server`
+鐜版湁浜戔啋璁惧锛歚hello / stt / tts / llm / mcp / system / alert / custom`
+
+### Phase 1
+
+涓嶅己鍒舵柊 `type`銆傚鐢細
+
+```text
+type=mcp tools/call self.mp3.play | stop
+鎴� snlp PLAY_MEDIA
+```
+
+### Phase 2 鏂板锛堜笌 tts/stt/mcp 骞跺垪锛�
+
+涓嬭锛�
+
+```text
+task.start / task.stop / task.pause / task.resume
+step.start
+skill.execute
+mode.set # CHAT | PLAY_AUDIO | FOLLOW_READ | LISTEN
+```
+
+涓婅锛�
+
+```text
+task.started / task.completed / task.failed
+step.started / step.completed / step.failed
+skill.completed / skill.failed
+audio.play.completed
+speech.started / speech.stopped # 鍙粠鐜版湁 VAD 浜嬩欢瀵煎嚭
+```
+
+杞借嵎蹇呴』甯︼細`session_id`锛汸hase 2 璧峰甫 `task_instance_id` / `step_id`銆�
+
+**涓嶈**鏂板缓绗洓濂� EventBus銆係ening `rpt/event` 鍙綔瀹炵幇锛屼絾璇箟搴斿榻愪笂琛ㄣ��
+
+鎻掑叆鐐癸細`Application` 涓� `protocol_->OnIncomingJson`锛坄application.cc` 绾� 647鈥�778 琛岋級锛宍unknown type` 鍒嗘敮鏃佸鍔� `task` / `mode`銆�
+
+---
+
+## 12. Event 涓庝富寰幆鎻掑叆鐐�
+
+鐜版湁鏈哄埗锛歚MAIN_EVENT_*` + 鍗忚鍥炶皟 + MCP JSON-RPC +锛圫ening锛塺pt/ack銆�
+
+Task Executor 鎸傝浇椤哄簭锛堜镜鍏ユ�т粠浣庡埌楂橈級锛�
+
+1. **MCP `tools/call`** 鈥� Phase 1 棣栭�夛紝Chat 璺緞涓嶅姩
+2. **`Application::Schedule`** 鈥� 绾跨▼瀹夊叏鎺ㄨ繘 UI / 闈為煶棰戝壇浣滅敤
+3. **OnIncomingJson 鏂� type** 鈥� Phase 2
+4. **绂佹**鎶� Task 鐘舵�佸啓鍏� `DeviceStateMachine::IsValidTransition` 鐨� Listening/Speaking 杈�
+
+鎸夐敭锛氫粖鏃ョ粦瀹� `ToggleChatState` / Start/Stop Listening銆侾hase 2 璧峰厑璁搞�屽彇娑堝綋鍓� Task / 璺宠繃銆嶈�屼笉鎷嗘帀 Chat 鎸夐敭銆�
+
+---
+
+## 13. 鏈湴鐘舵�侊紙钖勶級
+
+璁惧鍙寔涔呭寲蹇呰椤癸紙NVS / 鍐呭瓨锛夛細
+
+```text
+activeMode
+activeTaskInstanceId # 缂撳瓨锛屾潈濞佸湪 Server
+activeStepId # 缂撳瓨
+sdcard play 鐘舵�� # 宸叉湁
+```
+
+涓嶅湪璁惧淇濆瓨瀹屾暣 Step 鍘嗗彶銆佽瘎鍒嗐�丄SR 鏂囨湰銆�
+
+App 鐢熷懡鍛ㄦ湡 V1.0 鐨� INSTALL/LAUNCH/ACTIVE 鍏ㄥ **Phase 3** 鍐嶅仛銆侾hase 1 鐢氳嚦涓嶉渶瑕� `activeAppId`銆�
+
+---
+
+## 14. 鍒嗛樁娈碉紙Client锛�
+
+涓� Server Plan 闃舵缂栧彿瀵归綈銆�
+
+### Phase 1 鈥� 鍙屾ā寮忚瘉鏄庯紙蹇呴』涓嶇牬鍧� Chat锛�
+
+鐩爣锛氳嫳璇璇� N 閬嶃��
+
+Client 宸ヤ綔锛�
+
+- 鏂囨。鍖栧苟澶嶇敤 `self.mp3.play` / `PLAY_MEDIA` + `PauseSdCardMp3`
+- 绾﹀畾锛氭挱鏀炬湡闂村敜閱�/鎸夐敭鍙洖鍒� Chat锛堢幇鏈夎涓轰紭鍏堬級
+- **涓嶆敼** Opus 鐑矾寰勩�丏eviceState 杩佺Щ銆乭ello/listen/tts 璇箟
+- 涓嶅疄鐜� Task 鐘舵�佹満銆丄pp Manager銆佽窡璇诲綍闊虫枃浠�
+
+楠屾敹锛�
+
+1. 鏃� Task 鏃� Chat 涓庣幇缃戜竴鑷�
+2. 鑳藉惊鐜挱鏀炬寚瀹� SD/宸蹭笅杞介煶棰� N 娆�
+3. TTS/鍞ら啋浠嶈兘鎵撴柇鎴栨殏鍋滄挱鏀�
+
+### Phase 2 鈥� Task Executor + 璺熻閲囬泦
+
+- `mode.set` + `type=task` / `skill.execute`
+- FOLLOW_READ锛氭挱瀹屽啀 listen锛孲erver 璺宠繃 LLM
+- 涓婃姤 play/listen 瀹屾垚浜嬩欢锛堝甫 task/step id锛�
+- 褰曢煶鏈�灏忔柟妗堬細娴佸紡涓婁紶 + Server 褰掓。鍏宠仈锛屼笉寮哄埗璁惧 wav 鏂囦欢
+
+楠屾敹锛氫竴娆¤窡璇� Step 鑳藉湪 Server 鐪嬪埌 audio_id + asr_text銆�
+
+### Phase 3 鈥� Runtime 骞冲彴鍖�
+
+- 鏍稿績 `play_audio` / 涓嬭浇杩涢潪 Sening 鎶借薄锛堟垨鏄庣‘浠呮敮鎸� SD 鏉匡級
+- App 璧勬簮鐩綍涓庡閲� sync
+- snlp 涓� Xiaozhi 鍗忚鏀舵暃
+- 鍙�夛細鏈湴 pending 褰曢煶銆丵UIZ/GAME 浣滀负 Mode 鑰屼笉鏄‖缂栫爜涓氬姟
+- 鎸夐敭鍙粦瀹� Task 鍙栨秷
+
+---
+
+## 15. 绂佹浜嬮」锛圕lient锛�
+
+1. 閲嶅啓 AudioService 鐑矾寰勬垨銆屼负浜� Task 鎹竴濂楃紪瑙g爜銆�
+2. 鍦ㄥ浐浠跺啓鑻辫/鍙g畻/鎴愯涓氬姟鍒嗘敮
+3. 鐢� OTA 鍙戣绋� App
+4. Phase 1 鍚堝苟涓ゅ MQTT
+5. 鎶� Task 濉炶繘 Listening/Speaking
+6. 璁惧涓婂仛 ASR/璇勫垎/鎯呯华鍒嗘瀽
+7. 涓� Demo 鍐� `EnglishRepeatManager` 杩欑涓�娆℃�х被锛堝彲鐢ㄨ杽 `TaskExecutor` + Skill锛�
+8. 寮曞叆閫氱敤 EventBus / Redux
+9. 淇敼鐜版湁 Chat 鍗忚瀛楁鍚箟
+
+---
+
+## 16. AI Coding 绾︽潫锛堜繚鐣欏苟鏀剁揣 V1.0锛�
+
+1. 鍏堣鐜版湁 `Application` / `AudioService` / `Protocol` / `mcp_server`锛屽啀鏀广��
+2. 鏃� App/Task 鏃� Chat 蹇呴』鍙敤銆�
+3. 涓氬姟涓嶄笅娌夊埌 Client銆�
+4. 閰嶇疆/JSON 椹卞姩 InteractionMode 涓� Skill锛屼笉鍐欐 App銆�
+5. 鏈�灏忔敼鍔細鑳� MCP 瑙e喅鐨� Phase 1 涓嶆敼鍗忚銆�
+
+---
+
+## 17. 涓� Server Plan 鐨勫绾�
+
+Client 淇濊瘉锛�
+
+- CHAT 琛屼负鍚戝悗鍏煎
+- 鑳芥墽琛� `play_audio`锛堣嚦灏� Sening 璺緞锛�
+- Phase 2 鑳芥寜 mode 鍚�屼笉鑷姩褰� Chat 鍥炲寮曟搸
+- 浜嬩欢鍙叧鑱� `session_id`锛堝強鍚庣画 instance id锛�
+
+Client 瑕佹眰 Server锛�
+
+- 鏉冨▉鎺ㄨ繘 Step锛屼笉瑕佹寚鏈涜澶囪浣忔暣寮� Workflow
+- PLAY_AUDIO / FOLLOW_READ 涓嶈璇叆 `ConnectionHandler.chat()`
+- 闊抽瀵硅薄涓� step 鍏宠仈鍦ㄤ簯绔畬鎴�
+- App JSON 涓庤祫婧� URL 鐢变簯涓嬪彂锛屼笉瑕佹眰鍥轰欢鍐呯疆璇剧▼
+
+涓嬩竴浠藉彲鎵ц鏂囨。搴旀槸 **Phase 1 Coding Plan锛圕lient 鍒囩墖锛�**锛岃�屼笉鏄户缁姞 Mode銆�
--
Gitblit v1.9.1