In smart home automation, large language models (LLMs) offer transformative potential for enabling intelligent, context-aware interactions. However, deploying LLMs on edge devices is constrained to small-scale LLMs (typically 1B–8B parameters) with limited performance, while cloud-based models, though powerful, raise privacy, latency, and cost concerns. To effectively balance these trade-offs, an intelligent routing strategy is needed to determine when to use local inference and when to offload queries to the cloud. Despite growing attention to this problem, there remains a lack of standardized and comprehensive benchmarks that reflect realistic smart home use cases and support the evaluation of such routing strategies. To address this gap, we propose LLMSwitch-bench, a novelQAbenchmark dataset comprising 6,000 queries across five smart home domains, including dietary, sleep, home security, pet care, and plant care. The dataset includes a wide range of question types with varying levels of difficulty, while been human-validated to ensure accuracy and relevance. Building on this benchmark, we introduce LLMSwitch-router, a privacy-preserving routing method that dynamically balances local and cloud-based inference. LLMSwitch-router prioritizes on-device inference to reduce cost and protect user privacy, while selectively leveraging cloud LLMs for handling complex queries that exceed local LLM capabilities. Experimental results show that LLMSwitch-router achieves a 48% reduction in LLM API cost while maintaining cloud-level accuracy, effectively optimizing the trade-off among performance, cost, and privacy.