Relaxed energy is a core physical descriptor of catalytic reactivity, and its prediction accuracy directly determines the reliability of machine-learning-driven catalyst screening. While $E(3)$-equivariant graph neural networks (GNNs) can effectively model three-dimensional atomic structures and achieve strong performance, language-model-based approaches offer human-readable representations and reduce reliance on explicit graphs, but remain limited in energy-prediction accuracy and discriminating multiple configurations within the same system, even with GAP-CatBERTa-style graph-assisted pretraining. To address these limitations, we propose \textbf{QE-Catalytic}, a multimodal framework that tightly integrates a large language model (Qwen) with an $E(3)$-equivariant graph Transformer (Equiformer-V2) for relaxed adsorption-energy prediction and inverse design on complex catalytic surfaces. Across multiple benchmarks and evaluation metrics, QE-Catalytic consistently outperforms baseline models. Our code is available at https://github.com/1716757342/QE-Catalytic