Accurate driver intention prediction is essential for intelligent vehicle decision-making. Existing methods typically model in-cabin behavior or external traffic context independently, overlooking their dynamic interaction. This study introduces InOut-IntentNet, a bidirectional cross-modal fusion framework that jointly models driver behaviors and environmental interactions. The framework includes: (1) an Inside Behavior Capture Module extracting driving-operation and head-pose features via 3D ResNet and Transformer encoders; (2) an Outside Environment Encoding Module modeling spatiotemporal interactions and lane semantics based on stacked graph convolutions; (3) a Bidirectional Cross-Modal Fusion Module aligning in-cabin and external cues using cross-attention and gated fusion; and (4) an Intention Prediction Module trained with class-balanced loss. Experiments on the Brain4Cars show that InOut-IntentNet achieves 89.77% accuracy and 89.20% F1-score, outperforming state-of-the-art methods across early prediction intervals. In-cabin cues provide stronger intention signals, while fusion with external context improves robustness and class-wise reliability. Cross-dataset evaluation on DAAD further confirms strong generalization and real-time efficiency.