Zhirong Yang

Machine learning (ML) models show promise for predicting perceived indoor environmental quality (IEQ) and building-related symptoms (BRS), but their generalizability across different buildings remains limited. This study examines whether ML models trained in different school contexts provide consistent predictions across three schools. Concurrent subjective feedback on IEQ dissatisfaction, IEQ complaints, and BRS was combined with objective indoor and outdoor measurement data. Localized Random Forest models trained using single-school data were compared with generalized models trained using a combined multi-school dataset. The localized models achieved high predictive performance within their source school (R² > 0.85) but performed poorly when transferred to the other participating schools (R² < 0.30). By contrast, the generalized models achieved good training result (R² ~ 0.80) and more consistent validation performance across the three schools (R² up to 0.69). SHAP analysis showed that the composition of the training dataset affected the predictive importance assigned to different feature groups. IEQ complaints remained important predictors of BRS, while between-school variation allowed building-related factors to contribute to the multi-school predictions. These findings indicate that high within-school accuracy does not necessarily imply cross-school transportability. Multi-school training therefore provides a more suitable basis for achieving balanced predictive performance across heterogeneous school contexts. The findings remain preliminary because the study included only three schools, and the generalized models were validated on held-out observations from schools represented during training. Future work should include a larger and more diverse building sample, hyperparameter tuning, and real-time occupancy variables.

Sequential recommendation systems aim to provide personalized recommendations by analyzing dynamic preferences and dependencies within user behavior sequences. Recently, Transformer models can effectively capture user preferences. However, their quadratic computational complexity limits recommendation performance on long interaction sequence data. Inspired by the State Space Model (SSM)representative model, Mamba, which efficiently captures user preferences in long interaction sequences with linear complexity, we find that Mamba's recommendation effectiveness is limited in short interaction sequences, with failing to recall items of actual interest to users and exacerbating the data sparsity cold start problem. To address this issue, we innovatively propose a new model, MaTrRec, which combines the strengths of Mamba and Transformer. This model fully leverages Mamba's advantages in handling long-term dependencies and Transformer's global attention advantages in short-term dependencies, thereby enhances predictive capabilities on both long and short interaction sequence datasets while balancing model efficiency. Notably, our model significantly improves the data sparsity cold start problem, with an improvement of up to 33% on the highly sparse Amazon Musical Instruments dataset. We conducted extensive experimental evaluations on five widely used public datasets. The experimental results show that our model outperforms the current state-of-the-art sequential recommendation models on all five datasets. The code is available at https://github.com/Unintelligentmumu/MaTrRec.

Self-supervised pretraining (SSP) has been recognized as a method to enhance prediction accuracy in various downstream tasks. However, its efficacy for DNA sequences remains somewhat constrained. This limitation stems primarily from the fact that most existing SSP approaches in genomics focus on masked language modeling of individual sequences, neglecting the crucial aspect of encoding statistics across multiple sequences. To overcome this challenge, we introduce an innovative deep neural network model, which incorporates collaborative learning between a `student' and a `teacher' subnetwork. In this model, the student subnetwork employs masked learning on nucleotides and progressively adapts its parameters to the teacher subnetwork through an exponential moving average approach. Concurrently, both subnetworks engage in contrastive learning, deriving insights from two augmented representations of the input sequences. This self-distillation process enables our model to effectively assimilate both contextual information from individual sequences and distributional data across the sequence population. We validated our approach with preliminary pretraining using the human reference genome, followed by applying it to 20 downstream inference tasks. The empirical results from these experiments demonstrate that our novel method significantly boosts inference performance across the majority of these tasks. Our code is available at https://github.com/wiedersehne/FinDNA.

Sequential data naturally have different lengths in many domains, with some very long sequences. As an important modeling tool, neural attention should capture long-range interaction in such sequences. However, most existing neural attention models admit only short sequences, or they have to employ chunking or padding to enforce a constant input length. Here we propose a simple neural network building block called ChordMixer which can model the attention for long sequences with variable lengths. Each ChordMixer block consists of a position-wise rotation layer without learnable parameters and an element-wise MLP layer. Repeatedly applying such blocks forms an effective network backbone that mixes the input signals towards the learning targets. We have tested ChordMixer on the synthetic adding problem, long document classification, and DNA sequence-based taxonomy classification. The experiment results show that our method substantially outperforms other neural attention models.

Due to their intrinsic properties, DNA molecules commonly exhibit long-range interactions along a linear sequence representation. Taking this information into account when modeling DNA sequences is therefore important for obtaining more accurate sequence-based inference. Many deep learning methods have recently been developed for this purpose, but they still suffer from two major issues. First, the existing methods can only handle short DNA fragments, thereby losing longerrange interactions. Second, the current methods require massive supervised labeling while missing most order information within the sequences. Consequently, there is a need to develop an efficient deep neural network modeling framework to extract wide contextual information for more accurate sequence-based inference tasks. Our new framework, named Revolution, takes full DNA sequences as input, without any condensation, and can give accurate predictions for DNA sequences up to 10kbp. In variant effect prediction, our method increases the Area Under the Receiver Operating Characteristics (AUROC) by 19.61% on 49 human tissues on average. Revolution is also demonstrated to work on the plant sequences by improving 2.36% AUROC on average for predicting open chromatin regions (OCRs). The data, models, and code can be freely accessed at https://github.com/wiedersehne/Revolution-DNAPretraining .