Reza Nematirad

Time series forecasting plays a crucial role in many real-world applications, and numerous complex forecasting models have been proposed in recent years. Despite their architectural innovations, most state-of-the-art models report only marginal improvements -- typically just a few thousandths in standard error metrics. These models often incorporate complex data embedding layers to transform raw inputs into higher-dimensional representations to enhance accuracy. But are data embedding techniques actually effective in time series forecasting? Through extensive ablation studies across fifteen state-of-the-art models and four benchmark datasets, we find that removing data embedding layers from many state-of-the-art models does not degrade forecasting performance. In many cases, it improves both accuracy and computational efficiency. The gains from removing embedding layers often exceed the performance differences typically reported between competing models. Code available at: https://github.com/neuripsdataembedidng/DataEmbedding

Time series forecasting is an important application in various domains such as energy management, traffic planning, financial markets, meteorology, and medicine. However, real-time series data often present intricate temporal variability and sharp fluctuations, which pose significant challenges for time series forecasting. Previous models that rely on 1D time series representations usually struggle with complex temporal variations. To address the limitations of 1D time series, this study introduces the Times2D method that transforms the 1D time series into 2D space. Times2D consists of three main parts: first, a Periodic Decomposition Block (PDB) that captures temporal variations within a period and between the same periods by converting the time series into a 2D tensor in the frequency domain. Second, the First and Second Derivative Heatmaps (FSDH) capture sharp changes and turning points, respectively. Finally, an Aggregation Forecasting Block (AFB) integrates the output tensors from PDB and FSDH for accurate forecasting. This 2D transformation enables the utilization of 2D convolutional operations to effectively capture long and short characteristics of the time series. Comprehensive experimental results across large-scale data in the literature demonstrate that the proposed Times2D model achieves state-of-the-art performance in both short-term and long-term forecasting.

Machine learning (ML) has been extensively applied in various disciplines. However, not much attention has been paid to data heterogeneity in databases and number of samples used to train ML models in hydrology. In this study, we addressed these issues and their impacts on the accuracy and reliability of ML models in the estimation of saturated hydraulic conductivity, K_s. We selected 17,990 soil samples from the USKSAT database and created random subsets N = 2,000, 4,000, 6,000, 8,000, 10,000, 12,000, 14,000, 16,000, and 17,990, 80% of which were used for training. The random subset selection was repeated 50 times. The extreme gradient boosting (XGBoost) algorithm was used to estimate K_s from other soil properties, such as bulk density, soil depth, texture, and organic content. For each subset, we conducted the learning curve analysis on the training and cross‐validation data sets. Results showed that for all training sample sizes the number of samples was not enough for the training and cross‐validation curves to reach a plateau. We also applied the concept of representative elementary volume by plotting the average coefficient of determination, R², and root mean square log‐transformed error, RMSLE, against the training sample size. For the testing data set, as the number of training sample size increased from 1,600 to 14,392 the average R² value increased from 0.74 to 0.90, while the average RMSLE value decreased from 1.08 to 0.69. Either the learning curve or representative sample size analysis is required to investigate whether the number of samples is enough or not.

Integrating photovoltaic (PV) systems plays a pivotal role in the global shift toward renewable energy, offering significant environmental benefits. However, the PV installation should provide financial benefits for the utilities. Considering that the utility companies often incur costs for both energy and peak demand, PV installations should aim to reduce both energy and peak demand charges. Although PV systems can reduce energy needs during the day, their effectiveness in reducing peak demand, particularly in the early morning and late evening, is limited, as PV generation is zero or negligible at those times. To address this limitation, battery storage systems are utilized for storing energy during off-peak hours and releasing it during peak times. However, finding the optimal size of PV and the accompanying battery remains a challenge. While valuable optimization models have been developed to determine the optimal size of PV–battery systems, a certain gap remains where peak demand reduction has not been sufficiently addressed in the optimization process. Recognizing this gap, this study proposes a novel statistical model to optimize PV–battery system size for peak demand reduction. The model aims to flatten 95% of daily peak demands up to a certain demand threshold, ensuring consistent energy supply and financial benefit for utility companies. A straightforward and effective search methodology is employed to determine the optimal system sizes. Additionally, the model’s effectiveness is rigorously tested through a modified Monte Carlo simulation coupled with time series clustering to generate various scenarios to assess performance under different conditions. The results indicate that the optimal PV–battery system successfully flattens 95% of daily peak demand with a selected threshold of 2000 kW, yielding a financial benefit of USD 812,648 over 20 years.

Cooling fans are one of the critical components of air-forced dry-type transformers for regulating internal temperatures. Therefore, effective malfunction detection is crucial to maintain the transformer temperature within an acceptable range and prevent overheating. Current malfunction detection of cooling fans in certain types of transformers relies on complementary indicators, such as top-oil temperature, oil convection, dissolved gas, and oil quality. However, these conventional indicators are not directly applicable to air-forced transformers, which primarily use cooling fans as their cooling system. To overcome this challenge, this study utilizes cooling fan audio records as indicators. The audio signals are classified into normal and malfunctioning classes using advanced learning algorithms, including convolutional neural networks and random forests. Learning algorithms require transforming recorded audio data into proper formats. Accordingly, convolutional neural networks are trained based on spectrogram images derived from audio signals. For random forests, various time-frequency feature extraction methods are used to derive meaningful representations from audio signals. Besides, multiple data augmentation techniques are employed to enhance the dataset size and diversity. Algorithmic performance is optimized through hyperparameter tuning and classifier threshold adjustment. To further validate the model, a test is conducted on another dataset to evaluate the fitted learning model applicability in real-world applications. Simulations reveal that convolutional neural networks outperform random forests, whereas the latter provides superior interpretability of acoustic features compared to the former.

Residential consumers can optimize their participation in demand response programs (DRPs) using home energy management systems (HEMS). By automatically adjusting air conditioning (AC) setpoints and shifting certain appliances to off-peak hours, HEMS can lead to significant cost reductions. While HEMS aims to adjust AC temperature setpoints, it is important to consider the occupants thermal comfort. This study aims to develop a multi-objective model for the application of DRPs in a smart residential house. The objectives of the model are to achieve (a) reduction in electrical load demand, (b) adjustment in thermal comfort temperature setpoints, and (c) minimization of consumer costs subject to the related constraints. Determining occupancy status through HEMS provides more economic benefits and thermal comfort for consumers. However, traditional methods such as direct occupancy monitoring are often costly, inaccurate, and can intrusively collect data on residents’ activities, locations, and routines, compromising their privacy. To tackle these challenges, this study introduces advanced forecasting algorithms such as random forest, light gradient boosting machine, and multilayer perception artificial neural networks to predict occupancy by utilizing indirect data sources, such as energy consumption patterns. This approach enables the prediction of residential presence without direct monitoring. However, inherent uncertainty associated with predicted parameters can compromise the effectiveness of DRPs, and potentially lead to non-optimal energy savings, jeopardizing consumer comfort, and even system instability. To address these uncertainties, this study integrates robust counterpart optimization techniques, augmented with uncertainty budgets to control uncertainty variation making them less conservative. Simulations show uncertainty increases costs by 36% and reduces AC temperature setpoints. Further, implementing DRPs reduces demand by shifting some appliances to off-peak hours and lowers costs by 13.2%.

Cooling fans are one of the critical components of air-forced (AF) dry-type transformers for regulating internal temperatures. Therefore, effective malfunction detection is crucial to maintain the transformer temperature within an acceptable range and prevent overheating. Regular maintenance occurs periodically and issues with cooling fans may arise between maintenance periods, leading to prolonged operation under malfunctioning conditions and potential failures. In addition, utilities typically have online information about whether a fan works or not without providing information about cooling fan malfunctioning circumstances. To address these challenges, this study proposes learning-based online monitoring techniques to detect malfunctions in AF transformer cooling fans. Random forests (RFs) and convolutional neural networks (CNNs) are developed to classify the audio signals from cooling fans into normal and malfunctioning classes. Unlike RFs, which require separate feature extraction, CNNs are trained based on spectrogram images derived from audio signals. Thus, various time-frequency techniques are utilized for feature extraction in RFs. Besides, multiple data augmentation techniques are employed to enhance the dataset size and diversity. Algorithmic performance is optimized through hyperparameter tuning and classifier threshold adjustment. Simulations reveal that CNNs outperform RFs, whereas the latter provides superior interpretability of acoustic features compared to the former.

Integrating photovoltaic (PV) systems plays a pivotal role in the global shift toward renewable energy, offering significant environmental benefits. In many instances, utility companies pay for both energy and peak demand. PV systems can effectively reduce energy consumption during their operational periods, while, their contribution during peak hours, particularly in the early morning and late evening, is often limited. To address this issue, battery storage systems are utilized for storing energy during off-peak hours and releasing it during peak times. However, finding the optimal size of PV-battery systems remains a challenge. This study proposes a novel statistical model to optimize PV-battery system sizing for enhanced peak demand reduction. The model aims to flatten 95% of peak demand days up to a certain demand threshold, ensuring consistent energy supply and financial benefit for utility companies. A straightforward and effective search methodology is employed to determine the optimal system sizes. Additionally, the model effectiveness is rigorously tested through a modified Monte Carlo simulation coupled with time series clustering to generate various scenarios to assess performance under different conditions. The results indicate that the optimal PV-battery system successfully flattens 95% of daily peak demand up to 2000 kW, yielding a financial benefit of $812,648 over 20 years.