SJShi Jin

Shi Jin

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Shi Jin has a total of 7 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Self-Learning and Efficient Health-Status Analysis for a Core Router System

    by Shi Jin, Zhaobo Zhang, Krishnendu Chakrabarty, Xinli Gu

    The health status of core router systems needs to be analyzed efficiently in order to ensure high reliability and timely error recovery. Although a large amount operational data is collected from core routers, due to high computational complexity and expensive labor cost, only a small part of this data is labeled by experts. The lack of labels is an impediment towards the adoption of supervised learning. We present an iterative self-learning procedure for assessing the health status of a core router. This procedure first computes a representative feature matrix to capture different characteristics of time-series data. Not only statistical-modeling-based features are computed from three general categories, but also a recurrent neural network-based autoencoder is utilized to capture a wider range of hidden patterns. Moreover, both minimum-redundancy-maximum-relevance (mRMR) method and fully-connected feedforward autoencoder are applied to further reduce dimensionality of extracted feature matrix. Hierarchical clustering is then utilized to infer labels for the unlabeled dataset. Finally, a classifier is built and iteratively updated using both labeled and unlabeled dataset. Field data collected from a set of commercial core routers are used to experimentally validate the proposed health-status analyzer. The experimental results show that the proposed feature-based self-learning health analyzer achieves higher precision and recall than the traditional supervised health analyzer as well the currently deployed rule-based health analyzer. Moreover, it achieves better performance than the three anomaly detection baseline methods under the transformed binary classification scenario.


  • Anomaly Detection and Health-Status Analysis in a Core Router System

    by Shi Jin, Zhaobo Zhang, Krishnendu Chakrabarty, Xinli Gu

    Data analytics and real-time monitoring are essential to ensure high reliability and rapid error recovery in commercial core router systems. This paper describes how timeseries analysis and machine-learning techniques can be used to analyze health status and detect anomalies in complex core router systems. Traditional techniques fail to identify abnormal or suspect patterns when the monitored data involves temporal measurements and exhibits significantly different statistical characteristics for its constituent features. This paper thus not only describes a feature-categorization-based hybrid method and a changepoint-based method to detect anomalies in timevarying features with different statistical characteristics, but also introduces a symbol-based health analyzer to obtain a full picture of the health status of monitored core routers. A comprehensive set of experimental results is presented for data collected during 30 days of field operation from over 20 core routers deployed by customers of a major telecom company.


  • Hierarchical Symbol-Based Health-Status Analysis Using Time-Series Data in a Core Router System

    by Shi Jin, Zhaobo Zhang, Krishnendu Chakrabarty, Xinli Gu

    To ensure high reliability and rapid error recovery in commercial core router systems, a health-status analyzer is essential to monitor the different features of core routers. However, traditional health analyzers need to store a large amount of historical data in order to identify health status. The storage requirement becomes prohibitively high when we attempt to carry out long-term health-status analysis for a large number of core routers. We describe the design of a symbol-based health status analyzer that first encodes, as a symbol sequence, the long-term complex time series collected from a number of core routers, and then utilizes the symbol sequence to do health analysis. The symbolic aggregation approximation (SAX), 1d-SAX, moving-average-based trend approximation, and non-parametric symbolic approximation representation methods are implemented to encode complex time series in a hierarchical way. Hierarchical agglomerative clustering and sequitur rule discovery are implemented to learn important global and local patterns. Three classification methods including a vector-space-model-based approach are then utilized to identify the health status of core routers. Data collected from a set of commercial core router systems are used to validate the proposed health-status analyzer. The experimental results show that our symbol-based health status analyzer requires much lower storage than traditional methods, but can still maintain comparable diagnosis accuracy.


  • Changepoint-Based Anomaly Detection for Prognostic Diagnosis in a Core Router System

    by Shi Jin, Zhaobo Zhang, Krishnendu Chakrabarty, Xinli Gu

    Prognostic diagnosis is desirable for commercial core router systems to ensure early failure prediction and fast error recovery. The effectiveness of prognostic diagnosis depends on whether anomalies can be accurately detected before a failure occurs. However, traditional anomaly detection techniques fail to detect “outliers” when the statistical properties of the monitored data change significantly as time proceeds. We describe the design of a changepoint-based anomaly detector that first detects changepoints from collected time-series data, and then utilizes these changepoints to detect anomalies. Different changepoint detection approaches are implemented to detect various types of changepoints. A clustering method is then developed to identify normal/abnormal patterns from changepoint windows. Data collected from a set of commercial core router systems are used to validate the proposed anomaly detector. Experimental results show that our changepoint-based anomaly detector achieves better performance than traditional methods in terms of two metrics, namely success ratio and non-false-alarm ratio.