KMKanak Mahadik

Kanak Mahadik

*This is an algorithmically generated author record based on publicly available information.

Kanak Mahadik has a total of 30 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    NotebookOS: A Replicated Notebook Platform for Interactive Training with On-Demand GPUs

    by Benjamin Carver, Jingyuan Zhang, Haoliang Wang, Kanak Mahadik, Yue Cheng


  • Open Access

    Probabilistic Hypergraph Recurrent Neural Networks for Time-series Forecasting

    by Hongjie Chen, Ryan A. Rossi, Sungchul Kim, Kanak Mahadik, Hoda Eldardiry


  • Open Access

    Evaluation-free Time-series Forecasting Model Selection via Meta-learning

    by Mustafa Abdallah, Ryan A. Rossi, Kanak Mahadik, Sungchul Kim, Handong Zhao, Saurabh Bagchi

    Time-series forecasting models are invariably used in a variety of domains for crucial decision-making. Traditionally these models are constructed by experts with considerable manual effort. Unfortunately, this approach has poor scalability while generating accurate forecasts for new datasets belonging to diverse applications. Without access to skilled domain-knowledge, one approach is to train all the models on the new time-series data and then select the best one. However, this approach is nonviable in practice. In this work, we develop techniques for fast automatic selection of the best forecasting model for a new unseen time-series dataset, without having to first train (or evaluate) all the models on the new time-series data to select the best one. In particular, we develop a forecasting meta-learning approach called AutoForecast that allows for the quick inference of the best time-series forecasting model for an unseen dataset. Our approach learns both forecasting models’ performances over time horizon of the same dataset and task similarity across different datasets. The experiments demonstrate the effectiveness of the approach over state-of-the-art (SOTA) single and ensemble methods and several SOTA meta-learners (adapted to our problem) in terms of selecting better forecasting models (i.e., 2 \(\times\) gain) for unseen tasks for univariate and multivariate testbeds. AutoForecast has also significant reduction in inference time compared to the naïve approach (doing inference using all possible models and then selecting the best one), with median of 42 \(\times\) across the two testbeds. We release our meta-learning database corpus (348 datasets), performances of the 322 forecasting models on the database corpus, meta-features, and source codes for the community to access them for forecasting model selection and to build on them with new datasets and models which can help advance automating time-series forecasting problem. In our released database corpus, we unveil new traces of Adobe computing cluster usage for production workloads.


  • Open Access

    Pandora with Inaccurate Priors

    by Kiarash Banihashem, Xiang Chen, MohammadTaghi Hajiaghayi, Sungchul Kim, Kanak Mahadik, Ryan Rossi, Tong Yu

    We investigate the role of inaccurate priors for the classical Pandora's box problem. In the classical Pandora's box problem we are given a set of boxes each with a known cost and an unknown value sampled from a known distribution. We investigate how inaccuracies in the beliefs can affect existing algorithms. Specifically, we assume that the knowledge of the underlying distribution has a small error in the Kolmogorov distance, and study how this affects the utility obtained by the optimal algorithm.

  • Open Access

    Tackling Long-Tail Entities for Temporal Knowledge Graph Completion

    by Mehrnoosh Mirtaheri, Ryan A. Rossi, Sungchul Kim, Kanak Mahadik, Tong Yu, Xiang Chen, Mohammad Rostami

    Most Temporal Knowledge Graphs (TKGs) exhibit a long-tail entity distribution, where the majority of entities have sparse connections. Existing TKG completion methods struggle with managing new or unseen entities that often lack sufficient connections. In this paper, we introduce a model-agnostic enhancement layer that can be integrated with any existing TKG completion method to improve its performance. This enhancement layer employs a broader, global definition of entity similarity, transcending the limitations of local neighborhood proximity found in Graph Neural Network (GNN) based methods. Additionally, we conduct our evaluations in a novel, realistic setup that treats the TKG as a stream of evolving data. Evaluations on two benchmark datasets demonstrate that our framework surpasses existing methods in overall link prediction, inductive link prediction, and in addressing long-tail entities. Notably, our approach achieves a 10% improvement in MRR on one dataset and a 15% increase on another.


  • Evolving Super Graph Neural Networks for Large-Scale Time-Series Forecasting

    by Hongjie Chen, Ryan Rossi, Sungchul Kim, Kanak Mahadik, Hoda Eldardiry


  • Open Access

    Graph Deep Factors for Probabilistic Time-series Forecasting

    by Hongjie Chen, Ryan A. Rossi, Kanak Mahadik, Sungchul Kim, Hoda Eldardiry

    Effective time-series forecasting methods are of significant importance to solve a broad spectrum of research problems. Deep probabilistic forecasting techniques have recently been proposed for modeling large collections of time-series. However, these techniques explicitly assume either complete independence (local model) or complete dependence (global model) between time-series in the collection. This corresponds to the two extreme cases where every time-series is disconnected from every other time-series in the collection or likewise, that every time-series is related to every other time-series resulting in a completely connected graph. In this work, we propose a deep hybrid probabilistic graph-based forecasting framework called Graph Deep Factors (GraphDF) that goes beyond these two extremes by allowing nodes and their time-series to be connected to others in an arbitrary fashion. GraphDF is a hybrid forecasting framework that consists of a relational global and relational local model. In particular, a relational global model learns complex non-linear time-series patterns globally using the structure of the graph to improve both forecasting accuracy and computational efficiency. Similarly, instead of modeling every time-series independently, a relational local model not only considers its individual time-series but also the time-series of nodes that are connected in the graph. The experiments demonstrate the effectiveness of the proposed deep hybrid graph-based forecasting model compared to the state-of-the-art methods in terms of its forecasting accuracy, runtime, and scalability. Our case study reveals that GraphDF can successfully generate cloud usage forecasts and opportunistically schedule workloads to increase cloud cluster utilization by 47.5% on average. Furthermore, we target addressing the common nature of many time-series forecasting applications where time-series are provided in a streaming version; however, most methods fail to leverage the newly incoming time-series values and result in worse performance over time. In this article, we propose an online incremental learning framework for probabilistic forecasting. The framework is theoretically proven to have lower time and space complexity. The framework can be universally applied to many other machine learning-based methods.


  • Open Access

    Correlated Stochastic Knapsack with a Submodular Objective

    by Sheng Yang, Samir Khuller, Sunav Choudhary, Subrata Mitra, Kanak Mahadik

    We study the correlated stochastic knapsack problem of a submodular target function, with optional additional constraints. We utilize the multilinear extension of submodular function, and bundle it with an adaptation of the relaxed linear constraints from Ma [Mathematics of Operations Research, Volume 43(3), 2018] on correlated stochastic knapsack problem. The relaxation is then solved by the stochastic continuous greedy algorithm, and rounded by a novel method to fit the contention resolution scheme (Feldman et al. [FOCS 2011]). We obtain a pseudo-polynomial time $(1 - 1/\sqrt{e})/2 \simeq 0.1967$ approximation algorithm with or without those additional constraints, eliminating the need of a key assumption and improving on the $(1 - 1/\sqrt[4]{e})/2 \simeq 0.1106$ approximation by Fukunaga et al. [AAAI 2019].

  • Open Access

    Lerna: transformer architectures for configuring error correction tools for short- and long-read genome sequencing

    by Atul Sharma, Pranjal Jain, Ashraf Mahgoub, Zihan Zhou, Kanak Mahadik, Somali Chaterji

    Background: Sequencing technologies are prone to errors, making error correction (EC) necessary for downstream applications. EC tools need to be manually configured for optimal performance. We find that the optimal parameters (e.g.,k-mer size) are both tool- and dataset-dependent. Moreover, evaluating the performance (i.e., Alignment-rate or Gain) of a given tool usually relies on a reference genome, but quality reference genomes are not always available. We introduceLernafor the automated configuration ofk-mer-based EC tools.Lernafirst creates a language model (LM) of the uncorrected genomic reads, and then, based on this LM, calculates a metric called theperplexity metricto evaluate the corrected reads for different parameter choices. Next, it finds the one that produces the highest alignment ratewithoutusing a reference genome. The fundamental intuition of our approach is that the perplexity metric is inversely correlated with the quality of the assembly after error correction. Therefore,Lernaleverages the perplexity metric for automated tuning ofk-mer sizes without needing a reference genome.Results: First, we show that the bestk-mer value can vary for different datasets, even for the same EC tool. This motivates our design that automatesk-mer size selection without using a reference genome. Second, we show the gains of our LM using its component attention-based transformers. We show the model’s estimation of the perplexity metric before and after error correction. The lower the perplexity after correction, the better thek-mer size. We also show that the alignment rate and assembly quality computed for the corrected reads are stronglynegativelycorrelated with the perplexity, enabling the automated selection ofk-mer values for better error correction, and hence, improved assembly quality. We validate our approach on both short and long reads. Additionally, we show that our attention-based models have significant runtime improvement for the entire pipeline—18 $$\times$$× faster than previous works, due to parallelizing the attention mechanism and the use of JIT compilation for GPU inferencing.Conclusion: Lerna improvesde novogenome assembly by optimizing EC tools. Our code is made available in a public repository at:https://github.com/icanforce/lerna-genomics.


  • Open Access

    Scalable Genome Assembly through Parallel de Bruijn Graph Construction for Multiple k-mers

    by Kanak Mahadik, Christopher Wright, Milind Kulkarni, Saurabh Bagchi, Somali Chaterji

    Remarkable advancements in high-throughput gene sequencing technologies have led to an exponential growth in the number of sequenced genomes. However, unavailability of highly parallel and scalable de novo assembly algorithms have hindered biologists attempting to swiftly assemble high-quality complex genomes. Popular de Bruijn graph assemblers, such as IDBA-UD, generate high-quality assemblies by iterating over a set of k-values used in the construction of de Bruijn graphs (DBG). However, this process of sequentially iterating from small to large k-values slows down the process of assembly. In this paper, we propose ScalaDBG, which metamorphoses this sequential process, building DBGs for each distinct k-value in parallel. We develop an innovative mechanism to “patch” a higher k-valued graph with contigs generated from a lower k-valued graph. Moreover, ScalaDBG leverages multi-level parallelism, by both scaling up on all cores of a node, and scaling out to multiple nodes simultaneously. We demonstrate that ScalaDBG completes assembling the genome faster than IDBA-UD, but with similar accuracy on a variety of datasets (6.8X faster for one of the most complex genome in our dataset).