Justin Sybrandt
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Justin Sybrandt has a total of 28 co-authors. Here are the top co-authors by number of shared publications:
- Ilya Safro
Department of Computer and Information Sciences, University of Delaware, Newark, DE, USA
14 shared publications - Michael Shtutman
Department of Drug Discovery and Biomedical Sciences, College of Pharmacy, University of South Carolina, Columbia, USA
5 shared publications - Ilya Tyagin
University of Delaware
2 shared publications Laura Redmond
2 shared publicationsSez Atamturktur
2 shared publicationsWilliam Locke
2 shared publications
Latest Publications
- Open Access
Accelerating COVID-19 research with graph mining and transformer-based learning
In 2020, the White House released the, "Call to Action to the Tech Community on New Machine Readable COVID-19 Dataset," wherein artificial intelligence experts are asked to collect data and develop text mining techniques that can help the science community answer high-priority scientific questions related to COVID-19. The Allen Institute for AI and collaborators announced the availability of a rapidly growing open dataset of publications, the COVID-19 Open Research Dataset (CORD-19). As the pace of research accelerates, biomedical scientists struggle to stay current. To expedite their investigations, scientists leverage hypothesis generation systems, which can automatically inspect published papers to discover novel implicit connections. We present an automated general purpose hypothesis generation systems AGATHA-C and AGATHA-GP for COVID-19 research. The systems are based on graph-mining and the transformer model. The systems are massively validated using retrospective information rediscovery and proactive analysis involving human-in-the-loop expert analysis. Both systems achieve high-quality predictions across domains (in some domains up to 0.97% ROC AUC) in fast computational time and are released to the broad scientific community to accelerate biomedical research. In addition, by performing the domain expert curated study, we show that the systems are able to discover on-going research findings such as the relationship between COVID-19 and oxytocin hormone.
Open AccessAccelerating COVID-19 research with graph mining and transformer-based learning
In 2020, the White House released the, "Call to Action to the Tech Community on New Machine Readable COVID-19 Dataset," wherein artificial intelligence experts are asked to collect data and develop text mining techniques that can help the science community answer high-priority scientific questions related to COVID-19. The Allen Institute for AI and collaborators announced the availability of a rapidly growing open dataset of publications, the COVID-19 Open Research Dataset (CORD-19). As the pace of research accelerates, biomedical scientists struggle to stay current. To expedite their investigations, scientists leverage hypothesis generation systems, which can automatically inspect published papers to discover novel implicit connections. We present an automated general purpose hypothesis generation systems AGATHA-C and AGATHA-GP for COVID-19 research. The systems are based on graph-mining and the transformer model. The systems are massively validated using retrospective information rediscovery and proactive analysis involving human-in-the-loop expert analysis. Both systems achieve high-quality predictions across domains (in some domains up to 0.97% ROC AUC) in fast computational time and are released to the broad scientific community to accelerate biomedical research. In addition, by performing the domain expert curated study, we show that the systems are able to discover on-going research findings such as the relationship between COVID-19 and oxytocin hormone.
- Open Access
- 6 July 2021
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 16 (7) , e0253905
CBAG: Conditional biomedical abstract generation
Biomedical research papers often combine disjoint concepts in novel ways, such as when describing a newly discovered relationship between an understudied gene with an important disease. These concepts are often explicitly encoded as metadata keywords, such as the author-provided terms included with many documents in the MEDLINE database. While substantial recent work has addressed the problem of text generation in a more general context, applications, such as scientific writing assistants, or hypothesis generation systems, could benefit from the capacity to select the specific set of concepts that underpin a generated biomedical text. We propose a conditional language model following the transformer architecture. This model uses the “encoder stack” to encode concepts that a user wishes to discuss in the generated text. The “decoder stack” then follows the masked self-attention pattern to perform text generation, using both prior tokens as well as the encoded condition. We demonstrate that this approach provides significant control, while still producing reasonable biomedical text.
- 6 July 2021
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 16 (7) , e0253905
Open AccessCBAG: Conditional biomedical abstract generation
Biomedical research papers often combine disjoint concepts in novel ways, such as when describing a newly discovered relationship between an understudied gene with an important disease. These concepts are often explicitly encoded as metadata keywords, such as the author-provided terms included with many documents in the MEDLINE database. While substantial recent work has addressed the problem of text generation in a more general context, applications, such as scientific writing assistants, or hypothesis generation systems, could benefit from the capacity to select the specific set of concepts that underpin a generated biomedical text. We propose a conditional language model following the transformer architecture. This model uses the “encoder stack” to encode concepts that a user wishes to discuss in the generated text. The “decoder stack” then follows the masked self-attention pattern to perform text generation, using both prior tokens as well as the encoded condition. We demonstrate that this approach provides significant control, while still producing reasonable biomedical text.
- Open Access
Hypergraph Partitioning With Embeddings
Problems in scientific computing, such as distributing large sparse matrix operations, have analogous formulations as hypergraph partitioning problems. A hypergraph is a generalization of a traditional graph wherein "hyperedges" may connect any number of nodes. As a result, hypergraph partitioning is an NP-Hard problem to both solve or approximate. State-of-the-art algorithms that solve this problem follow the multilevel paradigm, which begins by iteratively "coarsening" the input hypergraph to smaller problem instances that share key structural features. Once identifying an approximate problem that is small enough to be solved directly, that solution can be interpolated and refined to the original problem. While this strategy represents an excellent trade off between quality and running time, it is sensitive to coarsening strategy. In this work we propose using graph embeddings of the initial hypergraph in order to ensure that coarsened problem instances retrain key structural features. Our approach prioritizes coarsening within self-similar regions within the input graph, and leads to significantly improved solution quality across a range of considered hypergraphs. Reproducibility: All source code, plots and experimental data are available at https://sybrandt.com/2019/partition.
Open AccessHypergraph Partitioning With Embeddings
Problems in scientific computing, such as distributing large sparse matrix operations, have analogous formulations as hypergraph partitioning problems. A hypergraph is a generalization of a traditional graph wherein "hyperedges" may connect any number of nodes. As a result, hypergraph partitioning is an NP-Hard problem to both solve or approximate. State-of-the-art algorithms that solve this problem follow the multilevel paradigm, which begins by iteratively "coarsening" the input hypergraph to smaller problem instances that share key structural features. Once identifying an approximate problem that is small enough to be solved directly, that solution can be interpolated and refined to the original problem. While this strategy represents an excellent trade off between quality and running time, it is sensitive to coarsening strategy. In this work we propose using graph embeddings of the initial hypergraph in order to ensure that coarsened problem instances retrain key structural features. Our approach prioritizes coarsening within self-similar regions within the input graph, and leads to significantly improved solution quality across a range of considered hypergraphs. Reproducibility: All source code, plots and experimental data are available at https://sybrandt.com/2019/partition.
- 17 August 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 34 (6) , 2771-2782
Hypergraph Partitioning With Embeddings
Problems in scientific computing, such as distributing large sparse matrix operations, have analogous formulations as hypergraph partitioning problems. A hypergraph is a generalization of a traditional graph wherein "hyperedges" may connect any number of nodes. As a result, hypergraph partitioning is an NP-Hard problem to both solve or approximate. State-of-the-art algorithms that solve this problem follow the multilevel paradigm, which begins by iteratively "coarsening" the input hypergraph to smaller problem instances that share key structural features. Once identifying an approximate problem that is small enough to be solved directly, that solution can be interpolated and refined to the original problem. While this strategy represents an excellent trade off between quality and running time, it is sensitive to coarsening strategy. In this work we propose using graph embeddings of the initial hypergraph in order to ensure that coarsened problem instances retrain key structural features. Our approach prioritizes coarsening within self-similar regions within the input graph, and leads to significantly improved solution quality across a range of considered hypergraphs. Reproducibility: All source code, plots and experimental data are available at https://sybrandt.com/2019/partition.
- 17 August 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 34 (6) , 2771-2782
Hypergraph Partitioning With Embeddings
Problems in scientific computing, such as distributing large sparse matrix operations, have analogous formulations as hypergraph partitioning problems. A hypergraph is a generalization of a traditional graph wherein "hyperedges" may connect any number of nodes. As a result, hypergraph partitioning is an NP-Hard problem to both solve or approximate. State-of-the-art algorithms that solve this problem follow the multilevel paradigm, which begins by iteratively "coarsening" the input hypergraph to smaller problem instances that share key structural features. Once identifying an approximate problem that is small enough to be solved directly, that solution can be interpolated and refined to the original problem. While this strategy represents an excellent trade off between quality and running time, it is sensitive to coarsening strategy. In this work we propose using graph embeddings of the initial hypergraph in order to ensure that coarsened problem instances retrain key structural features. Our approach prioritizes coarsening within self-similar regions within the input graph, and leads to significantly improved solution quality across a range of considered hypergraphs. Reproducibility: All source code, plots and experimental data are available at https://sybrandt.com/2019/partition.
- Open Access
Unsupervised Hierarchical Graph Representation Learning by Mutual Information Maximization
Graph representation learning based on graph neural networks (GNNs) can greatly improve the performance of downstream tasks, such as node and graph classification. However, the general GNN models do not aggregate node information in a hierarchical manner, and can miss key higher-order structural features of many graphs. The hierarchical aggregation also enables the graph representations to be explainable. In addition, supervised graph representation learning requires labeled data, which is expensive and error-prone. To address these issues, we present an unsupervised graph representation learning method, Unsupervised Hierarchical Graph Representation (UHGR), which can generate hierarchical representations of graphs. Our method focuses on maximizing mutual information between "local" and high-level "global" representations, which enables us to learn the node embeddings and graph embeddings without any labeled data. To demonstrate the effectiveness of the proposed method, we perform the node and graph classification using the learned node and graph embeddings. The results show that the proposed method achieves comparable results to state-of-the-art supervised methods on several benchmarks. In addition, our visualization of hierarchical representations indicates that our method can capture meaningful and interpretable clusters.
Open AccessUnsupervised Hierarchical Graph Representation Learning by Mutual Information Maximization
Graph representation learning based on graph neural networks (GNNs) can greatly improve the performance of downstream tasks, such as node and graph classification. However, the general GNN models do not aggregate node information in a hierarchical manner, and can miss key higher-order structural features of many graphs. The hierarchical aggregation also enables the graph representations to be explainable. In addition, supervised graph representation learning requires labeled data, which is expensive and error-prone. To address these issues, we present an unsupervised graph representation learning method, Unsupervised Hierarchical Graph Representation (UHGR), which can generate hierarchical representations of graphs. Our method focuses on maximizing mutual information between "local" and high-level "global" representations, which enables us to learn the node embeddings and graph embeddings without any labeled data. To demonstrate the effectiveness of the proposed method, we perform the node and graph classification using the learned node and graph embeddings. The results show that the proposed method achieves comparable results to state-of-the-art supervised methods on several benchmarks. In addition, our visualization of hierarchical representations indicates that our method can capture meaningful and interpretable clusters.
- Top Cited Papers
- 1 March 2020
- Published by Elsevier in Journal of Sound and Vibration
- Vol. 468
Using drive-by health monitoring to detect bridge damage considering environmental and operational effects
- 1 March 2020
- Published by Elsevier in Journal of Sound and Vibration
- Vol. 468
Top Cited PapersUsing drive-by health monitoring to detect bridge damage considering environmental and operational effects
- Open Access
CBAG: Conditional Biomedical Abstract Generation
Biomedical research papers use significantly different language and jargon when compared to typical English text, which reduces the utility of pre-trained NLP models in this domain. Meanwhile Medline, a database of biomedical abstracts, introduces nearly a million new documents per-year. Applications that could benefit from understanding this wealth of publicly available information, such as scientific writing assistants, chat-bots, or descriptive hypothesis generation systems, require new domain-centered approaches. A conditional language model, one that learns the probability of words given some a priori criteria, is a fundamental building block in many such applications. We propose a transformer-based conditional language model with a shallow encoder "condition" stack, and a deep "language model" stack of multi-headed attention blocks. The condition stack encodes metadata used to alter the output probability distribution of the language model stack. We sample this distribution in order to generate biomedical abstracts given only a proposed title, an intended publication year, and a set of keywords. Using typical natural language generation metrics, we demonstrate that this proposed approach is more capable of producing non-trivial relevant entities within the abstract body than the 1.5B parameter GPT-2 language model.
Open AccessCBAG: Conditional Biomedical Abstract Generation
Biomedical research papers use significantly different language and jargon when compared to typical English text, which reduces the utility of pre-trained NLP models in this domain. Meanwhile Medline, a database of biomedical abstracts, introduces nearly a million new documents per-year. Applications that could benefit from understanding this wealth of publicly available information, such as scientific writing assistants, chat-bots, or descriptive hypothesis generation systems, require new domain-centered approaches. A conditional language model, one that learns the probability of words given some a priori criteria, is a fundamental building block in many such applications. We propose a transformer-based conditional language model with a shallow encoder "condition" stack, and a deep "language model" stack of multi-headed attention blocks. The condition stack encodes metadata used to alter the output probability distribution of the language model stack. We sample this distribution in order to generate biomedical abstracts given only a proposed title, an intended publication year, and a set of keywords. Using typical natural language generation metrics, we demonstrate that this proposed approach is more capable of producing non-trivial relevant entities within the abstract body than the 1.5B parameter GPT-2 language model.
- Open Access
AGATHA: Automatic Graph-mining And Transformer based Hypothesis generation Approach
Medical research is risky and expensive. Drug discovery, as an example, requires that researchers efficiently winnow thousands of potential targets to a small candidate set for more thorough evaluation. However, research groups spend significant time and money to perform the experiments necessary to determine this candidate set long before seeing intermediate results. Hypothesis generation systems address this challenge by mining the wealth of publicly available scientific information to predict plausible research directions. We present AGATHA, a deep-learning hypothesis generation system that can introduce data-driven insights earlier in the discovery process. Through a learned ranking criteria, this system quickly prioritizes plausible term-pairs among entity sets, allowing us to recommend new research directions. We massively validate our system with a temporal holdout wherein we predict connections first introduced after 2015 using data published beforehand. We additionally explore biomedical sub-domains, and demonstrate AGATHA's predictive capacity across the twenty most popular relationship types. This system achieves best-in-class performance on an established benchmark, and demonstrates high recommendation scores across subdomains. Reproducibility: All code, experimental data, and pre-trained models are available online: sybrandt.com/2020/agatha
Open AccessAGATHA: Automatic Graph-mining And Transformer based Hypothesis generation Approach
Medical research is risky and expensive. Drug discovery, as an example, requires that researchers efficiently winnow thousands of potential targets to a small candidate set for more thorough evaluation. However, research groups spend significant time and money to perform the experiments necessary to determine this candidate set long before seeing intermediate results. Hypothesis generation systems address this challenge by mining the wealth of publicly available scientific information to predict plausible research directions. We present AGATHA, a deep-learning hypothesis generation system that can introduce data-driven insights earlier in the discovery process. Through a learned ranking criteria, this system quickly prioritizes plausible term-pairs among entity sets, allowing us to recommend new research directions. We massively validate our system with a temporal holdout wherein we predict connections first introduced after 2015 using data published beforehand. We additionally explore biomedical sub-domains, and demonstrate AGATHA's predictive capacity across the twenty most popular relationship types. This system achieves best-in-class performance on an established benchmark, and demonstrates high recommendation scores across subdomains. Reproducibility: All code, experimental data, and pre-trained models are available online: sybrandt.com/2020/agatha
- Open Access
- 4 December 2019
- Published by Springer Nature in Journal of Neuroimmune Pharmacology
- Vol. 15 (2) , 209-223
Inhibition of the Dead Box RNA Helicase 3 Prevents HIV-1 Tat and Cocaine-Induced Neurotoxicity by Targeting Microglia Activation
HIV-1 Associated Neurocognitive Disorder (HAND) is a common and clinically detrimental complication of HIV infection. Viral proteins, including Tat, released from infected cells, cause neuronal toxicity. Substance abuse in HIV-infected patients greatly influences the severity of neuronal damage. To repurpose small molecule inhibitors for anti-HAND therapy, we employed MOLIERE, an AI-based literature mining system that we developed. All human genes were analyzed and prioritized by MOLIERE to find previously unknown targets connected to HAND. From the identified high priority genes, we narrowed the list to those with known small molecule ligands developed for other applications and lacking systemic toxicity in animal models. To validate the AI-based process, the selective small molecule inhibitor of DDX3 helicase activity, RK-33, was chosen and tested for neuroprotective activity. The compound, previously developed for cancer treatment, was tested for the prevention of combined neurotoxicity of HIV Tat and cocaine. Rodent cortical cultures were treated with 6 or 60 ng/ml of HIV Tat and 10 or 25 μM of cocaine, which caused substantial toxicity. RK-33 at doses as low as 1 μM greatly reduced the neurotoxicity of Tat and cocaine. Transcriptome analysis showed that most Tat-activated transcripts are microglia-specific genes and that RK-33 blocks their activation. Treatment with RK-33 inhibits the Tat and cocaine-dependent increase in the number and size of microglia and the proinflammatory cytokines IL-6, TNF-α, MCP-1/CCL2, MIP-2, IL-1α and IL-1β. These findings reveal that inhibition of DDX3 may have the potential to treat not only HAND but other neurodegenerative diseases. RK-33, selective inhibitor of Dead Box RNA helicase 3 (DDX3) protects neurons from combined Tat and cocaine neurotoxicity by inhibition of microglia activation and production of proinflammatory cytokines.
- 4 December 2019
- Published by Springer Nature in Journal of Neuroimmune Pharmacology
- Vol. 15 (2) , 209-223
Open AccessInhibition of the Dead Box RNA Helicase 3 Prevents HIV-1 Tat and Cocaine-Induced Neurotoxicity by Targeting Microglia Activation
HIV-1 Associated Neurocognitive Disorder (HAND) is a common and clinically detrimental complication of HIV infection. Viral proteins, including Tat, released from infected cells, cause neuronal toxicity. Substance abuse in HIV-infected patients greatly influences the severity of neuronal damage. To repurpose small molecule inhibitors for anti-HAND therapy, we employed MOLIERE, an AI-based literature mining system that we developed. All human genes were analyzed and prioritized by MOLIERE to find previously unknown targets connected to HAND. From the identified high priority genes, we narrowed the list to those with known small molecule ligands developed for other applications and lacking systemic toxicity in animal models. To validate the AI-based process, the selective small molecule inhibitor of DDX3 helicase activity, RK-33, was chosen and tested for neuroprotective activity. The compound, previously developed for cancer treatment, was tested for the prevention of combined neurotoxicity of HIV Tat and cocaine. Rodent cortical cultures were treated with 6 or 60 ng/ml of HIV Tat and 10 or 25 μM of cocaine, which caused substantial toxicity. RK-33 at doses as low as 1 μM greatly reduced the neurotoxicity of Tat and cocaine. Transcriptome analysis showed that most Tat-activated transcripts are microglia-specific genes and that RK-33 blocks their activation. Treatment with RK-33 inhibits the Tat and cocaine-dependent increase in the number and size of microglia and the proinflammatory cytokines IL-6, TNF-α, MCP-1/CCL2, MIP-2, IL-1α and IL-1β. These findings reveal that inhibition of DDX3 may have the potential to treat not only HAND but other neurodegenerative diseases. RK-33, selective inhibitor of Dead Box RNA helicase 3 (DDX3) protects neurons from combined Tat and cocaine neurotoxicity by inhibition of microglia activation and production of proinflammatory cytokines.
- 24 January 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 2018, 1494-1503
Large-Scale Validation of Hypothesis Generation Systems via Candidate Ranking
The first step of many research projects is to define and rank a short list of candidates for study. In the modern rapidity of scientific progress, some turn to automated hypothesis generation (HG) systems to aid this process. These systems can identify implicit or overlooked connections within a large scientific corpus, and while their importance grows alongside the pace of science, they lack thorough validation. Without any standard numerical evaluation method, many validate general-purpose HG systems by rediscovering a handful of historical findings, and some wishing to be more thorough may run laboratory experiments based on automatic suggestions. These methods are expensive, time consuming, and cannot scale. Thus, we present a numerical evaluation framework for the purpose of validating HG systems that leverages thousands of validation hypotheses. This method evaluates a HG system by its ability to rank hypotheses by plausibility; a process reminiscent of human candidate selection. Because HG systems do not produce a ranking criteria, specifically those that produce topic models, we additionally present novel metrics to quantify the plausibility of hypotheses given topic model system output. Finally, we demonstrate that our proposed validation method aligns with real-world research goals by deploying our method within MOLIERE, our recent topic-driven HG system, in order to automatically generate a set of candidate genes related to HIV-associated neurodegenerative disease (HAND). By performing laboratory experiments based on this candidate set, we discover a new connection between HAND and Dead Box RNA Helicase 3 (DDX3).Reproducibility: code, validation data, and results can be found at sybrandt.com/2018/validation.
- 24 January 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 2018, 1494-1503
Large-Scale Validation of Hypothesis Generation Systems via Candidate Ranking
The first step of many research projects is to define and rank a short list of candidates for study. In the modern rapidity of scientific progress, some turn to automated hypothesis generation (HG) systems to aid this process. These systems can identify implicit or overlooked connections within a large scientific corpus, and while their importance grows alongside the pace of science, they lack thorough validation. Without any standard numerical evaluation method, many validate general-purpose HG systems by rediscovering a handful of historical findings, and some wishing to be more thorough may run laboratory experiments based on automatic suggestions. These methods are expensive, time consuming, and cannot scale. Thus, we present a numerical evaluation framework for the purpose of validating HG systems that leverages thousands of validation hypotheses. This method evaluates a HG system by its ability to rank hypotheses by plausibility; a process reminiscent of human candidate selection. Because HG systems do not produce a ranking criteria, specifically those that produce topic models, we additionally present novel metrics to quantify the plausibility of hypotheses given topic model system output. Finally, we demonstrate that our proposed validation method aligns with real-world research goals by deploying our method within MOLIERE, our recent topic-driven HG system, in order to automatically generate a set of candidate genes related to HIV-associated neurodegenerative disease (HAND). By performing laboratory experiments based on this candidate set, we discover a new connection between HAND and Dead Box RNA Helicase 3 (DDX3).Reproducibility: code, validation data, and results can be found at sybrandt.com/2018/validation.