Plug-and-Play Controllable Generation for Discrete Masked Models
Preprint
Abstract
This article makes discrete masked models for the generative modeling of discrete data controllable. The goal is to generate samples of a discrete random variable that adheres to a posterior distribution, satisfies specific constraints, or optimizes a reward function. This methodological development enables broad applications across downstream tasks such as class-specific image generation and protein design. Existing approaches for controllable generation of masked models typically rely on task-specific fine-tuning or additional modifications, which can be inefficient and resource-intensive. To overcome these limitations, we propose a novel plug-and-play framework based on importance sampling that bypasses the need for training a conditional score. Our framework is agnostic to the choice of control criteria, requires no gradient information, and is well-suited for tasks such as posterior sampling, Bayesian inverse problems, and constrained generation. We demonstrate the effectiveness of our approach through extensive experiments, showcasing its versatility across multiple domains, including protein design.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- De novo protein design—From new structures to programmable functionsCell, 2024
- Design of complicated all-α protein structuresNature Structural & Molecular Biology, 2024
- Protein Design: From the Aspect of Water Solubility and StabilityChemical Reviews, 2022
- Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequencesProceedings of the National Academy of Sciences, 2021
- DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genomeBioinformatics, 2021
- G protein-coupled receptors: structure- and function-based drug discoverySignal Transduction and Targeted Therapy, 2021
- Post-translational modifications in the context of therapeutic proteinsNature Biotechnology, 2006
- Fabrication of novel biomaterials through molecular self-assemblyNature Biotechnology, 2003
- Correlation between stability of a protein and its dipeptide composition: a novel approach for predicting in vivo stability of a protein from its primary sequenceProtein Engineering, Design and Selection, 1990
- A simple method for displaying the hydropathic character of a proteinJournal of Molecular Biology, 1982