TMTomas Mikolov

Tomas Mikolov

*This is an algorithmically generated author record based on publicly available information.

Tomas Mikolov has a total of 57 co-authors. Here are the top co-authors by number of shared publications:

  • Armand Joulin

    Facebook AI Research Ecole Normale Supérieure Facebook AI Research Facebook AI Research Facebook AI Research Facebook AI Research Facebook AI Research PSL University Inria Mathild Université Grenoble Alpes Inria University College London

    12 shared publications
  • Edouard Grave

    Facebook AI Research Ecole Normale Supérieure Facebook AI Research Facebook AI Research Facebook AI Research Facebook AI Research Facebook AI Research PSL University Inria Mathild Université Grenoble Alpes Inria University College London

    8 shared publications
  • Piotr Bojanowski

    Facebook AI Research Ecole Normale Supérieure Facebook AI Research Facebook AI Research Facebook AI Research Facebook AI Research Facebook AI Research PSL University Inria Mathild Université Grenoble Alpes Inria University College London

    7 shared publications
  • Armand Joulin

    Facebook AI Research

    3 shared publications
  • Sumit Chopra

    Facebook AI Research

    2 shared publications
  • Aaron Sloman

    School of Computer Science , University of Birmingham , Edgbaston, Birmingham , UK

    1 shared publication

Latest Publications

  • Open Access

    Evolving Structures in Complex Systems

    by Hugo Cisneros, Josef Sivic, Tomas Mikolov

    In this paper we propose an approach for measuring growth of complexity of emerging patterns in complex systems such as cellular automata. We discuss several ways how a metric for measuring the complexity growth can be defined. This includes approaches based on compression algorithms and artificial neural networks. We believe such a metric can be useful for designing systems that could exhibit open-ended evolution, which itself might be a prerequisite for development of general artificial intelligence. We conduct experiments on 1D and 2D grid worlds and demonstrate that using the proposed metric we can automatically construct computational models with emerging properties similar to those found in the Conway's Game of Life, as well as many other emergent phenomena. Interestingly, some of the patterns we observe resemble forms of artificial life. Our metric of structural complexity growth can be applied to a wide range of complex systems, as it is not limited to cellular automata.

  • Open Access

    Special Issue “On Defining Artificial Intelligence”—Commentaries and Author’s Response

    by Dagmar Monett, Colin W. P. Lewis, Kristinn R. Thórisson, Joscha Bach, Gianluca Baldassarre, Giovanni Granato, Istvan S. N. Berkeley, François Chollet, Matthew Crosby, Henry Shevlin, et al.

    Sciendo provides publishing services and solutions to academic and professional organizations and individual authors. We publish journals, books, conference proceedings and a variety of other publications.


  • Open Access

    Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment

    by Piotr Bojanowski, Onur Celebi, Tomas Mikolov, Edouard Grave, Armand Joulin

    In this paper, we focus on the problem of adapting word vector-based models to new textual data. Given a model pre-trained on large reference data, how can we adapt it to a smaller piece of data with a slightly different language distribution? We frame the adaptation problem as a monolingual word vector alignment problem, and simply average models after alignment. We align vectors using the RCSLS criterion. Our formulation results in a simple and efficient algorithm that allows adapting general-purpose models to changing word distributions. In our evaluation, we consider applications to word embedding and text classification models. We show that the proposed approach yields good performance in all setups and outperforms a baseline consisting in fine-tuning the model on new data.

  • Open Access

    Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion

    by Armand Joulin, Piotr Bojanowski, Tomas Mikolov, Herve Jegou, Edouard Grave

    Continuous word representations learned separately on distinct languages can be aligned so that their words become comparable in a common space. Existing works typically solve a least-square regression problem to learn a rotation aligning a small bilingual lexicon, and use a retrieval criterion for inference. In this paper, we propose an unified formulation that directly optimizes a retrieval criterion in an end-to-end fashion. Our experiments on standard benchmarks show that our approach outperforms the state of the art on word translation, with the biggest improvements observed for distant language pairs such as English-Chinese.

  • Open Access

    Learning Word Vectors for 157 Languages

    by Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, Tomas Mikolov

    Distributed word representations, or word vectors, have recently been applied to many tasks in natural language processing, leading to state-of-the-art performance. A key ingredient to the successful application of these representations is to train them on very large corpora, and use these pre-trained models in downstream tasks. In this paper, we describe how we trained such high quality word representations for 157 languages. We used two sources of data to train these models: the free online encyclopedia Wikipedia and data from the common crawl project. We also introduce three new word analogy datasets to evaluate these word vectors, for French, Hindi and Polish. Finally, we evaluate our pre-trained word vectors on 10 languages for which evaluation datasets exists, showing very strong performance compared to previous models.


  • A Roadmap Towards Machine Intelligence

    by Tomas Mikolov, Armand Joulin, Marco Baroni


  • Open Access

    Efficient Large-Scale Multi-Modal Classification

    by D. Kiela, E. Grave, A. Joulin, T. Mikolov

    While the incipient internet was largely text-based, the modern digital world is becoming increasingly multi-modal. Here, we examine multi-modal classification where one modality is discrete, e.g. text, and the other is continuous, e.g. visual representations transferred from a convolutional neural network. In particular, we focus on scenarios where we have to be able to classify large quantities of data quickly. We investigate various methods for performing multi-modal fusion and analyze their trade-offs in terms of classification accuracy and computational efficiency. Our findings indicate that the inclusion of continuous information improves performance over text-only on a range of multi-modal classification tasks, even with simple fusion methods. In addition, we experiment with discretizing the continuous features in order to speed up and simplify the fusion process even further. Our results show that fusion with discretized features outperforms text-only classification, at a fraction of the computational cost of full multi-modal fusion, with the additional benefit of improved interpretability.


  • Open Access

    Advances in Pre-Training Distributed Word Representations

    by Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, Armand Joulin

    Many Natural Language Processing applications nowadays rely on pre-trained word representations estimated from large text corpora such as news collections, Wikipedia and Web Crawl. In this paper, we show how to train high-quality word vector representations by using a combination of known tricks that are however rarely used together. The main result of our work is the new set of publicly available pre-trained models that outperform the current state of the art by a large margin on a number of tasks.


  • Learning Simpler Language Models with the Differential State Framework

    by Alexander G. Ororbia Ii, Tomas Mikolov, David Reitter

    Learning useful information across long time lags is a critical and difficult problem for temporal neural models in tasks such as language modeling. Existing architectures that address the issue are often complex and costly to train. The differential state framework (DSF) is a simple and high-performing design that unifies previously introduced gated neural models. DSF models maintain longer-term memory by learning to interpolate between a fast-changing data-driven representation and a slowly changing, implicitly stable state. Within the DSF framework, a new architecture is presented, the delta-RNN. This model requires hardly any more parameters than a classical, simple recurrent network. In language modeling at the word and character levels, the delta-RNN outperforms popular complex architectures, such as the long short-term memory (LSTM) and the gated recurrent unit (GRU), and, when regularized, performs comparably to several state-of-the-art baselines. At the subword level, the delta-RNN's performance is comparable to that of complex gated architectures.


  • Open Access

    Enriching Word Vectors with Subword Information

    by Piotr Bojanowski, Edouard Grave, Armand Joulin, Tomas Mikolov

    Continuous word representations, trained on large unlabeled corpora are useful for many natural language processing tasks. Popular models that learn such representations ignore the morphology of words, by assigning a distinct vector to each word. This is a limitation, especially for languages with large vocabularies and many rare words. In this paper, we propose a new approach based on the skipgram model, where each word is represented as a bag of character $n$-grams. A vector representation is associated to each character $n$-gram; words being represented as the sum of these representations. Our method is fast, allowing to train models on large corpora quickly and allows us to compute word representations for words that did not appear in the training data. We evaluate our word representations on nine different languages, both on word similarity and analogy tasks. By comparing to recently proposed morphological word representations, we show that our vectors achieve state-of-the-art performance on these tasks.