EBEtienne Barnard

Etienne Barnard

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Etienne Barnard has a total of 103 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Hidden Markov Model Decoding for LDPC Codes

    by Jan C Olivier, Etienne Barnard

    The paper proposes an iterative Hidden Markov Model (HMM) for decoding a Low Density Parity Check (LDPC) code. It is demonstrated that a first-order HMM provides a natural framework for the decoder. The HMM is time-homogeneous with a fixed transition matrix and is based on a random walk through the encoded frame bits. Each hidden state contains a pair of two encoded bits, and parity checks are naturally incorporated into the observation model. The paper shows that by implementing a forward-backward smoothing estimator for the hidden states, decoding is efficient and requires only a small number of iterations in most cases. The results show that the LDPC decoding threshold is significantly improved compared to belief propagation (BP) on a Tanner graph. Numerical results are presented showing that LDPC codes under the proposed decoder yield a frame error rate (FER) and decoding threshold comparable to that of a Polar code where Successive Cancellation List (SCL) - Cyclic Redundancy Check (CRC) decoding is deployed. This is shown to be achieved even if the frame length is short (on the order of $512$ bits or less) and a regular LDPC code is used. 1

  • Open Access

    Minimum Phase Linear Antenna Array Design

    by Jan C Olivier, Etienne Barnard

    The paper considers the design of minimum phase discrete linear arrays. The paper introduces recent advances for the design of minimum phase Finite Impulse Response filters, as applied to the design of minimum phase linear arrays. The minimum phase linear array is demonstrated to require the least number of elements of all linear arrays that are able to achieve a given magnitude pattern specification. Three example designs are presented and compared to results from the literature.

  • Open Access

    Manifold Characteristics That Predict Downstream Task Performance

    by Ruan van der Merwe, Gregory Newman, Etienne Barnard

    Pretraining methods are typically compared by evaluating the accuracy of linear classifiers, transfer learning performance, or visually inspecting the representation manifold's (RM) lower-dimensional projections. We show that the differences between methods can be understood more clearly by investigating the RM directly, which allows for a more detailed comparison. To this end, we propose a framework and new metric to measure and compare different RMs. We also investigate and report on the RM characteristics for various pretraining methods. These characteristics are measured by applying sequentially larger local alterations to the input data, using white noise injections and Projected Gradient Descent (PGD) adversarial attacks, and then tracking each datapoint. We calculate the total distance moved for each datapoint and the relative change in distance between successive alterations. We show that self-supervised methods learn an RM where alterations lead to large but constant size changes, indicating a smoother RM than fully supervised methods. We then combine these measurements into one metric, the Representation Manifold Quality Metric (RMQM), where larger values indicate larger and less variable step sizes, and show that RMQM correlates positively with performance on downstream tasks.

  • Open Access

    Benign interpolation of noise in deep learning

    by Marthinus Wilhelmus Theunissen, Marelie Davel, Etienne Barnard

    The understanding of generalisation in machine learning is in a state of flux, in part due to the ability of deep learning models to interpolate noisy training data and still perform appropriately on out-of-sample data, thereby contradicting long-held intuitions about the bias-variance tradeoff in learning. We expand upon relevant existing work by discussing local attributes of neural network training within the context of a relatively simple framework. We describe how various types of noise can be compensated for within the proposed framework in order to allow the deep learning model to generalise in spite of interpolating spurious function descriptors. Empirically, we support our postulates with experiments involving overparameterised multilayer perceptrons and controlled training data noise. The main insights are that deep learning models are optimised for training data modularly, with different regions in the function space dedicated to fitting distinct types of sample information. Additionally, we show that models tend to fit uncorrupted samples first. Based on this finding, we propose a conjecture to explain an observed instance of the epoch-wise double-descent phenomenon. Our findings suggest that the notion of model capacity needs to be modified to consider the distributed way training data is fitted across sub-units.


  • Open Access

    DNNs as Layers of Cooperating Classifiers

    by Marelie Davel, Marthinus Theunissen, Arnold Pretorius, Etienne Barnard

    A robust theoretical framework that can describe and predict the generalization ability of DNNs in general circumstances remains elusive. Classical attempts have produced complexity metrics that rely heavily on global measures of compactness and capacity with little investigation into the effects of sub-component collaboration. We demonstrate intriguing regularities in the activation patterns of the hidden nodes within fully-connected feedforward networks. By tracing the origin of these patterns, we show how such networks can be viewed as the combination of two information processing systems: one continuous and one discrete. We describe how these two systems arise naturally from the gradient-based optimization process, and demonstrate the classification ability of the two systems, individually and in collaboration. This perspective on DNN classification offers a novel way to think about generalization, in which different subsets of the training data are used to train distinct classifiers; those classifiers are then combined to perform the classification task, and their consistency is crucial for accurate classification.


  • Open Access

    DNNs as Layers of Cooperating Classifiers

    by Marelie H. Davel, Marthinus W. Theunissen, Arnold M. Pretorius, Etienne Barnard

    A robust theoretical framework that can describe and predict the generalization ability of deep neural networks (DNNs) in general circumstances remains elusive. Classical attempts have produced complexity metrics that rely heavily on global measures of compactness and capacity with little investigation into the effects of sub-component collaboration. We demonstrate intriguing regularities in the activation patterns of the hidden nodes within fully-connected feedforward networks. By tracing the origin of these patterns, we show how such networks can be viewed as the combination of two information processing systems: one continuous and one discrete. We describe how these two systems arise naturally from the gradient-based optimization process, and demonstrate the classification ability of the two systems, individually and in collaboration. This perspective on DNN classification offers a novel way to think about generalization, in which different subsets of the training data are used to train distinct classifiers; those classifiers are then combined to perform the classification task, and their consistency is crucial for accurate classification.

  • Efficient data selection for ASR

    by Neil Taylor Kleynhans, Etienne Barnard


  • Language independent search in MediaEval's Spoken Web Search task

    by Florian Metze, Xavier Anguera, Etienne Barnard, Marelie Davel, Guillaume Gravier


  • Introduction to the special issue on processing under-resourced languages

    by Laurent Besacier, Etienne Barnard, Alexey Karpov, Tanja Schultz


  • A smartphone-based ASR data collection tool for under-resourced languages

    by Nic J. de Vries, Marelie H. Davel, Jaco Badenhorst, Willem D. Basson, Febe de Wet, Etienne Barnard, Alta de Waal