JHJiaoyang Huang

Jiaoyang Huang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Jiaoyang Huang has a total of 10 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    $β$-Nonintersecting Poisson Random Walks: Law of Large Numbers and Central Limit Theorems

    by Jiaoyang Huang

    We study the $\beta$ analogue of the nonintersecting Poisson random walks. We derive a stochastic differential equation of the Stieltjes transform of the empirical measure process, which can be viewed as a dynamical version of the Nekrasov's equation in [7, Section 4]. We find that the empirical measure process converges weakly in the space of c\'adl\'ag measure-valued processes to a deterministic process, characterized by the quantized free convolution, as introduced in [11]. For suitable initial data, we prove that the rescaled empirical measure process converges weakly in the space of distributions acting on analytic test functions to a Gaussian process. The means and the covariances are universal, and coincide with those of $\beta$-Dyson Brownian motions with the initial data constructed by the Markov-Krein correspondence. Especially, the covariance structure can be described in terms of the Gaussian Free Field. Our proof relies on integrable features of the generators of the $\beta$-nonintersecting Poisson random walks, the method of characteristics, and a coupling technique for Poisson random walks.

  • Open Access

    Gradient Descent Finds Global Minima for Generalizable Deep Neural Networks of Practical Sizes

    by Kenji Kawaguchi, Jiaoyang Huang

    In this paper, we theoretically prove that gradient descent can find a global minimum of non-convex optimization of all layers for nonlinear deep neural networks of sizes commonly encountered in practice. The theory developed in this paper only requires the practical degrees of over-parameterization unlike previous theories. Our theory only requires the number of trainable parameters to increase linearly as the number of training samples increases. This allows the size of the deep neural networks to be consistent with practice and to be several orders of magnitude smaller than that required by the previous theories. Moreover, we prove that the linear increase of the size of the network is the optimal rate and that it cannot be improved, except by a logarithmic factor. Furthermore, deep neural networks with the trainability guarantee are shown to generalize well to unseen test samples with a natural dataset but not a random dataset.

  • Open Access

    Every Local Minimum Value Is the Global Minimum Value of Induced Model in Nonconvex Machine Learning

    by Kenji Kawaguchi, Jiaoyang Huang, Leslie Pack Kaelbling

    For nonconvex optimization in machine learning, this article proves that every local minimum achieves the globally optimal value of the perturbable gradient basis model at any differentiable point. As a result, nonconvex machine learning is theoretically as supported as convex machine learning with a handcrafted basis in terms of the loss at differentiable local minima, except in the case when a preference is given to the handcrafted basis over the perturbable gradient basis. The proofs of these results are derived under mild assumptions. Accordingly, the proven results are directly applicable to many machine learning models, including practical deep neural networks, without any modification of practical methods. Furthermore, as special cases of our general results, this article improves or complements several state-of-the-art theoretical results on deep neural networks, deep residual networks, and overparameterized deep neural networks with a unified proof technique and novel geometric insights. A special case of our results also contributes to the theoretical foundation of representation learning.


  • Open Access

    Eigenvalues for the Minors of Wigner Matrices

    by Jiaoyang Huang

    The eigenvalues for the minors of real symmetric ($\beta=1$) and complex Hermitian ($\beta=2$) Wigner matrices form the Wigner corner process, which is a multilevel interlacing particle system. In this paper, we study the microscopic scaling limit of the Wigner corner process both near the spectral edge and in the bulk, and prove they are universal. We show: (i) Near the spectral edge, the corner process exhibit a decoupling phenomenon, as first observed in [24]. Individual extreme particles have Tracy-Widom$_{\beta}$ distribution; the spacings between the extremal particles on adjacent levels converge to independent Gamma distributions in a much smaller scale. (ii) In the bulk, the microscopic scaling limit of the Wigner corner process is given by the bead process for general Sine$_\beta$ process, as constructed recently in [34].

  • Open Access

    Effect of Depth and Width on Local Minima in Deep Learning

    by Kenji Kawaguchi, Jiaoyang Huang, Leslie Pack Kaelbling

    In this paper, we analyze the effects of depth and width on the quality of local minima, without strong overparameterization and simplification assumptions in the literature. Without any simplification assumption, for deep nonlinear neural networks with the squared loss, we theoretically show that the quality of local minima tends to improve toward the global minimum value as depth and width increase. Furthermore, with a locally induced structure on deep nonlinear neural networks, the values of local minima of neural networks are theoretically proven to be no worse than the globally optimal values of corresponding classical machine learning models. We empirically support our theoretical observation with a synthetic data set, as well as MNIST, CIFAR-10, and SVHN data sets. When compared to previous studies with strong overparameterization assumptions, the results in this letter do not require overparameterization and instead show the gradual effects of overparameterization as consequences of general results.


  • Top Cited Papers
    Open Access

    Local Kesten–McKay Law for Random Regular Graphs

    by Roland Bauerschmidt, Jiaoyang Huang, Horng-Tzer Yau


  • β-Nonintersecting Poisson Random Walks: Law of Large Numbers and Central Limit Theorems

    by Jiaoyang Huang

    We study the $\beta $ analogue of the nonintersecting Poisson random walks. We derive a stochastic differential equation of the Stieltjes transform of the empirical measure process, which can be viewed as a dynamical version of the Nekrasov’s equation in [7, Section 4]. We find that the empirical measure process converges weakly in the space of cádlág measure-valued processes to a deterministic process, characterized by the quantized free convolution, as introduced in [11]. For suitable initial data, we prove that the rescaled empirical measure process converges weakly in the space of distributions acting on analytic test functions to a Gaussian process. The means and the covariances coincide with those of $\beta $-Dyson Brownian motions with the initial data constructed by the Markov–Krein correspondence. Especially, the covariance structure can be described in terms of the Gaussian free field. Our proof relies on integrable features of the generators of the $\beta $-nonintersecting Poisson random walks, the method of characteristics, and a coupling technique for Poisson random walks.


  • Rigidity and Edge Universality of Discrete β‐Ensembles

    by Alice Guionnet, Jiaoyang Huang

    We study discrete β‐ensembles as introduced in [17]. We obtain rigidity estimates on the particle locations; i.e., with high probability, the particles are close to their classical locations with an optimal error estimate. We prove the edge universality of the discrete β‐ensembles; i.e., for β ≥ 1, the distribution of extreme particles converges to the Tracy‐Widom β‐distribution. As far as we know, this is the first proof of general Tracy‐Widom β‐distributions in the discrete setting. A special case of our main results implies that under the Jack deformation of the Plancherel measure, the distribution of the lengths of the first few rows in Young diagrams converges to the Tracy‐Widom β‐distribution, which answers an open problem in [38]. Our proof relies on Nekrasov's (or loop) equations, a multiscale analysis, and a comparison argument with continuous β‐ensembles.


  • Open Access

    Invertibility of adjacency matrices for random $d$-regular graphs

    by Jiaoyang Huang

    Let $d\geq 3$ be a fixed integer and $A$ be the adjacency matrix of a random $d$-regular directed or undirected graph on $n$ vertices. We show there exist constants $\mathfrak d>0$, \begin{align*} {\mathbb P}(\text{$A$ is singular in $\mathbb R$})\leq n^{-\mathfrak{d}}, \end{align*} for $n$ sufficiently large. This answers an open problem by Frieze [12] and Vu [28,19]. The key idea is to study the singularity probability of adjacency matrices over a finite field ${\mathbb F}_p$. The proof combines a local central limit theorem and a large deviation estimate.