We present an attempt to measure the large angular scale fluctuations in the X-Ray Background (XRB) from the HEAO1 A2 data, expressed in terms of spherical harmonics. We model the harmonic coefficients assuming a power spectrum and an epoch-dependent bias parameter, and using a phenomenological scenario describing the evolution of the X-ray sources. From the few low-order multipoles detected (l<10), we estimate the power-spectrum normalisation on scales intermediate between those explored by local galaxy redshift surveys (~ 100 Mpc) and by the COBE Microwave Background measurements (~ 1000 Mpc). We find that the HEAO1 harmonics are consistent with present epoch rms fluctuations of the X-ray sources bx(0)sigma8 ~ 1-2 in 8 Mpc spheres. Therefore the observed fluctuations in the XRB are roughly as expected from just interpolating between the fluctuations in local galaxy surveys and the COBE CMB. We predict that an X-ray whole sky surface brightness survey which resolves sources a factor of 10 fainter than HEAO1, would allow us to measure the large scale fluctuations of the XRB to order l ~ 20 (on angular scale theta ~ pi/20), and therefore more strongly constrain the large scale structure of the Universe on scales of hundreds of Mpcs.