The refractive index varies slightly with wavelength which is why we have the phenomena of chromatic aberration. The formula of refractive index is as follows:

Where, n is the index of refraction, c is the speed of light in vacuum and v is the speed of light in the medium.
The refractive index of x-rays is slightly less than 1 and hence less than that of visible light. Thus, the speed of X-rays is more in glass (and in other materials) than in vacuum.