Constraining the Nature of the Most Distant Gamma-Ray Burst Host Galaxies