There were in Litlington some strips of unenclosed but cultivated land bordering on the north side of Ashwell Street, part of the Icknield Way.
In 1821, some workmen digging for gravel came upon a wall of flint and brick. Excavations in the following year uncovered several pottery urns containing ashes of the dead, which showed the site to have been a Roman walled cemetery of the type known as an ustrinum.
According to G. L. Gomme in Folklore as an Historical Science (1908), the place had been known since “time immemorial” as ‘Heaven’s Walls’. The village children were afraid to go near it after dark because it was said to be haunted. Gomme gives this as an example of folk memory; more likely, the name reflected earlier finds of funerary urns, and the tradition of the haunting followed.