Apparently the field that this stone sits in is just as important to the folklore stories as the stone itself. The field is called Cae Lleidr Dyfrydog which translate tos the field of the Dyfrydog thief.
There is a story relating to the stone and the field:
About 105 years ago there lived in the parish of Llandyfrydog, near Llannerch y Med, in Anglesey, a man named Ifan Gruffyd, whose cow happened to disappear one day. Ifan Gruffydd was greatly distressed, and he and his daughter walked up and down the whole neighbourhood in search of her. As they were coming back in the evening from their unsuccessful quest, they crossed the field called after the Dyfrydog thief, Cae Lleidr Dyfrydog, where they saw a great number of little men on ponies quickly galloping in a ring. They both drew nigh to look on; but Ifan Gruffyd's daughter, in her eagerness to behold the little knights more closely, got unawares within the circle in which their ponies galloped, and did not return to her father.
The latter now forgot all about the loss of the cow, and spent some hours in searching for his daughter; but at last he had to go home without her, in the deepest sadness. A few days afterwards he went to Mynadwyn to consult John Roberts, who was a magician of no mean reputation. That 'wise man' told Ifan Gruffyd to be no longer sad, since he could get his daughter back at the very hour of the night of the anniversary of the time when he lost her. He would, in fact, then see her riding round in the company of the Tylwyth Teg whom he had seen on that memorable night. The father was to go there accompanied by four stalwart men, who were to aid him in the rescue of his daughter. He was to tie a strong rope round his waist, and by means of this his friends were to pull him out of the circle when he entered to seize his daughter.
He went to the spot,and in due time he beheld his daughter riding round in great state. In he rushed and snatched her, and, thanks to his friends, he got her out of the fairy ring before the little men had time to think of it. The first thing Ifan's daughter asked him was, if he had found the cow, for she had not the slightest reckoning of the time she had spent with the fairies.