The custom is based on a passage in the Zohar which speaks of the great significance of this night as the time of preparation for the marriage of God and Israel, when the Torah is given again, as it were, on Shavuot. Actually the Zohar refers to the community of Israel on high, the name given to the Schechina of which Israel is the counterpart here below. In other words, at this special time of grace the “sacred marriage” takes place on high, between the Holy One blessed be he and His Bride which is a highly charged mythological way of expressing the unity that then reigns in all creation with harmony restored, as it were, in the divine realm.
Rabbi Simeon b. Yohai and his associates, we are told, therefore spent the night in vigil in order to prepare the ornaments for the Bride. There are 24 in number and represent the 24 books of the Bible. To assist the harmonisation of the supernal powers is called by the Cabalists this tikkun (“putting right,” “perfecting.”) The mystics of Sefad in the 16th century elaborated on this idea and produced a special Tikkun, that is still used, containing passages from the bible the Mishna the Zohar and other classical works.
In some circles however, instead of selections from the classics, a more detailed and rigourous study of a particular passage was preferred. As the Maggid of Dubnow is reported to have said, samples of goods for sale are only of value if the seller can deliver the goods. Nowadays he argued we cannot pretend by quoting selections that we have the whole of the Torah at our fingertips.
A less mystical reason given for the custom of spending the whole night in study is that according to the midrash the people of Israel all fell asleep on the night before the Torah was to be given. By staying up all night before the anniversary of the giving of the Torah we make sure that Israel will not be found asleep again.