In the 14th century, its walls repeatedly withstood the Crusaders attacks. For the first time German knights appeared here in 1392. The wars of the mid 17th – early 18th centuries befell Lida. In the summer of 1659 the castle was sieged by the 30 thousand Moscow army. In the beginning of the 18th century, during the Great Northern War, the Sweden troops heavily damaged the castle. Both towers were blown up, and the fortress lost its defensive role. Gradually, the walls of the Lida Castle fell into disrepair. In the summer of 1794, insurgents of Tadevuš Kasciuška fought with the Russian troops in the ruins of the Lida Castle. In 1891, there was a big fire in Lida. After it, locals even tried to take the castle walls to bricks. In the early 20th century, conservation works took place. The construction commission for the restoration of the Lida Castle was created. From 1909 to 1911, the East, West and North walls were brought to the proper condition. The South wall did not require significant repair work. Between the world wars, Polish restorers led by a young architect Teodor Bursche carried out some restoration works.