In the place of today´s castle, there was a Slavnik´s fortified settlement, mentioned in documents as long ago as 981, that was soon converted into a Přemyslid castle. In 1567, Litomyšl was acquired by Vratislav of Pernštejn, who started to build a representative family residence in the Italian Renaissance style.

The face of the castle is covered with rich sgraffiti and topped by stepped Renaissance gables. The Renaissance arcade castle with a complex of farm buildings, castle garden and the piaristic Church of the Holy Cross creates a unique image of rich feudal residence from 16th and 17th century.