A Fleming, Wizo, who died in 1130 founded at Wiston a motte and bailey fortification, forerunner of the stone castle, for protection against the Welsh warlords: the Flemings were reportedly unpopular wherever they settled.
The precarious position of Normans and Flemings was demonstrated in 1136 when the Normans, having already lost 500 men in battle at Loughor, re-recruited from Lordships from all over South Wales and led by Robert fitz Martin at Crug Mawr near Cardigan attacked Owain Gwynedd and his army.
It is recorded that the Constable of the castle in 1207 was Itohert, son of Richard Tancard, possibly a descendant of the first Tancred.
The Flemish presence, reputed to result from floods in the Low Countries, was more likely to have consisted initially of Flemish mercenaries originally in the invading army of William the Conqueror, who in reward for their part in William's victory were granted lands in parts of Northern Britain, and in Wales in the Gower, and Geraldus Cambrensis recorded their presence in the Hundred of Roose in Pembrokeshire.
The castle is recorded as having been founded in 1100 by the Norman Gilbert de Clare.
The Flemings, said to have arrived in three groups in 1107, 11, are likely to have participated in its later development for their own and the Normans' protection from the Welsh warlords.
The Roman legionary headquarters at Caerleon were roofed with slates from the lower slopes of the Preseli Hills.
On the southern spur, the High Street ascends steeply from the river and forms the core of St Mary's parish.
From the foot of each spur, ancient bridges cross the river to Prendergast: St Martin's Bridge ("the Old Bridge") and St Mary's Bridge ("the New Bridge", built in 1835).
However archaeological discoveries in Pembrokeshire suggest otherwise. 1695) refers to a valuable find of silver coins at Llanboidy, the latest coin being one of Domitian struck in AD 91.
In the 1920s Sir Mortimer Wheeler partially excavated a Roman dwelling or villa at Wolfscastle; work was restarted in 2002 by Professor Merroney.