Yesterday (Thursday) the Wales Audit Office said that more than £50,000 of payments to chief executive Bryn Parry-jones and another senior officer in lieu of pension contributions, were unlawful.
Mr Crabb told the Western Telegraph: "This is damaging for Pembrokeshire, no question.
"At a time when people are trying to make the case for Pembrokeshire to keep its local authority, this report is a gift to those who want to see the County abolished.”
"As we saw with the Estyn and CSSIW reports into child safeguarding, the culture of weak oversight and lack of transparency is still our Council's fatal weakness. It does feel like 'last chance saloon' for Pembrokeshire County Council. It is now up to Councillors to show that they understand the seriousness of the situation.”
Mr Crabb said some of the blame lay with the Welsh Government not introducing transparency measures along the lines of those introduced in England.
“The UK Government used the Localism Act to introduce new measures in England to increase financial transparency and give people greater power over the pay practises of their councils," he said.
"Sadly, the Welsh Government did not introduce the same measures in Wales and there have been problems across numerous Welsh local authorities. At a time when finances are being cut, we need a bigger spotlight on how ratepayers’ money is being used.”