The Scottish government has called for drivers moving perishable goods such as seafood and salmon to be prioritised as France eases it travel ban.
France and the UK have reached agreement over their shared border which was closed amid concerns over a new coronavirus variant.
EU nationals and hauliers are among those now able to travel – if they have had a recent negative test.
But there have been warnings that it could take days to clear the backlog.
Soldiers have joined NHS Test and Trace staff in Kent to carry out rapid tests on thousands of stranded lorry drivers.
There have also been reports of clashes between some stranded lorry drivers and police.
The travel ban was imposed on Sunday after the UK government warned of a new, fast-spreading variant of coronavirus and introduced strict tier four – “stay at home” – restrictions across large parts of southern and eastern England in response to a new variant of coronavirus.
UK Communities Secretary Robert Jenrick told BBC Radio 4’s Today programme there were more than 3,000 HGVs at Kent’s Manston Airport as of 19:00 GMT on Tuesday, and several hundred more have joined the queue since then.
He said it was likely to take “a few days” to clear the backlog of lorries, some of which are carrying millions of pounds worth of Scottish seafood.
Scottish Transport Secretary Michael Matheson said UK minister should now “urgently prioritise those drivers with perishable goods, such as Scottish seafood and salmon”.
He added: “What has always been of concern is the impact on exporters, this is their most critical time of year and the dismay caused by the uncertainty and length of delay has been avoidable and regrettable.”
Exports to France for the first nine months of 2020 are already down 11.3% on the same period the previous year.
Mr Withers said: “Contrary to an upbeat assessment from the prime minister yesterday, the situation has been deteriorating with a growing backlog of lorries.
“That is an irrecoverable loss of income and I fear about this being a fatal blow to some of the smaller businesses after the horrendous year they have already had.”
Donna Fordyce, chief executive of Seafood Scotland, agreed: “The window wherein companies would be able to salvage anything from the last couple of days is now, to all intents and purposes, closed for premium seafood, which has been perishing by the roadside since Sunday night.
“Millions has been lost, much of it by small companies that were depending on this trade for survival.
She said the situation turned thoughts to the end of the Brexit transition period.
She said: “The last 48 hours has given us a terrifying insight into what the situation could be come 1 January. While passage may not be formally blocked by then, there remains a “red tape blockade” which will likely have exactly the same impact as the last 48 hours.”