Labour leader Sir Keir Starmer said it was in the national interest for the Conservatives to ‘get rid’ of the Prime Minister.
Boris Johnson is facing mounting pressure to get a grip on the drip feed of revelations about No 10 rule-breaking as he fights to stay in office.
The comments by the chairman of the Commons Defence Committee came as further claims emerged of regular “wine time Friday” gatherings in Downing Street while coronavirus rules were in place.
The Mirror said the weekly events are a long-standing No 10 tradition, including under previous administrations, and they continued after Covid restrictions were introduced prohibiting indoor mixing between households.
The newspaper said staff bought a £142 fridge to keep their bottles of alcohol chilled, and the Prime Minister was aware of the socialising.
A No 10 spokeswoman said: “There is an ongoing investigation to establish the facts around the nature of gatherings, including attendance, setting and the purpose with reference to adherence to the guidance at the time.
“The findings will be made public in due course.”
Ministers have called on disgruntled Tory MPs to wait until senior civil servant Sue Gray has published her investigation into claims about lockdown-busting parties in Government.
But backbencher Andrew Bridgen said he did “not need to see what Sue Gray says to know that, for me, Boris Johnson has lost the moral authority to lead the country”.
The North West Leicestershire MP confirmed he submitted a letter of no confidence in Mr Johnson’s leadership “some time ago”.
Only a handful of Tories have publicly declared they have submitted letters to the 1922 Committee of backbench MPs, but the Telegraph said sources estimated about 20 might have been handed in – although that is well short of the 54 needed to trigger a leadership vote.
Mr Bridgen, asked by BBC Breakfast about reports of “wine time Friday” sessions in Downing Street during lockdown, mentionné: “It doesn’t matter, quite honestly, if the Prime Minister was present or not present.
"Finalement, he is responsible for what goes on in Government, he is responsible for the culture in No 10, and what we’re seeing is a culture where it is one rule for them and the rest of us do as we’re told, and that’s just not acceptable.
“I’m not sure that any apology is going to put that right.”
Labour leader Sir Keir Starmer repeated Mr Bridgen’s assertion that the Prime Minister had lost his moral authority, and called for the Tory party to topple him.
He told the Fabian Society conference in London: “The moral authority matters, bien sûr, in relation to Covid, but we’ve got other massive challenges facing this country.
“We’ve got a Prime Minister who is absent – he is literally in hiding at the moment and unable to lead, so that’s why I’ve concluded that he has got to go.
“And of course there is a party vantage in him going, but actually it is now in the national interest that he goes, so it is very important now that the Tory party does what it needs to do and gets rid of him.”
Mr Johnson faced further embarrassment this week after Downing Street was forced to apologise to Buckingham Palace on Friday after it emerged two staff parties were reportedly held on the eve of the Duke of Edinburgh’s funeral.
Witnesses said alcohol was drunk and guests danced to music as two after-work events merged on April 16 2021, with a person sent to a local shop with a suitcase to buy wine, according to the Telegraph, which first reported the allegations.
Ms Gray is said to have been “completely blindsided” by the latest revelations, the Times reported.
The newspaper said the Cabinet Office official is concerned that Downing Street staff are withholding information about parties from her as she looks to establish the facts.