The protesters entered a test and trace centre, where one person was seen damaging and taking equipment.
Sajid Javid’s comments follow an incident in which dozens of marchers entered the facility and appeared to damage and remove testing equipment in videos shared on social media.
“I was appalled to see Piers Corbyn and his mob threaten NHS test and trace staff who are working so hard to keep people safe,” Mr Javid tweeted.
“This kind of vile behaviour is unacceptable.”
The Home Secretary, Priti Patel condemned the scenes as “completely unacceptable”, adding that the police had her “full support” to take action against those involved.
“It is more important than ever that people have their vaccinations and booster jabs and it is completely unacceptable that conspiracy theorists are intimidating and harassing those administering tests or delivering the life-saving vaccine programme,” Ms Patel tweeted.
“The police have my full support to take swift action where necessary.
“We’ll always give the police the powers (and) resources they need to keep the British people safe.
“That’s why we’re bringing forward measures in the Police, Crime, Sentencing & Courts Bill to crack down on public nuisance & disruptive protests that impact the law-abiding majority.”
Anti-vaxxer Piers Corbyn was seen speaking at the demonstration. He was arrested on suspicion of encouraging people to attack MPs’ offices earlier this month.
The group also entered Milton Keynes Theatre, where a pantomime was being staged.
Alfie Drake, a 19-year-old student who was visiting the theatre with his family, told the PA news agency that children cried while the protesters were “shouting and blowing whistles”.
“(I was) shocked (at) what we had seen, this panto was for families to enjoy with their kids,” Mr Drake said.
“It’s disgraceful, older men and women were shouting and blowing whistles and making some kids cry.
“It is a panto, not the parliament, they came across as very aggressive and (were) shouting at families with their kids to take off their masks.”
Sir John Bell regius professor of medicine at Oxford University and a member of the Vaccine Task Force, told Times Radio the group were “crazy”.
“That’s completely unacceptable behaviour by any standard. There are a group of crazy people who don’t want to have vaccines and who you’ll never persuade to have vaccines, and as far as I’m concerned, good luck to them,” he said.
On Wednesday Thames Valley Police had called for the public to share information with them about the protest by calling 101 or finding them online and quoting reference 604 29/12/2021, adding that no arrests had been made.
PA approached the force on Thursday for an update on its investigation but had not received a response at the time of publication.