The 61-year-old president, nicknamed “The Bulldozer”, is serving his second term in office.
Opposition leader Tundu Lissu said that according to his sources the president is being treated in hospital for coronavirus in Kenya.
Mr Magufuli has faced criticism for his handling of Covid-19, with his government refusing to buy vaccines.
The East African nation has not published its coronavirus cases since May.
Earlier this month, at a funeral for a top presidential aide, Mr Magufuli said Tanzania had defeated Covid-19 last year and would win again this year.
The aide died hours after the vice-president of the country’s semi-autonomous islands of Zanzibar, who was being treated for Covid-19.
Mr Lissu says he was told that President Magufuli had been flown to Kenya for treatment at Nairobi Hospital on Monday night.
According to the opposition leader, the president has suffered a cardiac arrest and is in a critical condition.
There has been no official response from the government, which has warned against publishing unverified information about the Tanzanian leader, who was last seen at an official event in Dar es Salaam on 27 February.
Nairobi Hospital also said it could not comment.
Mr Lissu said that the government’s silence was fuelling rumours and was irresponsible, and the president’s health should not be a private matter.
It would not be a surprise to Tanzanians that Mr Magufuli had contracted coronavirus as he had been reckless in the face of the virus, he said.
We all should pray for president John magufuli