Capita is "not comparable" to Carillion, the government said, following a profit warning that wiped more than £1bn off its market value.
Cabinet Office Minister Oliver Dowden told the House of Commons his officials met senior executives of Capita to discuss the impact of its announcement.
He said Capita was not "in a comparable position" to Carillion, which went bust last month.
One Labour MP called the government's thinking "muddled and complacent".
Capita said on Wednesday that its profits for 2018 would be about a third lower than expected, prompting it to axe dividend payments to shareholders and raise £700m by issuing new shares.
The company operates the London congestion charge, runs the government's Jobseeker's Allowance helpline and administers the teachers' pension scheme. It also collects the TV licence fee on behalf of the BBC.
Its shares plunged by nearly 50% following the profit warning. Capita's shares fell another 10% to 164p on Thursday.
Answering an urgent question from Business Select Committee chair Rachel Reeves on whether Capita posed a risk to public services and finances, Mr Dowden said it was "in a very different situation" to that of Carillion.
"We continue to work closely with the company to monitor the execution of its plan, and of course to ensure the continued delivery of public services," he said.
"We regularly monitor the financial stability of all our strategic suppliers including Capita and we do not believe that they are in a comparable position to Carillion.
"The measures that Capita has announced as designed to strengthen its balance sheet, reduce its pension deficit and to invest in core elements of its business. Arguably these are the measures that could have prevented Carillion from getting into the difficulties that they did."
Carillion, the construction and services company, collapsed in January after it was unable to secure financial support to continue trading.
'More similarities than differences'
However, Ms Reeves described the government's thinking on the differences between Capita and Carillion as "muddled and complacent".
The Labour MP said: "Both Capita and Carillion have debts of more than £1bn and pensions deficits in the hundreds of millions. Both Capita and Carillion paid out dividends of more than £1bn in the last five years.
"Both Capita and Carillion rely on the public purse for half of their contracts. Both Capita and Carillion were audited by KPMG. Both Capita and Carillion grew through acquisition and not through organic growth.
"It seems there are more similarities than differences between these two companies," she added.