Zuma has endured months of criticism and growing calls for him to step down after a series of corruption scandals as the country battles falling economic growth and record unemployment.
National Prosecuting Authority (NPA) director Shaun Abrahams announced the decision to challenge a High Court order to reinstate 783 charges against Zuma, but denied there had been any political pressure.
The charges, relating to a multi-billion dollar arms deal, were dropped in 2009, clearing the way for Zuma to be elected president just weeks later.
At the time, state prosecutors justified dropping the case by saying that tapped phone calls between officials in then-president Thabo Mbeki's administration showed undue interference.
But the Pretoria High Court last month dismissed the decision to discontinue the charges as "irrational" and said it should be reviewed by the NPA.
"The judgement affects... the discretionary powers of the prosecutor," Abrahams told a press conference on Monday. "It is so important that I believe it needs a decision of an appeal court."
Abrahams railed against suggestions that the NPA was reluctant to prosecute Zuma.
"I will always do what is correct, irrespective of whether the individual concerned is an ordinary citizen, a cabinet minister or a sitting president," he said.
"Any suggestion that I may have succumbed to any pressure to make my decision – I can assure the public today that it is absolutely ridiculous and completely unfounded."
Zuma later announced that he too was appealing the ruling that he should face the charges.
"The president believes that the decision of the court affects him directly and is of a strong view that the court erred in several respects in its decision," the presidency said in a statement.
The tapped phone recordings, which became known as the "spy tapes", were kept secret until they were released in 2014 after a legal battle fought by the main opposition party, the Democratic Alliance (DA).
The DA dismissed the prosecutors' appeal bid as "a blatant delaying tactic to shield Jacob Zuma".
"Abrahams is preoccupied with protecting president Zuma at all costs," it said. "The DA will not let this matter rest. President Zuma will face justice."
Pressure on the president would increase if some or all of the charges – which relate to alleged corruption, racketeering, fraud and money laundering – were reinstated.
"It will take a year or so before the case is heard by the Supreme Court of Appeal so for now he is safe," Gareth Newham, of the Institute of Security Studies in Pretoria, told Agence France-Presse.
He added: "It is highly unlikely that the NPA will win the appeal."
The ruling African National Congress (ANC) faces tricky local elections in August, but Zuma retains widespread support within the party and has appointed many loyalists to key positions nationwide.
Last month, a commission that Zuma set up cleared all government officials – including himself – of corruption over the 1999 arms deal.
Zuma, who was appointed deputy ANC leader in 1997, was accused of having accepted bribes from international arms manufacturers.
The DA hopes to make major gains in the August elections, tapping into discontent over the ANC's struggle to deliver jobs, houses and education 22 years since the end of apartheid rule.
In March, the president lost another major legal case when South Africa's highest court found he violated the constitution over the use of public funds to upgrade his private residence.
The so-called "security" work, which cost taxpayers $24 million, included a swimming pool, chicken run, cattle enclosure and an amphitheater.