MANILA, Philippines – Is the camp of vice-presidential candidate Ferdinand Marcos making a mountain out of a molehill?

Francesca Huang of Marcos’ legal team earlier claimed in a press conference that a new script was introduced into the Comelec Transparency Server stored at Pope Pius center in Manila at around 7:30 pm on May 9. Huang made a big deal out of the fact that the change altered the hash code of the results files. 

She also claimed that the senator’s lead over rival Leni Robredo began to “erode at a rather distinctive pattern.” 

It looks like what happened was nothing more than human error. 

The error lies in the failure of Comelec and Smartmatic to immediately regenerate a new hash for the results file after they had corrected a mistake in the way names with the character Ñ were displayed, according to Marlon Garcia of Smartmatic.

IT experts including Rappler’s tech team agreed. Had Smartmatic immediately produced a new hash after the correction and before it transmitted the corrected results file, nobody would have noticed this issue.  

‘Cosmetic’ change

The diagram below reconstructs what happened based on a briefing conducted by Comelec and Smartmatic at the Transparency server room on Thursday evening, May 12. 

It is based on a sketch made by Joben Ilagan, one of the IT experts working with Rappler to analyze the results. Ilagan is the founder of Seer Technologies, one of Rappler’s IT partners. 

Human error

As shown in the diagram, each of the results files, according to Smartmatic’s Garcia, are transmitted by the transparency and mirror servers in zip file with the hash. 

The hash is a security feature of the system. “Hash codes are similar to the tamper-evident labels or seals you see in day-to-day items you buy in groceries,” according to Ilagan. 

“Manufacturers warn you not to consume an item if the seal is broken,” Ilagan explained. “In software, if a file’s tamper-evident seal is different from what we expect, we have reason to question the setup.”

To check for the correct seal, you need to run the file and the hash through a hash validator.

Each of the workstations at Pope Pius has a hash validator where groups can check this. 

So every zipped file of election results contains folders that will show the results file and the hash for every file.

In this particular batch of results past 7 pm of Monday, it was the second hash in the zipped file that showed a mismatch. 

Before this, the Rappler IT team in Pope Pius noticed the ? sign in names that contained the letter Ñ. We notified the Comelec and Smartmatic personnel about this error, and they immediately announced to all the groups present in the server room (including media, watchdogs and accredited political parties) that the correction would be in the next batch of files.

None of the groups present, including the political parties represented, raised an objection to the correction. 

After the correction was made, the server continued to transmit results with the old hash. That’s because Comelec and Smartmatic personnel forgot to regenerate a new hash results file.

Thus, hash files that accompanied the succeeding results files transmitted by the 2 servers to media and poll watchdog groups failed to match when they were run through a hash validator.

“We didn’t think much of it. It was not the code that was changed. It was hash of the results file. It did not affect the results, ” according to William Yu, IT Director of the Parish Pastoral Council for Responsible Voting (PPCRV). 

Had the hash files been regenerated, the succeeding results files would have been resealed. The hash validators would not have detected any problem. And the Marcos camp would have not screamed about it.  

Even a hash mismatch does not necessarily mean cheating, according to IT experts. A single character change, like what happened in this case, can also change the hash. 

“Hash codes can also change through human error or some other reason,” Ilagan explained.

A lawyer of the opposition United Nationalist Alliance acknowledged that they observed the “error” in the hash validation from 7:30 pm of May 9 through 8 am of May 10, 2016.

“We were validating the hash of the results file,” according to Ivan Uy, lawyer for the United Nationalist Opposition (UNA).

This was reported to the Comelec and by May 10, at 8 am, a new hash was generated.

Hash demo

To demonstrate that the Ñ character correction is really the only part of the application that was changed, the Comelec and election automation vendor Smartmatic explained how results files were generated and security features of the system, including the hash to media and observers at the Transparency server room on Thursday night.

The scripts inside the server, including scheduled tasks and how the hash looked, were shown using a projector to everyone in the briefing. 

Among the groups present were some IT professionals. 

Smartmatic’s Garcia explained that, for security reasons, Smartmatic and Comelec split the password to access the server into 2: one part is kept by Comelec and the other by Smartmatic.

Each time a change needs to be done, a Smartmatic representative has to input one half of the characters in the password while the other half is inputted by the authorized Comelec representative.

This means both Smartmatic and the Comelec cannot make changes to the server applications on their own. 

Garcia said it was he who corrected the Ñ error in the results generating script which generated results files received by accredited groups. (See timeline graphic above.) 

Garcia also said he will demonstrate how the hash changed. (Go to our livestream page: Canvassing of votes, 2016 Philippine elections to watch the demo) – Rappler.com