“The Emergence of Political Polling Through Kalye Surveys” is a study presented by the authors in a #FactsFirstPH research briefing held on March 25, 2022. The full copy of the research is reposted with permission from the authors.

Conclusion

Read the other parts of this series:
Part 1: The emergence of political polling through ‘kalye surveys’
Part 2: ‘Kalye surveys’ and deviation from scientific standards of research

Apart from the standards that address scientific rigor, surveys must persistently follow ethical standards of practice across the entire phase of research. The ethical norms of surveys are moored on the general principles of autonomy, nonmaleficence, beneficence, and justice.

Prescription 5. Obtain informed consent

First and foremost, autonomy must be observed and protected by acquiring informed consent from respondents before conducting the survey. Informed consent includes providing the respondents with sufficient information about the survey project, the group conducting the project, its funders, terms of survey participation (e.g., scope of response task; statement on beneficence and nonmaleficence, which articulates potential benefits and elimination of potential harm), and management of collected data (e.g., confidentiality, data privacy, data storage). It should also contain the project’s contact information and an agreement form on informed voluntary participation. These are all key to enable respondents to make informed decisions whether to participate in the survey project or not.

We observed from the videos that kalye survey enumerators tend to have a very casual and straightforward manner of approaching prospective respondents and getting them to participate by simply announcing that they are conducting a kalye survey and then dropping the question on presidential vote choice. It is designed to be a “fast survey”, a quick ask-and-go method of data collection. However, the principle of autonomy must always uphold the agency of respondents to make an informed decision on survey participation, even if the survey is designed to be quick. Thus, training content creators on the basic information needed to establish informed consent is necessary.

We likewise observed that most survey enumerators would show a board, which functions as a “show card” to present the list of candidates and, at the same time, as a recording device on which enumerators tally the responses. Designed like so, every succeeding respondent would then see the latest tally of votes. Thus, we heard respondents comment that, “Ah malakas ang [Candidate X] ngayon;” “Parang si [Candidate X] ang mahina diyan ah;” “Maraming undecided.” One respondent who noticed that the whiteboard marker was running out of ink even quipped, “Ayaw nang sumulat, baka matalo.”

In addition, in an effort to claim authenticity, an enumerator expressed, “Ito ‘yung tunay na survey,” as he showed the board with the current tally of votes to the camera.

The practice of using the board as a show card and a tally board does not only run contrary to the data privacy clause that is core to survey data collection, it also potentially creates a bandwagon effect (when respondents follow the vote of the majority) or underdog effect (when respondents opt to choose the minority) on succeeding respondents, which ultimately affects the validity and reliability of the survey results.

Moreover, the non-private manner of administering the survey can also undermine the independence of the respondents when they express their choice. There are occasions when the survey was administered to a congregation of individuals, and these individuals hear and can comment on the choices of the others. There is also an instance where the enumerator just shouted his question to a crowd and they shouted their answers back.

This manner of survey administration can create groupthink effect among the respondents, and some respondents may express a choice that appears to be socially desirable among the other respondents in the group. We noted occasions when some respondents changed their vote choice after they heard or saw the answer of another respondent. As such, the collected data may reflect voting preference that some respondents did not independently make.

It is also quite concerning to observe how the reviewed kalye surveys appear to assume that the respondents consented to have their participation recorded on video, and worse, be later uploaded on YouTube. Ethical practice emphasizes that survey data collection should be treated as a private and confidential exercise. The act of video documenting the process and later on uploading it without express permission runs against the ethical principles of informed consent, confidentiality, and data privacy.

We observed instances when respondents would be visibly hesitant to participate (e.g., they would walk away from the camera, they would hide their faces) but the enumerator would still proceed with the survey and in filming them. These respondents would eventually relent but their hesitation to participate is still visible. One respondent expressed his agreement to participate but requested not to be filmed. The enumerator replied, “Itatanong na lang. Hindi na kita i-video… ‘Yan narinig niyo na lang ah. Ayaw magpa-video eh.” However, the uploaded video still displayed the respondent’s face, without even any effort of blurring it.

We also noted at least two occasions when respondents were surprised that they were being filmed. One respondent said, “Naka-camera pala…baka pagalitan ako ah;” while another respondent, upon finding out that the survey will be uploaded on YouTube, commented, “Baka mamaya ma-shoot-to-kill ako ah.”

Some videos also inadvertently capture children’s faces and these were not blurred in the uploaded video. Minors are offered a greater degree of protection and data privacy. Acquiring informed consent for them to appear in a publicly released video must first be discussed with a parent or legal guardian. Lastly, we observed that the inside space of some houses were also inadvertently filmed without apparent prior informed consent.

Overall, while it appears to contribute to the kalye survey’s authenticity, the casual and unconsented means of video recording respondents is a violation of informed consent, confidentiality, and data privacy. Survey literacy training will teach content creators to be more mindful and respectful of these non-negotiable principles, regardless of whether they intend their surveys to be entertaining or not. 

Prescription 6. Honor respondents’ voluntary participation and voluntary non-participation

The ethical standard of justice means honoring the articulations in the informed consent form, respecting the decision of those who opted not to participate and those who decided to terminate their participation during the course of survey enumeration or project duration. It also means honoring the goodwill of survey respondents who volunteered their data.

Any survey, if improperly done, can generate misinformation.

Survey literacy training should highlight that surveys, as a tool that is highly dependent on informed voluntary participation, should always place a premium on the welfare of the respondents across the entire phase of the process.

Literacy training should also emphasize that the goal is not only to design and implement valid and reliable surveys but also to make the entire experience a pleasant and safe one for those who agreed to participate as respondents as well as for those who declined at the point of recruitment or those who decided to terminate their involvement during the actual course of the survey.

Way forward

We reiterate that surveys, if scientifically and ethically done, can validly and reliably contribute to our sense-making. While kalye surveys are generally packaged as an alternative to mainstream political polling, they are not excused from the standards expected from social science research methods given the agenda of their inquiry, the scope of their dissemination, and their potential to create actual voting effects. And while kalye surveys may be packaged as “for fun”, they are prone to cause misinformation if handled carelessly. Any survey, if improperly done, can generate misinformation.

Our freedom of speech and expression allows anyone of us to conduct surveys. Thus, while our review surfaced areas where kalye surveys tend to deviate from the gold standard, we also argue that scientific and ethical prescriptions can be learned and incorporated in kalye survey practice. Thus, we advocate for survey literacy and our six basic prescriptions highlighted areas where training can help kalye survey content creators in advancing their understanding of the science and ethics of surveys and how these can be applied across the design, administration, analysis, and dissemination phases of the practice.

Similarly, our freedom of speech and expression allows every one of us to be informed by surveys. As such, our advocacy for survey literacy also extends to the members of the public who are the ultimate consumers of survey reports. Survey literacy should raise the public’s awareness of the science and ethics of surveys to allow them to sensibly interrogate and prudently utilize survey results.

Survey literacy should make the tools for evaluating the validity, reliability, and ethicality of surveys more accessible and easily understandable to enable the public to protect themselves from all possible forms of misinformation. Most of all, survey literacy should empower the agency of the public so that they can demand accountability among survey groups that deliberately misuse surveys for misinformation. – Rappler.com

Ma. Rosel S. San Pascual is an associate professor at the Department of Communication Research of the University of the Philippines Diliman College of Mass Communication and a member of the Philippine Media Monitoring Laboratory. She primarily teaches quantitative research methods, measurement and sampling, and quantitative data analysis in both undergraduate and graduate levels. She also actively engages in academic research on communication and new media, online discourse, media effects, mass media, ICT and development, political communication, and transnational migration and family communication.

Anjenica U. Durana is an MA in Communication student at the Department of Communication Research of the University of the Philippines Diliman College of Mass Communication. She previously received her Bachelor of Science, Major in Psychology degree from Ateneo de Manila University, and has been a licensed psychometrician since 2017.

Myrnelle A. Cinco is currently aiding the Department of Communication Research of the University of the Philippines Diliman College of Mass Communication in a number of research projects. She is also a research assistant for an international nonprofit organization working on human rights. She has a background in data analytics, data science, and research. She has a bachelor’s degree in communication research from the University of the Philippines Diliman.