Search engine effects on news consumption: Ranking and representativeness outweigh familiarity in news selection. Roberto Ulloa and Celina Sylwia Kacperski. New Media & Society, Mar 6 2023. https://doi.org/10.1177/14614448231154926
Abstract: While individuals’ trust in search engine results is well-supported, little is known about their preferences when selecting news. We use web-tracked behavioral data across a 2-month period (280 participants) and we analyze three competing factors, two algorithmic (ranking and representativeness) and one psychological (familiarity), that could influence the selection of search results. We use news engagement as a proxy for familiarity and investigate news articles presented on Google search pages (n = 1221). We find a significant effect of algorithmic factors but not of familiarity. We find that ranking plays a lesser role for news compared to non-news, suggesting a more careful decision-making process. We confirm that Google Search drives individuals to unfamiliar sources, and find that it increases the diversity of the political audience of news sources. We tackle the challenge of measuring social science theories in contexts shaped by algorithms, demonstrating their leverage over the behaviors of individuals.
Discussion
We tested the hypothesis of news engagement (as a proxy for familiarity with news sources) as a predictor of news article selection in the Google search engine (RQ1). We did not find evidence supporting this. Instead, we found a significant effect for two factors that are decided by the search engine alone: the position in which the result is presented (ranking) and the number of times the news source appears (representativeness). While ranking has previously been demonstrated to play a strong role (Pan et al., 2007; Urman and Makhortykh, 2021), we show for the first time that its effect is weaker for news article selection compared to non-news selection (RQ3). This may well suggest a more careful decision-making process of individuals when selecting news (e.g. reading the titles and excerpts more attentively).
Research has indicated a higher representation of “mainstream” news sources in search results (Puschmann, 2019), while, at the same time, a positive effect in the diversity of news consumption (Fletcher et al., 2021; Fletcher and Nielsen, 2018). Our results align with this seemingly counter-intuitive evidence: representativeness reduced the likelihood of news article selection (RQ1). This might be an indication that once individuals have decided not to visit a result belonging to a specific news source, they also discard subsequent results from the same source, suggesting that the individual is actively avoiding such sources (Mukerjee and Yang, 2021).
In line with previous research (RQ2a), we found that Google Search increases the diversity of participants’ news consumption (Fletcher et al., 2021; Fletcher and Nielsen, 2018; Scharkow et al., 2020). It is possible that participants use Google Search when they are actively looking for novel news sources, though we also show that Google Search facilitates a discovery process by presenting a variety of news sources among the results. In addition, we show that Google Search increases the political audience diversity that news sources receive (RQ2b). Given that Google has its own news quality controls in place (Google Developers, 2021), the finding can explain recent research showcasing that political audience diversity can be used as a sign of news source reliability, and that it should be incorporated into ranking algorithms (Bhadani et al., 2022). Instead, our results suggest that it is Google Search (including its ranking) that drives this effect. More broadly, researchers should consider that online news browsing behavior is heavily shaped by online platforms, for example, we demonstrated that there are differences in the consistency of our familiarity metric depending on whether it is measured including traffic referred by Google or not. Considering the dominance of a few large search engines and their role in driving users to news, the situation may lead to a concentration of power and influence within the media landscape in which news organizations prioritize visibility on search engines (and similar platforms) by following SEO guidelines, instead of focusing on journalistic norms (Hackett, 1984; Muñoz-Torres, 2012).
Furthermore, we build on previous research investigating the phenomenon of mere-exposure (Montoya et al., 2017) and trust in news sources (Fletcher and Park, 2017), and investigated familiarity (RQ1), that is, the participants’ acquaintance with the news sources they are presented in the search results. We proposed news engagement as a proxy and measured it using a section of the browsing history that is independent from the analyzed news articles that are selected (and visited) in the search results. This allowed us to quantify the existent relationship between the individual preferences toward news sources through the number of visits of the individual to each news domain; thus, capturing three modes of news engagement: routinary visits, social media referrals, and intentional search (Möller et al., 2020).
To summarize, we find no evidence in favor of filter bubbles, nor do we find an effect of selective exposure toward news sources based on familiarity. We find that individuals place their trust in Google, and that in turn, Google steers them toward sources that are different to their routinary visits. In this process, we tackle the challenge of developing reliable measurement models when integrating social science theory with digital behavioral data (Wagner et al., 2021).
Limitations and future directions
We would like to point out several limitations. First, our engagement metric does not fully capture when individuals are familiar and trust a given source, that is, when they do not regularly consume news or mainly do it offline. In addition, the engagement for the time independent analysis was calculated using an arguably short period of time (~1 month). Future data collections should enable analyses across timespans of multiple months up to years, though we highlight that our results did not change when considering a time-dependent analysis (~2 months).
Second, our analysis is limited to Google, which we chose due to its market dominance (StatCounter, 2021). Our findings should not be generalized to other search engines due to differences in how search results are displayed, though it is likely that effects specifically related to ranking and representativeness will remain similar across different interfaces. Our findings should also not be generalized to contexts beyond the top-10 organic results of web searches, such as carousels and top stories, news aggregators (Google News), or recommendation systems such as Google Discover. The latter does not provide results based on prompts input by the user, but presents content (including news) personalized from individuals’ preferences, which may be extracted from previous search behavior. Since search choices are heavily influenced by ranking, the algorithmic representations of preferences might also reflect the ranking effects as individual traits. This raises questions about the extent to which preferences can be captured by these systems, or if they merely expand the influence of the search engine. It highlights a challenge for study of new media: how can we better understand interdependencies between the digital services, rather than studying them in isolation?
Third, characteristics of our sample should be kept in mind when interpreting our results. The sample size used in the above research is relatively small. Out of the 739 individuals that participated in the web tracking study overall, only 280 are represented in our subsample, due to a relatively low number of news visits (including the ones driven by Google Search) in web tracking data—consistent with previous literature (Scharkow et al., 2020; Wojcieszak et al., 2021), and the strict data quality constraints for including a search page for the analysis. Our sample is relatively uniform, only including German individuals with a Chrome or Firefox browser installed on their desktop computers; while this reduces noise, it also affects generalizability of the findings. Finally, many individuals refuse to participate in web tracking studies due to privacy concerns (Makhortykh et al., 2021), which might indicate that the sample is pre-selected based on factors that we don’t yet fully understand and cannot control for.
Despite these limitations, the logged web browsing behavior that we capture occurs in a real environment with minimal intervention; we argue that cognitive awareness of the presence of the web tracker is likely to only affect the very initial browsing behavior. Moreover, web tracking studies that include the website content remain rare, and our method is exceptional as it deterministically identifies referrals by tracing the tab activity of the browser and matching the presence of the URLs among the results.
Some promising directions emerge for new media research. Our results suggest that ranking is less important for news selection: research should further confirm if individuals are more selective when choosing news from the top-10 search results, which could for example be achieved by analyzing the duration of the interaction with the individual choices or by applying eye-tracking methodologies. In addition, we find that representativeness is negatively associated with news selection. A further inspection of the reasons is of interest, for example, this behavior might signal an active avoidance of specific news sources (Mukerjee and Yang, 2021). Finally, it is important to examine search choices within the broader context of the layout of web search pages, including features like video carousels and top stories, to gain a deeper understanding of how individuals make decisions for information acquisition.
No comments:
Post a Comment