University Students’ News Literacy on Social Media

*This is the research article I wrote with my partner for grad school. It wasn’t easy, but it wasn’t as hard as I thought it would be. We collaborated on the whole paper, but she did the Literature Review while I did the statistics and results. This was presented to multiple review boards and presented at the Shippensburg University Research Fair in 2017*

Barbara Schindo & Zeb Carbaugh, Shippensburg University

COM 520: Applied Mass Communication Research

May 7, 2017

Abstract

This research study evaluated the popularity of social media sites and their contribution to the spread of fake news. Participants were asked how they got their news and how often they shared news articles on social media. They were shown various social media articles and then asked to rate each articles’ credibility. Real news articles were used in the survey. Then those same articles were altered to suggest that they came from fake news sources. The participants (N = 110) did give the real articles a higher rating of credibility and trust. There was no significant relationship between the credibility of an article and the participants’ likelihood to share the article. On average, participants also rated Facebook as their most frequently used news source.

University Students’ News Literacy on Social Media

The term “fake news” has been making headlines in real news and all over social media. Fake news is the deliberate publication of false information and purporting it to be real news. Social media is a new way for consumers to get news in a time when public trust in traditional media has seen a strong decline (Turcotte, York, Irving, Scholl, & Pingree, 2015). A Pew Research Center survey (2016) shows that 62% of U.S. adults get their news on social media, but how much of the news on social media is authentic and factual? Out of the 62% of adults who get their news on social media, the majority of them get it from only one site, and they are not actively seeking it (Pew, 2016).

Social media, aside from being a source of news, has changed and continues to change the landscape of journalism. Citizen journalism, or the collection, sharing, and analysis of news by the general public on the Internet, is emerging as a competitor for mainstream media sources. In this changing landscape, anyone with access to the Internet has the potential to write and share information. Media consumers need to be aware that not every news item they see on social media is correct or is coming from a credible source. It is important for consumers to be vigilant and media literate before they like and share articles on Facebook.

This research addresses news consumption on social media, specifically, Facebook. As the use of social media as a news source increases, it is important to study users’ habits. If university students are more apt to share news articles that are not credible, that is significant and could mean a bleak future for social media as a news source.

This purpose of this study is to evaluate if the popularity of social media sites contributes to fake and incorrect news stories going viral. This study also aims to determine how well university students understand media literacy, and if they are able to tell the difference between credible and not credible news source.

Literature Review

A number of studies address how participants view media credibility and believability. Many found that consumers are more likely to find the news not trustworthy or biased (Golan & Baker, 2012; Turcotte, York, Irving, Scholl, & Pinegree, 2015; Zuniga & Hinsley, 2013). Overwhelmingly, Mormon college students viewed the media as not credible, not trustworthy, immoral, and incorrect (Golan and Baker, 2012). Fisher, Magee, and Mohammed-Baksh (2015), and Zuniga and Hinsley (2013) concluded that people working within the news industry see their work as more trustworthy and believable than the public did. One thing particularly alarming about Fisher, Magee, and Mohammed-Baksh’s (2015) conclusion is that the media consumers do not care if the information seems credible or not.

Turcotte, York, Irving, Scholl, and Pingree (2015) say there is a strong decline in the public’s trust in traditional media outlets, but social media is emerging as a new avenue for consumers to find information. As social media emerges as a news source, it also has created a new form of newsgathering: Citizen Journalism. The general public now has the ability to gather, write, post, share and analyze current events. Tweets, Facebook posts, and online messages have become news sources (Fisher, Magee & Mohammed-Baksh, 2015). More than half of adults surveyed said they get news from social media, which is up from 49 percent in 2012 (Pew, 2016). Chung, Nam, and Stefanone (2012) said younger people are the largest consumers of online news, and news consumption habits begin during the college age years. With the increase in use of social media as a news source, and the ability of social media to make almost anyone a content creator, there is concern that users are apt to believe and share information that is false or incorrect.

Studies show an increase of social media use as a news source over the last two decades. Jo (2005) said that the news source type and content had significant effects on how the consumer viewed credibility. Consumers were more likely to believe a newspaper article over an online press release. Golan and Baker (2012) and Jo (2005) found that consumers gave newspapers the highest ratings in credibility. Online news consumption opens the door for more sources of news; in addition to citizen journalism, users have access to mainstream media sources, independent news sources, and also index-type news sources, such as news.google.com and yahoo.news.com. Users like index-type news sources for their interactivity, but continue to rate mainstream sources as the most credible. Independent news sources like The Drudge Report and Axis of Logic were rated the least credible (Chung, Nam, & Stefanone, 2012).

Chung, Nam and Stefanone (2012) said young adults like online news, particularly index-type news sources. Users liked that places like Google allowed them to find a lot of information in one place, and that the index sources had interactivity and hypertextuality, meaning that one article would have hyper links to several other articles or sources of information. Users like online news for the convenience.

Some of how users see credibility of news articles shared on social media has to do with which outlet or friend is sharing the information. Users were more likely to find an article more trustworthy if it was shared by one of their friends on Facebook, rather that if it was shared by the news outlet itself (Turcotte, York, Irving, Scholl, & Pingree, 2015). Turcotte, York, Ivring, Scholl, and Pingree (2015) found that if a friend a person viewed as an “opinion leader” shared a news article on Facebook, the user was likely to trust that article and also seek out news articles form the source the article was shared from. If a friend who was not viewed as an “opinion leader” shared an article, users were more likely to see that article and news source as not trustworthy.

What’s interesting is users seem to care more about news, and find it more believable, if the news itself is about social media (Fisher, Magee, & Muhammed-Baksh, 2015). Fisher, Magee, and Muhammed-Baksh (2015) found that college students thought news stories about social media technology were more credible than news stories about economics. Other than stories about social media, students cared little about whether the information was credible or not. Survey participants had no difference in attitude about if the information came from a journalistic source or an external source.

News consumption seems to be at an interesting juxtaposition where the majority of users are getting their information from online sources or social media, but they think broadcast media and print media are more trustworthy. (Chung, Nam and Stefanone, 2012; Golan & Baker, 2013; Pew, 2016). What does this mean for the future landscape of news consumption and news sharing? In a time period where news outlets already have low credibility ratings with users, spreading fake news through social media will be even more damaging to credible outlets’ reputations.

H1: University students are as likely to share non-credible news articles as they are to share credible news articles.

H2: University students are more likely to get news from social media than from other sources.

RQ1: Do university students trust non-credible news sources on Facebook as much as credible news sources?

RQ2: Do university students rate fake Facebook articles with higher overall credibility than real Facebook articles?

Method

To retrieve data from as many students as possible, a survey was conducted by sending a Survey Monkey link to over 50 Shippensburg University professors. These professors then gave the link to the students of at least one of their courses, producing 110 participants. This is a form of snowball sampling, a non-probability type of sampling. Although this study used non-probability sampling, students were recruited from multiple departments around campus. Participants did not just consist of communication journalism majors or any other single major. Each participant indicated that they were 18 years of age or older, and that they were a current student of Shippensburg University of Pennsylvania.

When they first opened the questionnaire, participants were first asked to consent to the informed consent form. The next two questions asked participants, through multiple choice questions, to provide their current academic year (freshman, sophomore, etc.) and whether they had a Facebook account or not. The next question (question number 4) was also multiple choice, and asked how frequently each participant used seven different news sources. Participants were asked to indicate how frequently they used each news source. Question 4 asked the participants to choose their frequency of use by selecting “Never, Monthly, Weekly, or Daily” for each news source. Facebook, Twitter, and Instagram were combined, but their Cronbach’s Alpha value was 0.686.

The remaining questions referred to the three real and three altered news articles. For each article the participant was to rate on a Likert scale (strongly disagree, disagree, neutral, agree, or strongly agree) how believable, trustworthy, and biased they thought the article was. For each article the participant was also asked to rate how likely they would be to share the article. This was also measured on a Likert scale (very unlikely, unlikely, neutral, likely, or very likely).

Three separate articles were used in this survey. After creating an altered version of each article, they were labeled Real1, Fake1, Real2, Fake2, Real3, and Fake3. So, Real1 and Fake1 were the same article with only minor differences in the article’s source. For example, Real1 was an article about a naked man who drove a stolen cab through Rittenhouse Square from 6ABC.com, and Fake1 was the same article from TBSDaily.com. The news source TBS Daily was a source fabricated by the researchers in this study. Real2 was an article about the Ford Motor Company recalling 36,000 vehicles for an air bag defect from ABC27.com, while Fake2 was the same article from NewNowDaily.com. Real3 was an article about the Australian Competition and Consumer Commission’s report on raising hearing aid prices from ABC.net.au, while Fake3 was the same article from AboveAverage.com. Articles Fake2 and Fake3 were given fabricated news source logos as well as the fabricated new source web address.

Asking the participant to rate how believable, trustworthy, and biased each article was, was used to calculate the overall perceived credibility each participant had for each article. The responses for each article’s believability, trustworthiness, and biased ratings were combined through factor analysis to create a single credibility rating for each article. The combined credibility ratings for the articles were labeled Real1Cred, Fake1Cred, Real2Cred, Fake2Cred, Real3Cred, and Fake3Cred. The factor analysis for Real1Cred yielded 0.640 for a Cronbach’s Alpha value. Fake1Cred yielded a 0.664 Cronbach’s Alpha value. Real2Cred had a 0.815 Cronbach’s Alpha value. Fake2Cred’s value was 0.837. Real3Cred’s value was 0.830, and Fake3Cred had a Cronbach’s Alpha value of 0.853. The Cronbach’s Alpha value threshold needed to have a reliable factor analysis is 0.7. Any value under 0.7 is not considered reliable by most standards. This means that Real1 and Fake1, the articles about a naked man driving a stolen cab through a town square, had less than reliable Cronbach’s Alpha values. This could affect statistical data and should be noted when analyzing the results that included Real1Cred and Fake1Cred.

Results

Question number 2 of the survey asked the participants to indicate their current academic year. Out of the 108 participants who indicated their academic year, 31.5% indicated that they were freshmen, 23.1% indicated they were sophomores, 24.1% indicated they were juniors, 15.7% indicated they were seniors, 5.6% indicated they were graduate students, and no respondents indicated that they were not a student of Shippensburg University of Pennsylvania (see Table 1). There were 110 respondents to question number 3, which asked participants to indicate whether they had a Facebook account or not, and 104 indicated that they did have an account.

RQ1 asked if students trust real articles as much as fake articles. To answer this question, a paired sample t-test was done. The trust levels for each article were labeled TrustR1, TrustF1, TrustR2, TrustF2, TrustR3, and TrustF3 to match each corresponding real and fake articles. TrustR1 and TrustF1 were paired (t=6.460, df=109, p<0.01) (see Table 3). TrustR1 had a mean of 3.41 on a 1 to 5 scale, and a standard deviation of 0.980. TrustF1 had a mean of 2.75 on a 1 to 5 scale, and a standard deviation of 0.999. TrustR2 and TrustF2 were paired (t=8.202, df=109, p<0.01) (see Table 4). TrustR2 had a mean of 3.86 on a 1 to 5 scale, and a standard deviation of 0.872. TrustF2 had a mean of 2.87 on a 1 to 5 scale, and a standard deviation of 1.093. TrustR3 and TrustF3 were paired (t=9.767, df=108, p<0.01) (see Table 5). TrustR3 had a mean of 3.57 on a 1 to 5 scale, and a standard deviation of 0.886. TrustF3 had a mean of 2.48 on a 1 to 5 scale, and a standard deviation of 0.939. These results show that on average, participants gave a higher trust rating for real articles than fake articles. The answer to RQ1 is that participants rated fake articles with significantly lower trust than real articles.

RQ2 asked if students rated fake articles with the same credibility rating as real articles. To answer the question, a paired sample t-test was done. Combining the participants’ ratings of each article’s trustworthiness, believability, and level of bias created the overall credibility levels. The credibility levels for each article were labeled Real1Cred, Fake1Cred, Real2Cred, Fake2Cred, Real3Cred, and Fake3Cred to match each corresponding real and fake articles. Real1Cred and Fake1Cred were paired (t=6.382, df=109, p<0.01) (see Table 6). Real1Cred had a mean of 10.48 on a 1 to 15 scale, and a standard deviation of 2.208. Fake1Cred had a mean of 9.14 on a 1 to 15 scale, and a standard deviation of 2.406. Real2Cred and Fake2Cred were paired (t=6.912, df=109, p<0.01) (see Table 7). Real2Cred had a mean of 11.54 on a 1 to 15 scale, and a standard deviation of 2.208. Fake2Cred had a mean of 9.64 on a 1 to 15 scale, and a standard deviation of 2.920. Real3Cred and Fake3Cred were paired (t=9.195, df=107, p<0.01) (see Table 8). Real3Cred had a mean of 10.29 on a 1 to 15 scale, and a standard deviation of 2.453. Fake3Cred had a mean of 7.85 on a 1 to 15 scale, and a standard deviation of 2.699. These results show that on average, participants gave a higher credibility rating for real articles than fake articles. The answer to RQ2 is that participants rated fake articles with significantly lower credibility than real articles.

H1 predicted that students would share articles regardless of their credibility rating. To test this, a correlation test was done. The values of participants’ likeliness to share each article was labeled Real1Share, Fake1Share, Real2Share, Fake2Share, Real3Share, and Fake3Share to match each corresponding real and fake articles. Real1Cred and Real1Share were compared (r=0.146, df=106, p=ns) (see Table 9). Real1Cred had a mean of 10.48 on a scale of 1 to 15, and a standard deviation of 2.208. Real1Share had a mean of 1.39 on a scale of 1 to 5, and a standard deviation of 0.783. Fake1Cred and Fake1Share were compared (r=0.118, df=108, p=ns) (see Table 10). Fake1Cred had a mean of 9.14 on a scale of 1 to 15, and a standard deviation of 2.406. Fake1Share had a mean of 1.36 on a scale of 1 to 5, and a standard deviation of 0.763. Real2Cred and Real2Share were compared (r=0.220, df=108, p<0.05) (see Table 11). Real2Cred had a mean of 11.54 on a scale of 1 to 15, and a standard deviation of 2.208. Real2Share had a mean of 1.87 on a 1 to 5 scale, and a standard deviation of 1.220. Fake2Cred and Fake2Share were compared (r=0.348, df=107, p<0.01) (see Table 12). Fake2Cred had a mean of 9.64 on a scale of 1 to 15, and a standard deviation of 2.920. Fake2Share had a mean of 1.61 on a scale of 1 to 5, and a standard deviation of 1.018. Real3Cred and Real3Share were compared (r=0.186, df=106, p=ns) (see Table 13). Real3Cred had a mean of 10.25 on a scale of 1 to 15, and a standard deviation of 2.451. Real3Share had a mean of 1.62 on a scale of 1 to 5, and a standard deviation of 0.904. Fake3Cred and Fake3Share were compared (r=0.322, df=105, p<0.01) (see Table 14). Fake3Cred had a mean of 7.85 on a scale of 1 to 15, and a standard deviation of 0.749. Fake3Share had a mean of 1.42 on a scale of 1 to 5, and a standard deviation of 0.749. The correlation test values for each comparison are either negligible or very weak. These low correlation test values and the fact that half the tests are inconclusive due to their significance values makes H1 difficult to support or refute. The few tests that were significant and had weak correlation values do suggest a slight overall positive relationship between an articles perceived credibility and likelihood to share.

H2 predicted that most participants would get their news from social media. To test this, simple descriptive statistics were analyzed (see Table 2). It should be noted that a chi-square test was completed, but no tests came back significant. The first news source, television, had a mean of 2.62 on a scale of 1 to 4, and a standard deviation of 1.040. Newspaper had a mean of 1.46 on a scale of 1 to 4, and a standard deviation of 0.700. Radio had a mean of 2.36 on a scale of 1 to 4, and a standard deviation of 1.148. News websites had a mean of 2.78 on a scale of 1 to 4, and a standard deviation of 1.053. Facebook had a mean of 2.90 on a scale of 1 to 4, and a standard deviation of 1.194. Twitter had a mean of 2.31 on a scale of 1 to 4, and a standard deviation of 1.400. Instagram had a mean of 2.60 on a scale of 1 to 4, and a standard deviation of 1.415. Although Facebook had the highest mean value, Twitter and Instagram had mean values below some of the non-social media news sources. As a whole, participants did not rate social media higher than all other news sources.

Discussion

Facebook was the highest rated news source, overall there was a weak or no significant correlation between the articles’ credibility and likelihood to share, and participants gave higher levels of trust and credibility ratings to real articles than they did to fake articles. The Shippensburg University students who participated in this study were more literate on social media than predicted. This is good news for the future of social media as a place where users find news. If students are able to determine articles that others post and share are not credible, that will be helpful in stopping the spread of fake news on social media. It is a positive implication.

This study used a non-probability snowball sample. Non-probability sampling does not allow a study to generalize their findings to a larger population. Even if a probability sample was taken, limited resources such as time and financial means only allowed this study to recruit participants from Shippensburg University.

Other than sampling issues, this study’s questions could have caused confounding variables. For example, the topics of the articles used were about a naked man crashing a car, a Ford Motor Company recall, and a hearing aid scandal. If a participant had a personal experience involving someone crashing a car they might have given those articles a lower likelihood-to-share rating. The other two articles (Ford recall and hearing aid articles) were meant to have a more neutral topic, but these articles may come off as dull. Some participants may have given these articles a low likelihood-to-share rating, because they wouldn’t want a dull or boring article on their social media accounts.

There were time constraints on this study; the survey was active for only about a week and half. Given more time and an opportunity for a larger sample, the research may have been able to get a more accurate picture of social media use as a news source and whether more students would share fake news. The time constraints and sampling limitations also limited how the researchers conducted the study. Being able to show the participants full articles, rather than just Facebook posts, or having the ability to tell if the participants would actually click on and read full articles before posting or sharing them would be useful. There also could have been a more comprehensive result if researchers had asked participants general questions about how they view the media. This study did not ask what attitude the participants had toward the media, or whether they generally find the media trustworthy, credible, or biased. It also did not ask if the participants generated any citizen journalism themselves, or if that was something university students are interested in. A follow up study could be done on those subjects.

This study focused only on Facebook. A future study expanding to other social media outlets would be useful, as research shows that Reddit and Twitter are two of the biggest news sources for adults in the U.S. (Pew, 2016). Studies could be done on those sites individually, or they could be grouped together in a larger study. There are also some things learned during the literature review that would be good ideas for future studies expanding on this topic. Turcotte, York, Irving, Scholl, and Pingree (2015) found that people were more likely to find news articles shared by friends more trustworthy than when they were shared directly by the media outlet that reported them. This survey did not touch on how influential friends and family postings on social media are on university students’ credibility ratings.

The Pew study (2016) also shows that users who find news on social media are not actively seeking it; it is more of something that just pops up in their timelines as they are scrolling. Studies could be done about why social media users don’t seem to care to seek out news. Do they not think current events are important? What would get younger media consumers interested in news?

There was another limitation with the sampling. As college students, there is a big possibility that participants are more in tune with media credibility on social media. While presenting survey results, the conductors learned that at least one professor taught students about how to spot fake news articles. Another study could be done that only uses incoming freshman as a sample, and survey them before and after they learn about fake news and how to be sure of a news source’s credibility. That could be useful in researching if and how teachers address media literacy with students.

It would be useful to expand this study outside of Shippensburg University. A study involving participants outside of an academic setting could be vastly different. A general study of adults would be interesting.

For future studies, a survey format is not recommended. An experiment method may have produced more controlled results. Instead of asking all participants to rate both real and fake articles, maybe have the experimental group rate fake articles’ credibility and likelihood to share and have a control group rate real articles. As discussed, there are several opportunities to expand on or follow this study.

In conclusion, the results of this survey, though not what the researchers predicted, are positive. Media literacy will continue to be important as the future of the news industry is gravitating to online and social media sources; it is good news that younger generations understand it.

References

Chung, C. J., Nam, Y., & Stefanone, M. A. (2012). Exploring online news credibility: The relative influence of traditional and technological factors. Journal of Computer-Mediated Communication, 17(2), 171-186.

Fisher, H. D., Magee, S., & Mohammed-Baksh, S. (2015). Do they care? An experiment exploring millennials’ perception of source credibility in radio broadcast news. Journal of Radio & Audio Media, 22(2), 304-324.

Golan, G. J., & Baker, S. (2012). Perceptions of media trust and credibility among mormon college students. Journal of Media and Religion, 11(1), 31-43.

Jo, S. (2005). The effect of online media credibility on trust relationships. Journal of Website Promotion, 1(2), 57-78.

Pew Research Center. (2016). News use across social media platforms. Retrieved from: http://www.journalism.org/2016/05/26/news-use-across-social-media-platforms-2016/

Turcotte, J., York, C., Irving, J., Scholl, R. M., & Pingree, R. J. (2015). News recommendations from social media opinion leaders: Effects on media trust and information seeking. Journal of Computer-Mediated Communication, 20(5), 520-535.

Zúñiga, H. G., & Hinsley, A. (2013). The press versus the public. Journalism Studies, 14(6), 926-942.

Table 1

Descriptive Statistics for Participants’ Academic Standing (Question 2)

Academic Standing	n	%
Freshman	34	31.5%
Sophomore	25	23.1%
Junior	26	24.1%
Senior	17	15.7%
Graduate student	6	5.6%

Table 2

Means and Standard Deviations for News Sources (H2)

	M	SD	N
Television^a	2.62	1.040	110
Newspaper^a	1.46	0.700	110
Radio^a	2.36	1.148	108
News Websites^a	2.78	1.053	110
Facebook^a	2.90	1.194	109
Twitter^a	2.31	1.400	110
Instagram^a	2.60	1.415	109

^a Measured on a scale from 1 (Never) to 4 (Daily).

Table 3

Paired-Samples t-test for Differences Between TrustR1 and TrustF1 (RQ1)

	M	SD	t
TrustR1 ^a	3.41	0.980
TrustF1^a	2.75	0.999	6.460**

^a Measured on a scale from 1 (Strongly Disagree) to 5 (Strongly Agree).

**p < .01

Table 4

Paired-Samples t-test for Differences Between TrustR2 and TrustF2 (RQ1)

	M	SD	t
TrustR2^a	3.86	0.872
TrustF2^a	2.87	1.093	8.202**

^a Measured on a scale from 1 (Strongly Disagree) to 5 (Strongly Agree).

**p < .01

Table 5

Paired-Samples t-test for Differences Bes tween TrustR3 and TrustF3 (RQ1)

	M	SD	t
TrustR3^a	3.57	0.886
TrustF3^a	2.48	0.939	9.767**

^a Measured on a scale from 1 (Strongly Disagree) to 5 (Strongly Agree).

**p < .01

Table 6

Paired-Samples t-test for Differences Between Real1Cred and Fake1Cred (RQ2)

	M	SD	t
Real1Cred^a	10.48	2.208
Fake1Cred^a	9.14	2.406	6.382**