Monday, April 4, 2016

Thoughts on the Reuters "Expose" About SAT Cheating-Part One

If you've been a reader of this blog for awhile, you know that I was concerned with the topic of standardized tests even before I started work with Method Test Prep. It's no secret that I have been critical of, among other things, the role of standardized testing in college admissions, the value of standardized tests, and the overall competence of the College Board and the ACT. However, I'm even less of a fan of bad journalism.  Last week I read a pair of articles from Reuters that, to me, went far beyond criticism and veered all the way over to "hatchet job" territory. In response, I have compiled my own two-part piece. You can find the second one here.

On Monday, March 28, Reuters published a set of very long articles (seriously, they combine to be over 8300 words) by Renee Dudley, Steve Stecklow, Alexandra Harney and Irene Jay Liu. The first one, "As SAT Was Hit By Security Breaches, College Board Went Ahead With Tests That Had Leaked" was accompanied by part two, menacingly titled "How Asian Test-Prep Companies Swiftly Exposed The Brand-New SAT".  Part one of my response will focus on what seems like inconsistent editing and reasoning within the article, and on the hints of a "Yellow Peril" nativism within the Reuters pieces. The authors seem convinced that Chinese (and, to a lesser extent) South Koreans are gaming the system to "take" places at American colleges and universities from (presumably honest) Americans.


Here is a brief synopsis: the piece begins with the tale of Xingyuan Ding, a sophomore at UCLA ("one of America's most exclusive public universities") who got a perfect 800 on the Critical Reading section of the (now defunct) old SAT as a result of illicit aid from "a Shanghai test-preparation school" that took advantage of the College Board's recycling of previously administered tests for overseas administration.  According to the article, Ding says that "...he already knew the answers to about half of the reading section when he took the test in Hong Kong". The article goes on to refer to a leaked internal PowerPoint from the College Board that seems to acknowledge that "half of the SATs in inventory" in June 2013 were "compromised--the College Board's term to describe exams whose contents have leaked...outside the organization. Four of the exams had been compromised by an unnamed 'Chinese website.'"

The article goes on to describe the rising numbers of international students in American colleges and universities (particularly from Asia) and asserts that the College Board reliance on old tests is a major security hole that calls into question the validity of thousands of test scores. The articles are solely about the College Board and the SAT, apparently because "[t]he SAT is taken by far more foreign students applying to U.S. colleges than the ACT is." The authors are very hung up on the growth of international student enrollment, to the point where they seem to have got carried away. At one point they call China "the SAT's largest market by far", and then note that "29.000 [students] from China" took the test in 2013-14. But considering that almost 2,000,000 Americans took it that year it's hard to see that China is a market that challenges the US in size.

The article attempts to lay out economic motivations for the College Board's inaction in the face of what Reuters believes to be widespread dishonesty that gives "foreign applicants" an "unfair advantage" that leads to "displacing Americans". Blame is also apportioned to what are variously (and perhaps carelessly) labeled "test-preparation centers" and "Asian cram schools", as well as to less than virtuous American teenagers who take to Reddit and College Confidential as soon as they leave a test center to violate the confidentiality agreement they signed only hours before.


As I noted above, these articles seem to substitute what Stephen Colbert would call "truthiness" for actual facts, mostly in the use of insinuation that provides what may be a false sense of cause and effect, as well as lapses in logic that expose weakness of some of the "evidence" supplied by the authors.

One example of insinuation appears when the authors discuss the economic implications of increased test security. As noted above, the article is based on the revelation that when the SAT is administered overseas, they use previously administered tests, the contents of which have, by then, leaked into the public domain, and the article implies that the College Board is too cheap to pay money to prevent this. Discussing the costs related to reducing the number of international test dates (which would require the use of fewer old tests):
"...Reuters calculated that limiting the number of sittings in China would have cost the College Board roughly $1.2 million to $1.5 million in lost revenue over the next fiscal year. The College Board declined to comment on that estimate. The organization reported net income of $99 million on revenue of $841 million in the year ended June 2014."
The College Board is officially a non-profit, though figures like the one above seem to make that point somewhat laughable. But let's assume that Reuters is right, and the College Board was aware of serious security issues on their tests. It doesn't seem likely that a major corporation would risk their reputation and market share for 1.2% of their net annual income (.14% of total revenue), especially when the solution could be as simple as commissioning more original tests.

According to the article, if the College Board were to stop reusing old exams, it might be forced to "double the current fee" to cover the cost of creating new tests.  The authors note that "it costs $54.50 to take the SAT; students in East Asia also pay a $53 surcharge. That's a fraction of the thousands of dollars many Asian students pay to attend test-prep centers." So international students already pay nearly 100% extra to take the test; does that mean they would pay 200% extra for secure tests? Or everyone would face a 100% increase? The article isn't clear.

Another example of the article being oddly vague is how they refer to the Educational Testing Service (ETS). Most people think of the College Board as the company behind the SAT, but it is actually the product of the Educational Testing Service. According to their website, "ETS is a private nonprofit organization devoted to educational measurement and research, primarily through testing. We develop and administer more than 50 million achievement and admissions tests each year at more than 9,000 locations in the United States and 180 other countries." In other words, ETS devises and creates the SAT to the specifications of the College Board. However, according to the article, the ETS is "the College Board's Security Contractor". I found this to be curiously less than fully accurate, and I still can't figure out why Reuters chose this appellation, especially since the article keeps berating the College Board for security lapses, not ETS.

Another example of insinuation is that, even though the authors acknowledge that SAT content has been compromised in "South Korea, Egypt, Saudi Arabia and China", the focus, right from the beginning of the article is on China (and to a much lesser degree, South Korea). The authors state plainly that:

"Security breaches abroad are increasingly significant for U.S. higher education because schools are allocating more seats than ever to foreign students. About a third of the 761,000 degree seeking foreign students in America come from China, according to the Institute of International Education. Overseas students are especially attractive because most don't qualify for financial aid and thus pay full price. Chinese students spent almost $10 billion on tuition and other goods and services in America in 2014, Department of Commerce statistics show."

$10 billion dollars is a lot of money, but the American GDP is $18 trillion, so Chinese student spending is not even a drop in the overall bucket--I think this is another example of the article trying to imply something that may not be as significant as it seems at first blush.

As far as international enrollment goes, the always indispensable Jon Boeckenstedt has compiled information from the Integrated Postsecondary Data System (IPEDS) confirming that from Fall 2004-Fall 2014 international enrollment at the bachelor's level has increased 66%. To dive more deeply into this, one can run one's mouse over Reuters' accompanying graphs, which reveals that at the undergraduate level (i.e., the students who would take the SAT), there were 124,000 Chinese-born freshmen enrolled in American colleges and universities in 2014. That's a big number, but the article doesn't put it in perspective: there were nearly 18 million total college students, so it's not like U.S. citizens are getting elbowed out of the classroom.

All of the information in the charts and the text seem to conflate "international" with Chinese. As I noted above, this is misleading, and furthermore, the article seems to imply something more sinister. The authors suggest that "Evidence that some foreign applicants are displacing Americans because of an unfair advantage on the SAT could add to a backlash against standardized testing in college admissions." They quote Douglas L. Christiansen, Dean of Admissions at Vanderbilt and chair of the College Board Board of Trustees, who says "It is hurting all students when someone cheats on any aspect of their application. It is displacing someone else."

The implication, of course, is that "someone" is Asian and "someone else" is American. This seems similar to some of the divisive and inflammatory language that we have seen in this year's Presidential election, and it does not seem like something I'd expect from a leading journalistic outlet. Let's take the "top hosting institutions" listed in the graph above. None of these are what you would call "small, residential colleges". While they all might guarantee housing for first-year students, their admissions offices are not limited by available beds. Furthermore, while it might seem logical that every international student who enrolls reduces the number of spaces for domestic students by one, that is not actually the case. Colleges typically only allocate a small percentage of places to international students, so in reality the students from other countries are competing with each other, not with Americans. Colleges and universities make their decisions on how many students they want to enroll based on many complex factors, which makes this analysis rather simplistic. And if anyone knows that standardized test scores are not the most important factor in admissions, it is the admissions people themselves.

Please read the rest of my thoughts on the Reuters articles here, and feel free to share your thoughts in the comments below.

No comments:

Post a Comment