Saturday, June 16, 2018

Thoughts About The New ACT/SAT Concordance Tables-Summer 2018

Longtime readers of this blog will know that I work with Method Test Prep, a national ACT/SAT preparation company whose mission is to level the playing field of standardized testing. As such, I have previously written in this space about developments relating to the ACT and SAT (here's just one example) and will continue to do so when developments call for it. The joint publication of concordance tables by the ACT and SAT will be a great source of answers to questions for people about how to compare scores across the two tests, but it also raises some questions that I hope will encourage discussion among all stakeholders. My colleague Evan Wessler and I collaborated on an article for the Method Test Prep blog about this and I've reposted it here as well. I hope that you will find it interesting and useful.


In Accordance With Concordance

by Evan Wessler and Ethan Lewis

Where some see significant change, those who look deeper see something quite different. When it comes to SAT and ACT concordance, it's important to know what's at stake.
Let's [Concor]dance!
Because the ACT and SAT are different tests with distinct scoring scales, students' results are not automatically easily comparable. But there needs to be a way to reconcile scores. Students who have taken both exams naturally want to know if their scores on one test are higher than their scores on the other; counselors want to be able to advise their students properly; colleges, universities, and scholarship providers want to make sure that student scores meet or exceed their cutoff criteria. To accomplish this, we need a document called a concordance. When the College Board released a new SAT in 2016, it changed the the test's scoring scale––shifting from 2400 points back to 1600 points––and unilaterally released a concordance that converted new SAT scores to old ones, and then converted these to ACT scores. This provoked the ire of the ACT, which dismissed the new tables as invalid due to a lack of available score data from the new SAT. Eventually, the College Board committed to cooperating with the ACT to establish new (and, in the eyes of the ACT, credible) concordance tables; two years later, the new concordance is now available, and should be used by all parties interested in comparing scores across the two exams. 
The more things change...
The whole idea behind this spat was that the SAT changed in a big way––so big, in fact, that the previously determined concordance between the exams would no longer hold. While the new SAT is genuinely a very different exam than its predecessor, the more things change, the more they stay the same. Here's a snapshot of the former and current SAT-ACT concordance tables, with old and new conversions shown. 
Adapted from Guide to the 2018 ACT®/SAT® Concordance, The College Board, 2018.
The yellow cells highlight the scores that have apparently shifted. Looks like a lot of change, doesn't it? The conclusion most organizations have drawn is that things have gotten "better" for SAT students and "worse" for ACT students. For an example that shows why, take a look at the 1340 SAT score. According to the table, this used to be equivalent to a 28 on the ACT, but is now worth a 29. Conversely, in the reverse direction, a 28 on the ACT now lands a student the equivalent of a 1320 on the SAT, whereas it used to be worth as much as a 1340. This interpretation, however, is a bit too simplistic.
Free Samples!
When we take a deeper look, however, we begin to see how such small differences are all but irrelevant. To understand why, we must learn more about the statistical methods used to generate these concordance tables.
In order to produce a concordance, the College Board and ACT must collect data by sampling. That is, because it would be impractical for the organizations to use data from every single SAT and ACT examinee or test, they instead make inferences from a subset of the available data (in this case, 589,753 members of the class of 2017 who took both tests). Regardless of the statistical methods used to generate average score equivalences across exams, sampling inherently generates a certain degree of variability, known by statisticians as standard error, in the final numbers. You're probably familiar with standard error of a sampling statistic: when you see a "±" value, that's the standard error talking.
In this document, the College Board states the standard error of the score conversion values as follows.
When using the SAT Total and ACT Composite concordance table to estimate a student’s proximal ACT Composite score from their SAT Total score, the estimates in the table have a standard error of approximately ± 2.26 (2) ACT Composite score points on its 1–36 point scale. When using this table to estimate a student’s proximal SAT Total score from their ACT Composite score, the estimates have a standard error of approximately ± 79.57 (80) SAT Total score points on its 400–1600 point scale. (The emphasis is my own.)
Let's return to the example scores we used before to demonstrate how things supposedly got "better" for SAT takers and "worse" for ACT takers. Using the table alone, we might conclude that an SAT score of 1340 used to concord to a 28 on the ACT, but now concords to a 29. But the 29 in this table is not really a 29: it's 29 ± 2. Because of the way standard error is calculated, the practical interpretation of the measurement plus-or-minus the standard error is this: we are 68% confident that a score of 1340 on the SAT concords to an ACT score between 27 and 31. Notice how this range comfortably includes the 28 that the 1340 used to "equal". It doesn't take long to see that, when extended to all of the other values in the table, the standard error erases the apparent changes in the tables, placing them well within the ranges of confidence produced by the sampling method.
The long and short of it is this: any sampling method used to generate concordance produces not "exact" numbers, but instead ranges within an acceptable degree of confidence, or certainty. Thus, the concordance table alone does not tell the whole story. When standard error of the numbers in this table is taken into account, we reach a simple conclusion: the concordance table hasn't really changed, and things have not gotten markedly "better" or "worse" for either SAT or ACT takers. 
So You're Saying I Have A Chance...
Despite the mathematical fact that scores on the two tests are essentially the same (in relation to each other) as they were before the new concordance tables were published, the story doesn't end there. Many colleges and universities publish score thresholds for scholarships based on the old, unadjusted concordance. Similarly, some states offer their residents reduced (or free) tuition based on test scores, and unless they speedily change their documentation, the new concordance tables might seem to advantage one test over another. Let's take a look at a few examples:
At Louisiana State University, recipients of the Academic Scholars Award get $15,500 per year based on an ACT score of 30-32 or an SAT score of 1330-1430 and a cumulative 3.0 GPA. With the new concordance, 30-32 ACT concords to 1360-1440. So, well-meaning advisors might tell students that they should take the SAT because they can score lower (by getting a 1330, which concords to a 29) and still be awarded the scholarship. 
At Liberty University, students can get into the Honors Program with a 28 ACT or a 1330 SAT. Since the new concordance equates a 1330 to a 29 ACT, if Liberty doesn't change its documentation, students might conclude that it would be wiser to take the ACT and shoot for a 28.
At the University of Arizona, the "Wildcat Excellence" award criteria are on a sliding scale based on ACT or SAT scores and high school GPA. As you can see in the table below, a small difference in test scores can be worth a lot of money.
Screen Shot 2018-06-15 at 3.05.29 PM
For instance, a student with a 3.8 GPA and an 29 ACT stands to receive $18,000. With the new concordance, the 29 on the ACT is a 1330-1350 on the SAT. But a 1380 on the SAT now concords to a 30 on the ACT, which based on the chart, would get our student $25,000. What should our student do? Take the ACT again and shoot for an actual 30? Take the SAT and try for a 1390? Either option might work, but for $7,000, it would make sense to do something, unless Arizona updates their table.
Similarly, the state of Florida's Bright Futures Program is a wonderful tool for ensuring college access and rewarding students with high test scores and grades. "Florida Academic Scholars" get 100% free tuition plus a stipend for books at state universities with a weighted cumulative GPA of 3.50 and a 29 ACT or 1290 SAT.
As we just saw, under the new concordance, a 29 ACT equates to an SAT score between 1330-1380. So if all a student must do to be a Florida Academic Scholar is get a 1290 (which is now a 27 ACT), it would seem like they should eschew the ACT and pursue the SAT instead, shooting for that 1330.  As we now know, the standard error makes the apparent difference mathematically insignificant, but if the state of Florida doesn't update its criteria, then there is an effective difference in "real life".
Final Thoughts
Because the SAT and ACT generate so much stress for students and uncertainty for everyone in the college admissions process, any change in the tests can generate a disproportionate level of anxiety. The hubbub over the concordance tables is understandable, and is surely something that should be understood by anyone involved in the college process. It's good that there is now an official, universally agreed upon conversion between the two college admissions tests, but it is crucial that applicants, advisors, and advocates make sure that colleges, universities, and scholarship agencies have updated their score thresholds so that students can pursue the test preparation that makes the most sense for them

Monday, February 19, 2018

"So You Don't Have To": A Visit to Duke University

It's been a long time since I was able to visit a college! Last summer my wife and I bought a new house and moving and getting settled pretty much consumed my free time for the last six months. But things are calming down, and I expect to be able to do a few more college visits in 2018. 

In mid-February I had the chance to visit Duke University. I was very impressed with the school and also with the town of Durham, which looks like a really excellent college town. And located as it is in the "Research Triangle" with close proximity to the University of North Carolina, it would be a great place for a family to do a multi-school college visit.  There's a lot to like about Duke, and while it is one of the most selective institutions in America, for students who are looking at that kind of college it would be an excellent option.

Duke University At A Glance

Size:Just over 6,800 undergraduates (approximately 49% women/ 51% men). Duke University is one of the most selective colleges in America, having accepted about 3,300 of their 34,800 applicants to fill a first-year class of 1,750-- an overall acceptance rate of around 9.5%. 
Programs of Study:53 majors and 52 minors for undergraduates with 33 interdisciplinary certificates; Duke is a research university with graduate programs in business, law and medicine (among others), and students interested in pursuing careers in these fields are well prepared.
Sports:Duke has 25 NCAA Division I sports (13 women's/12 men's). Duke also has 37 club sports (which compete intercollegiately) and numerous intramural athletic options.
Campus Life:Duke's website lists over 700 student activity organizations. Over 100 clubs and organizations on campus; 22 fraternities and 17 sororities. On campus housing is guaranteed for all four years, and students are required to live on campus for the first 3 years. 
Costs & Aid:Tuition, room & board and fees total just about $72,200.  Parents need to fill out the FAFSA (Free Application for Federal Student Aid) and CSS Profile. Duke has a financial aid budget of nearly $120 million and are fully "need-blind"; they guarantee to meet 100% of the need for admitted students who are American citizens or permanent residents.
Deadlines:Duke University applicants can choose binding Early Decision, with a deadline of November 1 or Regular Decision with a deadline of January 2.  Students use the Common App or the Coalition App. The application fee is $85.
Tests:Duke requires considerable testing--students must submit either ACT (with optional writing section) or SAT (with optional essay). Students who take the SAT are encouraged to take two Subject Tests as well.  

©2018 Ethan Lewis
Duke University traces it's history back to the 19th century when it was a Methodist school called Trinity College. But the generosity of the Duke family (founders of American Tobacco, an original stock on the Dow Jones Industrial Average) financed massive expansion. Today, Duke University boasts a 9,350 acre campus including a 7,000 acre forest, a 55 acre garden, 2 undergraduate colleges and 9 graduate and professional schools. There is also a new campus in China.

Undergraduate applicants must specify whether they want to be in the Trinity College of Arts and Sciences or the Pratt College of Engineering. Approximately 80% of students choose Trinity leaving only a fifth of students in Pratt (though students can minor, or even double-major in the other college); this makes Duke a good choice for talented students interested in engineering or computer science who seek a small, close-knit community. According to the admissions presentation, 12% of Duke students double major; 12% have a major and an interdisciplinary certificate and fully half of all students have a major and a minor. According to my tour guide, Duke (especially Pratt) is generous with awarding Advanced Placement credit, which helps students to carry such heavy loads. Overall, the most popular majors include:

  • Public Policy Studies
  • Economics
  • Biomedical Engineering
  • Psychology
  • Biology

The admissions presentation also shared the most popular careers for Duke graduates (I wish everyone did this!) and they include: consulting, education, engineering, finance and health.

Duke proudly touts a commitment to diversity; 25% of the class of 2021 identify as Asian, 13% as African-American and 14% as Hispanic/Latinx.  Additionally, Duke has students from over 180 countries and all 50 states; the states sending the most students to Durham last year were:

  • North Carolina
  • New York
  • Florida
  • California
  • Texas

As you might know, North Carolina has been in the news over the last year or more due to issues involving inclusion for LGBT people. The Research Triangle area (along with Charlotte, Greensboro and Asheville) is more liberal than the rest of the state. Along those lines, I found it noteworthy that Assistant Director of Admissions Chris Briggs (who gave an excellent presentation for the info session) immediately introduced his preferred pronouns and invited us to visit the welcome center bathrooms ("two gender specific restrooms; use whichever you feel most comfortable with"). This was in a crowded room of nearly 400 visitors and I can imagine admissions reps at other schools being more cautious to avoid seeming too "liberal"; but I interpreted it as a very warm welcome and gesture of inclusion and community that gave me very positive feelings about Duke.

© 2018 Ethan Lewis
The tour only covered the West Campus (home to lovely neo-Gothic architecture). While there are quite a few residential options on West Campus, all first-years live on East Campus, which I didn't get to see (our tour guide encouraged visitors to approach students at random and ask to see their dorms, but I decided not to). After my visit the Admissions Office sent me a link to a webpage that shows all the Freshman housing options. My tour guide, Liz, was very enthusiastic in her description of Freshman housing; two things that stood out to me were that every dorm has a resident faculty member to add mentorship and support to the students and each dorm has a librarian attached to it so students always have someone to go to for help with research. As a veteran of 19 years' teaching at boarding schools who is married to a school librarian, these are both excellent features. Way to go, Duke!

© 2018 Ethan Lewis
The West Campus is not small, but it is quite walkable with a great deal of lighting--lampposts were everywhere and I expect that even students on campus late at night (the libraries are open 24/7) won't have to worry about it being too dark. Liz told us that she thinks it takes 20 minutes maximum to walk from one end to the other.

Shuttle buses run all day and most of the night (Liz said they don't run from 4-7am) to help students get from place to place (including the other campuses and downtown Durham). For the automotively inclined, all students, including first-years, can have cars on campus.

Due to the large number of visitors, the list of things NOT shown on this tour was lengthy. We were not shown:

  • the inside of any of the academic spaces (usually these are included)
  • the inside of any of the libraries (usually these are included)
  • the inside of any dining or health facility (usually these are included)
  • the inside or outside of any dorms
  • the inside or outside of any athletic/recreational athletic facility

 © 2018 Ethan Lewis
I've been on a LOT of campus tours, and it's not unusual for guides to skip some of these, and with hundreds of visitors on the day I was there I can understand the wish to avoid crowding or inconveniencing students and staff. That said, I think these are all important things to see. If possible (and especially if you are traveling a great distance), you might want to try to contact the admissions staff prior to your visit to make sure that you can see some or all of these spaces. I did make a point of eating lunch on campus at the new Brodhead Center, which contains over a dozen really awesome restaurants--I got an amazing BBQ Seitan dish at a vegan place, which was next to an Indian place, next to a Southern place, near an Italian place, and on and on. Do yourself a favor and check this place out if you are on campus--your stomach will thank you!

Student life wise, Liz told us that about 1/3 of students participate in Greek life, but she noted that pledging doesn't start until winter of the first year, so new arrivals can focus on academics. This policy also times it to coincide with basketball season, which is a pretty big deal at Duke. I didn't get any pictures (I was pressed for time), but my visit was smack in the middle of "tenting", when students set up a 24-hour per day campsite for six weeks to make sure that they are at the front of the line for tickets against arch-rival UNC in their annual basketball game. The tent city is known as Krzyzewski-ville after legendary coach Mike Krzyzewski. This was held out as a particular example of school spirit, but it seems like Duke students are extremely proud of being Duke students, and not just for basketball.

© 2018 Ethan Lewis
Academically, Duke is clearly a top-notch educational institution. The admissions presentation spent some time talking about an interesting program called "Duke Engage", which is a competitive (students have to apply) program that pays all the expenses for a student-planned educational trip with a social engagement component for a summer. Chris Briggs described it as "a fully funded chance to do something in the world; an 8-10 week summer experience to go humbly to learn and serve". Funded by the Bill and Melinda Gates Foundation (Mrs. Gates is a double alumn of Duke), this is a VERY exciting program. I read this article in the Duke magazine about it, which indicates that 80% of students who do Duke Engage "say that the experience influenced their career plans". I have no doubt that this would be a very strong reason to consider Duke for many students, especially those who are already committed to community service and engagement. 

Chris Briggs told the audience at the info session that there are "no minimums or cutoffs" and "no formulas" for admission to Duke. He said that Duke is looking for "talented, engaged, impactful, ambitious, thoughtful and diverse students". Students should take 5 academic courses per year, including 3 years of foreign language and applicants to Pratt College of Engineering need to take Calculus, with physics being "strongly recommended". We were told that students should take the most rigorous courses available to them and that they should aim for "not straight A's, but more A's than any other grade".  While he was saying this, I was reading the admissions guide they hand out to visitors that notes that "[m]ost students who apply to Duke are in the top 10% of their class." This indicates to me that students who can't take the most rigorous courses at their school might not stand a good chance. In other words, a student can be on a path where the most rigorous courses she can take are not the hardest at the high school, and that student might have difficulty gaining admission.

Due to my job with Method Test Prep I am especially attuned to how colleges employ standardized tests in the admissions process, and Duke has definitely gone "all in" on these tests. While Chris Briggs tried to downplay the tests by saying "we know it's just one Saturday in your life", the admissions guide says that students have to submit "either the ACT with writing or the SAT with essay. We also strongly recommend that students who submit the SAT also submit two Subject Test scores of their choice."  Realistically speaking, it stands to reason that most students applying to Duke will do the ACT and/or SAT multiple times, and two Subject Tests would be another test date; this means that Duke applicants will be paying a lot of extra money to the test agencies and will need to start doing it no later than winter of Junior year to be able to fit everything in.

And then when you look at the actual test scores Duke receives, the picture becomes more complicated. I totally believe them when they say that "there is no minimum score requirement", but these numbers somewhat belie that:

Trinity College of Arts and Sciences Middle 50% SAT: 1440-1570 | Middle 50% ACT: 31-35
Pratt College of Engineering: Middle 50% SAT: 1490-1570 | Middle 50% ACT: 33-35

In other words, 25% of admitted students get perfect 36 scores on the ACT, something that only one-tenth of one percent of test takers manage to do; the rate is similar for the students who score in the 99th percentile on the SAT (which is true for scores over 1530).  Of course it also means that 25% of students score lower but how much lower? And do those students stand out on campus? I don't think that people with average scores have "no chance" at Duke, but this is definitely a place where higher scores will really help.

Applicants can use either the Common Application or the Coalition Application. Applicants to Duke can choose between Early Decision and Regular Decision, and while the official line makes clear that there is a great advantage to applying early. Take a look at last year's numbers:

Early Decision Total Applicants: 3,503 |  Regular Decision Total Applicants: 30,985
Early Decision Total Accepted:      864  |  Regular Decision Total Accepted:      2,423
Early Decision % Accepted:          25%  |  Regular Decision % Accepted:               8%

Enrolled: 859 from ED, 892 from RD

Half the class came from the Early Decision pool. Does this mean that it's "easier" to get in by applying early? I doubt it, but it shows how very difficult it is to get into Duke through the regular admissions path. Oh, and Duke claims not to track "demonstrated interest" in admissions (though I think it's still worth it to visit such a super place).

Duke is beyond generous with Financial Aid. Families must submit the FAFSA and the CSS/Profile to allow Duke to best estimate their need. Duke is completely need-blind and they guarantee to meet 100% of need (for American citizens and permanent residents). Families who earn less than $60,000 per year will have NO parent contribution; that said, "half of families receiving aid" earned over $100,000. Aid comes primarily in the form of grants and work-study; loans are capped at $5,000 per year. The average amount of student debt at graduation is $18,000 (over $12,000 less than the national average). Applicants are automatically considered for over 100 merit scholarships, over half of which have "a need based component".  So yes, the "sticker price" of over $72,000 per year is eye-popping, but very few people will actually pay that much to go to Duke, so excellent students of limited means should definitely consider Duke when looking at colleges.

Duke may not be for everyone. While my tour guide Liz made a point of saying how helpful and supportive the faculty and fellow students are, my guess is that students who are not Type A, highly motivated, hard-working people might struggle at Duke. Further, while there seem to be ample support resources, people who aren't already very good students would probably fall behind. Based on hearing Liz' stories, reading some issues of the campus newspaper and the alumni magazine, Duke students seem to be proud of how "hard" Duke is and many people aren't looking for that in a college. But future doctors, lawyers, and businesspeople could do much worse than Duke. Factor in the charming city of Durham and the opportunities for jobs and cultural experience in the Research Triangle (one of America's fastest-growing metro areas) and Duke looks like a very good choice. 

Friday, September 22, 2017

What I Saw At NACAC 2017: A Summary Of "Testing Achievement and the Future of College Admission"

I attended the 2017 National Association for College Admission Counseling conference in Boston in mid-September. This was the third straight year I've been able to attend, thanks to my day job with Method Test Prep, a leading ACT/SAT preparation company. I spoke to lots of people (old friends, customers, colleagues) and the consensus seems to be that the college visits, the exhibits and the conference sessions were all very valuable.

I was able to attend two sessions:
  1. What Admission Deans Think-Results of an Inside Higher Ed and Gallup Survey
  2. Testing Achievement and the Future of College Admission 

The presenters for the session "Testing Achievement and the Future of College Admission" were:
  • Ben Wildavsky, College Board Senior Fellow & Executive Director
  • Lynn Letukas: College Board Associate Research Scientist
  • Edgar Sanchez: ACT Research Scientist in Statistical & Applied Research
  • Jerome Lucido: Professor at USC Center for Enrollment Research; Associate Dean of Strategic Enrollment Services
  • Jack Buckley: American Institutes for Research Vice President
Working as I do for a company that prepares thousands of students for the ACT and SAT while simultaneously recognizing the inherent flaws of standardized testing, I was quite excited for this session. The description of the talk was promising, though in the end I think the actual presentation did not live up to its potential. According to the brochure:
Despite widespread media coverage, underlying claims about the benefits of “going test-optional” have largely escaped empirical scrutiny, and support for these claims tends to be of limited generalizability and/or fail to adequately control for student selectivity and other factors. Hear a rigorous and balanced approach to the contemporary debate on standardized testing and the test-optional movement. Explore how test-optional practices emerged and expanded, how widespread grade inflation has made it increasingly difficult for institutions to solely rely on students' prior achievement, and how standardized admission tests can be used in predicting student retention, achievement, and graduation.
Looks good, right? Unfortunately, I found the session to be a disappointment. Instead of a debate, the members of the panel are contributors or editors to a forthcoming book called "Measuring Success" from Johns Hopkins University Press, and they spent the next hour giving lengthy summaries of a few of the chapters from the book. It didn't take long for the suspicious part of my nature (which is never far from the surface) to begin to wonder if this book would be "rigorous and balanced" the same way that Fox News is "fair and balanced" and would actually be a vehicle for the College Board and ACT to cast doubt on "test optional" policies to preserve their core businesses.

Ben Wildavsky introduced the panelists and then described the book. I'm amused when I look at my notes from the session; at first I wrote "PREORDER THIS BOOK!!!!", but as the talk went on I crossed that out and wrote "this book is going to be VERY dry and boring with lots of numbers. To help clarify? Or to obfuscate?" (there's that suspicious nature again). Wildavsky said that the premise of the book is that "arguments for test optional policies have not been studied rigorously; that they are based on isolated, non-reproducible case studies, and that they may have a downside, especially in a time of widespread grade inflation."

This led to the first presentation. Jack Buckley (whose biography notes that he "helped lead the redesign of the SAT") began by saying that he had eight minutes to discuss two topics: 

  1. To what extent should tests play a role in admissions?
  2. How should an institution best assess students in a manner that promotes fairness?

Interesting questions to be sure, but he didn't really talk about them. Instead, Buckley spent the next seven minutes giving a general overview of the forthcoming book, and then dove into a series of graphs describing what he saw as rising grade inflation at American high schools. According to his data, students' self-reported grades on SAT forms rose over time, while SAT scores decreased. Buckley says this has been confirmed by reviewing actual high school transcripts. Further, he said that research showed that "wealthier, less diverse schools have faster grade inflation than otherwise". More than once Buckley mentioned an article that had appeared in that morning's Inside Higher Ed--he seemed somewhat unhappy with the article and if you read it you will see why. I will be interested to read his research, but to me there are quite a few, if not "flaws", then "problematic areas" of this data. Specifically, he seems to use falling SAT scores as evidence of a negative trend, as opposed to seeing rising high school grades as a cause for celebration. Especially since the SAT has been completely revised (by Buckley!) to more closely hew to typical high school curricula since the data was recorded, doesn't that indicate that he might be viewing this through a distorted filter?

The next speaker was Edgar Sanchez of the ACT. He definitely spoke for longer than eight minutes and attempted to prove the following assumptions:

  1. There has been little research to show the impact of test prep and coaching.
  2. "Many studies" highlight the predictive power of the ACT. High School GPA is the strongest predictor of college success, but tests are valuable.
  3. Many institutions choose to go test optional out of concerns about diversity, but it's impossible to tell if test optional policies result in increased diversity because "other factors such as recruitment and awareness are present."

Sanchez told us that 25%-30% of the population had "discrepant ACT and High School GPAs" and that the only way to solve this "mixed message" was to consider both. Tests, he said, focus on cognitive skills while grades shed light on non-cognitive factors such as engagement, self-regulation, discipline and habits of inquiry. In other words, test optional schools lose the full picture. That might be true, but if he's right I could give up insights into "cognitive skills" if I still had all the other stuff he described. 

Part of the argument Sanchez made was that students with "high or standard GPAs and low ACT scores are more likely to suppress test scores and are less likely to be admitted. Not only that, but the ones who choose test optional are more likely to be women, poor or minorities and are less likely to come back for year two of college." I'll have to see the data, but this seems somewhat dubious to me. First of all, what's the proof that students with high grades who don't send test scores are less likely to be admitted? And secondly, he's basically saying that students with low ACTs are more likely to suppress those scores. But of course they are! And if the scores really are low, how would sending them help? It beggars the imagination to think that if high grades/no ACT is more likely to be denied that high grades/ low ACT are more likely to be accepted.

Sanchez summed up his point by concluding that "We stand behind the belief that more information is better than less information." Sure, that makes sense. But have the researchers at the ACT actually ever worked in college admissions?

Probably not, but the next speaker has. Jerry Lucido is both a scholar and a dean of college admissions, so I started to hope that he might engage with the topic in a more practical, less numbers-heavy way. Apparently his chapter in the book deals with "practitioners", but his presentation was rather problematic. First of all, he spent most of his time giving a long description of what test optional means and why institutions choose it, which only summarized what everyone in the room already knew. Lucido's point was that he "never doubted the sincerity of motivation to choose test optional", but that he was skeptical of the value of the policy. He concluded by saying that "the test optional practitioner is a pragmatic idealist" and that test optional admissions was "neither a panacea or the only way". In other words, he was the third straight presenter who told us that colleges should decide what's best for them, but they should decide on test optional policies after careful study. 

During the first two speakers I began to be aware of a certain, shall we say, restiveness on the part of the audience. Murmuring, people getting up and leaving, and lots of gesturing at friends in the audience increased steadily. Lucido's presentation didn't help with this, in part because of his constant stammering and repetition. He looked uncomfortable at the podium and this seemed to effect his speaking skills. The final speaker was Lynn Letukas of the College Board. Her presentation was marred by an inability to speak into the microphone and by the content. Basically she spent ten minutes describing every chapter of the book. That would have been fine if the purpose of the session was a book launch, but it purported to be a chance to engage with an important topic.

Just as I was becoming convinced that Letukas' role in the session was to eat up time and prevent questions, Wildavsky came back to the podium. He noted that there was 20 minutes left and that he wanted to leave "at least 15" for questions from the audience. By now, if you've been catching the tone of this summary you won't be shocked to learn that he only left seven minutes and there were already six people lined up at the microphones. Before asking for audience questions Wildavsky asked a couple of questions of the panel; the answer that seemed most emblematic of the whole session was when Buckley said that "case studies are not good ways to evaluate whether there is causality between test optional policies and student outcomes". 

When the seven minutes of audience questions began, I began to wonder even more if the fix was in. The first speaker was sloppily dressed and after asking if any research has been done to see why low GPA/high ACT scores exist (answer:no) wouldn't leave the microphone. At this point one member of the audience angrily shouted at Wildavsky (I didn't catch what he said) and stalked out of the room. 

The next question was from Jon Reider of University High in San Francisco. He asked "if the argument that test prep helps isn't rigorously proven, then why do ACT and SAT now urge test prep (from their preferred providers)?" He asserted that "research by anyone not from a test prep company shows that the impact of test prep is next to zero."  Ben Wildavsky responded with a short promotional statement about Khan Academy (the chosen partner of the College Board). Reider responded by saying that he thinks that "test prep is harmful, because it's a lost opportunity to study other things". Sanchez chimed in by agreeing that "any class time spent on test prep is inappropriate and has the chance to harm students." He said that people should do "some test prep, to avoid stress" but not more. 

That exchange was quite interesting to me. I work with hundreds of schools around the country, a large number of which use my company's ACT/SAT prep program. In many cases they are in states which require ALL juniors to take the ACT or SAT and reach a particular score--often the results of these tests are a big part of the school's official rating from the state Education Department and can impact the funding they receive. While this is almost certainly a misuse of the tests, it has also become an incredibly lucrative part of the test companies' business models. It's pretty bald-faced for an employee of the ACT to complain about declining test scores and then also complain about schools trying to raise them.

The next question came from an independent consultant from Brooklyn who asked if the book will be "promoted in a fair and balanced way". There was no clear answer, but since the question was so vague it didn't really deserve one. The final questioner was Wayne Camara who is a VP at the ACT. But guess what? Wildavsky interrupted to say that time was up and we had to clear the room right away to make space for the next session.  The other people waiting at the microphone were obviously disappointed. Jon Boeckenstedt, VP of Enrollment Management at DePaul (whose own writings are longtime favorite sources of mine) was next in line behind Camara and he made his way to the microphone to ask "How do you advise readers to consider research when writers and presenters have vested economic interest in the outcome?" As Wildavsky chided him for the tone of his question, Boeckenstedt smiled seraphically and walked away. 

As the speakers began packing up to leave, Camara and Boeckenstedt had a lengthy discussion at the back of the room. As this was going on, Jay Rosner of the Princeton Review Foundation told me that some of the back story of the tension in the room came from the fact that the editors had sought (largely in vain) for admissions professionals to write for the book but that it was evident that the articles would skew heavily against test optional policies. Hearing that made me even more willing to believe that the entire presentation was carefully structured to prevent any audience participation or, in fact, any argument in favor of test optional policies. In an era where people are increasingly sensitive to, and prejudiced against perceived bias, I can't help but worry that this book will be like lots of "fake news" and generate lots of heat, but shed little light on an important subject.

Thursday, September 21, 2017

What I Saw At NACAC 2017: A Summary Of "What Admission Deans Think"

I attended the 2017 National Association for College Admission Counseling conference in Boston in mid-September. This was the third straight year I've been able to attend, thanks to my day job with Method Test Prep, a leading ACT/SAT preparation company. I spoke to lots of people (old friends, customers, colleagues) and the consensus seems to be that the college visits, the exhibits and the conference sessions were all very valuable.

I was able to attend two sessions:
  1. What Admission Deans Think-Results of an Inside Higher Ed and Gallup Survey
  2. Testing Achievement and the Future of College Admission 

The presenters for the session on "What Admission Deans Think" were:

  • Scott Jaschik, the editor of Inside Higher Ed
  • David Hawkins, Executive Director for Educational Content and Policy for NACAC
  • Mary Ann Willis, the Director of College Counseling at Bayside Academy (Alabama)
I have to admit that I didn't read the full description of this session, and thought it would be more along the lines of "what kind of students do colleges look for", but I was pleasantly surprised to find out what it was really about. The purpose of the session was to reveal the findings of Inside Higher Ed's annual survey (in conjunction with Gallup) of admission deans. You can download the full report at their website. The focus of the survey and the presentation were on the difficulty of meeting enrollment goals, the "Trump Effect" on international recruitment and retention, and efforts to increase racial and ethnic diversity on campus. There was also brief discussion of challenges facing liberal arts institutions, rising student debt and the move toward "free" tuition at public institutions. The survey goes into depth on these and other topics and is worth the read.

The survey was sent to 3,547 admissions directors and enrollment managers, of whom 13% responded. The respondents came from 200 public institutions, 245 private ones, and 8 for-profit schools. 13% sounds kind of low, but if you've ever given a survey you know that it can be hard to get people to participate fully. 

One of the first findings discussed was the difficulty of meeting recruitment goals by the national decision day of May 1st. According to the survey, the number of colleges reporting that they met their target by May 1st is small, and shrinking fast:

This was quite surprising, and led to an interesting discussion. Scott observed that "if you didn't meet your goal by May 1 you're the norm in higher ed". He noted that the joke in the field is that "June 1 is the new May 1" and that some have argued that SEPTEMBER 1 is the new May 1. The panelists tried to discuss this development and the conclusion seemed to be that the enrollment goals that are being set (sometimes by offices higher up the chain from admissions) are unrealistically high. Plus families seem to be waiting longer to send in deposits (perhaps to try to play institutions off each other). Regardless of why, Scott noted that he feared that "some institutions will be pressured to do unwise things" to meet their goals by May 1. In response to an audience question, Scott noted that they will ask more questions next year to try to elicit just how "realistic" admission goals are.

The only sector that overwhelmingly met their enrollment goals was public doctoral institutions. Reasons could range from the large number of in-state students for whom these are the "first-choice" schools to the (perceived) prestige of these universities by international students and their families. Scott noted that even as states have eroded their support for universities, these institutions have great advantages in fundraising, donations, applicant pool and federal grants for research.

One question in the poll was "which group(s) will be the target of increased recruiting?" and the results were quite interesting:

I was very surprised to see the increased attention that private colleges plan to pay to transfer students; I mean, the pool of college age students may not be growing as rapidly as it was a few years ago, but overall enrollment in colleges continues to grow year over year. This effort to harvest transfer students represents almost a Malthusian world view that seems to reflect a zero-sum competition that I'm not sure exists. Also telling is the comparative de-emphasis on the part of private colleges and universities on veterans and minority students. It almost seems that public colleges and universities are more comfortable with diverse student populations compared to the privates. 

 At this point I want to point out that I was impressed by Scott's willingness to inject politics into the discussion. I personally believe that every topic is a political topic, but I can imagine that some people might have been nonplussed by this. Scott passionately addressed the data on the preceding chart by saying "We should think about these priorities. If private college's emphasis is non-need based merit aid, are they really going after the population not currently represented in American higher ed? Colleges measure demonstrated interest by favoring those who visit. But this has an obvious, unstated economic meaning--wealthier kids can afford college visits more." Especially since the topic of access to higher ed was a major theme of the conference, I welcomed Scott's focus on this issue.

You'll notice that the last item in the table shows a remarkable divergence on expected focus on international students.  They went into more detail later on:

This was quite interesting to me. I was a college counselor for eight years at a boarding school with a substantial international population and in many cases their default was to consider large public universities. But the leaders in the field clearly seem to be ceding international students to private institutions who in turn are preparing to pay heavily to increase international enrollment. Could this be due to a backlash against international kids at publics? Or could it be a "Trump Effect" that predicts a generally more hostile environment for immigrants? It was noted that international enrollment was flat this year, and expectations are for it to fall this year; apparently some schools are down 20-50%. Considering how many places depend on full-pay international students to help finance their discounts to American students, this could be a looming financial disaster for some schools.

Related to this is another side of the "Trump Effect": a concern that too many colleges (and college towns) are "excessively liberal". Apparently, independent consultants have reported that more parents are vetoing college choices due to a worry about the political views of the campus and area. Scott asked us to consider what it means when people perceive higher education this way. Personally, I don't think this is new. Deriding someone as a "college boy" (implying an effete elitism out of touch with common people) goes back for at least a century. That said, it is no secret that college populations (and university towns) are often isolated blue islands in the overwhelmingly red American landscape; this of course explains Republican efforts to disfranchise college students.

The twin topics of liberal arts colleges and student debt were addressed in an interesting way. Apparently, only 8% of students and 9% of parents "understand the value of liberal arts". As a result, there is overwhelming pressure to focus on college as a "pre-professional" education. Tied with this is the idea of getting one's money's worth and fear of graduating with debts no honest man can pay (as Bruce Springsteen would say). 36% of public institutions and a startling 78% of privates believe that "public discussion of student debt has discouraged applications to my college". Wow.

Taking a deeper dive into the topic of student debt, there is a growing divide between what admissions people think is "reasonable debt" for students:

What a difference! 36% of admission leaders at private colleges and universities seem comfortable with students graduating with at least $30,000 in debt, while only 10% of public institution admissions heads would find that reasonable. And 63% of publics think a maximum of $20,000 in debt is reasonable compared to 3% at privates. Quite a chasm between those two! Obviously the fact that privates cost less than publics might have something to do with this, but it's clear to me that going forward, the focus on containing costs will be at the public level. In fact, the survey took a look at that and the fear on the part of private institutions was palpable:

The only conclusion I have is that these limousine liberals running private colleges are probably secretly happy that Bernie Sanders' "free public college for all" idea didn't go anywhere!


After the data was presented, Mary Ann Willis and David Hawkins spoke briefly. Mary Ann agreed that "the general public doesn't understand liberal arts" and think college is for career preparation. She urged that admissions officers work harder to recruit students at smaller, rural schools like hers. She counseled that they shouldn't make assumptions about who goes to these schools: "I have students with their family name on buildings, and I have some who are Pell eligible". She also noted that at her school, kids apply to an average of 3.5 schools.

David Hawkins continued the focus on some political issues. He noted that America "confront(s) a real crisis with race, with 'have vs. have not'" He said that NACAC will be stepping up its effort to do a better job to communicate the value of higher education, not just economic, but personal and intellectual value. He also raised an interesting point that I would have liked to hear more about: he said that in 2002 the average yield rate for colleges was 48% but is now 36%. Why? I wonder if it's reliance on "write once, send many" tools like the Common Application, the Coalition Application and the Universal College Application. He closed with the frightening image of a "harsh new equilibrium in college admissions where the admission cycle is so long it reaches around and touches itself on the other end."

There were a number of questions from the audience which elicited more elaboration on these topics. The final question came from a secondary school counselor from Wisconsin who asked what the survey will look like in five years. Scott's response was glum: "I fear that five years from now we will have fewer small private colleges, more over-crowded regional public universities and flagship public universities will be more like elite private colleges. I'm worried about accessibility going forward."

Heavy stuff for sure! What have you or your students seen in your college admission experiences? Share your thoughts in the comments space below.

Thursday, July 6, 2017

"So You Don't Have To": A Visit To Columbia College Chicago

In mid-June I spent a few days in Chicago, and as always I thought I'd look for a college to visit. I didn't have to look far--in fact, the admissions office for Columbia College was LITERALLY right next to my hotel! I stopped by for a tour and informational meeting with a member of the Admissions team and I'm glad that I did--Columbia strikes me as a "hidden gem" for creative, confident students who are interested in pursuing studies in the arts with an eye towards becoming professionals, either creatively or on the business side of things. Columbia College's downtown, South Loop location in Chicago is as urban a campus as you'll find, so it's not for "country mouse" students, but for people looking at a city school, this is a great option. I recommend adding it to your list of possibilities.

Columbia College Chicago At A Glance

Size:Just over 8,600 undergraduates (approximately 58% women/ 42% men). Columbia College is less selective, having accepted about 7,300 of their 8,300 applicants to fill a first-year class of 1,550-- an overall acceptance rate of around 89%(including transfers). 
Programs of Study:99 majors and 46 minors for undergraduates; Columbia College is a pre-professional institution that prepares students to enter creative professions (as performers, creators, managers and anything in between).
Sports:No NCAA or NAIA sports, but there are some intramural teams.
Campus Life:Over 100 clubs and organizations on campus; no fraternities or sororities. On campus housing is NOT guaranteed so first year students desiring campus housing will want to pay their deposits as early as possible. Chicago is America's third largest city and offers a wide range of neighborhoods to live in, all connected by an excellent public transport system. Approximately 68% of students live in campus housing.
Costs & Aid:Tuition, room & board and fees total just about $37,800 (depending on housing and meal plan options).  Parents need to fill out the FAFSA (Free Application for Federal Student Aid). 
Deadlines:Columbia College uses Rolling Admissions, with a priority deadline for scholarships of January 15th.  Students use the Common App or Columbia College's own application. The application fee is a remarkably affordable $35.
Tests:Columbia College is test-optional, so students do not need to send ACT or SAT scores if the tests don't show them at their best. 

© Ethan Lewis 2017
Columbia College Chicago traces its history back to the Columbian Exposition (World's Fair) of 1893 when it began as a school to train public speakers. The performative aspect of the school's function has never gone away, and now Columbia College is a liberal arts institution with a focus on creative disciplines. Located in the South Loop of Chicago (mere blocks from the Art Institute, Millennium Park and other tourist destinations) the college serves over 8,600 students. The School of Liberal Arts and Sciences houses "traditional" departments such as English and History, but the majority of students major in the School of Fine and Performing Arts or in the School of Media Arts. According to the federal government's College Scorecard, 64% of students are in Visual and Performing Arts, with another 13% in Business, Management and related fields. Noteworthy programs include Comedy Studies, where students devote a full semester to study performance, writing and improv at Second City. One of the tour guides when I was there was a poised, together, funny young woman who was a rising senior majoring in comedy and management and was probably pretty representative of Columbia College students. 

Columbia College prides itself on its faculty, and that they are all "active in their industries". There are a good number of full-time faculty members at the college, but the number of part time instructors is remarkable; for instance, in Dance there are 6 full-time professors, 2 full time lecturers and 42 part time instructors. Obviously this means that students won't have trouble finding someone to work with and to mentor them, but it does make one wonder if there is a qualitative difference in the instruction received. I was later told that most departments have a ratio of 70% permanent faculty and 30% adjunct, but obviously this varies by discipline. 

Columbia College appears to be a very welcoming, diverse campus. Among other examples, they offer "Gender Inclusive Housing", which "welcomes students across all gender identities and/or expressions, including transgender, questioning, or gender non-conforming students." The college is also relatively diverse economically and ethnically; according to the College Scorecard, 38% of Columbia College's students come from families that earn less than $40,000--that is a large number (compared to, say, University of Chicago which comes up at 12% or Northwestern University, which has 14% of students from low-income families). 26% of Columbia College students are Black or Hispanic, compared to 14% at U of Chicago and 17% at Northwestern.  

© Ethan Lewis 2017
The campus is about as urban as it gets--spread across a rectangle about 8 blocks long by 6 blocks wide between S. Michigan Avenue and S. Clark St. The area is full of bus stops and elevated train stations, and Columbia College students can get a U-Pass from the Chicago Transit Authority that gives free, unlimited rides on public transportation.  Whenever I go to a city I always use the public transportation and in my (limited) experience, Chicago's buses and trains are some of the best in the country. Around the time of my visit, Chicago's mayor wrote an article in the New York Times explaining why, and the answer boils down to spending money to keep the system healthy. That said, there is no "campus" per se, and while the area is well policed (due in no small part to the thousands of tourists staying in nearby hotels) and according to my tour guide was in "the safest section of Chicago", this is definitely for students who are mature and streetwise, or who aren't afraid of becoming so quickly. That said, the architecture is really lovely in this part of the city, and Lake Michigan is literally a short walk away; for students actively interested in city living, Columbia College should definitely go on the short list.

Dormitory choices include traditional dorms, loft apartments and suites in several buildings throughout the neighborhood. One interesting living option is the University Center, a large building that has housing (on separate floors) for Columbia College, Roosevelt University, DePaul University and Robert Morris (IL) students. It also holds the dining hall for campus residents who opt for a meal plan. While these schools do not have a traditional consortium (such as the Five College Consortium in Massachusetts) I was told that students can take a class for their major at another nearby institution. 

© Ethan Lewis 2017
Academically, Columbia College is rigorous and demanding. All students must take core courses in the School of Liberal Arts and Sciences, including English composition, college-level math, science, social science, history, and literature, along with an "oral communication" course. While some students rush to complete these requirements, most students spread them across their four years. First year students take a seminar to learn "how to use the city as a tool for learning", and the choices are all incredibly interesting sounding! All students also must complete a capstone project to graduate--most of these are presented during the "Manifest" festival every spring. A glance at this year's listings shows the variety of projects undertaken at Columbia College.

Admissions-wise, Columbia College accepts the vast majority of applicants, but I suspect that this is partly due to the self-selecting nature of the applicant pool (creative people seeking a career in the arts) more than being "easy".  Standardized test scores are optional, but students who have a portfolio of their creative works are encouraged to submit it. According to the admissions representative to whom I spoke a great deal of attention is paid to the students' application essays (whether students use the Common Application or Columbia College's own application, the prompts are the same). Columbia College has rolling admissions, which means that while there is no fixed deadline, students would be wise to apply earlier in the cycle while the most spaces are still open--I would suggest that they aim to apply by the end of November to get a decision by Christmas.

Courtesy Google Image Search
The admissions team member told me that the "typical" student was "hands-on, minds on" and who embraces (and is good at) learning collaboratively. They seek "self-directed" students who are "progressive thinkers" who have "mastered both academic and creative writing" and are "ready to roll up their sleeves and start as first-years". They especially seek students who have taken AP courses or were in dual degree programs.

Perhaps as a consequence of seeking students who are so mature, Columbia College accepts a LOT of transfer students every year (815 last year compared to about 1,800 first year students). Columbia could be a GREAT option for your artsy students who are dissatisfied with their current colleges and are looking for a change.  

Since I started working for Method Test Prep in 2015 college visits have been a bit different for me--instead of having a number of active students in mind to "shop for", I find that I look at colleges through a much more "general interest" point of view.  While the students at the schools where I previously taught were predisposed to apply to much more selective institutions, it's important to remember that, as Jon Boeckenstedt of DePaul University notes, selectivity is more of an "input" measurement (related to the number of applicants) than any indication of quality. And the vast majority of college students in America attend institutions with admissions profiles closer to Columbia College Chicago's than, say, Stanford's.  So I have enjoyed visiting less selective schools and seeing just how passionate the students, faculty and staffs are about the work that they do. If you profess to value "fit" in the admissions process, looking at these kinds of colleges and universities could be an eye-opener for you and your students.

While I don't think that Columbia College Chicago is for everyone (the would-be engineer, doctor, or Wall Street Master of the Universe might look elsewhere), for students interested in a career in a creative field, this could be a great choice. Certainly the performance, exhibition and internship opportunities in America's third-largest city are nearly limitless as is the chance to build a meaningful personal Rolodex for future career opportunities. If you find yourself in downtown Chicago, do yourself a favor and pay a visit to Columbia College--I think you will be quite impressed.