As students and researchers, having to include standardized tests as part of our lives can bring a great deal of stress and anxiety. The stress that these tests generate is not foreign to us, in fact it adds on to our daily struggle to survive. Just think of the amount of material you had to study for the GRE, or the number of times you taught yourself tricks on improvising for the speaking session of the TOEFL. Not to mention the economic burden these tests impose on us: $200 for registration, plus roughly $200 for preparation materials. And if this is not enough, it requires your attention for a significant time of the day. The number of hours you have to spend on learning for the GRE is 3 per day for three months, or 1-2 hours a day for the TOEFL. In order to dedicate such a significant part of your day, you have to compromise on work, thus to earn less in order to study more. Affording these tests is not a trivial task for many students, in fact it’s a perfect tool to filter out minorities, women, and middle to low class citizens. So much for equal opportunities for all.
Well, you might think, if it manages to predict successful students from less successful ones, then perhaps the means justify the end. But this is wrong. Studies have shown that standardized tests failed time and again to predict the quality of future graduate students. It turns out that 50% of graduate students never complete their studies. In that case, the university’s admissions committee might as well flip a coin to select applicants – that’s faster, economical and gets the same results.
In order to give you a taste of the irrelevance of these tests to the skills they claim to assess, take the Test of English as a Foreign Language (TOEFL) for example. The test’s speaking session expects you to come up with a meticulous answer to an emotionally loaded question in 15 seconds and speak about it in a coherent fashion for 45 seconds. Any respected linguist or cognitive psychologist would tell you humans, let alone native English speakers themselves, do not speak this way. In conversations, sentences are constructed simultaneously as they are conceptualized. And the concepts themselves are associative in nature. Within the 60 second time frame, you are expected to accomplish the impossible – to deliver a short speech in an amount of time that’s enough just to begin pondering its topic and say a few sentences about it. Clearly, philosophical and creative students, the ones that question the status quo and move science forward are the ones to be penalized for answering this type of question.
Another example comes from the Graduate Record Examinations (GRE). It tests you on skills that are most likely to be irrelevant to your specialty. A mathematician should not be tested on their verbal skills just as a linguist or a literature student should not be tested on their mathematical skills. Even if you are going to study in the humanities or social sciences, being tested on low-frequency, obsolete and literary words will definitely not extrapolate your language skills. And being tested on math questions is not going to extrapolate your statistical analysis abilities, a task that nowadays requires a minimal knowledge of math thanks to statistics programs such as SPSS.
Then who are the big benefactors of standardized tests? The answer is private companies and conservative universities. Educational Testing Service (ETS), for instance, earns millions of dollars on the backs of prospective graduate students. Not to mention the secondary industry that earns even more from preparation courses for these tests. Standardized tests are carried out by private companies for private companies. By inventing standardized tests, private companies created a new market that deepens inequality and increases injustice. This very market, like any evolutionary product, is the product of unrestrained natural selection carried out by an unregulated economic system. Decisions in academia should ultimately be backed up by science and not be compromised by power, conservatism or outright greed.
The future of standardized tests
So what’s the alternative? First, we need an assessment tool that can actually predict applicants’ success rates, a tool that is based on science. Secondly, this tool must be economically and socially just. Luckily, several academic institutions in the US have had successful experiences with alternative assessment tests. In their STEM (science, technology, engineering and maths) Ph.D programmes, the University of South Florida in Tampa and Fisk–Vanderbilt in Nashville, Tennessee managed to achieve a completion rate of 80% (among the highest in the country) in addition to boosting the participation rate of women and minorities. Moreover, all graduated students ended up employed as post-docs, university faculty members or scientists in national or industry labs.
How did they do it? They filtered out applicants on the basis of skills and psychological traits instead of GRE scores. In a half-hour interview, each applicant discussed their “past research experiences, key relationships, leadership experience, service to community and life goals. The result is a good indication of the individual’s commitment to scientific research and a good assessment of traits such as maturity, perseverance, adaptability and conscientiousness atop a solid academic foundation. The combination of academic aptitude and these other competencies points to the likelihood of high achievement in graduate school and in a STEM career.” (Nature 510, 303-304; 2014)
Hampshire college, a liberal arts school in Massachusetts has decided not to accept SAT/ACT scores from applicants. Instead, they evaluate students based on essays, interviews and recommendations. They assess characteristics such as motivation, discipline, self-reflection, community engagement and entrepreneurism. The results: a 10% increase in class diversity, 18% increase in students’ rates of acceptance to enroll and more mature disciplined and motivated students than before. Moreover, this shift in assessment method turned out to be more economical: “having a smaller but more targeted, engaged, passionate, and robust applicant pool, we are able to streamline our resources”.
The fact that these traits predict success does not come as a surprise to me. Science and history show us that people who were able to split their time wisely between work and rest are the people who we consider the geniuses that progressed science forward. Alex Soojung-Kim Pang shows in his book Rest some staggering facts – according to a survey of scientists’ working lives which was conducted in the early 50s (by Raymond Van Zelst and Willard Kerr), it was found that “scientists who spent 25 hours in the workplace were no more productive than those who spent 5”. Moreover, “scientists working 35 hours a week were half as productive as their 20-hours-a-week colleagues”, and finally “the 60-plus-hour-a-week researchers were the least productive of all”. In fact, Darwin, Dickens, Henri Poincaré, Ingmar Bergman and many others worked no more than 4 hours a day, leaving the rest of the day for deliberate rest (more on it the article, or the book).
So what’s the future of standardized tests? Intelligent tests. Tests that examine self-discipline, self-reflection, motivation, academic activity, social activity and rest/work balance alongside grades and academic achievements. Only this will guarantee that the test is reliable and moral.
Perhaps the future of standardized tests is in AI. An algorithm that will scan all applications in a matter of seconds and be able to grade them and pass on its recommendations to the admissions board. The algorithm will also be able to correct and improve itself the more it is being used. Hopefully, this idea will be implemented by the universities themselves and not private companies, thus creating an evaluation tool that is economical (free for applicants), just, fast and intelligent.