What Are The Differences Between Fixed-Form And Adaptive Test Design?
There are two fundamental test designs in language proficiency assessment: Fixed-Form Testing and Adaptive Testing. In a fixed form test, every test taker receives the same items (aka questions) that are delivered on paper or computer. An adaptive test can only be delivered on a computer. The computer algorithm scores each item as the test taker answers it, then based on the score of that item or cluster of items, the computer algorithm adjusts the level of the next item or cluster of items to the next “challenge” level. Until the era of computers, it was not possible to do adaptive language proficiency testing on a large scale. It is still not possible to apply computerized adaptive test design for speaking and writing tests on a large scale, although Artificial Learning might change this soon. But it is now quite easy and common to use adaptive testing for reading and listening tests.
What Are The Advantages Of Adaptive Language Proficiency Tests?
There are many advantages to adaptive language proficiency tests. Because the level of difficulty of the items adjusts to the level demonstrated by the test taker, the test taker is consistently challenged by items without being overwhelmed by overly difficult items or bored by easy ones. With an adaptive test, the test taker can rise to the top of the scale based on their performance. In an adaptive test, test-takers will take various paths through the test so that cheating is much more difficult (yes, it does happen anywhere). Because the adaptive algorithm hones in on the actual level of the test taker quickly, it is able to deliver more items across a wide range of levels that are at or close to the test taker’s skill, increasing the confidence that the final score is accurate – all in a shorter amount of time than a fixed form-test. In Avant STAMP, we also use the adaptive reading score to determine the level of prompts we deliver for the writing prompts, and the adaptive listening score in the same way for the speaking prompts, thus making the productive sections semi-adaptive and more appropriately leveled for each test taker.
What Is A Computer Adaptive Multi-Stage Test(CAMST)?
One test design strategy that Avant adopted early on is the Computer Adaptive Multi-Stage Test (CAMST) design pioneered by Dr. Rick Leucht, a Professor of Educational Research Methodology at the University of North Carolina at Greensboro. CAMST combines the benefits of adaptive and fixed-form testing by essentially lining up a succession of mini fixed form testlets. After each testlet, test takers move up, down or stay at the same level depending on their performance. In a purely item-by-item adaptive test, a test taker will drop down if they get a single item wrong. In each testlet, we cluster a mix of topics and item difficulties to make the adaptive algorithm more accurate and efficient over the length of the test. You can learn more about Avant STAMP’s CAMST algorithm by following this link: Computer Adaptive Multi-Stage Test Design.
Fixed-form Language Tests And Technology
Fixed-form tests are based on older or less sophisticated technology and have a number of shortcomings, some of which I have mentioned already. In order to assess a wide range of language levels, it is necessary to have items that cover the entire range of levels, but this means the test becomes very long and lower-level students will be faced with items that will overwhelm them and higher-level students are often bored or not challenged by the content. As a compromise, in order to prevent these problems, teachers are sometimes asked to select the level of the form their students will take, which places a ceiling on their performance.
However, fixed-form tests do have advantages. Because all test takers receive the same items, you can control the mix of topics. It is also easier to conduct the psychometric (statistical) analysis of items on fixed-form tests than adaptive tests for some rather technical reasons. Here is a Wikipedia article on Computer Adaptive Testing that addresses this issue if you are really a testing nerd like some of us. Follow this link to learn more about computerized adaptive testing.
Avant uses CAMST adaptive test design in all of its language proficiency tests. We strongly believe in its power to deliver accurate results and create an enjoyable testing experience for world language learners.
