Plain Language Summaries
Both for successful studies and in everyday confrontation with social and political topics, it is essential to be able to deal with a variety of information sources on one and the same topic. University students, for example, are often confronted with numerous books, studies, etc. and should then be able to integrate this amount of information from the texts into a meaningful whole in the form of a term paper or a presentation. But how well can students actually do this? And is this ability really encouraged in the course of school education?
In order to be able to investigate and answer such questions, one first needs a test with which one can measure this ability. For this reason, we have first looked at the mental requirements for understanding several texts on one topic (= multiple documents comprehension). On this basis, we then developed a computer-based test that measures the corresponding competencies with various tasks. In a next step, we administered this test to 310 students and found correlations between the test score and school and academic performance, indicating that multiple documents comprehension is a learnable skill that develops as the student progresses in their studies.
©PxHere
Being able to handle multiple texts appropriately presents many challenges. On the one hand, the contents have to be integrated, but at the same time, one should also consider source information, such as who wrote the text and what goals the author is pursuing with it. For the development of our test we derived four central competencies for multiple documents comprehension on the basis of empirical studies and theoretical considerations, namely:
The test we developed consists of six units with different content topics (e.g. the end of the universe). For each unit, two to three texts on the respective topic are provided, including brief source information. On the basis of these texts, various tasks are to be worked on, each of which requires one of the four competencies listed. The guiding idea was that one should need information from at least two texts at the same time to correctly solve the tasks. These tasks were either multiple-choice tasks or verification tests in which the truth of a statement had to be assessed.
With this test, we were able to investigate in the next step whether students are able to understand several texts on one topic and how this is reflected in their school and study performance. Since the ability measured by the test is already required in school to complete many tasks, we initially assumed that the test score was positively related to the Abitur grade. However, since this ability should improve during the course of study, we expected this relationship to be higher for bachelor's students than for master's students and that the test score would generally be higher for master's students.
A total of 310 students from various Bachelor's and Master's programs in the humanities and social sciences took part. The students took the test we had developed on a PC. In the process, each student worked on three of the six units, The assignment was random. In addition, other characteristics were recorded, such as the Abitur grade.
On the whole, our expectations were confirmed: The test score was indeed positively related to the Abitur grade for both bachelor's and master's students; however, the correlation was only slightly and not statistically significantly smaller for master's students. In addition, master's students generally performed better on the test.
Further data analysis also showed that the individual tasks differed in difficulty: Comparing content across texts was easiest for test takers, while combining source and content information was the most difficult.
Overall, the results suggest that multiple document comprehension is indeed a significant skill necessary for successful study. But perhaps more importantly, this skill can be learned or improved, as is shown by the the higher test scores of master's students.
Therefore, special emphasis should be placed on fostering this ability in schools and universities - not only for good academic performance, but also to better cope with today's information society. The test we developed also provides an important basis for further research on multiple documents comprehension.
Schoor, C., Hahnel, C., Artelt, C., Reimann, D., Kroehne, U., & Goldhammer, F. (2020). Entwicklung und Skalierung eines Tests zur Erfassung des Verständnisses multipler Dokumente von Studierenden. Diagnostica, 66, 123-135. https://doi.org/10.1026/0012-1924/a000231
The study was conducted within the project MultiTex, which was funded by the German Ministry of Education and Research.