thelinguist.uberflip.com
OCTOBER/NOVEMBER The Linguist 25
FEATURES
(format and layout, usability, accessibility,
visibility, compatibility).
We then used the rubric to evaluate the
top five apps from the 67 we had assessed
previously (see box, left). Two evaluators
analysed the five apps according to the rubric,
and the results were consistent between the
two. Interactivity and adaptability seemed to
be the weakest criterion across the board.
This was no surprise, as the specifications of
this criterion include aspects with which
foreign language teaching methods have
traditionally struggled.
Because of this weakness, we decided to
focus on EFL methodology as an initial step
to the design and development of EFL apps.
We therefore looked at the Common
European Framework of Reference (CEFR)
8
in order to establish a linguistic benchmark.
Phase 3: Adding value
We incorporated the CEFR into the rubric in
order to add value to the pedagogic
assessment of phases 1 and 2, developing it
according to the following diagram:
Out of the six levels considered by the CEFR,
our research focused on A2-B2 solely, since
these are the most substantial levels in EFL.
SO-CALL-ME has a clear focus on oral
competence, and consequently our Rubric
for the Evaluation of Apps in Language
Learning (REALL) gave priority to this skill,
starting with oral comprehension. This
resulted in a rubric that has been used to
evaluate the linguistic adequacy of EFL apps
for listening.
The scale applied includes the following
categories: level, types of texts, topics and
delivery. The evaluating process was parallel
to the one used in phase 2: two evaluators
analysed the five chosen apps in order to
ascertain their linguistic adequacy according to
the CEFR. The small number of apps
assessed may not allow us to reach definitive
conclusions but it enables us to hint at relevant
criteria and characteristics of EFL apps.
Throughout this research project it has
become evident that the pedagogic and
technical quality of an app does not
necessarily go hand in hand with its linguistic
value and adequacy for EFL teaching and
learning, since quite a few apps do not have
sound linguistic content that is adequate for
steady language learning. It is necessary to
consider both technical quality and linguistic
content if we want to achieve apps that
enable solid, long-lasting MALL. The rubrics
here described will soon be made public. In
the meantime, they can be requested from
the authors.
This article is based on a paper delivered at
Jornadas Internacionales de Tecnología Móvil
e Innovación en el Aula, Universidad de la
Rioja, 2-3 May 2013. For details about app
ratings, see the paper at www.researchgate.
net/publication/255702557_REALL_Rubric_for_
the_evaluation_of_apps_in_language_learning.
Elena Martín Monje and Pilar Rodríguez
Arancón are Lecturers in English Studies at
the Universidad Nacional de Educación a
Distancia (UNED). Jorge Arús Hita is
Professor in English Philology at the
Universidad Complutense de Madrid
(UCM). Cristina Calle Martínez teaches in
UCM's English Philology Department.
Notes
1 Avatar Generation (2012) 'Rubrics for Evaluating
Educational Apps'. www.avatargeneration.com/
2012/09/rubrics-for-evaluating-educational-apps
2 See www.eduapps.es
3 Social Ontology-driven Cognitively Augmented
Language Learning Mobile Environment project,
funded by the Spanish Ministry of Science and
Innovation (ref: FFI2011-29829)
4 Cohen, A (2007) 'Avatars and Education' in
Classrooms without Walls. http://acohen843.word
press.com/2007/11/11/ avatars-and-education
5 Rodríguez Arancón, P, Calle, C and Arús, J
(2013) 'The Use of Current Mobile Learning
Applications in EFL' in Procedia: Social and
Behavioral Sciences Journal. Proceedings of
IETC 2013, 1189-1196
6 Arús, J, Rodríguez-Arancón, P and Calle, C
(2013) 'A Pedagogic Assessment of Mobile
Learning Applications' in Proceedings of ICDE
2013: Mobilizing Distance Education, 1-15
7 Fernández-Pampillón, A et al (2012)
'Herramienta de Evaluación de la Calidad de
Objetos de Aprendizaje (herramienta COdA).
Guía para la producción y evaluación de
materiales didácticos digitales'
8 Council of Europe (2001) Common European
Framework of Reference for Languages:
Learning, teaching, assessment, Cambridge,
Cambridge University Press
Websites accessed 1/9/15
APPS IN CLASS:
A student uses the English Attack! app
(left); and Speakingpal (below left)
