CAPÍTULO 1. INTRODUCCIÓN

A lo largo del presente capítulo se expone la motivación general que ha llevado a la realización de los trabajos presentados en esta tesis. A continuación, se establecen los objetivos generales propuestos, que se irán desgranado a lo largo del documento. Por último, se incluye la estructura organizativa de la memoria.

1.1. Motivación

El Pensamiento Computacional (PC) es una destreza cognitiva para resolver problemas que debe adquirirse desde una edad temprana, de forma similar a la adquisición de una segunda lengua, para poder así adaptarse a la sociedad informatizada actual. Muchas de las habilidades comúnmente asociadas al PC pueden desarrollarse al aprender a programar como, por ejemplo, la abstracción, la descomposición de problemas, la algoritmia y la depuración de programas [1]. La importancia del PC reside en que estas habilidades para resolver problemas, que podrían parecer solamente relacionados con la informática, pueden emplearse para hacer frente a problemas en muchos de los ámbitos productivos de la sociedad actual, así como a problemas de la vida cotidiana [2]-[4].

Para incluir la enseñanza de la programación y el desarrollo de habilidades de PC en el currículo escolar, es necesario determinar qué debe aprenderse a cada edad y cómo. Sin embargo, al ser una competencia tan reciente, la definición del concepto es aún difusa [5], así como los marcos de referencia operativos que permitan establecer métodos de enseñanza y de evaluación eficaces [4]. Por este motivo, es necesario realizar una investigación teórica sobre este tema y marcar una base operacional en cuanto a definiciones, estrategias y entornos para el desarrollo del PC, así como de instrumentos o herramientas existentes para el aprendizaje y evaluación de esta competencia.

Para establecer si un método de enseñanza del PC es adecuado, es necesario medir, a través de un instrumento específico para la evaluación del PC, el efecto de ese determinado método de enseñanza en el aprendizaje de los estudiantes. En los últimos años han surgido nuevos instrumentos de evaluación del PC, los cuales no suelen orientarse a etapas tempranas [1], [5]. En el momento de la realización del presente estudio, no existía, en la comunidad científica, ninguna prueba de evaluación (test) independiente (no relacionada con un entorno de aprendizaje concreto) y enfocada a alumnos de Educación Primaria. Con esta motivación, en primer lugar, se plantea el diseño y la validación de un test de estas características que nos permita una correcta evaluación del PC en etapas tempranas, el Beginners Computational Thinking test (BCTt) y su adaptación a niños más mayores, competence Computational Thinking test (cCTt), así como su validación y límites de edad para su administración [6]-[8].

Además, con la motivación de evaluar una herramienta de aprendizaje basado en juegos [9]-[11], fundamentada en el constructivismo [12]-[15], para la enseñanza y evaluación automática del PC mediante analíticas de aprendizaje en juegos [16], [17], se desarrolló y validó Blue Ant Code (BAC) [18], [19], que incluye modos de juego individual y colaborativo.

Mediante los instrumentos de evaluación (BCTt y cCTt) y el entorno de aprendizaje y evaluación (BAC), es posible extraer y analizar datos para poder avanzar en el conocimiento del desarrollo de las habilidades de PC en diferentes edades y género. Mediante BAC en entornos escolares y no escolares (versión masiva de BAC), y analíticas de aprendizaje en juegos, es posible también extraer y analizar no solamente las habilidades estrictamente académicas, sino también la progresión en el aprendizaje y disposiciones y predisposiciones para el desarrollo del PC en cuanto a aspectos relativos a la motivación intrínseca, intereses, persistencia y actitudes frente a las recompensas para la enseñanza de la programación y el PC.

La validación de los instrumentos de evaluación y de BAC, permite su utilización con una buena fiabilidad y, además, facilita la realización de nuevos instrumentos y metodologías de enseñanza-evaluación del PC, gracias a las conclusiones que se han obtenido de los procesos de validación. De las experiencias realizadas, se han extraído conclusiones concretas sobre las habilidades de PC asociadas a cada edad y género del alumno y, más allá, sobre la motivación intrínseca de los usuarios para el aprendizaje del PC, sus intereses, persistencia, y actitud frente a las recompensas, también dependiendo de su edad y su género.

De esta forma, los instrumentos y herramientas desarrolladas en esta tesis, así como las experiencias realizadas para su validación han sido motivadas por la búsqueda de diferentes variables relacionadas con el desarrollo del PC para, una vez analizadas, poder no solamente ampliar el conocimiento de estas habilidades, sino también mejorar el diseño del currículo escolar con respecto a esta competencia, así como poder progresar en el desarrollo de herramientas de aprendizaje y evaluación del PC. En la Figura 1 se muestra un esquema de las variables a analizar que motivan el trabajo realizado para esta tesis, y su relación con los instrumentos desarrollados.

A partir de esta motivación, es importante establecer los objetivos generales y específicos de la tesis, que se abordarán en el siguiente apartado.

1.2. Objetivos generales

Actualmente la definición del concepto de PC es aún difusa, así como los marcos de referencia operativos que permitan establecer métodos de enseñanza y de evaluación eficaces, sobre todo a etapas tempranas. Por lo tanto, el objetivo principal de la tesis es:

OP: Avanzar en el conocimiento sobre el PC, sobre todo en su componente operacional, es decir, en cuanto a métodos de enseñanza y de evaluación eficaces, sobre todo en etapas tempranas.

El objetivo principal OP de la tesis se puede dividir en una serie de objetivos específicos que se describen a continuación:

Investigación teórica:

- OP1: Recopilar un amplio marco teórico y estado del arte de publicaciones y estudios sobre PC. Se deben considerar los siguientes aspectos:

○ OP1a: Definiciones teóricas y operacionales del PC.

○ OP1b: Marcos, currículos escolares e iniciativas para el desarrollo del PC a nivel internacional y nacional en etapas tempranas.

○ OP1c: Marco teórico y estrategias para el desarrollo/aprendizaje del PC, así como herramientas y entornos existentes.

○ OP1d: Marco teórico y estrategias para la evaluación del PC, así como instrumentos existentes.

Investigación empírica:

- OP2: Establecer una propuesta operativa del PC en base al OP1 y así constituir la dirección de la investigación empírica mediante hipótesis y objetivos base. Posteriormente, en cada experiencia empírica, se detallarán las hipótesis y objetivos específicos de la misma.

- OP3: Diseñar y validar, empíricamente, herramientas para el aprendizaje y la evaluación del PC:

○ OP3a: Diseñar y validar un instrumento de evaluación del PC en edades tempranas.

○ OP3b: Desarrollar y validar una herramienta de aprendizaje y evaluación del PC en edades tempranas.

- OP4: Realizar experiencias empíricas que permitan analizar datos relativos tanto a habilidades de PC como a disposiciones y predisposiciones para el PC:

○ OP4a: Extraer y analizar datos para mejorar el diseño del currículo escolar relativo a PC según edad y género.

○ OP4b: Extraer y analizar datos para mejorar el diseño de herramientas de aprendizaje relativo a PC según edad y género.

○ OP4c: Extraer y analizar datos para mejorar el diseño de herramientas de evaluación relativo a PC según edad y género.

Además, existe un objetivo común con otros investigadores a nivel internacional, que es el diseño y validación de instrumentos de evaluación del PC que cubran un amplio rango de edades y poblaciones, desde la etapa de Educación Infantil hasta la etapa universitaria. Dentro de este marco común, esta tesis se enfoca en edades tempranas: Educación Infantil y primaria. De aquí se desprende el objetivo común:

OC: Diseño y validación de instrumentos de evaluación del PC que cubran un amplio rango de edades y poblaciones

En la Figura 2 se resumen los objetivos generales de esta tesis:

1.3. Experiencias realizadas

En cuanto a la investigación empírica que se desprende de los objetivos OP2, OP3 y OP4, se han realizado varias experiencias que se resumen en la Figura 3. En cada uno de los capítulos a los que se direcciona en la tabla, se detallará cada una de las experiencias, no sin antes establecer una base operacional clara y las hipótesis base generales a todas ellas en el Capítulo 3.

1.4. Estructura del documento

Tras este primer capítulo de introducción, donde se detallan las hipótesis y los objetivos de este trabajo, el documento se divide en 8 capítulos más cuyo contenido se estructura de la siguiente forma:

- Capítulo 2: Se realiza un estado del arte y se establece un marco teórico, mediante una revisión crítica de la literatura, con respecto a definiciones teóricas y operacionales del PC; marco, currículos escolares e iniciativas para el desarrollo del PC a nivel internacional y nacional en etapas educativas tempranas; y marco teóricos y estrategias para el desarrollo del PC. También se exploran herramientas y entornos existentes; marcos teórico y estrategias para la evaluación del PC; así como instrumentos existentes.

- Capítulo 3: Se exponen las conclusiones que se extraen del capítulo 2 y la dirección operativa para la investigación empírica, especificando las hipótesis generales y las experiencias a realizar.

- Capítulo 4: Se describe la experiencia de desarrollo y validación del instrument de evaluación del PC: Beginners Computational Thinking Test (BCTt) dirigido a alumnos de Educación Primaria, así como la posterior adaptación del test para los alumnos más mayores en los que el BCTt resulta menos fiable, mediante una colaboración internacional que resulta en el instrumento de evaluación: Competence Computational Thinking Test (cCTt).

- Capítulo 5: Se describe la experiencia dirigida a alumnos de Educación Infantil, en colaboración internacional, mediante el uso de entornos de aprendizaje basados en la robótica, para establecer el límite inferior de edad para el que el BCTt es adecuado.

- Capítulo 6: Se describe la experiencia de desarrollo y validación de un entorno de aprendizaje y evaluación basado en juegos (BAC), con el objetivo general de analizar estrategias para el aprendizaje y evaluación del PC y las habilidades de PC de alumnos de Educación Primaria, entre otros objetivos.

- Capítulo 7: Se describe la experiencia de despliegue de BAC en un entorno no escolar, con el objetivo general de analizar estrategias para el aprendizaje y evaluación del PC, así como analizar las habilidades de PC de alumnos de Educación Primaria, con especial atención en las disposiciones y predisposiciones para el PC basadas en la motivación intrínseca y el juego voluntario.

- Capítulo 8: Se resumen las principales colaboraciones, contribuciones, conclusiones y hallazgos en el contexto de la tesis, así como producción científica a la que ha dado lugar el trabajo.

- Bibiografía y Anexos: Finalmente se incluyen las referencias bibliográficas citadas a lo largo de este documento, así como documentación adicional de la tesis.

CHAPTER 1. INTRODUCTION

Throughout this chapter, the general motivation that has led to the work presented in this thesis is set out. Next, the general objectives are established and be described in detail. Finally, the organisational structure of the document is presented.

1.5. Motivación

Computational Thinking (CT) is a cognitive problem-solving skill that must be acquired from an early age, similar to second language acquisition, in order to adapt to today's computerised society. Many of the skills commonly associated with CT can be developed by learning to program, such as abstraction, problem decomposition, algorithmia, and debugging [1]. The importance of CT relies on the fact that these problem-solving skills, which might seem only computer-related, can be used to address problems in many of the productive domains of today's society as well as problems in everyday life [2]-[4].

In order to include the teaching of programming and the development of CT skills in the school curricula, it is necessary to determine what should be learned at each age and how. However, being such a recent competence, the definition of the concept is still unclear [5], as well as the operational frameworks for effective teaching and assessment strategies [4]. For this reason, it is necessary to conduct theoretical research on this topic and to establish an operational basis in terms of definitions, strategies, and environments for the development of CT, as well as existing instruments or tools for the learning and assessment of this competence.

To establish whether a CT learning strategy is appropriate, it is first necessary to establish whether it impacts learning, which requires a CT assessment instrument that can measure such learning. In recent years, new assessment instruments have emerged, but these are not usually focused on the early stages [1], [5]. At the time of the present study, there was no stand-alone assessment test in the scientific community - not related to a specific learning environment - focused on primary school students. With this motivation, first, we proposed the design and validation of a test of this type that would allow us to properly assess CT in early stages, the Beginners Computational Thinking test (BCTt) and its adaptation to older children, the competence Computational Thinking test (cCTt), as well as its validation and age limits for its administration [6]-[8].

Furthermore, with the motivation of evaluating a game-based learning tool [9]-[11], based on constructivism [12]-[15], for the learning and assessment of CT through game learning analytics [16], [17], Blue Ant Code (BAC) [18], [19], which includes individual and collaborative game modes, was developed and validated.

Using the assessment instruments (both BCTt and cCTt) and the learning and assessment environment (BAC), it is possible to collect and analyse data in order to advance knowledge of the development of CT skills at different age groups and gender. Through BAC in school and non-school environments (massive version of BAC), and game learning analytics, it is also possible to collect and analyse not only strictly academic skills, but also learning progression and dispositions and predispositions for CT development in terms of aspects related to intrinsic motivation, interests, persistence, and behaviours in response to rewards for learning to program and developing CT.

The validation of the assessment instruments and BAC, allows their administration with a good reliability and, in addition, facilitates the development of new instruments and methodologies for learning-assessment of CT, according to the conclusions that have been obtained from the validation procedures. From the studies carried out, specific conclusions have been drawn about the CT skills associated with each student's age and gender and, furthermore, about the students' intrinsic motivation for learning CT, their interests, persistence, and behaviour towards rewards, also depending on their age and gender.

In addition, the instruments and tools developed in this thesis, as well as the studies carried out for their validation, have been motivated by the search for different variables related to the development of CT in order to, once analysed, not only broaden the knowledge of these skills, but also to improve the design of the school curricula with regard to this competence, as well as to progress in the development of tools for learning and assessing CT. Figure 1 shows an outline of the variables to be analysed that motivate the work carried out for this thesis, and their relationship with the instruments developed.

Based on this motivation, it is important to establish the general and specific aims of the thesis, which will be addressed in the following section.

1.6. General aims

Currently, the definition of the concept of CT is still unclear, as well as the operational frameworks for effective learning and assessment strategies, especially at early stages. Therefore, the main aim of the thesis is:

OP: To advance knowledge about CT, especially in its operational component, i.e., in terms of effective learning and assessment strategies, especially at early stages.

The main aim OP of the thesis can be divided into several specific aims which are described below:

Theoretical research:

- OP1: Compile a broad theoretical framework and state of the art of publications and studies on CT. The following aspects should be considered:

○ OP1a: Theoretical and operational definitions of CT.

○ OP1b: Frameworks, school curricula and initiatives for the development of CT at international and national level at early stages.

○ OP1c: Theoretical framework and strategies for CT development/learning, as well as existing tools and environments.

○ OP1d: Theoretical framework and strategies for the assessment of CT, as well as existing instruments.

Empirical research:

- OP2: Establish an operational proposal of CT based on OP1 and thus constitute the direction of the empirical research by means of hypotheses and base aims. Subsequently, in each empirical study, the specific hypotheses and aims of the study will be detailed.

- OP3: To design and validate, empirically, tools for learning and assessment of CT:

○ OP3a: To design and validate a CT assessment tool aimed at early ages.

○ OP3b: To develop and validate a tool for learning and assessment of CT at an early age.

- OP4: To carry out empirical studies to analyse data related to CT skills as well as CT dispositions and predispositions:

○ OP4a: To collect and analyse data to improve the design of the school curricula related to CT according to age and gender.

○ OP4b: To collect and analyse data to improve the design of learning tools related to CT according to age and gender.

○ OP4c: To collect and analyse data to improve the design of CT assessment tools according to age and gender.

In addition, there is a common aim with other researchers at international scale, which is the design and validation of CT assessment tools covering a wide range of ages and populations, from Early Childhood Education to University. Within this common framework, this thesis focuses on early ages: pre-school and primary school. Thus, the common aim is the following:

OC: Design and validation of CT assessment instruments covering a wide range of ages and populations.

Figure 2 summarises the general aims of this thesis:

1.7. Studies conducted

With regard to the empirical research arising from the aims OP2, OP3 and OP4, several studies have been conducted, which are summarised in Figure 3. In each of the chapters addressed in the table, each of the studies will be detailed, but before that, in Chapter 3, a clear operational basis and the general basic hypotheses for all of them will be established.

1.8. Document structure

After this first introductory chapter, which details the hypotheses and the aims of this work, the document is divided into 8 chapters structured as follows:

- Chapter 2: A state of the art and theoretical framework is conducted, through a critical literature review, with respect to theoretical and operational definitions of CT; framework, school curricula and initiatives for CT development at international and national levels in early educational stages; and theoretical framework and strategies for CT development. It also explores existing tools and environments; theoretical frameworks and strategies for CT assessment; and existing instruments.

- Chapter 3: The conclusions drawn from Chapter 2 and the operational direction for the empirical research are presented, specifying the general hypotheses and the studies to be carried out.

- Chapter 4: The study of the development and validation of the CT assessment instrument: Beginners Computational Thinking Test (BCTt) aimed at Primary School students is described, as well as the subsequent adaptation of the test for older students for whom the BCTt is less reliable, through an international collaboration resulting in the assessment instrument: Competence Computational Thinking Test (cCTt).

- Chapter 5: Describes the study aimed at pre-school pupils in an international collaboration using robotics-based learning environments to establish the lower age limit for which the BCTt is suitable.

- Chapter 6: Describes the study of the development and validation of a game-based learning and assessment environment (BAC), with the general aim of analysing strategies for learning and assessing CT, as well as CT skills of Primary School students, among other aims.

- Chapter 7: The study describes the BAC deployment in a non-formal setting, with the general aim of analysing strategies for learning and assessing CT, as well as analysing the CT skills of Primary School pupils, with a special focus on dispositions and predispositions for CT based on intrinsic motivation and voluntary play.

- Chapter 8: The main collaborations, contributions, conclusions, and findings in the context of the thesis are summarised, as well as the scientific production related to it.

- Bibliography and Annexes: Finally, the bibliographical references cited throughout this document are included, as well as additional documentation of the thesis.

Bibliografía

  [1]V. J. Shute, C. Sun, and J. Asbell-Clarke, "Demystifying computational thinking," Educational Research Review, vol. 22, pp. 142-158, 2017. Available: https://www.sciencedirect.com/science/article/pii/S1747938X17300350, doi: 10.1016/j.edurev.2017.09.003.

  [2]M. Berland and U. Wilensky, "Comparing virtual and physical robotics environments for supporting complex systems and computational thinking," Journal of Science Education and Technology, vol. 24, no. 5, pp. 628-647, 2015.

  [3]J. M. Wing, "Computational thinking and thinking about computing," Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 366, no. 1881, pp. 3717-3725, 2008. Available: http://rsta.royalsocietypublishing.org/content/366/1881/3717.abstract, doi: 10.1098/rsta.2008.0118.

  [4]T. Hsu, S. Chang, and Y. Hung, "How to learn and how to teach computational thinking: Suggestions based on a review of the literature," Computers & Education, vol. 126, pp. 296-310, 2018. Available: http://www.sciencedirect.com/science/article/pii/S0360131518301799, doi: //doi.org/10.1016/j.compedu.2018.07.004.

  [5]S. Grover and R. Pea, "Computational Thinking in K—12: A Review of the State of the Field," Educational Researcher, vol. 42, no. 1, pp. 38-43, 2013. Available: https://www.jstor.org/stable/23360476, doi: 10.3102/0013189X12463051.

  [6]M. Zapata-Cáceres and N. Fanchamps, "Using the beginners computational thinking test to measure development on computational concepts among preschoolers," in Proceedings of the 5th APSCE International Computational Thinking and STEM in Education Conference 2021, Singapore, National Institute of Education, 2021, pp. 32-37.

  [7]Zapata-Cáceres María, E. Martín-Barroso and M. Román-González, "BCTt: Beginners computational thinking test," in Raspberry Pi Foundation Research Seminars, Apr 2021, pp. 46-56. ISSN 2514-586X (16). Available: https://www.raspberrypi.org/app/uploads/2021/05/Understanding-computing-education-Volume-1-%E2%80%93-Raspberry-Pi-Foundation-Research-Seminars.pdf.

  [8]M. Zapata-Cáceres, E. Martín-Barroso, and M. Román-González, “Computational thinking test for beginners: Design and content validation,” in IEEE Global Engineering Education Conference (EDUCON’20), Porto, Portugal, pp. 1905–1914, 2020, doi: 10.1109/EDUCON45650.2020.9125368.

  [9]J. Kirriemuir and A. Mcfarlane, Literature Review in Games and Learning. Bristol: Futurelab, 2004. Available: https://telearn.archives-ouvertes.fr/hal-00190453.

 [10]M. Prensky, "Digital game-based learning," Computers in Entertainment (CIE), vol. 1, no. 1, pp. 21, 2003.

 [11]M. Qian and K. R. Clark, "Game-based Learning and 21st century skills: A review of recent research," Computers in Human Behavior, vol. 63, pp. 50-58, 2016. Available: https://www.sciencedirect.com/science/article/pii/S0747563216303491, doi: 10.1016/j.chb.2016.05.023.

 [12]L. S. Vygotsky, Mind in Society: The Development of Higher Psychological Processes. Cambridge, MA, USA: Harvard University Press, 1978.

 [13]J. Piaget, Psychology and Epistemology: Towards a Theory of Knowledge. CA, USA: Allen Lane, 1972.

 [14]D. H. Jonassen and L. Rohrer-Murphy, “Activity theory as a framework for designing constructivist learning environments,” Educational Technology Research & Development, vol. 47, no. 1, pp. 61–79, 1999. Available: https://www.jstor.org/stable/30220230, doi: 10.1007/BF02299477.

 [15]D. H. Jonassen, Learning to Solve Problems with Technology: A Constructivist Perspective. Upper Saddle River, N.J.: Merrill Prentice Hall, 2003.

 [16]C. Alonso-Fernandez et al, "Systematizing game learning analytics for serious games," in 2017 IEEE global engineering education conference (EDUCON), Apr 2017, pp. 1111-1118.

 [17]M. Freire et al, "Game Learning Analytics: Learning Analytics for Serious Games," Springer Nature Switzerland AG, pp. 1-29, Dec 1, 2016.

 [18]M. Zapata-Caceres, E. Martin, and M. Roman-Gonzalez, "Collaborative GameBased Environment and Assessment Tool for Learning Computational Thinking in Primary School: A Case Study," in IEEE Transactions on Learning Technologies, vol. 14, no. 5, pp. 576-589, 1 Oct. 2021, doi: 10.1109/TLT.2021.3111108.

 [19]M. Zapata-Cáceres and E. Martín-Barroso, "Applying Game Learning Analytics to a Voluntary Video Game: Intrinsic Motivation, Persistence, and Rewards in Learning to Program at an Early Age," in IEEE Access, vol. 9, pp. 123588-123602, 2021, doi: 10.1109/ACCESS.2021.3110475.

 [20]M. Binkley et al, "Defining twenty-first century skills," in Assessment and Teaching of 21st Century Skill. Dordrecht: Springer Netherlands, 2012, pp. 17-66.

 [21]B. Trilling and C. Fadel, 21st Century Skills: Learning for life in our times. San Francisco: John Wiley & Sons, 2009.

 [22]H. Kim et al, "Enhancing teachers' ICT capacity for the 21st century learning environment: Three cases of teacher education in Korea," Australasian Journal of Educational Technology, vol. 28, no. 6, 2012.

 [23]K. Tang, T. Chou, and C. Tsai, "A Content Analysis of Computational Thinking Research: An International Publication Trends and Research Typology," The Asia-Pacific Education Researcher, pp. 1-11, 2019. Available: https://search.proquest.com/docview/2195733480, doi: 10.1007/s40299-019-00442-8.

 [24]S. Grover et al, "A Framework for Using Hypothesis-Driven Approaches to Support Data-Driven Learning Analytics in Measuring Computational Thinking in Block-Based Programming Environments," ACM Transactions on Computing Education (TOCE), vol. 17, no. 3, pp. 1-25, 2017. Available: http://dl.acm.org/citation.cfm?id=3105910, doi: 10.1145/3105910.

 [25]S. Grover and A. Korhonen, "Unlocking the Potential of Learning Analytics in Computing Education," ACM Transactions on Computing Education (TOCE), vol. 17, no. 3, pp. 1-4, 2017. Available: http://dl.acm.org/citation.cfm?id=3122773, doi: 10.1145/3122773.

 [26]S. Grover. “The 5th ‘C’ of 21st Century skills? Try computational thinking (not coding)”, edsurge.com. http://bit.ly/346qK78 (accessed Feb. 1, 2018). Available: http://scholar.aci.info/view/1544f0f08b300170003/161ce93d34000012ef80942.

 [27]J. A. Velazquez-Iturbide, "Towards an analysis of computational thinking," in Sep 2018, pp. 1-6.

 [28]J.A. Velázquez-Iturbide, “Towards an analysis of computational thinking,” in Int. Symp. on Computers in Education (SIIE’18), pp. 1-6, Sep. 2018.

 [29]M. Israel et al, "Supporting all learners in school-wide computational thinking: A cross-case qualitative analysis," Computers & Education, vol. 82, pp. 263-279, 2015. Available: https://www.sciencedirect.com/science/article/pii/S0360131514002759, doi: 10.1016/j.compedu.2014.11.022.

 [30]C. Angevine et al, "Computational Thinking for a Computational World," 2017. Available: www.digitalpromise.org.

 [31]National Research Council, Report of a Workshop on the Pedagogical Aspects of Computational Thinking. National Academies Press, 2011.

 [32]Q. Burke, W. I. O'Byrne and Y. B. Kafai, "Computational participation: Understanding coding as an extension of literacy instruction," Journal of Adolescent & Adult Literacy, vol. 59, no. 4, pp. 371-375, 2016.

 [33]J. F. Sanford and J. T. Naidu, "Computational thinking concepts for grade school," Contemporary Issues in Education Research (CIER), vol. 9, no. 1, pp. 23-32, 2016.

 [34]K. Mills et al, Computational Thinking for an Inclusive World: A Resource for Educators to Learn and Lead. Digital Promise, 2021. Available: https://doi.org/10.51388/20.500.12265/140.

 [35]C. Sneider et al, "Computational thinking in high school science classrooms," The Science Teacher, vol. 81, no. 5, pp. 53, 2014.

 [36]N. Kelly and J. S. Gero, "Design thinking and computational thinking: a dual process model for addressing design problems," Design Science, vol. 7, 2021.

 [37]T. C. Pargman et al, "Teaching Computational Thinking in K-9: Tensions at the Intersection of Technology and Pedagogical Knowledge," in 14th International Conference of the Learning Sciences (ICLS) 2020, vol. 4, pp. 2409-2410, International Society of the Learning Sciences, 2020.

 [38]D. Kohen-Vacs and M. Milrad, "Computational thinking education for in-service elementary swedish teachers: Their perceptions and implications for competence development," in International Conference on Computational Thinking Education, 13-15 June 2019, Hong Kong, 2019.

 [39]V. Allan et al, "Computational thinking in high school courses," in Proceedings of the 41st ACM Technical Symposium on Computer Science Education, 2010.

 [40]P. Sengupta et al, "Integrating computational thinking with K-12 science education using agent-based computation: A theoretical framework," Education and Information Technologies, 2013, vol. 18, no 2, p. 351-380. Available: https://search.proquest.com/docview/1323979287, doi: 10.1007/s10639-012-9240-x.

 [41]V. Barr and C. Stephenson, "Bringing computational thinking to K-12: What is Involved and what is the role of the computer science education community?" ACM Inroads, vol. 2, no. 1, pp. 48-54, 2011.

 [42]P. Mishra, A. Yadav, and Deep-Play Research Group, "Rethinking technology & creativity in the 21st century," TechTrends, vol. 57, no. 3, pp. 10-14, 2013.

 [43]J. M. Wing, "Computational Thinking Test," CACM Viewpoint, pp. 33-35, 2006. Available: http://www.cs.cmu.edu/~wing/.

 [44]M. Román-González, J. Pérez-González and C. Jiménez-Fernández, "Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test," Computers in Human Behavior, vol. 72, pp. 678-691, 2017. Available: https://www.sciencedirect.com/science/article/pii/S0747563216306185, doi: 10.1016/j.chb.2016.08.047.

 [45]A. V. Aho, "Computation and computational thinking," The Computer Journal, vol. 55, no. 7, pp. 832-835, 2012.

 [46]K. Brennan, M. Resnick, and MIT Media Lab, "New frameworks for studying and assessing the development of computational thinking," American Educational Research Association Meeting, Vancouver, BC, Canada., 2012.

 [47]S. Papert, Mindstorms: Children, Computers, and Powerful Ideas. Basic Books, Inc., 1980.

 [48]J. J. Lu and G. H. Fletcher, "Thinking about computational thinking," in Proceedings of the 40th ACM Technical Symposium on Computer Science Education, 2009.

 [49]J. Wing, "Research notebook: Computational thinking—What and why," The Link Magazine, vol. 6, pp. 20-23, 2011.

 [50]S. Furber, "Shut down or restart? The way forward for computing in UK schools," The Royal Society, 2012. Available: https://royalsociety.org/-/media/education/computing-in-schools/2012-01-12-computing-in-schools.pdf.

 [51]M. Tedre and P. J. Denning, "The long quest for computational thinking," in Proceedings of the 16th Koli Calling International Conference on Computing Education Research, pp. 120-129, 2016.

 [52]J. Moreno et al, " No es lo mismo: un análisis de red de texto sobre definiciones de pensamiento computacional para estudiar su relación con la programación informática," Revista Interuniversitaria De Investigación En Tecnología Educativa, 2019, doi: 10.6018/riite.397151.

 [53]ISTE and CSTA, "Operational definition of computational thinking for K-12 education," 2011. Available: https://cdn.iste.org/www-root/Computational_Thinking_Operational_Definition_ISTE.pdf.

 [54]M. U. Bers et al, "Computational thinking and tinkering: Exploration of an early childhood robotics curriculum," Computers & Education, vol. 72, pp. 145-157, 2014. Available: https://search.datacite.org/works/10.1016/j.compedu.2013.10.020, doi: 10.1016/j.compedu.2013.10.020.

 [55]S. Atmatzidou and S. Demetriadis, "Advancing students’ computational thinking skills through educational robotics: A study on age and gender relevant differences," Robotics and Autonomous Systems, vol. 75, pp. 661-670, 2016. Available: https://www.sciencedirect.com/science/article/pii/S0921889015002420, doi: 10.1016/j.robot.2015.10.008.

 [56]N. D. Anderson, "A call for computational thinking in undergraduate psychology," Psychology Learning & Teaching, vol. 15, no. 3, pp. 226-234, 2016.

 [57]S. Rose, J. Habgood and T. Jay, " An exploration of the role of visual programming tools in the development of young children’s computational thinking," Electronic journal of e-learning, 2017, vol. 15, no 4, p. 297-309, Aug 1, 2017.

 [58]D. Weintrop and U. Wilensky, "Using commutative assessments to compare conceptual understanding in blocks-based and text-based programs," in ICER, pp. 101-110, Aug 9, 2015.

 [59]D. Statter and M. Armoni, "Teaching abstract thinking in introduction to computer science for 7th graders," in Proceedings of the 11th Workshop in Primary and Secondary Computing Education, pp. 80-83, 2016.

 [60]B. Zhong et al, " An Exploration of Three-Dimensional Integrated Assessment for Computational Thinking," Journal of Educational Computing Research, vol. 53, no. 4, pp. 562-590, 2016. Available: https://journals.sagepub.com/doi/full/10.1177/0735633115608444, doi: 10.1177/0735633115608444.

 [61]S. Y. Lye and J. H. L. Koh, "Review on teaching and learning of computational thinking through programming: What is next for K-12?" Computers in Human Behavior, vol. 41, pp. 51-61, 2014. Available: https://www.sciencedirect.com/science/article/pii/S0747563214004634, doi: 10.1016/j.chb.2014.09.012.

 [62]E. Hunsaker, Computational Thinking. The K–12 Educational Technology Handbook, A. Ottenbreit-Leftwich and R. Kimmons, Eds. EdTech Books, 2020, pp. 80–95. [Online]. Available: https://bit.ly/2UP0gZi.

 [63]P. J. Rich et al, "Coding in K-8: International Trends in Teaching Elementary/Primary Computing," TechTrends, pp. 1-19, 2018.

 [64]C. S. González-González, "State of the art in the teaching of computational thinking and programming in childhood education," Education in the Knowledge Society, vol. 20, pp. 1-15, 2019.

 [65]S. Bocconi, A. Chioccariello, G. Dettori, A. Ferrari, and K. Engelhardt, “Developing computational thinking in compulsory education,” Eur. Commission, JRC Sci. Policy Rep., pp. 68, 2016.

 [66]Unión Europea, “Plan de Acción de Educación Digital (2021-2027)”, Unión Europa, European Comission, European Education Area, 2021.[Online]. Available: https://ec.europa.eu/jrc/en/digcomp/digital-competence-framework. [Accessed: Dic. 12, 2021].

 [67]Unión Europea, “DigComp. The Digital Competence Framework 2.0”, EU SCIENCE HUB, 2021. [Online]. Available: https://ec.europa.eu/jrc/en/digcomp/digital-competence-framework. [Accessed: Dic. 12, 2021].

 [68]Code.org, “Learn computer science. Change the world”, Code.org. [Online]. Available: https://code.org/. [Accessed: Jan. 20, 2022].

 [69]La hora del código, “Celebramos las Ciencias de la Computación en todas partes”, hourofcode.com. [Online]. Available: https://hourofcode.com/es. [Accessed: Jan. 20, 2022].

 [70]J. Á Velázquez-Iturbide, "Report of the Spanish computing scientific society on computing education in pre-university stages," in Proceedings of the Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality, 2018.

 [71]Instituto Nacional de Tecnologías Educativas y de Formación del Profesorado (INTEF), "Programación, robótica y Pensamiento Computacional en el aula. Situación en España y propuesta normativa", Ministerio de Educación y Formación Profesional, Gobierno de España, Oct, 2018. [Online]. Available: https://code.intef.es/wp-content/uploads/2018/10/Ponencia-sobre-Pensamiento-Computacional.-Informe-Final.pdf.

 [72]Junta de Castilla y León, “Proyecto de innovación 'TIC STEAM' - Curso 2016-2017”, educacyl Portal de Educación, 2017. [Online]. Available: https://www.educa.jcyl.es/profesorado/es/formacion-profesorado/proyectos-relacionados-formacion-permanente-profesorado/competencia-digital/convocatorias/proyecto-innovacion-tic-steam-curso-2016-2017.

 [73]Instituto Nacional de Tecnologías Educativas y de Formación del Profesorado (INTEF), “Escuela de Pensamiento Computacional e Inteligencia Artificial”, Ministerio de Educación y Formación Profesional, Gobierno de España, 2021. [Online]. Available: https://intef.es/tecnologia-educativa/pensamiento-computacional/.

 [74]Programamos, “Es necesario que todo el mundo desarrolle su Pensamiento Computacional, desde niños a mayores”, programamos.com. [Online]. Available: https://programamos.es/. [Accessed: Jan. 20, 2022].

 [75]Boletín Oficial del Estado, “Marco de referencia de la competencia digital docente”, Ministerio de la Presidencia, Relaciones con las Cortes y Memoria Democrática, Gobierno de España. [Online]. Available: https://www.boe.es/diario_boe/txt.php?id=BOE-A-2020-7775#top.

 [76]E. Hunsaker, Integrating computational thinking. K-12 technology integration. Pressbooks. Available: https://k12techintegration.pressbooks.com/chapter/integrating-computational-thinking, 2018.

 [77]M. Zapata-Ros, "Computational thinking unplugged," Education in the Knowledge Society, vol. 20, pp. 1-29, 2019.

 [78]T. Bell et al, "Computer science unplugged: School students doing real computing without computers," The New Zealand Journal of Applied Computing and Information Technology, vol. 13, no. 1, pp. 20-29, 2009.

 [79]S. Adak, "Effectiveness of Constructivist Approach on Academic Achievement in Science at Secondary Level." Educational Research and Reviews, vol. 12, no. 22, pp. 1074-1079, 2017.

 [80]V. Richardson, "Constructivist pedagogy," Teachers College Record, vol. 105, no. 9, pp. 1623-1640, 2003.

 [81]C. T. Fosnot and R. S. Perry, "Constructivism: A psychological theory of learning," Constructivism: Theory, Perspectives, and Practice, vol. 2, no. 1, pp. 8-33, 1996.

 [82]S. Papert and I. Harel, "Situating constructionism," Constructionism, vol. 36, no. 2, pp. 1-11, 1991.

 [83]C. W. Istikomah and C. Budiyanto, "The contribution of educational robotics and constructivist approach to computational thinking in the 21st century," in Proceedings of the the 1st International Conference on Computer Science and Engineering Technology Universitas Muria Kudus, 2018.

 [84]S. Hartikainen et al, "The concept of active learning and the measurement of learning outcomes: A review of research in engineering higher education," Education Sciences, vol. 9, no. 4, pp. 276, 2019.

 [85]R. T. Johnson and D. W. Johnson, "Active learning: Cooperation in the classroom," The Annual Report of Educational Psychology in Japan, vol. 47, pp. 29-30, 2008.

 [86]P. Dillenbourg, Ed., Collaborative Learning: Cognitive and Computational Approaches. Oxford, U.K.: Pergamon, 1999.

 [87]M. Laal and S. M. Ghodsi, "Benefits of collaborative learning," Procedia - Social and Behavioral Sciences, vol. 31, pp. 486-490, 2012. Available: http://www.sciencedirect.com/science/article/pii/S1877042811030205, doi: //doi.org/10.1016/j.sbspro.2011.12.091.

 [88]D. W. Johnson and R. T. Johnson, "Effects of cooperative and individualistic learning experiences on interethnic interaction," Journal of Educational Psychology, vol. 73, (3), pp. 444-449, 1981. Available: https://search.proquest.com/docview/614371667. DOI: 10.1037/00220663.73.3.444.

 [89]M. Laal and M. Laal, "Collaborative learning: what is it?" Procedia - Social and Behavioral Sciences, vol. 31, pp. 491-495, 2012. Available: http://www.sciencedirect.com/science/article/pii/S1877042811030217. DOI: //doi.org/10.1016/j.sbspro.2011.12.092.

 [90]D. Boud and G. E. Feletti, Eds., The Challenge of Problem-Based Learning. London, U.K.: Routledge, 1997, doi: 10.4324/9781315042039.

 [91]J. Roschelle and S. D. Teasley, "The construction of shared knowledge in collaborative problem solving," in Computer Supported Collaborative Learning, 1995.

 [92]P. C. Blumenfeld et al, "Motivating Project-Based Learning: Sustaining the Doing, Supporting the Learning," Educational Psychologist, vol. 26, no. 3-4, pp. 369-398, 1991. Available: http://www.tandfonline.com/doi/abs/10.1080/00461520.1991.9653139, doi: 10.1080/00461520.1991.9653139.

 [93]L. Zhang and J. Nouri, "A systematic review of learning computational thinking through Scratch in K-9," Comput. Educ., vol. 141, pp. 103-607, 2019.

 [94]J. Moreno-León, G. Robles, and M. Román-González, "Towards data-driven learning paths to develop computational thinking with scratch," IEEE Transactions on Emerging Topics in Computing, vol. 8, no. 1, pp. 193-205, 2017.

 [95]J. Nouri et al, "Development of computational thinking, digital competence and 21st century skills when learning programming in K-9," Education Inquiry, vol. 11, no. 1, pp. 1-17, 2020.

 [96]J. Fagerlund et al, "Computational thinking in programming with scratch in primary schools: A systematic review," Computer Applications in Engineering Education, vol. 29, no. 1, pp. 12-28, 2021.

 [97]G. Falloon, "An analysis of young students' thinking when completing basic coding tasks using Scratch Jnr. On the iPad," J. Comput. Assisted Learn., vol. 32, no. 6, pp. 576-593, 2016.

 [98]Massachusetts Institute of Technology: MIT, “Scratch”, scratch.mit.edu, [Online]. Available: https://scratch.mit.edu/.

 [99]P. Mozelius and L. Öberg, "Play‐based learning for programming education in primary school: The östersund model," in European Conference on E-Learning-ECEL 2017, Porto, Portugal, 26-27 October 2017, 2017.

[100]A. G. Soosai Raj et al, "Does native language play a role in learning a programming language?" in Proceedings of the 49th ACM Technical Symposium on Computer Science Education, 2018.

[101]M. Jancheski, "Improving teaching and learning computer programming in schools through educational software," Olympiads in Informatics, vol. 11, pp. 55-75, 2017.

[102]E. F. Silva, B. J. Dembogurski and G. S. Semaan, "A Systematic Review of Computational Thinking in Early Ages," arXiv Preprint arXiv:2106.10275, 2021.

[103]J. M. Cañas Plaza et al, "Entorno docente universitario para la programación de los robots," Revista Iberoamericana De Automática E Informática Industrial, vol. 15, no. 4, pp. 404-415, 2018.

[104]A. Hamidi, M. Milrad and S. Tavajoh, "Combining maker technologies to promote computational thinking and heart-ware skills through project-based activities: Design considerations and empirical outputs," in Cte-Stem 2021, 2021.

[105]R. Zerega et al, "A co-design approach for developing computational thinking skills in connection to STEM related curriculum in swedish schools," in Cte-Stem 2021, 2021.

[106]M. Riedmiller and T. Gabel, "On experiences in a complex and competitive gaming domain: Reinforcement learning meets robocup," in 2007 IEEE Symposium on Computational Intelligence and Games, 2007.

[107]N. L. Fanchamps et al, "The influence of SRA programming on algorithmic thinking and self-efficacy using Lego robotics in two types of instruction," International Journal of Technology and Design Education, vol. 31, no. 2, pp. 203-222, 2021.

[108]N. L. Fanchamps et al, "The Impact of SRA-Programming on Computational Thinking in a Visual Oriented Programming Environment," Education and Information Technologies, vol. 26, no. 5, pp. 6479-6498, 2021.

[109]M. Montessori, J. M. Hunt and J. Valsiner, The Montessori Method. Routledge, 2017.

[110]M. Elkin, A. Sullivan, and M. U. Bers, "Programming with the KIBO robotics kit in preschool classrooms," Computers in the Schools, vol. 33, no. 3, pp. 169-186, 2016.

[111]L. L. Román, "Metodología para la enseñanza aprendizaje de la lógica de la programación orientada a objetos," in 5ta. Conferencia Iberoamericana en Sistemas, Cibernética e Informática CISCI, International Institute of Informatics and Systemics, EE. UU, 2007.

[112]J. Urquiza-Fuentes and J. Á Velázquez-Iturbide, "A survey of successful evaluations of program visualization and algorithm animation systems," ACM Transactions on Computing Education (TOCE), vol. 9, no. 2, pp. 1-21, 2009.

[113]L. Seiter and B. Foreman, "Modeling the learning progressions of computational thinking of primary grade students," in Proceedings of the Ninth Annual International ACM Conference on International Computing Education Research, 2013, Available: http://doi.acm.org/10.1145/2493394.2493403, doi: 10.1145/2493394.2493403.

[114]S. Grover, R. Pea and S. Cooper, "Designing for deeper learning in a blended computer science course for middle school students," Computer Science Education, vol. 25, no. 2, pp. 199-237, 2015.

[115]A. Basawapatna et al, "Recognizing computational thinking patterns," in Proceedings of the 42nd ACM technical symposium on Computer science education, pp. 245-250 Mar 9, 2011.

[116]S. Papadakis and M. Kalogiannakis, "Evaluating a course for teaching advanced programming concepts with scratch to preservice kindergarten teachers: A case study in greece," in Early Childhood EducationAnonymous 2018.

[117]A. Eguiluz et al, "Exploring the progression of early programmers in a set of computational thinking challenges via clickstream analysis," Tetc, pp. 1, 2017. Available: https://ieeexplore.ieee.org/document/8093668, doi: 10.1109/TETC.2017.2768550.

[118]G. Petri and C. Gresse von Wangenheim, "How games for computing education are evaluated? A systematic literature review," Computers & Education, vol. 107, pp. 68-90, 2017. Available: http://dx.doi.org/10.1016/j.compedu.2017.01.004, doi: 10.1016/j.compedu.2017.01.004.

[119]A. Vahldick, A. J. Mendes and M. J. Marcelino, "Dynamic Difficulty Adjustment through a Learning Analytics Model in a Casual Serious Game for Computer Programming Learning," EAI Endorsed Transactions on Serious Games, vol. 4, no. 13, 2017.

[120]C. Kroustalli and S. Xinogalos, "Studying the effects of teaching programming to lower secondary school students with a serious game: a case study with Python and CodeCombat," Education and Information Technologies, pp. 1-27, 2021.

[121]M. Csikszentmihalyi, Finding Flow: The Psychology of Engagement with Everyday Life. United States: 1997.

[122]S. H. Chuah, "Why and who will adopt extended reality technology? Literature review, synthesis, and future research agenda," Literature Review, Synthesis, and Future Research Agenda, Dec. 13, 2018.

[123]M. Montola, "Exploring the edge of the magic circle: Defining pervasive games," in Proceedings of DAC, 2005.

[124]C. Bonillo, J. Marco, and E. Cerezo, "Developing pervasive games in interactive spaces: the JUGUEMOS toolkit," Multimedia Tools Appl, vol. 78, no. 22, pp. 32261-32305, 2019.

[125]N. M. Medina et al, "Hacia una propuesta de evaluación heurística de experiencias de juego pervasivas," Revista De La Asociación Interacción Persona Ordenador (AIPO), vol. 2, no. 2, pp. 42-53, 2021.

[126]J. Marco, C. Bonillo, and E. Cerezo, "A tangible interactive space odyssey to support children learning of computer programming," in Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces, 2017.

[127]S. SUKIRMAN et al, "A Strategy of Learning Computational Thinking through Game Based in Virtual Reality: Systematic Review and Conceptual Framework," Informatics in Education, 2021.

[128]Y. Ching, Y. Hsu and S. Baldwin, "Developing computational thinking with educational technologies for young learners," TechTrends, vol. 62, no. 6, pp. 563-573, 2018.

[129]C. P. Brackmann et al, "Development of computational thinking skills through unplugged activities in primary school," in Proceedings of the 12th Workshop on Primary and Secondary Computing Education, 2017.

[130]S. Suh, M. Lee and G. Xia, "Coding strip: A pedagogical tool for teaching and learning programming concepts through comics," in 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), 2020.

[131]G. Aranda and J. P. Ferguson, "Unplugged Programming: The future of teaching computational thinking?" Pedagogika, vol. 68, no. 3, 2018.

[132]J. Waite et al, "Unplugged computing and semantic waves: Analysing crazy characters," in Proceedings of the 1st UK & Ireland Computing Education Research Conference, 2019.

[133]A. Dengel, "Computer science replugged: What is the use of virtual reality in computer science education?" in Proceedings of the 14th Workshop in Primary and Secondary Computing Education, 2019.

[134]X. Tang et al, "Assessing computational thinking: A systematic review of empirical studies," Computers & Education, vol. 148, pp. 103-798, 2020. Available: http://www.sciencedirect.com/science/article/pii/S0360131519303483, doi: https://doi.org/10.1016/j.compedu.2019.103798.

[135]J. Denner, L. Werner and E. Ortiz, "Computer games created by middle school girls: Can they be used to measure understanding of computer science concepts?" Computers & Education, vol. 58, no. 1, pp. 240-249, 2012. Available: https://www.sciencedirect.com/science/article/pii/S0360131511001849, doi: 10.1016/j.compedu.2011.08.006.

[136]I. Cetin and E. Dubinsky, "Reflective abstraction in computational thinking," Journal of Mathematical Behavior, vol. 47, pp. 70-80, 2017. Available: https://www.sciencedirect.com/science/article/pii/S0732312316301511, doi: 10.1016/j.jmathb.2017.06.004.

[137]S. Jun et al, "Assessing the computational literacy of elementary students on a national level in Korea," Educational Assessment, Evaluation and Accountability, vol. 26, no. 4, pp. 319-332, 2014.

[138]L. Werner et al, "The fairy performance assessment," in Proceedings of the 43rd ACM technical symposium on Computer Science Education, pp. 215-220, 2012.

[139]V. Garneli and K. Chorianopoulos, "Programming video games and simulations in science education: exploring computational thinking through code analysis," Interactive Learning Environments, vol. 26, no. 3, pp. 386-401, 2018. Available: http://www.tandfonline.com/doi/abs/10.1080/10494820.2017.1337036, doi: 10.1080/10494820.2017.1337036.

[140]A. Mühling, A. Ruf and P. Hubwieser, "Design and first results of a psychometric test for measuring basic programming abilities," in Proceedings of the workshop in primary and secondary computing education, pp. 2-10, Nov 9, 2015.

[141]Ö Korkmaz, R. Çakir and M. Y. Özden, "A validity and reliability study of the computational thinking scales (CTS)," Computers in Human Behavior, vol. 72, pp. 558-569, 2017. Available: https://www.sciencedirect.com/science/article/pii/S0747563217300055, doi: 10.1016/j.chb.2017.01.005.

[142]M. Román-González et al, "Extending the nomological network of computational thinking with non-cognitive factors," Computers in Human Behavior, vol. 80, pp. 441-459, 2018. Available: http://www.sciencedirect.com/science/article/pii/S0747563217305563, doi: https://doi.org/10.1016/j.chb.2017.09.030.

[143]M. Román-González et al, "Can computational talent be detected? Predictive validity of the Computational Thinking Test," International Journal of ChildComputer Interaction, vol. 18, pp. 47-58, 2018. Available: http://www.sciencedirect.com/science/article/pii/S221286891730034X, doi: https://doi.org/10.1016/j.ijcci.2018.06.004.

[144]AERA, APA and NCME, Standards for Educational and Psychological Testing. 2014. Available: http://www.fachportal-paedagogik.de/fis_bildung/suche/fis_set.html?FId=1050398.

[145]G. Chen et al, "Assessing elementary students’ computational thinking in everyday reasoning and robotics programming," Computers & Education, vol. 109, pp. 162-175, 2017. Available: http://www.sciencedirect.com/science/article/pii/S0360131517300490, doi: https://doi.org/10.1016/j.compedu.2017.03.001.

[146]E. Relkin, L. de Ruiter and M. U. Bers, "TechCheck: Development and Validation of an Unplugged Assessment of Computational Thinking in Early Childhood Education," Journal of Science Education and Technology, vol. 29, no. 4, pp. 482-498, 2020. Available: https://search.proquest.com/docview/2413230852, doi: 10.1007/s10956-020-09831-x.

[147]Y. Li, S. Xu, and J. Liu, "Development and Validation of Computational Thinking Assessment of Chinese Elementary School Students," Journal of Pacific Rim Psychology, vol. 15, pp. 18344909211010240, 2021. Available: https://doi.org/10.1177/18344909211010240, doi: https://doi.org/10.1177/18344909211010240.

[148]S. Grover, R. Pea and S. Cooper, "«Systems of assessments» for deeper learning of computational thinking in K-12," in Proceedings of the 2015 Annual Meeting of the American Educational Research Association, 2015.

[149]G. Siemens and P. Long, "Penetrating the Fog: Analytics in Learning and Education," EDUCAUSE Review, vol. 46, no. 5, pp. 30, 2011.

[150]A. Mathrani, S. Christian and A. Ponder-Sutton, "PlayIT: Game based learning approach for teaching programming concepts," Journal of Educational Technology & Society, vol. 19, no. 2, pp. 5-17, 2016.

[151]C. Malliarakis, M. Satratzemi and S. Xinogalos, "CMX: The effects of an educational MMORPG on learning and teaching computer programming," IEEE Transactions on Learning Technologies, vol. 10, no. 2, pp. 219-235, 2016.

[152]C. Malliarakis, M. Satratzemi and S. Xinogalos, "Integrating learning analytics in an educational MMORPG for computer programming," in 2014 IEEE 14th International Conference on Advanced Learning Technologies, 2014.

[153]W. Greller and H. Drachsler, "Translating learning into numbers: A generic framework for learning analytics," Journal of Educational Technology & Society, vol. 15, no. 3, pp. 42-57, 2012.

[154]M. A. Chatti et al, "Learning Analytics: Challenges and Future Research Directions," eleed, vol. 10, no 1. Nov 6, 2014.

[155]M. Román González, "Computational thinking test: Design guidelines and content validation," in Proceedings of EDULEARN15 conference, Barcelona, pp. 2436-2444, 201.

[156]D. Căprioară, "Emotions and the Learning of School Mathematics," Bulletin of the Transilvania University of Braşov, Series VII: Social Sciences and Law, vol. 10, no. 1, pp. 9-18, 2017. Available: http://www.ceeol.com/search/article-detail?id=566146.

[157]C. Chen and P. Herbst, "The interplay among gestures, discourse, and diagrams in students’ geometrical reasoning," Educational Studies in Mathematics, vol. 83, no. 2, pp. 285-307, 2013. Available: 10.1007/s10649-012-9454-2, doi: https://doi.org/10.1007/s10649-012-9454-2.

[158]H. Y. Durak and M. Saritepeci, "Analysis of the relation between computational thinking skills and various variables with the structural equation model," Computers & Education, vol. 116, pp. 191-202, 2018. Available: http://www.sciencedirect.com/science/article/pii/S0360131517302087, doi: //doi.org/10.1016/j.compedu.2017.09.004.

[159]T. Watanabe, "Visual Reasoning Tools in Action: Double Number Lines, Area Models, and Other Diagrams Power Up Students' Ability to Solve and Make Sense of Various Problems," Mathematics Teaching in the Middle School, vol. 21, no. 3, pp. 152, 2015.

[160]C. E. Lance, M. M. Butts and L. C. Michels, "The sources of four commonly reported cutoff criteria: What did they really say?" Organ. Res. Methods, vol. 9, no. 2, pp. 202-220, 2006.

[161]M. Román-González, J. Moreno-León and G. Robles, "Combining assessment tools for a comprehensive evaluation of computational thinking interventions," in Computational Thinking Education, S. Kong and H. Abelson, Eds. 2019, Available: https://doi.org/10.1007/978-981-13-6528-7_6, doi: 10.1007/978-981-13-6528-7_6.

[162]A. A. De Souza et al, "Data mining framework to analyze the evolution of computational thinking skills in game building workshops," IEEE Access, vol. 7, pp. 82848-82866, 2019.

[163]R. F. DeVellis, "Classical test theory," Med. Care, pp. S50-S59, 2006.

[164]R. K. Hambleton and R. W. Jones, "Comparison of classical test theory and item response theory and their applications to test development," Educational Measurement: Issues and Practice, vol. 12, no. 3, pp. 38-47, 1993.

[165]L. El-Hamamsy, M. Zapata-Caceres, E. Martín, F. Mondada, J. Dehler Zufferey, and B. Bruno, "The competent Computational Thinking test (cCTt): Development and validation of an unplugged Computational Thinking test for upper primary school," Journal of Educational Computing Research, 2022. In press.

[166]S. Kong, "A framework of curriculum design for computational thinking development in K-12 education," Journal of Computers in Education, vol. 3, no. 4, pp. 377-394, 2016.

[167]F. Buitrago Flórez et al, "Changing a Generation's Way of Thinking: Teaching Computational Thinking Through Programming," Review of Educational Research, vol. 87, no. 4, pp. 834, 2017. Available: https://search.proquest.com/docview/1917804191, doi: 10.3102/0034654317710096.

[168]J. Alvarez et al, "Blue Bot Project Experiment," 8th International Toy Research Association World Conference Jul 11, 2018.

[169]C. Sáez Fernández, G. Viera López and D. Pérez Marín, "Propuesta metodológica de la enseñanza de la programación en Educación Infantil con Cubetto," IE Comunicaciones: Revista Iberoamericana De Informática Educativa, no. 28, pp. 1-8, 2018. Available: https://dialnet.unirioja.es/servlet/oaiart?codigo=6750332.

[170]W. Wu et al, "Investigating the learning‐theory foundations of game‐based learning: a meta‐analysis," Journal of Computer Assisted Learning, vol. 28, no. 3, pp. 265-279, 2012. Available: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1365-2729.2011.00437.x, doi: 10.1111/j.1365-2729.2011.00437.x.

[171]E. Boyle, T. M. Connolly, and T. Hainey, "The role of psychology in understanding the impact of computer games," Entertainment Computing, vol. 2, no. 2, pp. 69-74, 2011. Available: https://www.sciencedirect.com/science/article/pii/S1875952110000200, doi: 10.1016/j.entcom.2010.12.002.

[172]M. Li and C. Tsai, "Game-Based Learning in Science Education: A Review of Relevant Research," Journal of Science Education and Technology, vol. 22, no. 6, pp. 877-898, 2013. Available: https://doi.org/10.1007/s10956-013-9436-x, doi: 10.1007/s10956-013-9436-x.

[173]W. Wu et al, "Re-exploring game-assisted learning research: The perspective of learning theoretical bases," Computers & Education, vol. 59, no. 4, pp. 1153-1161, 2012. Available: https://www.sciencedirect.com/science/article/pii/S0360131512001248, doi: 10.1016/j.compedu.2012.05.003.

[174]M. Csikszentmihalyi, "Flow and the psychology of discovery and invention," HarperPerennial, New York, vol. 39, 1997.

[175]T. Hainey et al, "A systematic literature review of games-based learning empirical evidence in primary education," Computers & Education, vol. 102, pp. 202-223, 2016.

[176]J. R. Savery and T. M. Duffy, "Problem based learning: An instructional model and its constructivist framework," Educational Technology, vol. 35, no. 5, pp. 31-38, 1995.

[177]J. R. Savery, "Overview of Problem-based Learning: Definitions and distinctions," Essential readings in problem-based learning: Exploring and extending the legacy of Howard S. Barrows, vol. 9, no 2, pp. 5-15, 2015. Available: https://library.biblioboard.com/viewer/620aa903-49eb-49d1-8fc2-b02dbeda193c.

[178]B. Haßler, L. Major, and S. Hennessy, "Tablet use in schools: A critical review of the evidence for learning outcomes," Journal of Computer Assisted Learning, vol. 32, no. 2, pp. 139-156, 2016.

[179]Y. Rogers et al, " Equal opportunities: Do shareable interfaces promote more group participation than single user displays?" Human–Computer Interaction, vol. 24, no. 1-2, pp. 79-116, 2009.

[180]D. Roldan-Alvarez et al, "DEDOS: An Authoring Toolkit to Create Educational Multimedia Activities for Multiple Devices," IEEE Transactions on Learning Technologies, vol. 11, no. 4, pp. 493-505, 2018. Available: https://ieeexplore.ieee.org/document/8244297, doi: 10.1109/TLT.2017.2788867.

[181]C. Lin, L. Wong and Y. Shao, "Comparison of 1: 1 and 1: m CSCL environment for collaborative concept mapping," Journal of Computer Assisted Learning, vol. 28, no. 2, pp. 99-113, 2012.

[182]M. Zapata-Cáceres, E. Martín-Barroso and M. Román-González, "Computational Thinking Test for Beginners: Design and Content Validation," 2020 IEEE Global Engineering Education Conference (EDUCON). in Press, 2020.

[183]R. S. Pressman, Software Engineering: A Practitioner's Approach. Palgrave macmillan, 2005.

[184]C. Larman, Agile and Iterative Development: A Manager's Guide. Addison-Wesley Professional, 2004.

[185]L. García-de-la-Fuente, "Tratamiento de ficheros generados por Blue Ant Code," Trabajo De Fin De Grado En Ingeniería Informática, Universidad Rey Juan Carlos, 2020.

[186]G. Matilla-Jiménez, "Desarrollo una aplicación web para facilitar la visualización y análisis se datos de Blue Ant Code," Trabajo De Fin De Grado En Ingeniería Informática, Universidad Rey Juan Carlos, 2020.

[187]J. Moreno and G. Robles, "Automatic detection of bad programming habits in scratch: A preliminary study," in 2014 IEEE Frontiers in Education Conference (FIE) Proceedings, pp. 1-4, Oct. 2014.

[188]N. Diana et al, "An instructor dashboard for real-time analytics in interactive programming assignments," in Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 272-279, 2017.

[189]L. Gal et al, "Suggesting a log-based creativity measurement for online programming learning environment," in Proceedings of the Fourth (2017) ACM Conference on Learning@ Scale, pp. 273-277, 2017.

[190]D. Olivares and C. Hundhausen, "Supporting learning analytics in computing education," in Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 584-585, 2017.

[191]M. Zapata-Ros and P. Pérez-Paredes, El Pensamiento Computacional. Análisis De Una Competencia Clave. New York: Create Space Independent Publishing, 2018.

[192]H. Søndergaard and R. A. Mulder, "Collaborative learning through formative peer review: Pedagogy, programs and potential," Computer Science Education, vol. 22, no. 4, pp. 343-367, 2012.

[193]M. R. Snodgrass, M. Israel and G. C. Reese, "Instructional supports for students with disabilities in K-5 computing: Findings from a cross-case analysis," Computers & Education, vol. 100, pp. 1-17, 2016.

[194]N. Ayuso et al, "Gender Gap in STEM: A Cross-Sectional Study of Primary School Students’ Self-Perception and Test Anxiety in Mathematics," IEEE Transactions on Education, vol. 64, no. 1, pp. 40-49, 2020.

[195]L. Sun et al, "Primary school students' perceptions of scaffolding in digital gamebased learning in mathematics," Learning, Culture and Social Interaction, vol. 28, pp. 100457, 2021.

[196]M. Cengiz et al, "Development of an interactive game-based learning environment to teach data mining," The International Journal of Engineering Education, vol. 33, no. 5, pp. 1598-1617, 2017.

[197]H. Lu, "Rethinking game-based learning from a gender perspective: a case study of a male English language learner in China," Gender, Technology and Development, vol. 16, no. 1, pp. 49-70, 2012.

[198]E. A. Boyle et al, "An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games," Computers & Education, vol. 94, pp. 178-192, 2016. Available: http://www.sciencedirect.com/science/article/pii/S0360131515300750, doi: //doi.org/10.1016/j.compedu.2015.11.003.

[199]R. J. Mislevy, L. S. Steinberg and R. G. Almond, "Focus Article: On the Structure of Educational Assessments," Measurement (Mahwah, N.J.), vol. 1, no. 1, pp. 3-62, 2003. Available: http://www.tandfonline.com/doi/abs/10.1207/S15366359MEA0101_02, doi: 10.1207/S15366359MEA0101_02.

[200]V. J. Shute and G. R. Moore, "Consistency and validity in game-based stealth assessment," Technology Enhanced Innovative Assessment: Development, Modeling, and Scoring from an Interdisciplinary Perspective, vol. 296, 2017.

[201]C. Alonso Fernández et al, "Improving serious games analyzing learning analytics data: lessons learned," in International Conference on Games and Learning Alliance. Springer, Cham Dec 5, 2018.

[202]G. Wallner and S. Kriglstein, "Comparative visualization of player behavior for serious game analytics," in Serious Games Analytics, Cham: Springer International Publishing, pp. 159-179, 2015.

[203]E. Zimmerman, "Narrative, interactivity, play, and games: Four naughty concepts in need of discipline," First Person: New Media as Story, Performance, and Game, vol. 154, 2004.

[204]E. Carratala Sánchez, "Análisis de la teoría de las metas de logro y de la autodeterminación en los planes de especialización deportiva de la Generalitat Valenciana." 2004.

[205]C. J. Ferguson and C. K. Olson, "Friends, fun, frustration and fantasy: Child motivations for video game play," Motivation and Emotion, vol. 37, no. 1, pp. 154-164, 2013.

[206]P. R. Pintrich and A. Zusho, "The development of academic self-regulation: The role of cognitive and motivational factors," in Development of Achievement Motivation, 2002.

[207]A. E. Gottfried, "Academic intrinsic motivation in young elementary school children." J. Educ. Psychol., vol. 82, no. 3, pp. 525, 1990.

[208]R. M. Ryan and E. L. Deci, "Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being." Am. Psychol., vol. 55, no. 1, pp. 68, 2000.

[209]E. L. Deci and R. M. Ryan, "Intrinsic motivation," The Corsini Encyclopedia of Psychology, pp. 1-2, 2010.

[210]J. R. Calvo-Ferrer, "Juegos, videojuegos y juegos serios: Análisis de los factores que favorecen la diversión del jugador," Miguel Hernández Communication Journal, no. 9, pp. 191-226, 2018.

[211]I. Granic, A. Lobel and R. C. Engels, "The benefits of playing video games." American Psychologist Journal, vol. 69, no. 1, pp. 66, 2014.

[212]M. Ventura, V. Shute and W. Zhao, "The relationship between video game use and a performance-based measure of persistence," Computers & Education, vol. 60, no. 1, pp. 52-58, 2013.

[213]R. M. Ryan, C. S. Rigby and A. Przybylski, "The motivational pull of video games: A self-determination theory approach," Motivation and Emotion, vol. 30, no. 4, pp. 344-360, 2006.

[214]M. Salminen and N. Ravaja, "Increased oscillatory theta activation evoked by violent digital game events," Neuroscience Letters - Journals, vol. 435, no. 1, pp. 69-72, 2008.

[215]R. Israel-Fishelson and A. Hershkovitz, "Shooting for the stars: Micro-persistence of students in game-based learning environments," in Early Warning Systems and Targeted Interventions for Student Success in Online Courses, 2020.

[216]C. Hart, "Factors associated with student persistence in an online program of study: A review of the literature." Journal of Interactive Online Learning, vol. 11, no. 1, 2012.

[217]R. Israel‐Fishelson and A. Hershkovitz, "Micro‐persistence and difficulty in a game‐based learning environment for computational thinking acquisition," Journal of Computer Assisted Learning, 2021.

[218]Y. Fang et al, "Online Learning Persistence and Academic Achievement." International Educational Data Mining Society, 2017.

[219]C. Alonso-Fernández et al, "Applications of data science to game learning analytics data: A systematic literature review," Computers & Education, vol. 141, pp. 103612, 2019. Available: https://search.datacite.org/works/10.1016/j.compedu.2019.103612, doi: 10.1016/j.compedu.2019.103612.

[220]P. R. Pintrich and A. Zusho, "Student motivation and self-regulated learning in the college classroom," in Higher Education: Handbook of Theory and ResearchAnonymous 2002.

[221]K. Weber and B. R. Patterson, "Student interest, empowerment and motivation," Communication Research Reports, vol. 17, no. 1, pp. 22-29, 2000.

[222]Á Serrano-Laguna et al, "Applying standards to systematize learning analytics in serious games," Computer Standards & Interfaces, vol. 50, pp. 116-123, 2017.

[223]Y. Tran, "Computational thinking equity in elementary classrooms: What third-grade students know and can do," Journal of Educational Computing Research, vol. 57, no. 1, pp. 3-31, 2019.

[224]R. Israel-Fishelson and A. Hershkovitz, "Persistence and achievement in acquiring computational thinking concepts: A large-scale log-based analysis," in E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, 2019.

[225]S. Grover et al, "Assessing problem-solving process at scale," in Proceedings of the Third (2016) ACM Conference on Learning@ Scale, pp. 245-248, Apr 25, 2016.

[226]M. Zapata-Cáceres and E. Martín-Barroso, "Blue Ant Code," 2020. Available: https://play.google.com/store/apps/details?id=com.artbabygame.blueAnt https://apps.apple.com/es/app/blue-ant-code/id1495021110.

[227]S. Kong, M. M. Chiu and M. Lai, "A study of primary school students' interest, collaboration attitude, and programming empowerment in computational thinking education," Computers & Education, vol. 127, pp. 178-189, 2018. Available: http://dx.doi.org/10.1016/j.compedu.2018.08.026, doi: 10.1016/j.compedu.2018.08.026.

[228]M. Guenaga et al, "How do students develop computational thinking? Assessing early programmers in a maze-based online game," Computer Science Education, vol. 31, no 2, pp. 259-289, 2021. Available: https://doi.org/10.1080/08993408.2021.1903248, doi: 10.1080/08993408.2021.1903248.

[229]T. P. Alloway, "How does working memory work in the classroom?" Educational Research and Reviews, vol. 1, no. 4, pp. 134-139, 2006.

[230]C. S. Dweck and D. C. Molden, "Their impact on competence motivation and acquisition," Handbook of Competence and Motivation: Theory and Application, vol. 2, pp. 135-154, 2017.

[231]L. S. Blackwell, K. H. Trzesniewski and C. S. Dweck, "Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention," Child Dev., vol. 78, no. 1, pp. 246-263, 2007.

[232]P. Sweetser and P. Wyeth, "GameFlow: a model for evaluating player enjoyment in games," Computers in Entertainment (CIE), vol. 3, no. 3, pp. 3, 2005.

[233]M. Lafuente Martínez et al, "Assessing Computational Thinking: Development and Validation of the Algorithmic Thinking Test for Adults," 2021.

[234]L. Narcisi et al, "Beginners Computational Thinking Test: Adaptation pour un public de moins de 5 ans." Service D'Ingénierie Pédagogique Et Du Numérique Éducatif, FPSE, UMONS, s.f., 2021.

[235]M. Lázaro-Álvarez, "Sudokus adaptativos para Educación Infantil o Primaria," Trabajo De Fin De Grado En Ingeniería Informática, Universidad Rey Juan Carlos, 2021.

[236]J. Plaza-Carro, "Blue-encode: Juego colaborativo de cifrado y descifrado de mensajes en Educación Primaria," Trabajo De Fin De Grado En Ingeniería Informática, Universidad Rey Juan Carlos, 2021.