CAPÍTULO 3. INVESTIGACIÓN EMPÍRICA: HIPÓTESIS Y EXPERIENCIAS

Una vez definido el marco teórico y estudiado el estado de arte, a continuación, se exponen las conclusiones que se extraen de esta investigación teórica y, en base a las mismas, la dirección operativa que se establece para la investigación empírica, tal y como se establece en el objetivo general OP2: Establecer una propuesta operativa del PC en base al OP1 y así constituir la dirección de la investigación empírica mediante hipótesis y objetivos base. Posteriormente, en cada experiencia empírica, se detallarán las hipótesis y objetivos específicos de la misma (ver apartado 1.2. Objetivos ).

De esta forma, en este capítulo se plantean, en relación con los objetivos generales de la tesis, las hipótesis, desarrollos y experiencias realizadas en la investigación empírica. En los capítulos siguientes, se irán detallando en profundidad dichos desarrollos y experiencias, así como las hipótesis y objetivos específicas relativas a cada una de ellas.

3.1. Propuesta de definición de PC

Dado que no existe una definición consensuada del PC, se ha realizado una revisión de la literatura y, basándonos en los puntos en común de las definiciones existentes (apartado 2.1), en esta tesis se propone la siguiente definición:

El pensamiento computacional es una habilidad cognitiva que permite resolver problemas utilizando estrategias computacionales.

De igual forma, en esta tesis, se propone la siguiente descomposición del PC en seis factores y sus definiciones: 1) abstracción (proceso de obtener algo simple desde algo complejo, obviando los detalles); 2) análisis de datos (buscar, seleccionar, organizar y analizar lógicamente los datos); 3) descomposición de problemas (descomponer problemas en otros más pequeños que pueden resolverse con mayor facilidad; 4) algoritmia (identificar instrucciones específicas y explícitas que, paso a paso, llevan a cabo un proceso); 5) depuración de errores (identificar y corregir los errores en la solución aportada); y 6) generalización (transferir un proceso de resolución de problemas a una variedad grande de problemas).

Además, se propone también la identificación de disposiciones y predisposiciones para el aprendizaje del PC, tal y como se plantea en la literatura [41], [53]. Esta identificación tendrá especial relevancia en el marco de la tesis, ya que se analizarán estos aspectos motivacionales y de persistencia ante los retos en el Capítulo 7.

En cuanto al marco de referencia educativo, se ha seleccionado uno de los marcos de PC más citados en la literatura y más utilizados empíricamente, que es el marco 3D de Brennan y Resnick (2012) [46]. Este marco clasifica el PC según tres dimensiones: 1) conceptos computacionales (conceptos que utilizan los programadores); 2) prácticas computacionales (prácticas de resolución de problemas que se necesitan/producen en el proceso de programación); y 3) perspectivas computacionales (perspectivas que se forman los diseñadores sobre sí mismos y el mundo que les rodea). Este marco es adecuado para considerar el PC en entornos que utilizan la programación informática y en Educación Primaria [61], por lo que parece adecuado como punto de partida en esta tesis.

3.2. Hipótesis generales de la investigación empírica

A continuación, se describen las hipótesis generales que se plantean tras la investigación teórica y que constituyen la base para la investigación empírica realizada. A continuación, se especifican las hipótesis empíricas (HE) planteadas y las experiencias realizadas en relación a cada una de ellas.

HE1 – sobre la evaluación del PC en Educación Primaria.

Es posible evaluar el PC en la etapa de Educación Primaria mediante un test tradicional independiente de un entorno concreto.

Como se ha evidenciado en el estado del arte, pese a que evaluar del progreso de los estudiantes es indispensable para la introducción del PC en el currículo [31], hay una falta de consenso sobre qué estrategias son las más adecuadas para esta evaluación y, además, existen pocos instrumentos en este sentido y no suelen incluir datos relativos a su fiabilidad y validez. Sin embargo, es manifiesta la necesidad de una evaluación fiable y válida del PC en todos los contextos educativos para que los investigadores puedan determinar si sus intervenciones son eficaces o no.

Al comienzo de este trabajo de investigación, en el año 2019, no existía ningún instrumento de tipo test tradicional, independiente de entornos concretos, para evaluar el PC en Educación Primaria, por lo que los esfuerzos en esta investigación se han orientado hacia esta dirección mediante el desarrollo del test de PC para principiantes: Beginners Computational Thinking test (BCTt), publicado en 2020 [8], y que se detalla en el Capítulo 4, para el que el Computational Thinking Test (CTt) [44] ha sido una base consolidada y firme.

El BCTt ha tenido gran repercusión internacional y, junto a otros equipos de investigación de países como Suiza, Portugal o Países Bajos, se están realizando esfuerzos para determinar los límites de edad para la aplicación del test o desarrollando nuevas versiones para adaptarse a estudiantes de diferentes edad y poblaciones. De esta forma, en función de los resultados de fiabilidad del BCTt para los ciclos superiores de Educación Primaria, y gracias a la colaboración internacional, se ha desarrollado y validado el competence Computational Thinking test (cCTt) para así, junto con el BCTt y el CTt, cubrir todos los rangos de edad correspondientes a esta etapa educativa.

Por otro lado, se plantea poder utilizar el BCTt en también en la etapa de Educación Infantil, por lo que en el Capítulo 5 se describe una experiencia en la que trata de obtener el límite inferior de edad en la que se puede administrar el test, validando su fiabilidad en niños de 4 y 5 años.

HE2 – sobre la relación de estrategias de aprendizaje con marco 3D.
Mediante diferentes estrategias de aprendizaje, pueden abarcarse las tres dimensiones del PC, de forma que la relación entre el marco 3D y las estrategias de aprendizaje es como se muestra en la Figura 10.

Del estado del arte, se ha detectado que la mayoría de las investigaciones y actuaciones para el desarrollo del PC en el aula, se centran en la dimensión conceptos computacionales del marco 3D de PC de Brennan y Resnick [46], [61], [93] y, solamente en algunas investigaciones muy recientes, sobre el aprendizaje y la evaluación de las dimensiones prácticas computacionales y perspectivas computacionales [93]-[97]. Esto nos lleva a la necesidad de más investigación y de prestar especial atención a estas dos últimas dimensiones, especialmente más allá del entorno de Scratch [98] en el cual se ubican la mayor parte de los casos de estudio en las investigaciones. Además, el PC es un conjunto interdisciplinario de habilidades y es necesario que existan métodos operativos de aprendizaje a través de diversas estrategias [61], [96].

Además, para dotar a los estudiantes de habilidades de resolución de problemas que puedan transferir posteriormente a dominios no relacionados con la programación [41], es necesario desarrollar el PC desde etapas tempranas y es especialmente importante desarrollar las prácticas y perspectivas computacionales. Estas dos últimas dimensiones del marco 3D son las que nos permiten trabajar la componente de esencial del PC que, en la descomposición propuesta en esta tesis, se ha descrito como la generalización.

En el marco teórico de esta tesis se exploraron las estrategias de aprendizaje basado en problemas, aprendizaje colaborativo y aprendizaje basado en juegos como las estrategias de aprendizaje más utilizadas y con más impacto en el aprendizaje del PC. En esta tesis se va a tratar de relacionar el posible impacto de cada una de estas estrategias de aprendizaje en cada dimensión del marco 3D del PC. En la Figura 10 se muestra nuestra hipótesis sobre dicha relación: las dimensiones computacionales se sitúan en los vértices del triángulo, de modo que cuanto más cerca de un vértice aparezca la estrategia, mayor será la probabilidad de que esta dimensión se vea afectada por la estrategia.

A diferencia de la estrategia de aprendizaje basada en problemas, en la estrategia de aprendizaje basada en juegos el alumno no recibe necesariamente una definición previa de los conceptos o contenidos a tratar, sino que guía su propio aprendizaje libremente, motivado por la sensación de logro (véase la Tabla 4), por lo que nuestra hipótesis es que la adquisición de conceptos podría no ser completa. Por el contrario, en el aprendizaje basado en juegos, las prácticas computacionales podrían verse especialmente potenciadas, ya que dominarlas es esencial para progresar en los retos o niveles que se proponen [61].

Por otro lado, la hipótesis es que las prácticas y perspectivas computacionales se pueden ver reforzadas si se utilizan una estrategia colaborativa. Esta estrategia se utiliza a menudo en los estudios que implican experiencias de programación y puede mejorar tanto las prácticas como las dimensiones de las perspectivas computacionales, debido a que los estudiantes revisan y piensan sobre su experiencia de programación, mejorando así su propio proceso de aprendizaje [61]. En un entorno colaborativo enfocado a aprender programación, los estudiantes no sólo reflexionan sobre el funcionamiento de su programa, sino que también revisan y piensan en los programas que han creado sus compañeros. El debate y discusión, en voz alta, de todas las soluciones de programación aportadas, propias y ajenas, puede mejorar estas dos dimensiones del PC, especialmente las perspectivas computacionales [15], [60], [61].

Posteriormente, mediante la experiencia en colegios descrita en el Capítulo 6, en el que se utilizarán las tres estrategias de aprendizaje simultáneamente, se comprobará esta hipótesis.

HE3 – sobre la estrategia basada en juegos para el aprendizaje del PC
Los entornos que utilizan la estrategia basada en juegos (GBL) potencian el aprendizaje de habilidades de PC en Educación Primaria en contextos escolares, sobre en los niños más pequeños y si se aplica una componente colaborativa.

Solamente un 20% de las experiencias que incorporan GBL en la literatura se orientan a la etapa de Educación Infantil y primaria, pese a que esta estrategia suele arrojar resultados muy positivos en el aprendizaje en un 85% de los casos [11]. La hipótesis planteada es que la enseñanza de habilidades de PC y su evaluación es posible y fiable en estas etapas escolares utilizando GBL.

Esta hipótesis se comprueba en el Capítulo 6, mediante la validación a través experiencias en colegios de un videojuego educativo, desarrollado específicamente en el marco de la tesis, para la enseñanza de la programación y desarrollo del PC.

HE4 – sobre la relación entre las estrategias de evaluación y el marco 3D.
Es posible abarcar las tres dimensiones del marco 3D del PC mediante diferentes estrategias de evaluación tal y como se muestra en la Figura 11.

Una limitación de la evaluación por medio de test tradicional es que no abarca todas las dimensiones del PC del marco 3D. Según Grover y Pea [148], es necesario contar con sistema de evaluación, para evaluar un aprendizaje más profundo combinando diferentes medidas de datos [142], además del test tradicional.

Por otro lado, la evaluación del PC también puede ser dinámica y así reflejar las capacidades y la progresión de los alumnos a lo largo del tiempo. Este aspecto es evaluable mediante analíticas de aprendizaje en juegos o Game Learning Analytics (GLA). Para alcanzar el objetivo de analizar el PC en todas sus dimensiones, en esta tesis se plantea el uso de GLA como parte del sistema de evaluación propuesto. La hipótesis de impacto de las diferentes estrategias de evaluación según el marco 3D se muestra en la Figura 11, en la que las dimensiones computacionales se sitúan en los vértices del triángulo, de modo que cuanto más cerca se represente la estrategia de evaluación de un vértice, mayor será la probabilidad de que esta dimensión se vea afectada por la estrategia.

En los capítulos 6 y 7, mediante la combinación de diferentes estrategias de evaluación, se explorará esta hipótesis.

HE5 – sobre el análisis de las predisposiciones y disposiciones del PC con GLA.

Los entornos de aprendizaje basados en juegos voluntarios pueden aportar información sobre disposiciones y predisposiciones para el aprendizaje del PC según las características del individuo en cuanto a edad y género.

La estrategia GBL se utiliza con éxito en entornos escolares, formalmente controlados y dirigidos, en tiempo y forma. La hipótesis que se plantea es que, mediante entornos GBL voluntarios, fuera del contexto escolar, y utilizando GLA, es posible la identificación de disposiciones y predisposiciones [41], [53] para el aprendizaje del PC, como por ejemplo intereses, motivación, persistencia ante los retos o comportamiento ante las recompensas. Esta identificación tendrá especial relevancia en el marco de la tesis y se detallará en el Capítulo 7.

3.3. Estructura de la investigación empírica

A continuación, se enumeran las experiencias que se plantean tras la investigación teórica y que constituyen la base de la investigación empírica realizada. Estas experiencias se relacionan directamente con los objetivos generales de la tesis y con las hipótesis previamente descritas. En los capítulos siguientes, se detallarán los objetivos específicos de cada una de las experiencias.

E1 – Desarrollo y validación de prueba independiente para la evaluación del PC en Educación Primaria: Beginners Computational Thinking test (BCTt).

En base a la HE1, se plantea el desarrollo de un test de PC que abarque la etapa de Educación Primaria y su validación en cuanto a fiabilidad, realizando para ello un procedimiento de validación de contenido a través de un panel de expertos y, después, una experiencia de aplicación en alumnos de Educación Primaria para comprobar su fiabilidad.

Además, según los resultados, se plantean otras experiencias, a través de colaboraciones internacionales, para el desarrollo y validación de una adaptación del test para los niños más mayores: competence Computational Thinking test (cCTt), de forma que, al finalizar todos los desarrollos y validaciones, se disponga de instrumentos fiables de evaluación que cubran la etapa completa de Educación Primaria.

E2 – Estimación de la validez del BCTt en alumnos de Educación Infantil.

Una vez desarrollado el test y sus adaptaciones en la experiencia anterior, cubriendo toda la etapa de Educación Primaria, se plantea, nuevamente a través de colaboraciones internacionales, el estudio de los límites inferiores de aplicación en cuanto a la edad de los niños. Se plantea la validación del BCTt mediante una experiencia en alumnos de Educación Infantil para concretar este límite inferior.

E3 – Validación y análisis del entorno educativo basado en el juego BAC, en modos individual y colaborativo y análisis de habilidades de PC en Educación Primaria.

En base a las hipótesis HE2, HE3 y HE4, se plantea el desarrollo de un videojuego que se basa en GBL e incluye GLA, como herramienta para el aprendizaje y la evaluación del PC en alumnos de Educación Primaria, así como su validación a través de experiencias en distintos colegios.

E4 – Análisis de motivación intrínseca, intereses, persistencia y comportamiento frente a recompensas en el aprendizaje de la programación en etapas tempranas utilizando el entorno BAC.

En base a las hipótesis HG4 y HG5, se plantea el despliegue masivo de BAC para que los usuarios puedan jugar voluntariamente y, de esta forma, analizar con GLA la progresión del aprendizaje a lo largo del tiempo, así como predisposiciones y disposiciones del PC relacionadas con la motivación intrínseca, como intereses, persistencia ante los retos, y comportamiento ante las recompensas.

En la Tabla 10 se muestra la relación entre las experiencias, los objetivos generales y las hipótesis base para la investigación empírica. Los objetivos generales mostrados están incluidos en el objetivo principal OP: Avanzar en el conocimiento sobre el PC, sobre todo en su componente operacional, es decir, en cuanto a métodos de enseñanza y de evaluación eficaces, sobre todo a etapas tempranas. Las experiencias, también persiguen transversalmente el objetivo común OC: Diseño y validación de instrumentos de evaluación del PC que cubran un amplio rango de edades y poblaciones. En la tabla también se indica en qué capítulo se describe cada experiencia.

En este punto, el grado de cumplimiento de los objetivos es el que se muestra en la Tabla 11.

CHAPTER 3. EMPIRICAL RESEARCH: HYPOTHESES AND CASE STUDIES

Once the theoretical framework has been defined and the state of the art has been reviewed, the conclusions drawn from this theoretical research are set out below and, based on these, the operational direction established for the empirical research, as set out in the general aim OP2: To establish an operational proposal for the CT based on OP1, and thus constitute the direction of the empirical research by means of hypotheses and basic objectives. Subsequently, in each empirical case study, the specific hypotheses and specific aims will be detailed (see section 1.2. General aims).

This chapter thus sets out, in relation to the general aims of the thesis, the hypotheses, developments and case studies carried out in the empirical research. In the following chapters, these developments and case studies will be described in depth, as well as the specific hypotheses and aims related to each one of them.

3.4. Suggested definition of CT

Given that there is no agreed definition of CT, a review of the literature has been carried out and, based on the commonalities of the existing definitions (section 2.1), the following definition is proposed in this thesis:

Computational thinking is a cognitive ability that enables problem solving using computational strategies.

Similarly, in this thesis, the following decomposition of CT into 6 components is suggested: 1) abstraction (process of obtaining something simple from something complex, obviating details); 2) data analysis (searching, selecting, organising and logically analysing data); 3) problem decomposition (decomposing problems into smaller ones that can be solved more easily; 4) algorithms (identifying specific and explicit instructions that, step by step, implement a process); 5) error debugging (identifying and correcting mistakes in a given solution); and 6) generalisation (transferring a problem-solving strategy to a large variety of problems).

In addition, the identification of dispositions and predispositions for learning CT, as proposed in the literature [41], [53]., is also addressed. This identification will be of particular relevance in the framework of the thesis, as these motivational aspects and persistence in the face of challenges will be analysed in Chapter 7.

Regarding the educational frame of reference, one of the most cited in the literature and most empirically used CT frameworks has been selected, which is Brennan and Resnick's (2012) 3D framework [46]. This framework classifies CT along three dimensions (see Table 3): 1) computational concepts (concepts that programmers use); 2) computational practices (problem-solving practices that are needed/produced in the programming process); and 3) computational perspectives (perspectives that designers form about themselves and the world around them). This framework is suitable for considering CT in environments that use computer programming and in Primary Education [61], so it seems appropriate as a starting point in this thesis.

3.5. General hypotheses of the empirical investigation

The following is a description of the general hypotheses that emerged from the theoretical research and that form the basis for the empirical research carried out. The empirical hypotheses (HE) and the studies carried out in relation to each of them are as specified below.

HE1 – on the assessment of CT in Primary Education.

It is possible to assess CT in primary education using a traditional test independent of a specific environment.

As has been shown in the state of the art, although assessment of student progress is essential for the introduction of CT in the curriculum [31], there is a lack of consensus on which strategies are the most appropriate for this assessment and, moreover, there are few such instruments and they do not usually include data on their reliability and validity. However, there is a clear need for reliable and valid assessment of CT in all educational contexts so that researchers can determine whether their interventions are effective.

At the beginning of this research work in 2019, there was no traditional, setting-independent test-type instrument for assessing CT in Primary Education, so efforts in this research have been directed in this direction by developing the Beginners CT test: Beginners Computational Thinking test (BCTt), published in 2020 [8], and detailed in Chapter 4, for which the Computational Thinking Test (CTt) [44] has been a consolidated and firm basis.

The BCTt has had considerable international impact and, together with other research teams in countries such as Switzerland, Portugal and the Netherlands, efforts are being made to determine the age limits for the application of the test or to develop new versions to adapt it to students of different ages and populations. In this way, based on the reliability results of the BCTt for upper stages of Primary Education, and thanks to international collaboration, the competence Computational Thinking test (cCTt) has been developed and validated to cover, together with the BCTt and the CTt, all the age ranges corresponding to this educational stage.

On the other hand, the BCTt can also be used in the Infant Education stage, so Chapter 5 describes a study in which the aim is to obtain the lower age limit at which the test can be administered, validating its reliability in children of 4 and 5 years of age.

HE2 – on the relationship of learning strategies to the 3D frame.
Through different learning strategies, the three dimensions of CT can be covered, so that the relationship between the 3D framework and the learning strategies is as shown in Figure 10. Expected impact of the learning strategy on the key dimensions of the CT 3D framework.

From the state of the art, it has been noticed that most of the research and actions for the development of CT in the classroom, focus on the computational concepts dimension of Brennan and Resnick's CT 3D framework [46], [61], [93] and, only in some very recent research, on the learning and assessment of computational practices and computational perspectives dimensions [93]-[97]. This leads us to the need for more research and special attention to the latter two dimensions, especially out of the Scratch environment [98] in which most of the case studies in the research are conducted. Moreover, CT is an interdisciplinary set of skills and there is a need for operational methods of learning through a combination of various strategies [61], [96].

Furthermore, to equip students with problem-solving skills that they can later transfer to non-programming domains [41], it is necessary to develop CT from early stages and it is especially important to develop computational practices and perspectives. These last two dimensions of the 3D framework are the ones that allow us to work on the essential component of the CT which, in the decomposition proposed in this thesis, has been described as generalisation.

In the theoretical framework of this thesis, problem-based learning, collaborative learning, and game-based learning strategies were identified as the most commonly used learning strategies with the greatest impact on CT learning. In this thesis we are going to try to link the possible impact of each of these learning strategies on each dimension of the CT 3D framework. Figure 10 shows our hypothesis about this relationship: the computational dimensions are located at the vertices of the triangle, so

that the closer to a vertex the strategy appears, the higher the probability that this dimension will be affected by the strategy.

Unlike the problem-based learning strategy, in the game-based learning strategy the learner does not necessarily receive a prior definition of the concepts or content to be covered, but guides his or her own learning freely, motivated by a sense of achievement (see Table 4), so the hypothesis is that the acquisition of concepts may not be complete. On the contrary, in game-based learning, computational practices could be particularly enhanced, since mastering them is essential to progress in the challenges or levels proposed [61].

On the other hand, we hypothesise that computational practices and perspectives can be enhanced by using a collaborative strategy. This strategy is often used in studies involving programming experiences and can enhance both the practices and computational perspectives dimensions, since students review and think about their programming experience, thus enhancing their own learning process [61]. In a collaborative environment focused on learning programming, students not only reflect on how their program works, but also review and think about the programs their peers have created. Debate and discussion, aloud, of their own and others' programming solutions, can enhance both dimensions of CT, especially computational perspectives [15], [60], [61].

Subsequently, through the case study in schools described in Chapter 6, in which all three learning strategies are used simultaneously, this hypothesis will be tested.

HE3 – on the game-based strategy for learning CT
Environments using the Game-Based Learning approach (GBL) enhance the learning of CT skills in Primary Education in school settings, especially in younger children and if a collaborative component is applied.

Just 20% of the studies that incorporate GBL in the literature are aimed at the pre-school and primary school stages, even though this strategy tends to yield very positive learning results in 85% of cases [11]. The hypothesis posed is that the teaching of CT skills and their assessment is possible and reliable at these school stages using GBL

This hypothesis is confirmed in Chapter 6, by validating through studies in schools an educational video game, developed specifically within the framework of the thesis, for the teaching of programming and CT development.

HE4 – on the relationship between assessment strategies and the 3D framework
It is possible to cover the three dimensions of the CT 3D framework through different assessment strategies as shown in Figure 11. Expected impact of the assessment strategy on the key dimensions of the CT 3D framework.

A limitation of traditional test-based assessment is that it does not cover all dimensions of the CT of the 3D framework. According to Grover and Pea [148], a system of assessments is needed to assess deeper learning by combining different data measures [142], in addition to the traditional test.

Moreover, the assessment of CT can also be dynamic, reflecting learners' abilities and progression over time. This aspect can be evaluated by means of Game Learning Analytics (GLA). In order to achieve the aim of analysing CT in all its dimensions, this thesis proposes the use of GLA as part of the proposed system of assessments. The impact hypothesis of the different assessment strategies according to the 3D framework is shown in Figure 11, in which the computational dimensions are placed at the vertices of the triangle, so that the closer the assessment strategy is represented to a vertex, the higher the probability that this dimension will be affected by the strategy.

In chapters 6 and 7, by combining different assessment strategies, this hypothesis will be explored.

HE5 – sobre el análisis de las predisposiciones y disposiciones del PC con GLA
Los entornos de aprendizaje basados en juegos voluntarios pueden aportar información sobre disposiciones y predisposiciones para el aprendizaje del PC según las características del individuo en cuanto a edad y género.

The GBL strategy is successfully used in school settings, formally controlled and directed, in time and form. It is hypothesised that through voluntary GBL settings, outside the school context, and using GLA, it is possible to identify dispositions and predispositions [41], [53] for CT learning, such as interests, motivation, persistence in the face of challenges or behaviour towards rewards. This analysis will be of relevance in the framework of the thesis and will be detailed in Chapter 7.

3.6. Empirical research structure

Following the theoretical research, a series of case studies were carried out that are directly related to the general aims of the thesis and to the hypotheses previously described. In the following chapters, the specific aims of each of the case studies will be detailed.

E1 – Development and validation of a stand-alone test for the assessment of CT in Primary Education: Beginners Computational Thinking test (BCTt).

Based on HE1, we propose the development of a CT test that covers the Primary Education stage and its validation in terms of reliability, carrying out a content validation procedure through a panel of experts and, later, an administration on Primary Education students to check its reliability.

In addition, depending on the results, other case studies are planned, through international collaborations, for the development and validation of an adaptation of the test for older children: competence Computational Thinking test (cCTt), so that, when all the developments and validations are completed, reliable assessment instruments covering the entire Primary Education stage would be available.

E2 – Estimation of the validity of the BCTt in Infant Education students.

Once the test and its adaptations have been developed in the previous study, covering the entire Primary Education stage, research on the lower limits of application of the tests in terms of the age of the children is planned, once again through international collaborations. Validation of the BCTt is proposed by means of a case study with pre-school students in order to determine this lower limit.

E3 – Validation and analysis of the educational environment based on the BAC game, in individual and collaborative modes and analysis of CT skills in Primary Education.

Based on hypotheses HE2, HE3 and HE4, we propose the development of a video game based on GBL and including GLA, as a tool for learning and assessment of CT in Primary Education students, as well as its validation through case studies in different schools.

E4 – Analysis of intrinsic motivation, interests, persistence, and behavior in response to rewards, in learning programming in early stages using the BAC environment.

Based on hypotheses HG4 and HG5, we propose the massive deployment of BAC so that users can play voluntarily and, in this way, analyse with GLA the progression of learning over time, as well as predispositions and dispositions of the CT related to intrinsic motivation, such as interests, persistence in the face of challenges, and behavior in response to rewards.

Table 10 shows the relationship between the case studies, the general aims, and the base hypotheses for the empirical research. The general aims shown are included in the main aim OP: To advance knowledge about CT, especially in its operational component, i.e., in terms of effective teaching and assessment methods, especially at early stages. The studies also pursue transversally the common aim OC: Design and validation of CT assessment instruments covering a wide range of ages and populations. The table also indicates in which chapter each case study is described.

At this point, the degree of achievement of the aims is as shown in Table 11.

Bibliografía

  [1]V. J. Shute, C. Sun, and J. Asbell-Clarke, "Demystifying computational thinking," Educational Research Review, vol. 22, pp. 142-158, 2017. Available: https://www.sciencedirect.com/science/article/pii/S1747938X17300350, doi: 10.1016/j.edurev.2017.09.003.

  [2]M. Berland and U. Wilensky, "Comparing virtual and physical robotics environments for supporting complex systems and computational thinking," Journal of Science Education and Technology, vol. 24, no. 5, pp. 628-647, 2015.

  [3]J. M. Wing, "Computational thinking and thinking about computing," Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 366, no. 1881, pp. 3717-3725, 2008. Available: http://rsta.royalsocietypublishing.org/content/366/1881/3717.abstract, doi: 10.1098/rsta.2008.0118.

  [4]T. Hsu, S. Chang, and Y. Hung, "How to learn and how to teach computational thinking: Suggestions based on a review of the literature," Computers & Education, vol. 126, pp. 296-310, 2018. Available: http://www.sciencedirect.com/science/article/pii/S0360131518301799, doi: //doi.org/10.1016/j.compedu.2018.07.004.

  [5]S. Grover and R. Pea, "Computational Thinking in K—12: A Review of the State of the Field," Educational Researcher, vol. 42, no. 1, pp. 38-43, 2013. Available: https://www.jstor.org/stable/23360476, doi: 10.3102/0013189X12463051.

  [6]M. Zapata-Cáceres and N. Fanchamps, "Using the beginners computational thinking test to measure development on computational concepts among preschoolers," in Proceedings of the 5th APSCE International Computational Thinking and STEM in Education Conference 2021, Singapore, National Institute of Education, 2021, pp. 32-37.

  [7]Zapata-Cáceres María, E. Martín-Barroso and M. Román-González, "BCTt: Beginners computational thinking test," in Raspberry Pi Foundation Research Seminars, Apr 2021, pp. 46-56. ISSN 2514-586X (16). Available: https://www.raspberrypi.org/app/uploads/2021/05/Understanding-computing-education-Volume-1-%E2%80%93-Raspberry-Pi-Foundation-Research-Seminars.pdf.

  [8]M. Zapata-Cáceres, E. Martín-Barroso, and M. Román-González, “Computational thinking test for beginners: Design and content validation,” in IEEE Global Engineering Education Conference (EDUCON’20), Porto, Portugal, pp. 1905–1914, 2020, doi: 10.1109/EDUCON45650.2020.9125368.

  [9]J. Kirriemuir and A. Mcfarlane, Literature Review in Games and Learning. Bristol: Futurelab, 2004. Available: https://telearn.archives-ouvertes.fr/hal-00190453.

 [10]M. Prensky, "Digital game-based learning," Computers in Entertainment (CIE), vol. 1, no. 1, pp. 21, 2003.

 [11]M. Qian and K. R. Clark, "Game-based Learning and 21st century skills: A review of recent research," Computers in Human Behavior, vol. 63, pp. 50-58, 2016. Available: https://www.sciencedirect.com/science/article/pii/S0747563216303491, doi: 10.1016/j.chb.2016.05.023.

 [12]L. S. Vygotsky, Mind in Society: The Development of Higher Psychological Processes. Cambridge, MA, USA: Harvard University Press, 1978.

 [13]J. Piaget, Psychology and Epistemology: Towards a Theory of Knowledge. CA, USA: Allen Lane, 1972.

 [14]D. H. Jonassen and L. Rohrer-Murphy, “Activity theory as a framework for designing constructivist learning environments,” Educational Technology Research & Development, vol. 47, no. 1, pp. 61–79, 1999. Available: https://www.jstor.org/stable/30220230, doi: 10.1007/BF02299477.

 [15]D. H. Jonassen, Learning to Solve Problems with Technology: A Constructivist Perspective. Upper Saddle River, N.J.: Merrill Prentice Hall, 2003.

 [16]C. Alonso-Fernandez et al, "Systematizing game learning analytics for serious games," in 2017 IEEE global engineering education conference (EDUCON), Apr 2017, pp. 1111-1118.

 [17]M. Freire et al, "Game Learning Analytics: Learning Analytics for Serious Games," Springer Nature Switzerland AG, pp. 1-29, Dec 1, 2016.

 [18]M. Zapata-Caceres, E. Martin, and M. Roman-Gonzalez, "Collaborative GameBased Environment and Assessment Tool for Learning Computational Thinking in Primary School: A Case Study," in IEEE Transactions on Learning Technologies, vol. 14, no. 5, pp. 576-589, 1 Oct. 2021, doi: 10.1109/TLT.2021.3111108.

 [19]M. Zapata-Cáceres and E. Martín-Barroso, "Applying Game Learning Analytics to a Voluntary Video Game: Intrinsic Motivation, Persistence, and Rewards in Learning to Program at an Early Age," in IEEE Access, vol. 9, pp. 123588-123602, 2021, doi: 10.1109/ACCESS.2021.3110475.

 [20]M. Binkley et al, "Defining twenty-first century skills," in Assessment and Teaching of 21st Century Skill. Dordrecht: Springer Netherlands, 2012, pp. 17-66.

 [21]B. Trilling and C. Fadel, 21st Century Skills: Learning for life in our times. San Francisco: John Wiley & Sons, 2009.

 [22]H. Kim et al, "Enhancing teachers' ICT capacity for the 21st century learning environment: Three cases of teacher education in Korea," Australasian Journal of Educational Technology, vol. 28, no. 6, 2012.

 [23]K. Tang, T. Chou, and C. Tsai, "A Content Analysis of Computational Thinking Research: An International Publication Trends and Research Typology," The Asia-Pacific Education Researcher, pp. 1-11, 2019. Available: https://search.proquest.com/docview/2195733480, doi: 10.1007/s40299-019-00442-8.

 [24]S. Grover et al, "A Framework for Using Hypothesis-Driven Approaches to Support Data-Driven Learning Analytics in Measuring Computational Thinking in Block-Based Programming Environments," ACM Transactions on Computing Education (TOCE), vol. 17, no. 3, pp. 1-25, 2017. Available: http://dl.acm.org/citation.cfm?id=3105910, doi: 10.1145/3105910.

 [25]S. Grover and A. Korhonen, "Unlocking the Potential of Learning Analytics in Computing Education," ACM Transactions on Computing Education (TOCE), vol. 17, no. 3, pp. 1-4, 2017. Available: http://dl.acm.org/citation.cfm?id=3122773, doi: 10.1145/3122773.

 [26]S. Grover. “The 5th ‘C’ of 21st Century skills? Try computational thinking (not coding)”, edsurge.com. http://bit.ly/346qK78 (accessed Feb. 1, 2018). Available: http://scholar.aci.info/view/1544f0f08b300170003/161ce93d34000012ef80942.

 [27]J. A. Velazquez-Iturbide, "Towards an analysis of computational thinking," in Sep 2018, pp. 1-6.

 [28]J.A. Velázquez-Iturbide, “Towards an analysis of computational thinking,” in Int. Symp. on Computers in Education (SIIE’18), pp. 1-6, Sep. 2018.

 [29]M. Israel et al, "Supporting all learners in school-wide computational thinking: A cross-case qualitative analysis," Computers & Education, vol. 82, pp. 263-279, 2015. Available: https://www.sciencedirect.com/science/article/pii/S0360131514002759, doi: 10.1016/j.compedu.2014.11.022.

 [30]C. Angevine et al, "Computational Thinking for a Computational World," 2017. Available: www.digitalpromise.org.

 [31]National Research Council, Report of a Workshop on the Pedagogical Aspects of Computational Thinking. National Academies Press, 2011.

 [32]Q. Burke, W. I. O'Byrne and Y. B. Kafai, "Computational participation: Understanding coding as an extension of literacy instruction," Journal of Adolescent & Adult Literacy, vol. 59, no. 4, pp. 371-375, 2016.

 [33]J. F. Sanford and J. T. Naidu, "Computational thinking concepts for grade school," Contemporary Issues in Education Research (CIER), vol. 9, no. 1, pp. 23-32, 2016.

 [34]K. Mills et al, Computational Thinking for an Inclusive World: A Resource for Educators to Learn and Lead. Digital Promise, 2021. Available: https://doi.org/10.51388/20.500.12265/140.

 [35]C. Sneider et al, "Computational thinking in high school science classrooms," The Science Teacher, vol. 81, no. 5, pp. 53, 2014.

 [36]N. Kelly and J. S. Gero, "Design thinking and computational thinking: a dual process model for addressing design problems," Design Science, vol. 7, 2021.

 [37]T. C. Pargman et al, "Teaching Computational Thinking in K-9: Tensions at the Intersection of Technology and Pedagogical Knowledge," in 14th International Conference of the Learning Sciences (ICLS) 2020, vol. 4, pp. 2409-2410, International Society of the Learning Sciences, 2020.

 [38]D. Kohen-Vacs and M. Milrad, "Computational thinking education for in-service elementary swedish teachers: Their perceptions and implications for competence development," in International Conference on Computational Thinking Education, 13-15 June 2019, Hong Kong, 2019.

 [39]V. Allan et al, "Computational thinking in high school courses," in Proceedings of the 41st ACM Technical Symposium on Computer Science Education, 2010.

 [40]P. Sengupta et al, "Integrating computational thinking with K-12 science education using agent-based computation: A theoretical framework," Education and Information Technologies, 2013, vol. 18, no 2, p. 351-380. Available: https://search.proquest.com/docview/1323979287, doi: 10.1007/s10639-012-9240-x.

 [41]V. Barr and C. Stephenson, "Bringing computational thinking to K-12: What is Involved and what is the role of the computer science education community?" ACM Inroads, vol. 2, no. 1, pp. 48-54, 2011.

 [42]P. Mishra, A. Yadav, and Deep-Play Research Group, "Rethinking technology & creativity in the 21st century," TechTrends, vol. 57, no. 3, pp. 10-14, 2013.

 [43]J. M. Wing, "Computational Thinking Test," CACM Viewpoint, pp. 33-35, 2006. Available: http://www.cs.cmu.edu/~wing/.

 [44]M. Román-González, J. Pérez-González and C. Jiménez-Fernández, "Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test," Computers in Human Behavior, vol. 72, pp. 678-691, 2017. Available: https://www.sciencedirect.com/science/article/pii/S0747563216306185, doi: 10.1016/j.chb.2016.08.047.

 [45]A. V. Aho, "Computation and computational thinking," The Computer Journal, vol. 55, no. 7, pp. 832-835, 2012.

 [46]K. Brennan, M. Resnick, and MIT Media Lab, "New frameworks for studying and assessing the development of computational thinking," American Educational Research Association Meeting, Vancouver, BC, Canada., 2012.

 [47]S. Papert, Mindstorms: Children, Computers, and Powerful Ideas. Basic Books, Inc., 1980.

 [48]J. J. Lu and G. H. Fletcher, "Thinking about computational thinking," in Proceedings of the 40th ACM Technical Symposium on Computer Science Education, 2009.

 [49]J. Wing, "Research notebook: Computational thinking—What and why," The Link Magazine, vol. 6, pp. 20-23, 2011.

 [50]S. Furber, "Shut down or restart? The way forward for computing in UK schools," The Royal Society, 2012. Available: https://royalsociety.org/-/media/education/computing-in-schools/2012-01-12-computing-in-schools.pdf.

 [51]M. Tedre and P. J. Denning, "The long quest for computational thinking," in Proceedings of the 16th Koli Calling International Conference on Computing Education Research, pp. 120-129, 2016.

 [52]J. Moreno et al, " No es lo mismo: un análisis de red de texto sobre definiciones de pensamiento computacional para estudiar su relación con la programación informática," Revista Interuniversitaria De Investigación En Tecnología Educativa, 2019, doi: 10.6018/riite.397151.

 [53]ISTE and CSTA, "Operational definition of computational thinking for K-12 education," 2011. Available: https://cdn.iste.org/www-root/Computational_Thinking_Operational_Definition_ISTE.pdf.

 [54]M. U. Bers et al, "Computational thinking and tinkering: Exploration of an early childhood robotics curriculum," Computers & Education, vol. 72, pp. 145-157, 2014. Available: https://search.datacite.org/works/10.1016/j.compedu.2013.10.020, doi: 10.1016/j.compedu.2013.10.020.

 [55]S. Atmatzidou and S. Demetriadis, "Advancing students’ computational thinking skills through educational robotics: A study on age and gender relevant differences," Robotics and Autonomous Systems, vol. 75, pp. 661-670, 2016. Available: https://www.sciencedirect.com/science/article/pii/S0921889015002420, doi: 10.1016/j.robot.2015.10.008.

 [56]N. D. Anderson, "A call for computational thinking in undergraduate psychology," Psychology Learning & Teaching, vol. 15, no. 3, pp. 226-234, 2016.

 [57]S. Rose, J. Habgood and T. Jay, " An exploration of the role of visual programming tools in the development of young children’s computational thinking," Electronic journal of e-learning, 2017, vol. 15, no 4, p. 297-309, Aug 1, 2017.

 [58]D. Weintrop and U. Wilensky, "Using commutative assessments to compare conceptual understanding in blocks-based and text-based programs," in ICER, pp. 101-110, Aug 9, 2015.

 [59]D. Statter and M. Armoni, "Teaching abstract thinking in introduction to computer science for 7th graders," in Proceedings of the 11th Workshop in Primary and Secondary Computing Education, pp. 80-83, 2016.

 [60]B. Zhong et al, " An Exploration of Three-Dimensional Integrated Assessment for Computational Thinking," Journal of Educational Computing Research, vol. 53, no. 4, pp. 562-590, 2016. Available: https://journals.sagepub.com/doi/full/10.1177/0735633115608444, doi: 10.1177/0735633115608444.

 [61]S. Y. Lye and J. H. L. Koh, "Review on teaching and learning of computational thinking through programming: What is next for K-12?" Computers in Human Behavior, vol. 41, pp. 51-61, 2014. Available: https://www.sciencedirect.com/science/article/pii/S0747563214004634, doi: 10.1016/j.chb.2014.09.012.

 [62]E. Hunsaker, Computational Thinking. The K–12 Educational Technology Handbook, A. Ottenbreit-Leftwich and R. Kimmons, Eds. EdTech Books, 2020, pp. 80–95. [Online]. Available: https://bit.ly/2UP0gZi.

 [63]P. J. Rich et al, "Coding in K-8: International Trends in Teaching Elementary/Primary Computing," TechTrends, pp. 1-19, 2018.

 [64]C. S. González-González, "State of the art in the teaching of computational thinking and programming in childhood education," Education in the Knowledge Society, vol. 20, pp. 1-15, 2019.

 [65]S. Bocconi, A. Chioccariello, G. Dettori, A. Ferrari, and K. Engelhardt, “Developing computational thinking in compulsory education,” Eur. Commission, JRC Sci. Policy Rep., pp. 68, 2016.

 [66]Unión Europea, “Plan de Acción de Educación Digital (2021-2027)”, Unión Europa, European Comission, European Education Area, 2021.[Online]. Available: https://ec.europa.eu/jrc/en/digcomp/digital-competence-framework. [Accessed: Dic. 12, 2021].

 [67]Unión Europea, “DigComp. The Digital Competence Framework 2.0”, EU SCIENCE HUB, 2021. [Online]. Available: https://ec.europa.eu/jrc/en/digcomp/digital-competence-framework. [Accessed: Dic. 12, 2021].

 [68]Code.org, “Learn computer science. Change the world”, Code.org. [Online]. Available: https://code.org/. [Accessed: Jan. 20, 2022].

 [69]La hora del código, “Celebramos las Ciencias de la Computación en todas partes”, hourofcode.com. [Online]. Available: https://hourofcode.com/es. [Accessed: Jan. 20, 2022].

 [70]J. Á Velázquez-Iturbide, "Report of the Spanish computing scientific society on computing education in pre-university stages," in Proceedings of the Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality, 2018.

 [71]Instituto Nacional de Tecnologías Educativas y de Formación del Profesorado (INTEF), "Programación, robótica y Pensamiento Computacional en el aula. Situación en España y propuesta normativa", Ministerio de Educación y Formación Profesional, Gobierno de España, Oct, 2018. [Online]. Available: https://code.intef.es/wp-content/uploads/2018/10/Ponencia-sobre-Pensamiento-Computacional.-Informe-Final.pdf.

 [72]Junta de Castilla y León, “Proyecto de innovación 'TIC STEAM' - Curso 2016-2017”, educacyl Portal de Educación, 2017. [Online]. Available: https://www.educa.jcyl.es/profesorado/es/formacion-profesorado/proyectos-relacionados-formacion-permanente-profesorado/competencia-digital/convocatorias/proyecto-innovacion-tic-steam-curso-2016-2017.

 [73]Instituto Nacional de Tecnologías Educativas y de Formación del Profesorado (INTEF), “Escuela de Pensamiento Computacional e Inteligencia Artificial”, Ministerio de Educación y Formación Profesional, Gobierno de España, 2021. [Online]. Available: https://intef.es/tecnologia-educativa/pensamiento-computacional/.

 [74]Programamos, “Es necesario que todo el mundo desarrolle su Pensamiento Computacional, desde niños a mayores”, programamos.com. [Online]. Available: https://programamos.es/. [Accessed: Jan. 20, 2022].

 [75]Boletín Oficial del Estado, “Marco de referencia de la competencia digital docente”, Ministerio de la Presidencia, Relaciones con las Cortes y Memoria Democrática, Gobierno de España. [Online]. Available: https://www.boe.es/diario_boe/txt.php?id=BOE-A-2020-7775#top.

 [76]E. Hunsaker, Integrating computational thinking. K-12 technology integration. Pressbooks. Available: https://k12techintegration.pressbooks.com/chapter/integrating-computational-thinking, 2018.

 [77]M. Zapata-Ros, "Computational thinking unplugged," Education in the Knowledge Society, vol. 20, pp. 1-29, 2019.

 [78]T. Bell et al, "Computer science unplugged: School students doing real computing without computers," The New Zealand Journal of Applied Computing and Information Technology, vol. 13, no. 1, pp. 20-29, 2009.

 [79]S. Adak, "Effectiveness of Constructivist Approach on Academic Achievement in Science at Secondary Level." Educational Research and Reviews, vol. 12, no. 22, pp. 1074-1079, 2017.

 [80]V. Richardson, "Constructivist pedagogy," Teachers College Record, vol. 105, no. 9, pp. 1623-1640, 2003.

 [81]C. T. Fosnot and R. S. Perry, "Constructivism: A psychological theory of learning," Constructivism: Theory, Perspectives, and Practice, vol. 2, no. 1, pp. 8-33, 1996.

 [82]S. Papert and I. Harel, "Situating constructionism," Constructionism, vol. 36, no. 2, pp. 1-11, 1991.

 [83]C. W. Istikomah and C. Budiyanto, "The contribution of educational robotics and constructivist approach to computational thinking in the 21st century," in Proceedings of the the 1st International Conference on Computer Science and Engineering Technology Universitas Muria Kudus, 2018.

 [84]S. Hartikainen et al, "The concept of active learning and the measurement of learning outcomes: A review of research in engineering higher education," Education Sciences, vol. 9, no. 4, pp. 276, 2019.

 [85]R. T. Johnson and D. W. Johnson, "Active learning: Cooperation in the classroom," The Annual Report of Educational Psychology in Japan, vol. 47, pp. 29-30, 2008.

 [86]P. Dillenbourg, Ed., Collaborative Learning: Cognitive and Computational Approaches. Oxford, U.K.: Pergamon, 1999.

 [87]M. Laal and S. M. Ghodsi, "Benefits of collaborative learning," Procedia - Social and Behavioral Sciences, vol. 31, pp. 486-490, 2012. Available: http://www.sciencedirect.com/science/article/pii/S1877042811030205, doi: //doi.org/10.1016/j.sbspro.2011.12.091.

 [88]D. W. Johnson and R. T. Johnson, "Effects of cooperative and individualistic learning experiences on interethnic interaction," Journal of Educational Psychology, vol. 73, (3), pp. 444-449, 1981. Available: https://search.proquest.com/docview/614371667. DOI: 10.1037/00220663.73.3.444.

 [89]M. Laal and M. Laal, "Collaborative learning: what is it?" Procedia - Social and Behavioral Sciences, vol. 31, pp. 491-495, 2012. Available: http://www.sciencedirect.com/science/article/pii/S1877042811030217. DOI: //doi.org/10.1016/j.sbspro.2011.12.092.

 [90]D. Boud and G. E. Feletti, Eds., The Challenge of Problem-Based Learning. London, U.K.: Routledge, 1997, doi: 10.4324/9781315042039.

 [91]J. Roschelle and S. D. Teasley, "The construction of shared knowledge in collaborative problem solving," in Computer Supported Collaborative Learning, 1995.

 [92]P. C. Blumenfeld et al, "Motivating Project-Based Learning: Sustaining the Doing, Supporting the Learning," Educational Psychologist, vol. 26, no. 3-4, pp. 369-398, 1991. Available: http://www.tandfonline.com/doi/abs/10.1080/00461520.1991.9653139, doi: 10.1080/00461520.1991.9653139.

 [93]L. Zhang and J. Nouri, "A systematic review of learning computational thinking through Scratch in K-9," Comput. Educ., vol. 141, pp. 103-607, 2019.

 [94]J. Moreno-León, G. Robles, and M. Román-González, "Towards data-driven learning paths to develop computational thinking with scratch," IEEE Transactions on Emerging Topics in Computing, vol. 8, no. 1, pp. 193-205, 2017.

 [95]J. Nouri et al, "Development of computational thinking, digital competence and 21st century skills when learning programming in K-9," Education Inquiry, vol. 11, no. 1, pp. 1-17, 2020.

 [96]J. Fagerlund et al, "Computational thinking in programming with scratch in primary schools: A systematic review," Computer Applications in Engineering Education, vol. 29, no. 1, pp. 12-28, 2021.

 [97]G. Falloon, "An analysis of young students' thinking when completing basic coding tasks using Scratch Jnr. On the iPad," J. Comput. Assisted Learn., vol. 32, no. 6, pp. 576-593, 2016.

 [98]Massachusetts Institute of Technology: MIT, “Scratch”, scratch.mit.edu, [Online]. Available: https://scratch.mit.edu/.

 [99]P. Mozelius and L. Öberg, "Play‐based learning for programming education in primary school: The östersund model," in European Conference on E-Learning-ECEL 2017, Porto, Portugal, 26-27 October 2017, 2017.

[100]A. G. Soosai Raj et al, "Does native language play a role in learning a programming language?" in Proceedings of the 49th ACM Technical Symposium on Computer Science Education, 2018.

[101]M. Jancheski, "Improving teaching and learning computer programming in schools through educational software," Olympiads in Informatics, vol. 11, pp. 55-75, 2017.

[102]E. F. Silva, B. J. Dembogurski and G. S. Semaan, "A Systematic Review of Computational Thinking in Early Ages," arXiv Preprint arXiv:2106.10275, 2021.

[103]J. M. Cañas Plaza et al, "Entorno docente universitario para la programación de los robots," Revista Iberoamericana De Automática E Informática Industrial, vol. 15, no. 4, pp. 404-415, 2018.

[104]A. Hamidi, M. Milrad and S. Tavajoh, "Combining maker technologies to promote computational thinking and heart-ware skills through project-based activities: Design considerations and empirical outputs," in Cte-Stem 2021, 2021.

[105]R. Zerega et al, "A co-design approach for developing computational thinking skills in connection to STEM related curriculum in swedish schools," in Cte-Stem 2021, 2021.

[106]M. Riedmiller and T. Gabel, "On experiences in a complex and competitive gaming domain: Reinforcement learning meets robocup," in 2007 IEEE Symposium on Computational Intelligence and Games, 2007.

[107]N. L. Fanchamps et al, "The influence of SRA programming on algorithmic thinking and self-efficacy using Lego robotics in two types of instruction," International Journal of Technology and Design Education, vol. 31, no. 2, pp. 203-222, 2021.

[108]N. L. Fanchamps et al, "The Impact of SRA-Programming on Computational Thinking in a Visual Oriented Programming Environment," Education and Information Technologies, vol. 26, no. 5, pp. 6479-6498, 2021.

[109]M. Montessori, J. M. Hunt and J. Valsiner, The Montessori Method. Routledge, 2017.

[110]M. Elkin, A. Sullivan, and M. U. Bers, "Programming with the KIBO robotics kit in preschool classrooms," Computers in the Schools, vol. 33, no. 3, pp. 169-186, 2016.

[111]L. L. Román, "Metodología para la enseñanza aprendizaje de la lógica de la programación orientada a objetos," in 5ta. Conferencia Iberoamericana en Sistemas, Cibernética e Informática CISCI, International Institute of Informatics and Systemics, EE. UU, 2007.

[112]J. Urquiza-Fuentes and J. Á Velázquez-Iturbide, "A survey of successful evaluations of program visualization and algorithm animation systems," ACM Transactions on Computing Education (TOCE), vol. 9, no. 2, pp. 1-21, 2009.

[113]L. Seiter and B. Foreman, "Modeling the learning progressions of computational thinking of primary grade students," in Proceedings of the Ninth Annual International ACM Conference on International Computing Education Research, 2013, Available: http://doi.acm.org/10.1145/2493394.2493403, doi: 10.1145/2493394.2493403.

[114]S. Grover, R. Pea and S. Cooper, "Designing for deeper learning in a blended computer science course for middle school students," Computer Science Education, vol. 25, no. 2, pp. 199-237, 2015.

[115]A. Basawapatna et al, "Recognizing computational thinking patterns," in Proceedings of the 42nd ACM technical symposium on Computer science education, pp. 245-250 Mar 9, 2011.

[116]S. Papadakis and M. Kalogiannakis, "Evaluating a course for teaching advanced programming concepts with scratch to preservice kindergarten teachers: A case study in greece," in Early Childhood EducationAnonymous 2018.

[117]A. Eguiluz et al, "Exploring the progression of early programmers in a set of computational thinking challenges via clickstream analysis," Tetc, pp. 1, 2017. Available: https://ieeexplore.ieee.org/document/8093668, doi: 10.1109/TETC.2017.2768550.

[118]G. Petri and C. Gresse von Wangenheim, "How games for computing education are evaluated? A systematic literature review," Computers & Education, vol. 107, pp. 68-90, 2017. Available: http://dx.doi.org/10.1016/j.compedu.2017.01.004, doi: 10.1016/j.compedu.2017.01.004.

[119]A. Vahldick, A. J. Mendes and M. J. Marcelino, "Dynamic Difficulty Adjustment through a Learning Analytics Model in a Casual Serious Game for Computer Programming Learning," EAI Endorsed Transactions on Serious Games, vol. 4, no. 13, 2017.

[120]C. Kroustalli and S. Xinogalos, "Studying the effects of teaching programming to lower secondary school students with a serious game: a case study with Python and CodeCombat," Education and Information Technologies, pp. 1-27, 2021.

[121]M. Csikszentmihalyi, Finding Flow: The Psychology of Engagement with Everyday Life. United States: 1997.

[122]S. H. Chuah, "Why and who will adopt extended reality technology? Literature review, synthesis, and future research agenda," Literature Review, Synthesis, and Future Research Agenda, Dec. 13, 2018.

[123]M. Montola, "Exploring the edge of the magic circle: Defining pervasive games," in Proceedings of DAC, 2005.

[124]C. Bonillo, J. Marco, and E. Cerezo, "Developing pervasive games in interactive spaces: the JUGUEMOS toolkit," Multimedia Tools Appl, vol. 78, no. 22, pp. 32261-32305, 2019.

[125]N. M. Medina et al, "Hacia una propuesta de evaluación heurística de experiencias de juego pervasivas," Revista De La Asociación Interacción Persona Ordenador (AIPO), vol. 2, no. 2, pp. 42-53, 2021.

[126]J. Marco, C. Bonillo, and E. Cerezo, "A tangible interactive space odyssey to support children learning of computer programming," in Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces, 2017.

[127]S. SUKIRMAN et al, "A Strategy of Learning Computational Thinking through Game Based in Virtual Reality: Systematic Review and Conceptual Framework," Informatics in Education, 2021.

[128]Y. Ching, Y. Hsu and S. Baldwin, "Developing computational thinking with educational technologies for young learners," TechTrends, vol. 62, no. 6, pp. 563-573, 2018.

[129]C. P. Brackmann et al, "Development of computational thinking skills through unplugged activities in primary school," in Proceedings of the 12th Workshop on Primary and Secondary Computing Education, 2017.

[130]S. Suh, M. Lee and G. Xia, "Coding strip: A pedagogical tool for teaching and learning programming concepts through comics," in 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), 2020.

[131]G. Aranda and J. P. Ferguson, "Unplugged Programming: The future of teaching computational thinking?" Pedagogika, vol. 68, no. 3, 2018.

[132]J. Waite et al, "Unplugged computing and semantic waves: Analysing crazy characters," in Proceedings of the 1st UK & Ireland Computing Education Research Conference, 2019.

[133]A. Dengel, "Computer science replugged: What is the use of virtual reality in computer science education?" in Proceedings of the 14th Workshop in Primary and Secondary Computing Education, 2019.

[134]X. Tang et al, "Assessing computational thinking: A systematic review of empirical studies," Computers & Education, vol. 148, pp. 103-798, 2020. Available: http://www.sciencedirect.com/science/article/pii/S0360131519303483, doi: https://doi.org/10.1016/j.compedu.2019.103798.

[135]J. Denner, L. Werner and E. Ortiz, "Computer games created by middle school girls: Can they be used to measure understanding of computer science concepts?" Computers & Education, vol. 58, no. 1, pp. 240-249, 2012. Available: https://www.sciencedirect.com/science/article/pii/S0360131511001849, doi: 10.1016/j.compedu.2011.08.006.

[136]I. Cetin and E. Dubinsky, "Reflective abstraction in computational thinking," Journal of Mathematical Behavior, vol. 47, pp. 70-80, 2017. Available: https://www.sciencedirect.com/science/article/pii/S0732312316301511, doi: 10.1016/j.jmathb.2017.06.004.

[137]S. Jun et al, "Assessing the computational literacy of elementary students on a national level in Korea," Educational Assessment, Evaluation and Accountability, vol. 26, no. 4, pp. 319-332, 2014.

[138]L. Werner et al, "The fairy performance assessment," in Proceedings of the 43rd ACM technical symposium on Computer Science Education, pp. 215-220, 2012.

[139]V. Garneli and K. Chorianopoulos, "Programming video games and simulations in science education: exploring computational thinking through code analysis," Interactive Learning Environments, vol. 26, no. 3, pp. 386-401, 2018. Available: http://www.tandfonline.com/doi/abs/10.1080/10494820.2017.1337036, doi: 10.1080/10494820.2017.1337036.

[140]A. Mühling, A. Ruf and P. Hubwieser, "Design and first results of a psychometric test for measuring basic programming abilities," in Proceedings of the workshop in primary and secondary computing education, pp. 2-10, Nov 9, 2015.

[141]Ö Korkmaz, R. Çakir and M. Y. Özden, "A validity and reliability study of the computational thinking scales (CTS)," Computers in Human Behavior, vol. 72, pp. 558-569, 2017. Available: https://www.sciencedirect.com/science/article/pii/S0747563217300055, doi: 10.1016/j.chb.2017.01.005.

[142]M. Román-González et al, "Extending the nomological network of computational thinking with non-cognitive factors," Computers in Human Behavior, vol. 80, pp. 441-459, 2018. Available: http://www.sciencedirect.com/science/article/pii/S0747563217305563, doi: https://doi.org/10.1016/j.chb.2017.09.030.

[143]M. Román-González et al, "Can computational talent be detected? Predictive validity of the Computational Thinking Test," International Journal of ChildComputer Interaction, vol. 18, pp. 47-58, 2018. Available: http://www.sciencedirect.com/science/article/pii/S221286891730034X, doi: https://doi.org/10.1016/j.ijcci.2018.06.004.

[144]AERA, APA and NCME, Standards for Educational and Psychological Testing. 2014. Available: http://www.fachportal-paedagogik.de/fis_bildung/suche/fis_set.html?FId=1050398.

[145]G. Chen et al, "Assessing elementary students’ computational thinking in everyday reasoning and robotics programming," Computers & Education, vol. 109, pp. 162-175, 2017. Available: http://www.sciencedirect.com/science/article/pii/S0360131517300490, doi: https://doi.org/10.1016/j.compedu.2017.03.001.

[146]E. Relkin, L. de Ruiter and M. U. Bers, "TechCheck: Development and Validation of an Unplugged Assessment of Computational Thinking in Early Childhood Education," Journal of Science Education and Technology, vol. 29, no. 4, pp. 482-498, 2020. Available: https://search.proquest.com/docview/2413230852, doi: 10.1007/s10956-020-09831-x.

[147]Y. Li, S. Xu, and J. Liu, "Development and Validation of Computational Thinking Assessment of Chinese Elementary School Students," Journal of Pacific Rim Psychology, vol. 15, pp. 18344909211010240, 2021. Available: https://doi.org/10.1177/18344909211010240, doi: https://doi.org/10.1177/18344909211010240.

[148]S. Grover, R. Pea and S. Cooper, "«Systems of assessments» for deeper learning of computational thinking in K-12," in Proceedings of the 2015 Annual Meeting of the American Educational Research Association, 2015.

[149]G. Siemens and P. Long, "Penetrating the Fog: Analytics in Learning and Education," EDUCAUSE Review, vol. 46, no. 5, pp. 30, 2011.

[150]A. Mathrani, S. Christian and A. Ponder-Sutton, "PlayIT: Game based learning approach for teaching programming concepts," Journal of Educational Technology & Society, vol. 19, no. 2, pp. 5-17, 2016.

[151]C. Malliarakis, M. Satratzemi and S. Xinogalos, "CMX: The effects of an educational MMORPG on learning and teaching computer programming," IEEE Transactions on Learning Technologies, vol. 10, no. 2, pp. 219-235, 2016.

[152]C. Malliarakis, M. Satratzemi and S. Xinogalos, "Integrating learning analytics in an educational MMORPG for computer programming," in 2014 IEEE 14th International Conference on Advanced Learning Technologies, 2014.

[153]W. Greller and H. Drachsler, "Translating learning into numbers: A generic framework for learning analytics," Journal of Educational Technology & Society, vol. 15, no. 3, pp. 42-57, 2012.

[154]M. A. Chatti et al, "Learning Analytics: Challenges and Future Research Directions," eleed, vol. 10, no 1. Nov 6, 2014.

[155]M. Román González, "Computational thinking test: Design guidelines and content validation," in Proceedings of EDULEARN15 conference, Barcelona, pp. 2436-2444, 201.

[156]D. Căprioară, "Emotions and the Learning of School Mathematics," Bulletin of the Transilvania University of Braşov, Series VII: Social Sciences and Law, vol. 10, no. 1, pp. 9-18, 2017. Available: http://www.ceeol.com/search/article-detail?id=566146.

[157]C. Chen and P. Herbst, "The interplay among gestures, discourse, and diagrams in students’ geometrical reasoning," Educational Studies in Mathematics, vol. 83, no. 2, pp. 285-307, 2013. Available: 10.1007/s10649-012-9454-2, doi: https://doi.org/10.1007/s10649-012-9454-2.

[158]H. Y. Durak and M. Saritepeci, "Analysis of the relation between computational thinking skills and various variables with the structural equation model," Computers & Education, vol. 116, pp. 191-202, 2018. Available: http://www.sciencedirect.com/science/article/pii/S0360131517302087, doi: //doi.org/10.1016/j.compedu.2017.09.004.

[159]T. Watanabe, "Visual Reasoning Tools in Action: Double Number Lines, Area Models, and Other Diagrams Power Up Students' Ability to Solve and Make Sense of Various Problems," Mathematics Teaching in the Middle School, vol. 21, no. 3, pp. 152, 2015.

[160]C. E. Lance, M. M. Butts and L. C. Michels, "The sources of four commonly reported cutoff criteria: What did they really say?" Organ. Res. Methods, vol. 9, no. 2, pp. 202-220, 2006.

[161]M. Román-González, J. Moreno-León and G. Robles, "Combining assessment tools for a comprehensive evaluation of computational thinking interventions," in Computational Thinking Education, S. Kong and H. Abelson, Eds. 2019, Available: https://doi.org/10.1007/978-981-13-6528-7_6, doi: 10.1007/978-981-13-6528-7_6.

[162]A. A. De Souza et al, "Data mining framework to analyze the evolution of computational thinking skills in game building workshops," IEEE Access, vol. 7, pp. 82848-82866, 2019.

[163]R. F. DeVellis, "Classical test theory," Med. Care, pp. S50-S59, 2006.

[164]R. K. Hambleton and R. W. Jones, "Comparison of classical test theory and item response theory and their applications to test development," Educational Measurement: Issues and Practice, vol. 12, no. 3, pp. 38-47, 1993.

[165]L. El-Hamamsy, M. Zapata-Caceres, E. Martín, F. Mondada, J. Dehler Zufferey, and B. Bruno, "The competent Computational Thinking test (cCTt): Development and validation of an unplugged Computational Thinking test for upper primary school," Journal of Educational Computing Research, 2022. In press.

[166]S. Kong, "A framework of curriculum design for computational thinking development in K-12 education," Journal of Computers in Education, vol. 3, no. 4, pp. 377-394, 2016.

[167]F. Buitrago Flórez et al, "Changing a Generation's Way of Thinking: Teaching Computational Thinking Through Programming," Review of Educational Research, vol. 87, no. 4, pp. 834, 2017. Available: https://search.proquest.com/docview/1917804191, doi: 10.3102/0034654317710096.

[168]J. Alvarez et al, "Blue Bot Project Experiment," 8th International Toy Research Association World Conference Jul 11, 2018.

[169]C. Sáez Fernández, G. Viera López and D. Pérez Marín, "Propuesta metodológica de la enseñanza de la programación en Educación Infantil con Cubetto," IE Comunicaciones: Revista Iberoamericana De Informática Educativa, no. 28, pp. 1-8, 2018. Available: https://dialnet.unirioja.es/servlet/oaiart?codigo=6750332.

[170]W. Wu et al, "Investigating the learning‐theory foundations of game‐based learning: a meta‐analysis," Journal of Computer Assisted Learning, vol. 28, no. 3, pp. 265-279, 2012. Available: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1365-2729.2011.00437.x, doi: 10.1111/j.1365-2729.2011.00437.x.

[171]E. Boyle, T. M. Connolly, and T. Hainey, "The role of psychology in understanding the impact of computer games," Entertainment Computing, vol. 2, no. 2, pp. 69-74, 2011. Available: https://www.sciencedirect.com/science/article/pii/S1875952110000200, doi: 10.1016/j.entcom.2010.12.002.

[172]M. Li and C. Tsai, "Game-Based Learning in Science Education: A Review of Relevant Research," Journal of Science Education and Technology, vol. 22, no. 6, pp. 877-898, 2013. Available: https://doi.org/10.1007/s10956-013-9436-x, doi: 10.1007/s10956-013-9436-x.

[173]W. Wu et al, "Re-exploring game-assisted learning research: The perspective of learning theoretical bases," Computers & Education, vol. 59, no. 4, pp. 1153-1161, 2012. Available: https://www.sciencedirect.com/science/article/pii/S0360131512001248, doi: 10.1016/j.compedu.2012.05.003.

[174]M. Csikszentmihalyi, "Flow and the psychology of discovery and invention," HarperPerennial, New York, vol. 39, 1997.

[175]T. Hainey et al, "A systematic literature review of games-based learning empirical evidence in primary education," Computers & Education, vol. 102, pp. 202-223, 2016.

[176]J. R. Savery and T. M. Duffy, "Problem based learning: An instructional model and its constructivist framework," Educational Technology, vol. 35, no. 5, pp. 31-38, 1995.

[177]J. R. Savery, "Overview of Problem-based Learning: Definitions and distinctions," Essential readings in problem-based learning: Exploring and extending the legacy of Howard S. Barrows, vol. 9, no 2, pp. 5-15, 2015. Available: https://library.biblioboard.com/viewer/620aa903-49eb-49d1-8fc2-b02dbeda193c.

[178]B. Haßler, L. Major, and S. Hennessy, "Tablet use in schools: A critical review of the evidence for learning outcomes," Journal of Computer Assisted Learning, vol. 32, no. 2, pp. 139-156, 2016.

[179]Y. Rogers et al, " Equal opportunities: Do shareable interfaces promote more group participation than single user displays?" Human–Computer Interaction, vol. 24, no. 1-2, pp. 79-116, 2009.

[180]D. Roldan-Alvarez et al, "DEDOS: An Authoring Toolkit to Create Educational Multimedia Activities for Multiple Devices," IEEE Transactions on Learning Technologies, vol. 11, no. 4, pp. 493-505, 2018. Available: https://ieeexplore.ieee.org/document/8244297, doi: 10.1109/TLT.2017.2788867.

[181]C. Lin, L. Wong and Y. Shao, "Comparison of 1: 1 and 1: m CSCL environment for collaborative concept mapping," Journal of Computer Assisted Learning, vol. 28, no. 2, pp. 99-113, 2012.

[182]M. Zapata-Cáceres, E. Martín-Barroso and M. Román-González, "Computational Thinking Test for Beginners: Design and Content Validation," 2020 IEEE Global Engineering Education Conference (EDUCON). in Press, 2020.

[183]R. S. Pressman, Software Engineering: A Practitioner's Approach. Palgrave macmillan, 2005.

[184]C. Larman, Agile and Iterative Development: A Manager's Guide. Addison-Wesley Professional, 2004.

[185]L. García-de-la-Fuente, "Tratamiento de ficheros generados por Blue Ant Code," Trabajo De Fin De Grado En Ingeniería Informática, Universidad Rey Juan Carlos, 2020.

[186]G. Matilla-Jiménez, "Desarrollo una aplicación web para facilitar la visualización y análisis se datos de Blue Ant Code," Trabajo De Fin De Grado En Ingeniería Informática, Universidad Rey Juan Carlos, 2020.

[187]J. Moreno and G. Robles, "Automatic detection of bad programming habits in scratch: A preliminary study," in 2014 IEEE Frontiers in Education Conference (FIE) Proceedings, pp. 1-4, Oct. 2014.

[188]N. Diana et al, "An instructor dashboard for real-time analytics in interactive programming assignments," in Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 272-279, 2017.

[189]L. Gal et al, "Suggesting a log-based creativity measurement for online programming learning environment," in Proceedings of the Fourth (2017) ACM Conference on Learning@ Scale, pp. 273-277, 2017.

[190]D. Olivares and C. Hundhausen, "Supporting learning analytics in computing education," in Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 584-585, 2017.

[191]M. Zapata-Ros and P. Pérez-Paredes, El Pensamiento Computacional. Análisis De Una Competencia Clave. New York: Create Space Independent Publishing, 2018.

[192]H. Søndergaard and R. A. Mulder, "Collaborative learning through formative peer review: Pedagogy, programs and potential," Computer Science Education, vol. 22, no. 4, pp. 343-367, 2012.

[193]M. R. Snodgrass, M. Israel and G. C. Reese, "Instructional supports for students with disabilities in K-5 computing: Findings from a cross-case analysis," Computers & Education, vol. 100, pp. 1-17, 2016.

[194]N. Ayuso et al, "Gender Gap in STEM: A Cross-Sectional Study of Primary School Students’ Self-Perception and Test Anxiety in Mathematics," IEEE Transactions on Education, vol. 64, no. 1, pp. 40-49, 2020.

[195]L. Sun et al, "Primary school students' perceptions of scaffolding in digital gamebased learning in mathematics," Learning, Culture and Social Interaction, vol. 28, pp. 100457, 2021.

[196]M. Cengiz et al, "Development of an interactive game-based learning environment to teach data mining," The International Journal of Engineering Education, vol. 33, no. 5, pp. 1598-1617, 2017.

[197]H. Lu, "Rethinking game-based learning from a gender perspective: a case study of a male English language learner in China," Gender, Technology and Development, vol. 16, no. 1, pp. 49-70, 2012.

[198]E. A. Boyle et al, "An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games," Computers & Education, vol. 94, pp. 178-192, 2016. Available: http://www.sciencedirect.com/science/article/pii/S0360131515300750, doi: //doi.org/10.1016/j.compedu.2015.11.003.

[199]R. J. Mislevy, L. S. Steinberg and R. G. Almond, "Focus Article: On the Structure of Educational Assessments," Measurement (Mahwah, N.J.), vol. 1, no. 1, pp. 3-62, 2003. Available: http://www.tandfonline.com/doi/abs/10.1207/S15366359MEA0101_02, doi: 10.1207/S15366359MEA0101_02.

[200]V. J. Shute and G. R. Moore, "Consistency and validity in game-based stealth assessment," Technology Enhanced Innovative Assessment: Development, Modeling, and Scoring from an Interdisciplinary Perspective, vol. 296, 2017.

[201]C. Alonso Fernández et al, "Improving serious games analyzing learning analytics data: lessons learned," in International Conference on Games and Learning Alliance. Springer, Cham Dec 5, 2018.

[202]G. Wallner and S. Kriglstein, "Comparative visualization of player behavior for serious game analytics," in Serious Games Analytics, Cham: Springer International Publishing, pp. 159-179, 2015.

[203]E. Zimmerman, "Narrative, interactivity, play, and games: Four naughty concepts in need of discipline," First Person: New Media as Story, Performance, and Game, vol. 154, 2004.

[204]E. Carratala Sánchez, "Análisis de la teoría de las metas de logro y de la autodeterminación en los planes de especialización deportiva de la Generalitat Valenciana." 2004.

[205]C. J. Ferguson and C. K. Olson, "Friends, fun, frustration and fantasy: Child motivations for video game play," Motivation and Emotion, vol. 37, no. 1, pp. 154-164, 2013.

[206]P. R. Pintrich and A. Zusho, "The development of academic self-regulation: The role of cognitive and motivational factors," in Development of Achievement Motivation, 2002.

[207]A. E. Gottfried, "Academic intrinsic motivation in young elementary school children." J. Educ. Psychol., vol. 82, no. 3, pp. 525, 1990.

[208]R. M. Ryan and E. L. Deci, "Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being." Am. Psychol., vol. 55, no. 1, pp. 68, 2000.

[209]E. L. Deci and R. M. Ryan, "Intrinsic motivation," The Corsini Encyclopedia of Psychology, pp. 1-2, 2010.

[210]J. R. Calvo-Ferrer, "Juegos, videojuegos y juegos serios: Análisis de los factores que favorecen la diversión del jugador," Miguel Hernández Communication Journal, no. 9, pp. 191-226, 2018.

[211]I. Granic, A. Lobel and R. C. Engels, "The benefits of playing video games." American Psychologist Journal, vol. 69, no. 1, pp. 66, 2014.

[212]M. Ventura, V. Shute and W. Zhao, "The relationship between video game use and a performance-based measure of persistence," Computers & Education, vol. 60, no. 1, pp. 52-58, 2013.

[213]R. M. Ryan, C. S. Rigby and A. Przybylski, "The motivational pull of video games: A self-determination theory approach," Motivation and Emotion, vol. 30, no. 4, pp. 344-360, 2006.

[214]M. Salminen and N. Ravaja, "Increased oscillatory theta activation evoked by violent digital game events," Neuroscience Letters - Journals, vol. 435, no. 1, pp. 69-72, 2008.

[215]R. Israel-Fishelson and A. Hershkovitz, "Shooting for the stars: Micro-persistence of students in game-based learning environments," in Early Warning Systems and Targeted Interventions for Student Success in Online Courses, 2020.

[216]C. Hart, "Factors associated with student persistence in an online program of study: A review of the literature." Journal of Interactive Online Learning, vol. 11, no. 1, 2012.

[217]R. Israel‐Fishelson and A. Hershkovitz, "Micro‐persistence and difficulty in a game‐based learning environment for computational thinking acquisition," Journal of Computer Assisted Learning, 2021.

[218]Y. Fang et al, "Online Learning Persistence and Academic Achievement." International Educational Data Mining Society, 2017.

[219]C. Alonso-Fernández et al, "Applications of data science to game learning analytics data: A systematic literature review," Computers & Education, vol. 141, pp. 103612, 2019. Available: https://search.datacite.org/works/10.1016/j.compedu.2019.103612, doi: 10.1016/j.compedu.2019.103612.

[220]P. R. Pintrich and A. Zusho, "Student motivation and self-regulated learning in the college classroom," in Higher Education: Handbook of Theory and ResearchAnonymous 2002.

[221]K. Weber and B. R. Patterson, "Student interest, empowerment and motivation," Communication Research Reports, vol. 17, no. 1, pp. 22-29, 2000.

[222]Á Serrano-Laguna et al, "Applying standards to systematize learning analytics in serious games," Computer Standards & Interfaces, vol. 50, pp. 116-123, 2017.

[223]Y. Tran, "Computational thinking equity in elementary classrooms: What third-grade students know and can do," Journal of Educational Computing Research, vol. 57, no. 1, pp. 3-31, 2019.

[224]R. Israel-Fishelson and A. Hershkovitz, "Persistence and achievement in acquiring computational thinking concepts: A large-scale log-based analysis," in E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, 2019.

[225]S. Grover et al, "Assessing problem-solving process at scale," in Proceedings of the Third (2016) ACM Conference on Learning@ Scale, pp. 245-248, Apr 25, 2016.

[226]M. Zapata-Cáceres and E. Martín-Barroso, "Blue Ant Code," 2020. Available: https://play.google.com/store/apps/details?id=com.artbabygame.blueAnt https://apps.apple.com/es/app/blue-ant-code/id1495021110.

[227]S. Kong, M. M. Chiu and M. Lai, "A study of primary school students' interest, collaboration attitude, and programming empowerment in computational thinking education," Computers & Education, vol. 127, pp. 178-189, 2018. Available: http://dx.doi.org/10.1016/j.compedu.2018.08.026, doi: 10.1016/j.compedu.2018.08.026.

[228]M. Guenaga et al, "How do students develop computational thinking? Assessing early programmers in a maze-based online game," Computer Science Education, vol. 31, no 2, pp. 259-289, 2021. Available: https://doi.org/10.1080/08993408.2021.1903248, doi: 10.1080/08993408.2021.1903248.

[229]T. P. Alloway, "How does working memory work in the classroom?" Educational Research and Reviews, vol. 1, no. 4, pp. 134-139, 2006.

[230]C. S. Dweck and D. C. Molden, "Their impact on competence motivation and acquisition," Handbook of Competence and Motivation: Theory and Application, vol. 2, pp. 135-154, 2017.

[231]L. S. Blackwell, K. H. Trzesniewski and C. S. Dweck, "Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention," Child Dev., vol. 78, no. 1, pp. 246-263, 2007.

[232]P. Sweetser and P. Wyeth, "GameFlow: a model for evaluating player enjoyment in games," Computers in Entertainment (CIE), vol. 3, no. 3, pp. 3, 2005.

[233]M. Lafuente Martínez et al, "Assessing Computational Thinking: Development and Validation of the Algorithmic Thinking Test for Adults," 2021.

[234]L. Narcisi et al, "Beginners Computational Thinking Test: Adaptation pour un public de moins de 5 ans." Service D'Ingénierie Pédagogique Et Du Numérique Éducatif, FPSE, UMONS, s.f., 2021.

[235]M. Lázaro-Álvarez, "Sudokus adaptativos para Educación Infantil o Primaria," Trabajo De Fin De Grado En Ingeniería Informática, Universidad Rey Juan Carlos, 2021.

[236]J. Plaza-Carro, "Blue-encode: Juego colaborativo de cifrado y descifrado de mensajes en Educación Primaria," Trabajo De Fin De Grado En Ingeniería Informática, Universidad Rey Juan Carlos, 2021.