Hi Judith,

Again I have the opportunity to have some graduate students helping me with my PhD. One of them is Eddy. But he will introduce himzelf in this guestblog:

Data Ecosystem Infrastructure LA4LD

(original posted on: https://eddyvandenaker.com/post/Data-Ecosystem-Infrastructure-LA4LD)

Hi, my name is Eddy van den Aker and I’m currently doing my graduate internship. My project is part of Marcel’s PhD research project (https://2bejammed.org/2017/01/03/the-basics-of-my-phd-research/) about learning analytics, learning dashboards, and learning design.

Background

One of the problems faced by faculties, course designers, and teachers is the lack of insight into the student experience. Faculties are rated based on two factors: the time it takes for students to get their degree and the student experience. The first factor is obvious and easy to measure, but student experience is harder.

Currently the faculty of ICT within Zuyd University of Applied Science has two ways of measuring student experience. The first is the Nationale Studenten Enquête (NSE), which is a national questionnaire filled in by students from all Universities (of Applied Science). The second is a questionnaire at the end of every course, these are faculty specific.

The results of the NSE are not linked to a specific course, and the course questionnaires are done after the course has ended, so the results also come after the fact. The feedback toward the students on what is done with the results is also limited, which probably (based on anecdotal evidence) contributes to lower participation numbers. All in all, not enough data is available to improve student experience, and students are not seeing enough actionable feedback to be more engaged with the courses and the faculty.

The Project

To solve this problem Marcel has suggested creating a data ecosystem in which students, teachers and course designers participate to collect and make use off more and more useful data. Several projects have bin done and are currently going on to develop systems to collect data (for example the IoT projects https://2bejammed.org/2018/01/02/5-student-teams-working-on-classroom-iot/). Another project is looking at ways to present the data gathered in a collection of dashboards (LINK NAAR SANDERS POST).

My project fits neatly between all projects mentioned before. I will be developing an open-source infrastructure that can catch, clean, structure and store all data gathered while also delivering the underlying services needed to present the data to the users through dashboards.

Because this system sits at the core of the data ecosystem and must be able to support many different kinds of systems, both current and in the future, it is vital to make the entire infrastructure modular. During my internship, a couple of modules will be developed.

The first module will be an end-point for collecting information on student attendance. This system could be an RFID reader on which students swipe their student-card. Another module connects to the digital learning environment, in this case Moodle and collects data on how students use the provided course material. A third module imports student results from a file. And finally a last module will collect and store data from questionnaires.

As said before it has to be possible to develop more modules later down the line, adding for example environmental variables from the classroom or students study room at home. Another example would be to track the view of students in the classroom, where they are looking on the slides, what draws their attention.

Any system that collects this amount of data, especially potentially sensitive private data, has to consider the privacy of it’s participants and thus the security of the system. A way has to be sought to ensure that no one but the student themselves are able to see their own personalized data. Teachers and course designers will only see anonymised group data. The general security of the system also has to be considered.

Methods

During the development of the system I will be using a couple of different methodologies.

Design based Research Process

This project will be using the design based research methodology[1] The first three phases (problem definition & motivation, objectives of solution, and design & development) will be completed during the project, the fourth phase (demonstration) will be started.

Systematic Mapping Review

At the start of the project, a systematic mapping review[2] will be done to see in which fields data ecosystems have been suggested and maybe even deployed. It’s also interesting to know if any effect studies have been published in cases where data ecosystems have been deployed.

Scrum and GitHub

For managing the project I will be using a slightly modified version of Scrum. Slightly modified because I’m the only person in the development team. For tracking all Scrum related information I will be using the issues, pull requests, projects and wiki pages on the GitHub page for the project.

I wanted to figure out how to automate the entire Scrum workflow on GitHub. I have made some decent progress on it, good enough for this project, but I still have to move “to-do” items manually to “in progress” and after that to “in review”. If you read my post on converting exam questions to flashcard (https://eddyvandenaker.com/post/Converting-Exam-Questions-to-Flashcards/), you know I’m lazy (in a good way, I hope) and I will be looking to automate as much as the workflow as possible, so maybe I can find a solution to these two manual actions.

Test Driven Development

For the development of the system I’ll be using Test Driven Development (TDD). The basic idea of TDD is to make testing an integral part of the development cycle. By developing automated functional and/or integration tests first, then developing smaller unit tests. at first these tests should fail (it would be weird if they didn’t). Only after having done all that, you write just enough code to get the tests to pass (or at least progress to the next step). When you have some passing tests you can refactor (improve) the code while using the previously passing tests to make sure the program does not regress. This process is often called Red, Green, Refactor.

Timeline

My internship lasts half a year (20 school weeks). The first 3 weeks are spend on clearly defining the project, choosing the methodologies, and planning the phases of the project. Week 4 and 5 are used for requirements analysis and the systematic review. From week 6 until week 16 the system will be designed & developed in a couple of Scrum sprints. The last 4 weeks are used to prepare for the presentation at the end of the internship and to finish up the project in general.

Sprints

The design & development phase consists of a number of sprints:

1. Setup (software architecture & base functionality like logins, database connections, etc.) – 2 weeks
2. Importing student results from file – 1 week
3. Student attendance – 1 week
4. Moodle/xAPI connection – 3 weeks
5. MSLQ or other questionnaire connection – 1 week
6. Admin panel – 2 weeks
7. Wrapping up (extended testing, deployment considerations, etc.) – 2 weeks

What now?

In about 5 or 6 weeks I’ll be posting a status update on where I’m at with the project. Another 5 or 6 weeks after that I will present my results. Finally when I’m (almost) done with my internship I’ll write a post about my experiences.

The repository for this project can be found on https://github.com/eddyvdaker/Zuyd-LA4LD-Dataecosystem

[1]     Peffers, K.; Tuunanen, T. (februari 2006). The Design Science Research Process. Opgeroepen van

wrsc.org op 26 februari 2018 via:

Klik om toegang te krijgen tot 000designscresearchproc_desrist_2006.pdf

[2]     Kitchenham, B. (2007). Guidelines for Performing Systematic Literature Reviews in Software Engineering. Opgeroepen op 14 maart 2018 via:

Klik om toegang te krijgen tot 525444systematicreviewsguide.pdf

Hi Judith,

Wouldn’t it be great to be able to give insight in how students perform and like a certain learning activity? As a teacher I often ask them how did they like several learning activities during a coffee break or one of our social activities. As an institution we ask them after an Educational period of 10 weeks with a survey. This survey (called blokenquete) has one question in it about all of the learning material in the course. Besides the extra comments students can make in general there isn’t a question on separate learning activities.

Wouldn’t it be great to be able to give teachers but also students insight in how students perform in and like the learning activities they are provided. I believe that this would help students to increase performance and results and that teachers and institutions are able to provide better fit learning activities. Now often I only use my gut feeling or the “results” of conversation with the students at the coffee machine. It would be great to have an extra instrument, which based on data and input from students generates information on a learning activity level.

In a literature review, soon to be published and presented at the EC-TEL 2017 conference, I searched peer reviewed journals for experiments with learning analytics for learning design. I have been trying to identify opportunities and challenges in this field. I found it in three opportunities, who each have their own opportunities and challenges within them.

1. Using on demand indicators for evidence based decisions on Learning Design
2. Intervening during the run-time of a course
3. Increasing student learning outcome and satisfaction

In the table below I have categorized the articles I found into the three opportunities and the subopportunities and subchallenges within. I have specific tried to look for experiments and research on Learning Design, Learning Analytics, Learning Dashboards and Metacognitive Competences. This last item is interesting to me because from a Knowledge Engineering perspective the topic of “learning to learn”, which we do as we use our meta-cognitive competences is especially interesting. Taking these competences into consideration while designing a Learning Analytics for Learning Design solution seems very relevant.

After publication of the article in the Conference Proceedings I will put a link to the article here and if my presentation is ready the slides will be available here to. For now I can only share the keynote presentation of Prof. Dr. Hendrik Drachsler who is my supervisor. He talks about the place of my and my colleagues research in the world of Learning Analytics.

After this literature review on the state of the art I am going to try to design a Learning Dashboard which incorporates a Learning Analytics for Learning Design solution, which can give information during the course and which tries to make the most of the meta-cognitive competences of the students.

If I project the opportunities and challenges on a model for a solution you will get the next figure:

I will keep you posted. But feel free to ask.

Marcel

Hi Judith,

Thanks for your last blog about “How to support educational teams with the design of education.”   It again illustrates the difficulty of the implementation of blended learning. In the picture you use, several elements are mentioned which have influenced succesfull implementation strategies. In that graphic I miss the element of the management.

No, this is not a blog in which I want to complain about the management. I just want to try to see their point of view on this case of trying to get more blended learning. A case, sometimes struggle, sometimes a nice match, in which we have had several roles in the last few years. Their point of view is: why do we want blended learning, and do we as teachers, students and thus management want blended learning?

Quality of education is the dreamed answer, Sir Ken Robinsons’ vision of education that is not factory, which we were able to see on stage, is a wish waiting to come true. Self regulating collaborating networks of teachers and students co creating educational ecosystems is still science fiction. The managers want to dream, wish with us, they want to make science fiction into reality as well. But most of all they are stuck in that reality and sometimes we seem to forget.

The questions we have to answer aren’t how can we implement Blended Learning, as you said it yourself, that we know already. No we have to find the definite answer why do we want blended learning?

As long as we can’t translate our dreams and visions into profits, turn-over, student succes rates, student satisfaction, teacher satisfaction or regionwide knoweldge increase, management will have too much other priorities. So our real case is to illustrate the evidence there is in the context we are in so that management can dream, vision and make blended learning reality with us.

I believe that if we do, they will have a lot of fun doing it with us.

As a PhD student I will do my share, at least I will try, but I need allies. As a Zuyd community we will have to collect evidence. Not to defend our own faculty or our own model or framework which we have chosen, but to create an environment that is able to use the qualities we have to evidence based adjust our education to the demands the context or our working environments need.

Let’s go to sleep and dream on that.

Good night, Marcel Schmitz