Educational Data Mining - Revision history

R.hosseini: /* Exploring Problem Solving Paths in a Java Programming Course */

2016-04-05T13:39:16Z

‎Exploring Problem Solving Paths in a Java Programming Course

R.hosseini: /* Publications */

2016-04-05T13:38:58Z

‎Publications

R.hosseini: /* Publications */

2016-04-05T13:38:33Z

‎Publications

R.hosseini: /* Exploring Problem Solving Paths in a Java Programming Course */

2016-04-05T13:37:52Z

‎Exploring Problem Solving Paths in a Java Programming Course

R.hosseini: /* Exploring Problem Solving Paths in a Java Programming Course */

2016-04-05T13:37:39Z

‎Exploring Problem Solving Paths in a Java Programming Course

R.hosseini: /* Exploring Problem Solving Paths in a Java Programming Course */

2016-04-05T13:37:24Z

‎Exploring Problem Solving Paths in a Java Programming Course

R.hosseini: /* Exploring Problem Solving Paths in a Java Programming Course */

2016-04-05T13:37:03Z

‎Exploring Problem Solving Paths in a Java Programming Course

R.hosseini at 13:33, 5 April 2016

2016-04-05T13:33:59Z

Yuh43 at 04:04, 5 April 2016

2016-04-05T04:04:24Z

Yuh43 at 04:03, 5 April 2016

2016-04-05T04:03:09Z

@@ Line 21: / Line 21: @@
 Assessment of students’ programming submissions has been the focus of interest in many studies. Although the final submissions capture the whole program, they often tell very little about how it was
 developed. In this paper, we are able to look at intermediate programming steps using a unique dataset that captures a series of snapshots showing how students developed their program over time. We assessed each of these intermediate steps and performed a fine-grained concept-based analysis on each step to identify the most common programming paths. Analysis of results showed that most of the students tend to incrementally build the program and improve its correctness. This finding provides us with evidence that intermediate programming steps are important, and need to be taken into account for not only improving user modelling in educational programming systems, but also for providing better feedback to students.
 [[Image:Programming_patterns.jpg|900x500px]]

@@ Line 27: / Line 27: @@
 * Sahebi, S., Huang, Y., and Brusilovsky, P. (2014) Parameterized Exercises in Java Programming: Using Knowledge Structure for Performance Prediction. In:  Proceedings of The second Workshop on AI-supported Education for Computer Science (AIEDCS) at 12th International Conference on Intelligent Tutoring Systems ITS 2014, Honolulu, Hawaii, June 6 2014. ([http://d-scholarship.pitt.edu/21915/ paper])([http://www.slideshare.net/chagh/parameterized-exercises-in-java-programming-using-knowledge-structure-for-performance-prediction presentation])
 * Sahebi, S., Huang, Y., and Brusilovsky, P. (2014) Predicting Student Performance in Solving Parameterized Exercises. In: S. Trausan-Matu, K. Boyer, M. Crosby and K. Panourgia (eds.) Proceedings of 12th International Conference on Intelligent Tutoring Systems (ITS 2014), Honolulu, HI, USA, June 5-9, 2014, Springer International Publishing, pp. 496-503, ([http://d-scholarship.pitt.edu/21916/ paper]) ([http://www.slideshare.net/chagh/its14-pitttemplate presentation])
-* Hosseini, R., Vihavainen, A., and Brusilovsky, P. (2014) Exploring Problem Solving Paths in a Java Programming Course. In: Proceedings of Psychology of Programming Interest Group Annual Conference, PPIG 2014, Brighton, UK, June 25-27, 2014, pp. 65-76. (paper) (presentation)
+* Hosseini, R., Vihavainen, A., and Brusilovsky, P. (2014) Exploring Problem Solving Paths in a Java Programming Course. In:  Proceedings of Psychology of Programming Interest Group Annual Conference, PPIG 2014, Brighton, UK, June 25-27, 2014, pp. 65-76. ([https://d-scholarship.pitt.edu/secure/26272/1/PPIG_2014_camera_ready.pdf paper]) ([http://www.slideshare.net/RoyaHosseini1/ppig2014-problem-solvingpaths presentation])

← Older revision		Revision as of 13:38, 5 April 2016
Line 27:		Line 27:
	* Sahebi, S., Huang, Y., and Brusilovsky, P. (2014) Parameterized Exercises in Java Programming: Using Knowledge Structure for Performance Prediction. In: Proceedings of The second Workshop on AI-supported Education for Computer Science (AIEDCS) at 12th International Conference on Intelligent Tutoring Systems ITS 2014, Honolulu, Hawaii, June 6 2014. ([http://d-scholarship.pitt.edu/21915/ paper])([http://www.slideshare.net/chagh/parameterized-exercises-in-java-programming-using-knowledge-structure-for-performance-prediction presentation])		* Sahebi, S., Huang, Y., and Brusilovsky, P. (2014) Parameterized Exercises in Java Programming: Using Knowledge Structure for Performance Prediction. In: Proceedings of The second Workshop on AI-supported Education for Computer Science (AIEDCS) at 12th International Conference on Intelligent Tutoring Systems ITS 2014, Honolulu, Hawaii, June 6 2014. ([http://d-scholarship.pitt.edu/21915/ paper])([http://www.slideshare.net/chagh/parameterized-exercises-in-java-programming-using-knowledge-structure-for-performance-prediction presentation])
	* Sahebi, S., Huang, Y., and Brusilovsky, P. (2014) Predicting Student Performance in Solving Parameterized Exercises. In: S. Trausan-Matu, K. Boyer, M. Crosby and K. Panourgia (eds.) Proceedings of 12th International Conference on Intelligent Tutoring Systems (ITS 2014), Honolulu, HI, USA, June 5-9, 2014, Springer International Publishing, pp. 496-503, ([http://d-scholarship.pitt.edu/21916/ paper]) ([http://www.slideshare.net/chagh/its14-pitttemplate presentation])		* Sahebi, S., Huang, Y., and Brusilovsky, P. (2014) Predicting Student Performance in Solving Parameterized Exercises. In: S. Trausan-Matu, K. Boyer, M. Crosby and K. Panourgia (eds.) Proceedings of 12th International Conference on Intelligent Tutoring Systems (ITS 2014), Honolulu, HI, USA, June 5-9, 2014, Springer International Publishing, pp. 496-503, ([http://d-scholarship.pitt.edu/21916/ paper]) ([http://www.slideshare.net/chagh/its14-pitttemplate presentation])
		+	* Hosseini, R., Vihavainen, A., and Brusilovsky, P. (2014) Exploring Problem Solving Paths in a Java Programming Course. In: Proceedings of Psychology of Programming Interest Group Annual Conference, PPIG 2014, Brighton, UK, June 25-27, 2014, pp. 65-76. (paper) (presentation)

@@ Line 21: / Line 21: @@
 Assessment of students’ programming submissions has been the focus of interest in many studies. Although the final submissions capture the whole program, they often tell very little about how it was
 developed. In this paper, we are able to look at intermediate programming steps using a unique dataset that captures a series of snapshots showing how students developed their program over time. We assessed each of these intermediate steps and performed a fine-grained concept-based analysis on each step to identify the most common programming paths. Analysis of results showed that most of the students tend to incrementally build the program and improve its correctness. This finding provides us with evidence that intermediate programming steps are important, and need to be taken into account for not only improving user modelling in educational programming systems, but also for providing better feedback to students.
-[[Image:Programming_patterns.jpg|700x400px]]
+[[Image:Programming_patterns.jpg|900x500px]]
 == Publications ==

@@ Line 21: / Line 21: @@
 Assessment of students’ programming submissions has been the focus of interest in many studies. Although the final submissions capture the whole program, they often tell very little about how it was
 developed. In this paper, we are able to look at intermediate programming steps using a unique dataset that captures a series of snapshots showing how students developed their program over time. We assessed each of these intermediate steps and performed a fine-grained concept-based analysis on each step to identify the most common programming paths. Analysis of results showed that most of the students tend to incrementally build the program and improve its correctness. This finding provides us with evidence that intermediate programming steps are important, and need to be taken into account for not only improving user modelling in educational programming systems, but also for providing better feedback to students.
-[[Image:Programming_patterns.jpg|600x300px]]
+[[Image:Programming_patterns.jpg|700x400px]]
 == Publications ==

← Older revision		Revision as of 13:33, 5 April 2016
Line 16:		Line 16:
	Although the tasks of domain modeling, predicting student performance, and student knowledge modeling are interrelated, these problems are usually addressed separately in the literature of educational data mining. Also, the traditional tensor and matrix factorization algorithms, that can extract the domain model while predicting student performance, are not designed for educational purposes: these models ignore the fact that students' knowledge in each step is related to her knowledge in previous steps and the concepts she is practicing while taking current learning resources. In this work, we propose a tensor factorization algorithm that can simultaneously perform all of the aforementioned tasks, while imposing a constraint to increase the knowledge of students using the feedback it receives from the current learning resources.		Although the tasks of domain modeling, predicting student performance, and student knowledge modeling are interrelated, these problems are usually addressed separately in the literature of educational data mining. Also, the traditional tensor and matrix factorization algorithms, that can extract the domain model while predicting student performance, are not designed for educational purposes: these models ignore the fact that students' knowledge in each step is related to her knowledge in previous steps and the concepts she is practicing while taking current learning resources. In this work, we propose a tensor factorization algorithm that can simultaneously perform all of the aforementioned tasks, while imposing a constraint to increase the knowledge of students using the feedback it receives from the current learning resources.

		+	[[Image:TensorFactorization.png\|650x1300px]]
		+
		+	== Exploring Problem Solving Paths in a Java Programming Course ==
		+	Assessment of students’ programming submissions has been the focus of interest in many studies. Although the final submissions capture the whole program, they often tell very little about how it was
		+	developed. In this paper, we are able to look at intermediate programming steps using a unique dataset that captures a series of snapshots showing how students developed their program over time. We assessed each of these intermediate steps and performed a fine-grained concept-based analysis on each step to identify the most common programming paths. Analysis of results showed that most of the students tend to incrementally build the program and improve its correctness. This finding provides us with evidence that intermediate programming steps are important, and need to be taken into account for not only improving user modelling in educational programming systems, but also for providing better feedback to students.
	[[Image:TensorFactorization.png\|650x1300px]]		[[Image:TensorFactorization.png\|650x1300px]]

← Older revision		Revision as of 04:04, 5 April 2016
Line 23:		Line 23:
	* Sahebi, S., Huang, Y., and Brusilovsky, P. (2014) Parameterized Exercises in Java Programming: Using Knowledge Structure for Performance Prediction. In: Proceedings of The second Workshop on AI-supported Education for Computer Science (AIEDCS) at 12th International Conference on Intelligent Tutoring Systems ITS 2014, Honolulu, Hawaii, June 6 2014. ([http://d-scholarship.pitt.edu/21915/ paper])([http://www.slideshare.net/chagh/parameterized-exercises-in-java-programming-using-knowledge-structure-for-performance-prediction presentation])		* Sahebi, S., Huang, Y., and Brusilovsky, P. (2014) Parameterized Exercises in Java Programming: Using Knowledge Structure for Performance Prediction. In: Proceedings of The second Workshop on AI-supported Education for Computer Science (AIEDCS) at 12th International Conference on Intelligent Tutoring Systems ITS 2014, Honolulu, Hawaii, June 6 2014. ([http://d-scholarship.pitt.edu/21915/ paper])([http://www.slideshare.net/chagh/parameterized-exercises-in-java-programming-using-knowledge-structure-for-performance-prediction presentation])
	* Sahebi, S., Huang, Y., and Brusilovsky, P. (2014) Predicting Student Performance in Solving Parameterized Exercises. In: S. Trausan-Matu, K. Boyer, M. Crosby and K. Panourgia (eds.) Proceedings of 12th International Conference on Intelligent Tutoring Systems (ITS 2014), Honolulu, HI, USA, June 5-9, 2014, Springer International Publishing, pp. 496-503, ([http://d-scholarship.pitt.edu/21916/ paper]) ([http://www.slideshare.net/chagh/its14-pitttemplate presentation])		* Sahebi, S., Huang, Y., and Brusilovsky, P. (2014) Predicting Student Performance in Solving Parameterized Exercises. In: S. Trausan-Matu, K. Boyer, M. Crosby and K. Panourgia (eds.) Proceedings of 12th International Conference on Intelligent Tutoring Systems (ITS 2014), Honolulu, HI, USA, June 5-9, 2014, Springer International Publishing, pp. 496-503, ([http://d-scholarship.pitt.edu/21916/ paper]) ([http://www.slideshare.net/chagh/its14-pitttemplate presentation])
−	* Huang, Y., Xu, Y., and Brusilovsky, P. (2014) Doing More with Less: Student Modeling and Performance Prediction with Reduced Content Models. In: V. Dimitrova, et al. (eds.) Proceedings of 22nd Conference on User Modeling, Adaptation and Personalization (UMAP 2014), Aalborg, Denmark, July 7-11, 2014, Springer Verlag, pp. 338-349. ([http://www.slideshare.net/pbrusilovsky/umap-v1 presentation][http://link.springer.com/chapter/10.1007%2F978-3-319-08786-3_30 paper])
−	* Huang, Y., González-Brenes, J. P., Kumar, R., Brusilovsky, P. (2015) A Framework for Multifaceted Evaluation of Student Models. In: Proceedings of the 8th International Conference on Educational Data Mining (EDM 2015), Madrid, Spain, pp. 203-210. ([http://www.educationaldatamining.org/EDM2015/uploads/papers/paper_164.pdf paper]) ([http://www.slideshare.net/huangyun/2015edm-a-framework-for-multifaceted-evaluation-of-student-models-polygon presentation])
−	* Huang, Y., González-Brenes, J. P., Brusilovsky, P. (2015) Challenges of Using Observational Data to Determine the Importance of Example Usage. In: Proceedings of the 17th International Conference on Artificial Intelligence in Education (AIED 2015), Madrid, Spain, pp. 633-637. ([http://d-scholarship.pitt.edu/26056/ paper])
−	* Gonzalez-Brenes, J. P., Huang, Y. (2014) The White Method: Towards Automatic Evaluation Metrics for Adaptive Tutoring Systems. In: Proceedings of NIPS 2014 Workshop on Human Propelled Machine Learning, Montreal, Canada, December 13, 2014 ([http://d-scholarship.pitt.edu/26061/ paper])
−	* Gonzalez-Brenes, J. P., Huang, Y. (2015) Your model is predictive— but is it useful? Theoretical and Empirical Considerations of a New Paradigm for Adaptive Tutoring Evaluation. In: Proceedings of the 8th International Conference on Educational Data Mining (EDM 2015), Madrid, Spain, pp. 187-194. ([http://d-scholarship.pitt.edu/26046/ paper] [http://www.slideshare.net/huangyun/2015-edm-leopard-for-adaptive-tutoring-evaluation presentation])
−	* Gonzalez-Brenes, J. P., Huang, Y. The Leopard Framework: Towards understanding educational technology interventions with a Pareto Efficiency Perspective. In: The ICML 2015 Workshop on Machine Learning for Education (ICML 2015), Lille, France, 2015. ([https://dsp.rice.edu/sites/dsp.rice.edu/files/leopard_evaluation(1).pdf paper])
−	* Gonzalez-Brenes, J. P., Huang, Y. Using Data from Real and Simulated Learners to Evaluate Adaptive Tutoring Systems. In: 2nd AIED Workshop on Simulated Learners at the 17th Intl. Conf. on Artificial Intelligence in Education (AIED 2015), Madrid, Spain, 2015. ([http://ceur-ws.org/Vol-1432/sl_pap4.pdf paper])

← Older revision		Revision as of 04:03, 5 April 2016
Line 18:		Line 18:
	[[Image:TensorFactorization.png\|650x1300px]]		[[Image:TensorFactorization.png\|650x1300px]]

−	~~== Content Model Reduction for Better Learner Model ==~~
−	When modeling student knowledge and predicting student performance, adaptive educational systems frequently rely on content models that connect learning content (i.e., problems) with its underlying domain knowledge (i.e., knowledge components, KCs) required to complete it. In some domains, such as programming, the number of KCs associated with advanced learning contents is quite large. It complicates modeling due to increasing noise and decreases efficiency. We argue that the efficiency of modeling and prediction in such domains could be improved without the loss of quality by reducing problems content models to a subset of most important KCs. To prove this hypothesis, we evaluate several KC reduction methods varying reduction size by assessing the prediction performance of Knowledge Tracing and Performance Factor Analysis. The results show that the predictive performance using reduced content models can be significantly better than using original one, with extra benefits of reducing time and space.
−
−	~~== Multifaceted Evaluation of Learner Models for Practitioners ==~~
−	We have proposed two state-of-the-art frameworks for evaluating student models in a data-driven manner for practitioners: the Polygon framework, and the Learner Effort-Outcomes Paradigm. We also explored challenges of using observational data to answer research question such as determine the importance of example usage.
−
−	~~=== Polygon framework ===~~
−	Latent variable models, such as the popular Knowledge Tracing method, are often used to enable adaptive tutoring systems to personalize education. However, finding optimal model parameters is usually a difficult non-convex optimization problem when considering latent variable models. Prior work has reported that latent variable models obtained from educational data vary in their predictive performance, plausibility, and consistency. Unfortunately, there are still no unified quantitative measurements of these properties. This paper suggests a general unified framework (that we call Polygon) for multifaceted evaluation of student models. The framework takes all three dimensions mentioned above into consideration and offers novel metrics for the quantitative comparison of different student models. These properties affect the effectiveness of the tutoring experience in a way that traditional predictive performance metrics fall short. The present work demonstrates our methodology of comparing Knowledge Tracing with a recent model called [[Feature-Aware Student knowledge Tracing (FAST)]] on datasets from different tutoring systems. Our analysis suggests that FAST generally improves on Knowledge Tracing along all dimensions studied.
−
−	~~[[Image:Polygon.jpg\|800x1600px]]~~
−
−	~~=== Learner Effort-Outcomes Paradigm (Leopard) ===~~
−	Classification evaluation metrics are often used to evaluate adaptive tutoring systems— programs that teach and adapt to humans. Unfortunately, it is not clear how intuitive these metrics are for practitioners with little machine learning background. Moreover, our experiments suggest that existing convention for evaluating tutoring systems may lead to suboptimal decisions. We propose the Learner Effort-Outcomes Paradigm (Leopard), a new framework to evaluate adaptive tutoring. We introduce Teal and White, novel automatic metrics that apply Leopard and quantify the amount of effort required to achieve a learning outcome. Our experiments suggest that our metrics are a better alternative for evaluating adaptive tutoring.
−
−	~~[[Image:Leopard.png\|500x300px]]~~

	== Publications ==		== Publications ==