Lei Bao a Edward F. Redish b

Transcription

1 Lei Bao a Edward F. Redish b a Departmet of Physics, The Ohio-State Uiversity b Departmet of Physics, Uiversity of Marylad August GIREP Barceloa

2 Itroductio: A Hybrid Model of Research Coduct qualitative research to idetify studet models uderlyig their spotaeous reasoig for a particular cotet area. Develop a theoretical framewor to model the studet reasoig process for that particular topic. Develop model-based istrumets to aalyze studet model use with quatitative measures. Use the results from the quatitative research to facilitate the desig of ew istructios as well as ew diagostic ad evaluatio tools. August GIREP Barceloa

3 Icosistecies i Studet Reasoig Studets ca appear to reaso icosistetly about physical problems. Problems see as equivalet by experts may ot be treated usig equivalet reasoig by studets. Qualitative research has documeted may differet clusters of semi-cosistet reasoig studets use i respodig to physics problems. This owledge has bee used i creatig attractive distracters for multiple-choice examiatios that allow oe to examie large populatios. The way that studets select wrog aswers o such tests cotais valuable iformatio o studet uderstadig. Traditioal aalyses of multiple-choice exams focus o the scores, ad possibly o the correlatio betwee correct aswers chose by studets. Such a aalysis ofte fails to extract all the iformatio about a class s stage of owledge the test ca provide. August GIREP Barceloa 3

4 Model Aalysis: A Quatitative Approach to Study Studets Coceptual Models We have developed algorithms to coveietly extract ad display iformatio o studets applyig their models. Our Model Aalysis allows us to aalyze complete studet resposes rather tha just the fractio of the time they are usig the correct approach. I this paper, we discuss a aalytical method for aalyzig the cocetratio / diversity of studet resposes to multiplechoice questios ad a method for evaluatig ad describig the mixed metal state of a class. These methods provide tools to extract iformatio from a multiple-choice test ad to be used i creatig such tests. August GIREP Barceloa 4

5 The defiitio The Cocetratio Factor It measures how the studets resposes are distributed o a particular multiple-choice questio. Type A B C D E I Radom II Two models III Oe model Possible distributios of resposes from a studets o a sigle 5-choice questio The formulatio of the Cocetratio Factor Overall cocetratio C m i m i ( ) m N m m i i N Cocetratio of the icorrect Γ m ( m m i i S (N S) ) m C ad Γ has a value betwee ad August GIREP Barceloa 5

6 Model the studet resposes with C Combiig C with scores, we ca ow model the differet types of the resposes: e.g., Three level codig scheme to model the resposes with SC, e.g. LH for low score ad high C. Score Level C Level ~.4 L ~. L.4~.7 M.~.5 M.7~. H.5~. H Implicatios of the patters HH Oe correct model Oe-Pea LH Oe domiat icorrect model LM Two possible icorrect models Two-Pea Two popular models (correct MM ad icorrect) No-Pea LL Near radom situatio Oe-Pea: Most of the resposes cocetrated o oe aswer. (LH or HH) Two-Pea: Most of the resposes cocetrated o two aswers, usually oe correct ad oe icorrect. (LM or MM) No-Pea: Most of the resposes somewhat evely distributed amog three or more aswers. (LL) August GIREP Barceloa 6

7 Cocetratio Aalysis with FCI ad FMCE Data The Studets FCI, 4 classes itro uiversity physics (UMd) FMCE, 4 classes itro uiversity physics (Cal. Polytech) The Iitial States FCI FMCE Types LL LM LH ML MM MH HH Patters No pea Questios 5, 4 Two peas 5, 9, 8, 8 Oe pea, 3, Wea oe-pea 3, 7,, 6 Two peas 6, 8,, 4, 7,, 3, 5 Oe pea Oe pea, 6, 9 Types Questios LH,,4,8,9,,,,3,4,6,,8,3,34,36,38,44 LM 3,5,6,7,7,8,9,,3,5,7,9,3,3,39,45,46,47 MM,4,6,35,37 HH 5,33,4,4,4,43, 4,, 9, 7 August GIREP Barceloa 7

8 FCI Questio # Respose Type LH Imagie a head-o collisio betwee a large truc ad a small compact car. Durig the collisio: A the truc exerts a greater amout of force o the car tha the car exerts o the truc. B the car exerts a greater amout of force o the truc tha the truc exerts o the car. C either exerts a force o the other, the car gets smashed simply because it gets i the way of the truc. D the truc exerts a force o the car but the car does t exert a force o the truc. E the truc exerts the same amout of force o the car as the car exerts o the truc. August GIREP Barceloa 8

9 Implicatios of the Cocetratio Aalysis (FCI) Force ad Motio Newto s Third Law Aswer % Type Aswer % Type 5-c 58% LM -a 66% LH 9-c 45% LM -d 43% MM 8-a 63% LM 3-c 68% LH -c 66% LH 8-d 5% LM A LH or LM type of respose ofte implies the existece of a commo icorrect studet model related to the physical cotext of the questio August GIREP Barceloa 9

10 The Studet Model State Idetifyig studet states Through qualitative research, we ca idetify a fiite set of commoly used studet modes for a particular cocept Correct expert model Icorrect commo studet models Null model For a sigle studet to solve a group of questios related to a sigle cocept, the studet ca use oe of the physical models cosistetly (pure states), or hold differet physical models at the same time ad use them icosistetly (mixed states). Model States The differet situatios of studets usig their models are described with differet studet Model States. A model state is a set of probabilities describig a studet s use of differet models. August GIREP Barceloa

11 Measurig Studet Model State Cotext depedet model triggerig ad applicatio q M q M Physical Cotext Model Triggerig q 3 M 3 Coditioal Radom Process q w M w With a set of questios desiged with a sigle physics cocept, we ca measure the probability for a studet to be triggered ito differet models. For differet studets, the distributios of probability will be differet. August GIREP Barceloa

12 August GIREP Barceloa Mathematical Represetatios Model Space Physical Models Studet Model States,,, M L M M e w e e η η w w q q q q, u u u M

13 Idetifyig the Physical Models i the Test (FCI) Force-Motio cocept group. (Questios 5, 9, 8,, 8) Model : It is ot always ecessary to have a force to maitai motio ad there is o such thig as a force i the directio of motio. (correct) Model : A force is eeded to maitai motio. (icorrect) Model 3: Other ideas. (ull model) Three physical models: Model <> () T e Model <> () T e Model 3 <> () T e 3 Item-based Modelig (IbM) of Studets Resposes Questio Model Model Model 3 5 d a, b, c e 9 a, d b, c e 8 b a, e c, d a, d b, c, e 8 c a, d, e b m 5, umber of questios w 3, umber of physical models August GIREP Barceloa 3

14 August GIREP Barceloa 4 The studet model desity matrix (w 3, m 5) The sigle studet model state vector u The sigle studet model desity matrix D Studet Class Model Desity Matrix m u u u 3 3 u { } ηµ m u u D N N N N D D N is the umber of studets i the class.

15 The Iformatio Retaied i D Examples of studet class model desity matrix Cosistet oe-model.5.3. Cosistet three-model (a) (b) (c)... Icosistet three-model Perform eigevalue decompositio o D. The resultig eigevectors represet the studet class model states. (a three-model example) σ v v v 3 w 3 H V D V [ Σ ] V v v v3 σ v3 v3 v33 σ3 Class model states represet the saliet features embedded i the idividual studet model states. The prevalece of such features is represeted by the eigevalues. (A large eigevalue idicates a popular class model state, ad i this case idividual studets will have similar model states.) August GIREP Barceloa 5

16 Displayig Class Model States Model Plot: Studet resposes are ofte cocetrated o two models, ofte a correct oe ad a icorrect oe. We the ca costruct a two dimesioal Model Plot usig the eigevalues ad the compoets of the class model states. Expert Model.5.4 Model Regio Secodary Regio Mixed Model Regio.4.5 Model Regio Icorrect Model Probability of Usig Model σ µ (-v 3µ ) σ µ v µ B σ µ v µ σ µ (-v 3µ ) Probability of Usig Model Usig a Model Plot, we ca visually represet features of a class s model states. August GIREP Barceloa 6

17 Results: Studet FCI Data (UMd) Results from 7 tutorial classes ad 7 traditioal classes, UMd Before istructio, both classes show a cosistet icorrect model. Model (Correct) Average Results of Model Plot for Force Motio Tut-Pre Tut-Post Trd-Pre Trd-Post. Model. (Icorrect)..5. After istructio, the tutorial classes have a fairly cosistet correct model; the traditioal classes have a very mixed model states showig that may studets are i a cofused stage. August GIREP Barceloa 7

18 Coclusio We have developed a ew method of aalyzig multiple-choice data based o our uderstadig of studet thiig. It maes full uses of the data, ad allows us to study the studet models i a quatitative way. It ca serve as a more comprehesive quatitative evaluatio for the studet performace with explicit iformatio o their coceptual models. It ca also be used to evaluate the effectiveess of differet istructios ad the features of diagostic istrumets August GIREP Barceloa 8