Test Design Criteria

Transcription

1 A E. Boehm, PhD The Boehm Test of Basic Cocepts, Thid Editio (Boehm 3) is a goup admiisteed assessmet fo studets i kidegate though secod gade. Boehm 3 was desiged to evaluate youg studet s udestadig of 50 impotat basic elatioal cocepts itegal fo success i school ad to idetify studets who may ot have had the leaig expeieces ecessay to develop a udestadig of key cocepts. The basic elatioal cocepts as defied by Boehm 3 iclude Oveview...1 Test Desig Citeia...1 Admiistatio ad Scoig...2 Stadadizatio Sample...2 Reliability & Validity...4 Spaish Editio...5 Summay...6 Oveview Test Desig Citeia size, diectio, ad positio i space, time, quatity, classificatio, ad geeal. Boehm 3 is a evisio of Boehm R, published i Ehacemets i the evisio iclude ew oms, colo illustatios, cocuet developmet of Spaish editio, tools to assist i commuicatio with paets, ad moe! The evisio will povide valuable ifomatio to elemetay educatos ad educatioal diagosticias, as well as speechlaguage pathologists, ad school psychologists. Sice the publicatio of Boehm R i 1986, thee have bee may chages i ealy school assessmet. This test was developed withi the cotext of the chagig eviomet ad was devised i a fomat that ca be taslated ito leaig objectives, as well as beig used fo cuiculum plaig ad wokig with paets. Outlied below ae the ehacemets that have bee icluded i Boehm 3: Fou measues povided Raw Scoes, Pecet Coect, Pefomace Rage, ad Pecetile The additio of a fouth espose choice fo each item to miimize guessig Modificatio ad the additio of colo i illustatios to appeal to studets takig the test Iceasig the divesity of people pictued i the illustatios by icludig divese acial goups, idividuals i o-steeotypical oles, ad diffeetly-abled idividuals Itoductio of a obsevatio fom that teaches ad cliicias ca use fo ogoig obsevatio of the geealized use of basic cocepts acoss cuiculum aeas Developmet of a paet epot fom to commuicate ifomatio to paets ad suggestios fo wokig with thei studets i the home eviomet Cocuet developmet of a Spaish editio with the Eglish editio Ehacemet of the tasitio fom Boehm 3 to Boehm-3 Peschool, the idividually admiisteed test of basic cocepts fo childe ages 3.0 to

2 ADMINISTRATION AND SCORING Thee ae two paallel foms of Boehm 3; ad. Each fom addesses the same 50 cocepts, i the same ode, usig diffeet cotexts. Admiisteig Boehm 3 Foms E ad F is simple ad staightfowad. Each fom cosists of a total of 50 pictoial items i oe test booklet. Thee pactice items ae peseted, followed by 25 test items. Thee is a beak poit afte the fist 25 items if you wish to admiiste the test i two sessios. Thee pactice items pecede the fial 25 items. iteested i the test ad ejoy takig it, so holdig the studet s attetio, eve at the kidegate level, is usually ot difficult. Boehm 3 esults ca be epoted as a aw scoe, pecet coect pefomace age, ad a pecetile. The aw scoe ca be detemied by summig the umbe of coect esposes. Usig the aw scoes, the pecet coect, pefomace age, ad pecetile ca be obtaied. Easie items ae itespesed with moe difficult items i ode to icease a studet s sese of success ad ehace attetio. Most studets become STANDARDIZATION SAMPLE Natioal fall oms wee developed fo the Boehm 3 by admiisteig Foms E ad F to studets i kidegate, fist- ad secod-gade i Octobe ad Novembe of Natioal spig testig to establish oms took place i Apil ad May of A attempt was made to test studets fom schools epesetative of the atio as a whole i tems of school distict size (betwee 600 to moe tha 20,000 studets), socioecoomic level of the esidets of the school distict, ubaicity by Gade ad Fom Testig Gade Kidegate Fist Gade Secod Gade , , , (uba, sububa, ual, o o-pubic), ad geogaphic egio. Studets who eceived special sevices, but wee maisteamed i egula classooms wee icluded i the stadadizatio. The Boehm 3 fall stadadizatio sample cosisted of ove 6,000 studets i kidegate, fist- ad secod-gade classooms ad the spig sample icluded moe tha 4,000 studets. by Gade ad Fom Spig Testig Gade Kidegate Fist Gade Secod Gade by Gede ad Fom Testig Gede Female Male Not Repoted 1, , , , by Gede ad Fom Spig Testig Gede Female Male 1, , , , by Race/Ethicity ad Fom Testig Race/Ethicity Afica Ameica Hispaic Othe White Not Repoted , , by Race/Ethicity ad Fom Testig Race/Ethicity Afica Ameica Hispaic Othe White , ,

3 Regio Notheast Noth Cetal South West by Socioecoomic Level of the Paticipatig School Disticts ad Schools ad Fom Testig Socioecoomic Level Low Middle High 1, , by Ubaicity ad Fom Testig Ubaicity Uba Sububa Rual by Regio ad Fom Testig , , by Regio ad Fom Spig Testig Regio Notheast Noth Cetal South West by Socioecoomic Level of the Paticipatig School Disticts ad Schools ad Fom Spig Testig Socioecoomic Level Low Middle High by Ubaicity ad Fom Spig Testig Ubaicity Uba Sububa Rual , , Bueau of the Cesus. (1998). Cuet populatio suvey, Octobe 1998: School Eollmet Supplemetal File [Machie-eadable data file]. Washigto, DC: Bueau of the Cesus (Poduce/Distibuto). 3

4 The eliability of the Boehm 3 was detemied by checkig iteal cosistecy, stadad eo of measuemet, test-etest eliability, ad alteate foms eliability. A test with a high level of iteal cosistecy idicates that the iteal stuctue of the test is such that esults ae epeatable. The coefficiet RELIABILITY AND VALIDITY 4 alphas fo Boehm 3 aged fom.80 to.91. The Stadad Eo of Measuemet (SEM) povides a alteative measue of eliability. The smalle the SEM, the geate level of cofidece attibuted to the accuacy of test scoes. The SEM fo Boehm 3 aged fom 1.14 to 2.43, idicatig oveall low vaiability. Iteal Cosistecy Reliability Coefficiets (Coefficiet Alpha) ad Stadad Eo of Measuemet by Gade ad Fom Gade Kidegate SEM Fist Gade SEM Secod Gade SEM , , , Alteate fom eliability is eflected i coefficiets of coelatio betwee two foms of a test. It is impotat to ote that ealy 94 of the studets 216 Oveall test-etest eliability coefficiets by gade/fom aged fom.80 to.89. A test ca be said to have evidece of validity if it measues what it states that it measues. The Boehm 3 cotais evidece of validity based o test cotet ad elatios to othe vaiables. Reseach outlied i the Boehm 3 Examie s Maual suppots the claim that a child s ability to make elatioal judgmets usig the Boehm-3 cocept tems, plus thei syoyms ad atoyms, is basic to udestadig classoom istuctio, whethe i the fom of followig teache diectios, completig cuicula tasks. Validity evidece based o elatioships of othe vaiables was compiled i five sepaate studies. The fist study looked at the elatioship betwee Boehm 3 ad its pedecesso, Boehm R. The coelatio betwee Boehm 3 ad Boehm R aged fom.61 to.96. It is impotat to ote that while both the Boehm 3 ad its pedecesso test 50 basic cocepts, ew cocepts have bee added to Boehm 3. Secodly, a study was coducted to look at the elatioship betwee Boehm 3 ad the Metopolita Achievemet Tests, Eighth Editio ( Educatioal Measuemet, Stadadizatio Editio 1999), a test used to assess studet achievemet i the aeas of souds ad pit (emeget liteacy skills), Atemate Foms Reliability Mea SD Spig paticipatig i the Boehm 3 alteate foms study had a diffeece of 4 o fewe aw scoe poits fom oe fom to the othe. Mea SD mathematics, ad laguage. The coelatio betwee these two tests aged fom.58 to.88, idicatig that kowledge of basic cocepts is impotat fo studets to comply with admiistatio diectios ad pefomace o the test tasks. A thid study ivolved a goup of 75 kidegate studets who wee admiisteed the Metopolita Readiess Test, Sixth Editio (Nuss & McGuava, 1995). This test evaluates the ealy skills equied i emeget eadig ad math. The coelatio betwee this test ad Boehm-3 aged fom.48 to.63. The coelatio betwee Boehm 3 ad the Otis-Leo School Ability Test, Seveth Editio (Otis & Leo, 1996) aged fom.45 to.68. The OLSAT 7 is used to measue thikig ad easoig by checkig studet pefomace o tasks. The modeate coelatio ca be expected due to the vaied cotet of the umeous aeas assessed with the OLSAT 7, suggestig that kowledge of basic cocepts is stogly elated to studet pefomace ad is a essetial compoet of thikig ad easoig. This futhe demostates the eed to make sue studets kow basic cocepts pio to admiisteig tests. The fial study was a Logitudial Study to Spig. A goup of 273 kidegate studets took pat i a compaiso of thei pefomace i the fall with thei pefomace i the spig o Boehm 3. The coelatio fo both Foms E ad F is.78, idicatig that studets who did well i the fall also did well i the spig.

5 Natioal fall oms fo the Spaish vesio of Boehm 3 wee developed by admiisteig Foms E ad F to studets i kidegate, fist, ad secod gade i Octobe ad Novembe of Spig stadadizatio took place i Apil ad May of Biligual teaches who wee expeieced i test admiistatio wee ivited to paticipate i the stadadizatio testig. Moe tha 1,200 Spaish-speakig studets i the Uited States paticipated i the fall stadadizatio ad the elated eliability ad validity studies of the Boehm 3 Spaish editio. Moe tha 400 Spaish-speakig studets paticipated i the spig stadadizatio. Gede was closely divided with 51.2 male ad 48.3 female compisig the fall oms, ad 56.3 female ad 43.7 male compisig the spig oms. Although cosideable ecuitmet of examies fom all ove the Uited States occued, few esposes wee eceived fom cetai aeas of the Uited States, paticulaly the Notheast egio. As a esult, a lage Regio Notheast South West SPANISH EDITION Boehm 3 Spaish Stadadizatio Sample by Regio ad Fom ad Spig Total pecetage of the sample cosists of studets fom those egios of the Uited States that typically have lage populatios of Spaish-speakig studets. School disticts paticipatig i the stadadizatio study wee chaacteized by the socioecoomic level of the distict. Thee wee thee levels: low, middle ad high. The levels wee detemied by eviewig the icome level ad educatio of the esidets withi a defied school distict. These classificatios wee desiged to place 33 of the public school distict populatio ito each of the thee categoies. Studets paticipatig i the stadadizatio studies that atted o-public school wee placed i a fouth categoy. Ubaicity ca be defied as the school distict s sevice aea elative to the Metopolita Sevice Aea ad is based o zip code ifomatio of the distict. Thee ae thee pimay levels of ubaicity: uba, sububa, ad ual Spig Boehm 3 Spaish Stadadizatio Sample by Socioecoomic Level of the School Distict ad Schools ad Fom ad Spig Socioecoomic Level Low Middle High Total Spig Ubaicity Uba Sububa Rual Boehm 3 Spaish Stadadizatio Sample by Ubaicity ad Fom ad Spig Total Bueau of the Cesus. (1998). Cuet populatio suvey, Octobe 1998: School Eollmet Supplemetal File [Machie-eadable data file]. Washigto, DC: Bueau of the Cesus (Poduce/Distibuto) Spig

6 6