Hi, Dr. Linacre. I am a green-hand, and have some problems with writing the Facets specification file. The study is about a Rasch analysis of data from a writing test for second language learners. Three facets are included: examinees, raters, and criteria (i.e., three analytic scores in rating scale of content, organization and language). The analytic score ranges from 0 to 10. But I also want to include the holistic score in the analysis, and the holistic score is not the simple sum of the three analytic score, but a weighted score ranging from 0 to 10. My question is how to write the models to combine analytic scores and holistic scores together. Thanks a lot for your help.
The models I have tried are as follows:
Title = Provincial Argumentative Essays (2014) Facets = 3; examinees, raters, criteria Positive =1; for examinees, higher score = higher measure Noncentered = 1; examinees don't have mean score set to zero Model = ?,?,?,R10 ; observations are ratings in R10. * Labels= 1,examinee 1-33 ; 33 otherwise anonymous examinees 1,S1 2,S2 ... * 2, Rater 1-3 ; 3 otherwise anonymous raters 1, R1 2, R2 3, R3 * 3, Criteria 1-3 ; 3 rating criteria 1, content 2, organization 3, language * data= 1,1,1-3,8.5,8.5,9,8.65 ; first rating: examinee 1, rater 1, criteria of 8.5,8.5,9, holistic rating of 8.65 1,2,1-3,8.5,8,9,8.5 1,3,1-3,8,8.5,8,8.15 ...
Your Facets specification file looks good. Here are some suggestions:
i) There is no need to define each element twice. For instance, * 3, Criteria 1-3 ; 3 rating criteria 1, content 2, organization 3, language * is the same as * 3, Criteria 1, content 2, organization 3, language * It is convenient to use * 3, Criteria 1-3 ; 3 rating criteria * when there is no label for the individual elements.
ii) Facets expects integer, independent ratings. This data line: 1,1,1-3,8.5,8.5,9,8.65 ; first rating: examinee 1, rater 1, criteria of 8.5,8.5,9, holistic rating of 8.65 has decimal ratings, and the holistic rating is not independent of the analytic ratings. It is a weighted sum.
Let's start with the analytic ratings 1,1,1-3,8.5,8.5,9 these are not integers. There are two ways to solve this problem:
A. double all the ratings: Model = ?,?,?,R20 ; observations are ratings in R20. * Data= 1,1,1-3,17, 17, 18
or B. half-weight the data, and enter twice: Data= R0.5, 1,1,1-3,8,8,9 R0.5, 1,1,1-3,9,9,9
Since your rating scale is already long, 0-10, my recommendation is B. If every line in the data file will be entered twice, then use:
Model = ?,?,?,R10, 0.5 ; observations are ratings in R10, weighted 0.5
Data= 1,1,1-3,8,8,9 ; these lines are weighted 0.5 in the Models= specification. 1,1,1-3,9,9,9
iii) Holistic rating. This is not independent of the analytic so including it in the Facets analysis does not make sense from a measurement perspective, but let's pretend it does:
The specifications would start as
3, Criteria 1, content 2, organization 3, language
Since the analytic item is idiosyncratic, I recommend a separate rating scale and a separate Models= specification:
Model = ?,?,1-3,R10, 0.5 ; analytic observations are ratings in R10, weighted 0.5, sharing the same rating scale ?,?,4,R10 ; holistic observations are ratings in R10, weighted in the data, with its own rating scale *
1,1,1-3,8,8,9 ; these lines are weighted 0.5 in the Models= specification. 1,1,1-3,9,9,9