
Post by on Dec 8, 2015 7:34:53 GMT
Hi, Dr. Linacre. I am a greenhand, and have some problems with writing the Facets specification file. The study is about a Rasch analysis of data from a writing test for second language learners. Three facets are included: examinees, raters, and criteria (i.e., three analytic scores in rating scale of content, organization and language). The analytic score ranges from 0 to 10. But I also want to include the holistic score in the analysis, and the holistic score is not the simple sum of the three analytic score, but a weighted score ranging from 0 to 10. My question is how to write the models to combine analytic scores and holistic scores together. Thanks a lot for your help.
The models I have tried are as follows:
Title = Provincial Argumentative Essays (2014) Facets = 3; examinees, raters, criteria Positive =1; for examinees, higher score = higher measure Noncentered = 1; examinees don't have mean score set to zero Model = ?,?,?,R10 ; observations are ratings in R10. * Labels= 1,examinee 133 ; 33 otherwise anonymous examinees 1,S1 2,S2 ... * 2, Rater 13 ; 3 otherwise anonymous raters 1, R1 2, R2 3, R3 * 3, Criteria 13 ; 3 rating criteria 1, content 2, organization 3, language * data= 1,1,13,8.5,8.5,9,8.65 ; first rating: examinee 1, rater 1, criteria of 8.5,8.5,9, holistic rating of 8.65 1,2,13,8.5,8,9,8.5 1,3,13,8,8.5,8,8.15 ...



Post by Mike Linacre on Dec 8, 2015 7:52:44 GMT
Welcome, suguess. You are doing well so far.
Your Facets specification file looks good. Here are some suggestions: i) There is no need to define each element twice. For instance, * 3, Criteria 13 ; 3 rating criteria 1, content 2, organization 3, language * is the same as * 3, Criteria 1, content 2, organization 3, language * It is convenient to use * 3, Criteria 13 ; 3 rating criteria * when there is no label for the individual elements.
ii) Facets expects integer, independent ratings. This data line: 1,1,13,8.5,8.5,9,8.65 ; first rating: examinee 1, rater 1, criteria of 8.5,8.5,9, holistic rating of 8.65 has decimal ratings, and the holistic rating is not independent of the analytic ratings. It is a weighted sum.
Let's start with the analytic ratings 1,1,13,8.5,8.5,9 these are not integers. There are two ways to solve this problem:
A. double all the ratings: Model = ?,?,?,R20 ; observations are ratings in R20. * Data= 1,1,13,17, 17, 18 ....
or B. halfweight the data, and enter twice: Data= R0.5, 1,1,13,8,8,9 R0.5, 1,1,13,9,9,9 ....
Since your rating scale is already long, 010, my recommendation is B. If every line in the data file will be entered twice, then use: Model = ?,?,?,R10, 0.5 ; observations are ratings in R10, weighted 0.5 * Data= 1,1,13,8,8,9 ; these lines are weighted 0.5 in the Models= specification. 1,1,13,9,9,9 ....
iii) Holistic rating. This is not independent of the analytic so including it in the Facets analysis does not make sense from a measurement perspective, but let's pretend it does: The specifications would start as Labels= ..... * 3, Criteria 1, content 2, organization 3, language 4, holistic * Data= 1,1,4, 8.65 1,2,4, 8.5 1,3,4, 8.15
but we need integers, so it is weighting again Data= R0.35, 1,1,4, 8 ; two lines = 8.65 R0.65, 1,1,4, 9 R0.5, 1,2,4, 8 ; two lines = 8.5 R0.5, 1,2,4, 9 R0.85, 1,1,4, 8 ; two lines = 8.15 R0.15, 1,1,4, 9
Since the analytic item is idiosyncratic, I recommend a separate rating scale and a separate Models= specification: Model = ?,?,13,R10, 0.5 ; analytic observations are ratings in R10, weighted 0.5, sharing the same rating scale ?,?,4,R10 ; holistic observations are ratings in R10, weighted in the data, with its own rating scale * Data= 1,1,13,8,8,9 ; these lines are weighted 0.5 in the Models= specification. 1,1,13,9,9,9 R0.35, 1,1,4, 8 ; two lines = 8.65 R0.65, 1,1,4, 9
1,2,13,8,8,9 1,2,13,9,8,9 R0.5, 1,2,4, 8 ; two lines = 8.5 R0.5, 1,2,4, 9 .....



Post by suguess on Dec 8, 2015 8:51:09 GMT
Hi, Dr. Linacre. Thanks a lot for your prompt reply. That's very helpful. I see how to do weighting. Now I am trying the method B. Thank you so much!



Post by bensmith on Dec 21, 2016 11:05:12 GMT
I stopped by to thank you for such an intriguing post! Also, I have an idea about how to construct a text, essay, article, or any other written piece  here is the special tool to be utilized:

