I have been tasked by my advisor to compare RSM and PCM for a particular type of language test we are trying to develop. I am re-analyzing data from eight different language tests and comparing (as per your suggestions) item design, predictive validity, fit (model and person/item), size of the data set, unobserved categories, etc. However, the sample sizes for all language tests are small, and our items are a bit unorthodox. Therefore, I would like to run simulations on the data to help support claims I might make and offset our small-sample state of affairs.
I can create a simulated data file and re-analyze it just fine, but how can I do this for 100 data files? If I use SINUMBER=100 and want to re-analyze them, is there a way to do this quickly (i.e., instead of one by one)?
Two quick follow-up questions to my earlier posting:
I was able to get the simulation working to analyze the simulated data sets from an initial analysis of some language tests. Although the language tests have small sample sizes, the test developers had some fairly solid test-development and administration procedures. I thought bootstrapping techniques would be a wise decision here given that we are satisfied with the data (though, correct me if this is horrible thinking).
(1) I want to make sure I am entering specifications in the right places of the data files during the batch routine to ensure that data are (a) simulated from the data and (b) persons are resampled. In the original data file (BANG.txt), I am entering the following specification: SIMEASURE=No SIRESAMPLE=1. Will these specifications cause each simulated data set that is analyzed during the batch routine to be drawn from the original data set and resampled persons? I guess my question is: Am I putting these specifications in the right place?
(2) What is the best way to extract the crucial statistics from the simulated data sets after the batch routine? I entered TFILE=* 3.1 * into a line in the .bat file to output these tables, but is there a quick way to grab the outputted data? My colleague and I are trying to wrap our heads around some commands/coding to help to this end, but neither of us is terribly savvy in this department. By trial and error, we are working a few things out, though.
As for (2), the approach my colleague and I are taking is to (a) combine files using a command prompt (Copy file*.txt combinedfile.txt) and (b) exporting data to Excel and moving/deleting undesired rows and columns. If there is an easier way to do this (i.e., a way that most people in Comp Sci 101 know that we don't), we would appreciate the tips!