Abstract:
Analysis of students’ response and response-time data availed test-developers, pyschometricians and researchers the opportunity toimpartially measureexaminees’learning outcomes with the advent of Item Response Theory (IRT). Records showed that the usage of 1-, 2- and 3-Parameter Logistic (PL) models inthe calibration and estimationof examinees’ ability and item parameters had actually enhanced students’ accurate estimates of their academic performances. However, the newly invented4-PL and Lognormal Response-time (LNIRT) models with their inherent advantages have not been sufficientlyexplored, whereas, the capacity they have to eliminate biases make them stand out. This study, therefore, was designed to explore the applicability of 4-PL and LNIRTmodels in calibratingComputer-Based Mathematics Achievement Test (CBMAT) among senior secondary school (SSS) students in Oyo and Lagos States, Nigeria.
Instrumentation design wasadopted and the study, hinged on IRT approach, was carried out in two phases. Fourteen senior secondary schools in Oyo State, having functional computers were purposively selected for Phase I, which involvedconstruction, validation and calibration of pooled 114-item CBMAT. Stratified sampling in proportion to number of available computers was used to select 731 SSS II students. Forty-item CBMAT with marginal reliability of 0.89 scaled through validation process. Phase II entailed a purposive selection of Lagos State based on the availability of large numbers of functional computers in Agege, Ifako/Ijaye and Alimosho Local Government Areas. Three schools each in Agege and Ifako/Ijaye and two from Alimosho were selected. In each of the eightschools, CBMAT was administered to 874 examinees (in two batches). Data wereanalysed with Dimtest statistic, Yen Q3 test, IRT Logistic Models, LNIRTmodel and Pearson product moment correlation at α = 0.05.
Both pooled and final CBMAT revealed that only mathematics ability trait is dominant in the Dimtest results (unidimensionality) (T=1.028; T=0.06).Two pairs of items were locally dependent (Yen Q3values = 0.38, 0.31;both were >|0.2|).Model-fit analysis indicated that the test data found a better fit with 4-PL model (-2loglikelihood=97274, AIC=97282 and BIC=97293). Comparisons of parameter estimates among 1-, 2-, 3- and 4PL models were significant (discrimination; T=122.68; difficulty: T=24.45; guessing: T=2.09; ability estimate=16.89)although, estimates of 4-PL model performed better. Also, time-intensity in the LNIRT model estimated an approximation of 60minutes to correctly respond to the final CBMAT. The observed response time showed that 4% of the examinees exhibited aberrance responses. There is a negatively low relationship between examinee parameters (r_θζ=-0.06) and moderate and significant correlations existed among item parameter estimates of the LNIRT model(r12=0.39, r34=0.48). LNIRT model produced better examinees’ ability estimates (x ̅=0.0015) when compared to that of the conventional IRT models (x ̅=0.0003).
Calibrating with 4-parameter logistic in the unidimensional category and lognormal response-time models were effective in estimating examinees’ mathematics ability in Oyo and Lagos States. Test-developers are encouraged to use lognormal model for a more objective measurement of examinees’ ability.
