The purpose of this study was to examine the effect of language simplification on pupils’ performance on the Primary School Leaving Certificate Examination (PSLCE) Mathematics word problems. The study was conducted in view of observations made regarding PSLCE pupils’ poor performance in Mathematics in general and their differential performance on mechanical and word problems in particular. It sought to find out changes in difficulty levels of test items after simplifying their language, differences in pupils’ performance on the two test forms and pupils’ perceptions on the differences in the language of the two test forms. The study made use of both quantitative and qualitative data obtained from a test and an interview conducted on equivalent groups of a sample of Standard 8 pupils drawn from six schools from the South East Education Division (SEED) using a two-stage cluster sampling procedure. It was found that pupils who took the simplified version of the test performed significantly better than those who sat for the original language version. The differences were statistically significant in 6 of the twelve items that became less difficult. Pupils who were interviewed observed that items in the simplified version were easier as they had shorter and more straight-forward sentences in plain language. The findings of this study, therefore, support the view that complexity of language in word items is a critical factor in influencing examinees’ performance more especially in circumstances where the language of the test is not the examinee’s mother tongue. Hence, great caution on language should be taken during setting and moderation of word items in Malawi, where almost all examinees have English as their second or third language.