Objective　To compare the scores from the simplified comfort scale with those from the bispectral index (BIS) and analyze the variation among different evaluators in children in an intensive care unit, and to investigate the reliability; validity of simplified comfort scale. Methods Twenty children between 3 and 16 years of ages on mechanical ventilation and sedation were simultaneously classified based on the BIS and the simplified comfort scale evaluated by 4 independent evaluators. The Kappa test was performed and the correlation between the two methods (Pearson correlation) was tested. Results In total, 70 observations were performed on 20 patients. Based on the Kappa coefficient, the agreement among evaluators ranged from 0.62 to 0.77 (p<0.001). There was a correlation between the BIS and the simplified Comfort scale (r=0.53-0.61，p<0.01). Conclusions Due to the strong consistency between the independent evaluators and the correlation between the two methods, the reliability and validity of simplified comfort scale are preferable and simplified comfort scale is useful in classifying the level of sedation in children on mechanical ventilation.