By Dr. Patricia L. Roberts, Chair
Following the success of its June 2011 pilot study, the Society’s Operative Competency Evaluation Committee recently completed a larger technical skills assessment examination, COSATS (Colorectal Objective Structure Assessment of Technical Skill), in combination with the oral board examination of the American Board of Colon and Rectal Surgery.
The COSATS was modeled on the OSATS examination and was developed to assess technical skills at the time of board certification. The examination consists of eight stations of models relative to colon and rectal surgery such as ileoanal J-pouch, colonoscopy and laparoscopic mobilization of the colon.
Thirty-seven volunteers who had completed a colon and rectal residency were evaluated on the COSATS stations in September 2012 at the Northwestern Skills Center in Chicago. Expert colorectal surgeons evaluated performance using both a task specific checklist and global rating scale.
Preliminary results indicate the portability of the COSATS as well as the feasibility of administering a large scale technical skills exam in a high-stakes setting. Of the 37 candidates, our initial analysis indicates that 34 candidates passed and 3 failed (for a failure rate of 8%). Final analysis of the data is currently underway.
Our initial pilot study, performed at the University of Toronto skills lab in June 2011, demonstrated that the COSATS has good reliability and evidence of validity with the ability to discriminate between the performance of a graduating colorectal residents and graduating general surgery residents. Until now, written and oral examinations and other tools have been used to assess a trainee’s cognitive knowledge, judgment, and professionalism.
The COSATS will be the first test of its kind assessing subspecialty surgical skills. If implemented as a component of the American Board of Colon and Rectal Surgery certification examination, it would also be the first test to be incorporated into subspecialty certification.
In addition to Dr. Roberts, members of the Operative Competency Evaluation Committee are Vice Chair Dr. Ann Lowry, St. Paul, MN, Advisor Dr. Richard Reznick, Kingston, ON, Canada, Council Rep Dr. Steven Wexner, Weston, FL, and Drs. Glenn Ault, Los Angeles, CA, Jennifer Beaty, Omaha, NE, Marcus Burnstein, Toronto, ON, Canada, Peter Cataldo, Burlington, VT, Bradley Champagne, Cleveland, OH, Hueylan Chern, San Francisco, CA, Bradley Davis, Cincinnati, OH, Conor Delaney, Cleveland, OH, Sandra de Montbrun, Toronto, ON, Canada, Eric Dozois, Rochester, MN, Gary Dunn, Oklahoma City, OK, James Fleshman, Dallas, TX, Rebecca Hoedema, Grand Rapids, MI, Gerald Isenberg, Philadelphia, PA, Helen MacRae, Toronto, ON, Canada, Najjia Mahmoud, Philadelphia, PA, Peter Marcello, Burlington, MA, Jorge Marcet, Tampa, FL, Michelle Olson, Danville, PA, Jan Rakinic, Springfield, IL, Craig Reickert, Detroit, MI, David Schoetz, Jr. (ex-officio), Burlington, MA, Scott Steele, Fort Lewis, WA, Scott Strong, Cleveland, OH, Judith Trudel, St. Paul, MN, and Eric Weiss, Weston, FL.