Piloting IBM Watson Oncology within Memorial Sloan Kettering’s regional network.

Health Services Research
Session Type and Session Title: 
This abstract will not be presented at the 2014 ASCO Annual Meeting but has been published in conjunction with the meeting.
Abstract Number: 
J Clin Oncol 32, 2014 (suppl; abstr e17653)
Marjorie Glass Zauderer, Ayca Gucalp, Andrew S. Epstein, Andrew David Seidman, Aryeh Caroline, Svetlana Granovsky, Julia Fu, Jeffrey Keesing, Scott Lewis, Heather Co, John Petri, Mark Megerian, Thomas Eggebraaten, Peter Bach, Mark G. Kris; Memorial Sloan Kettering Cancer Center, New York, NY; IBM, Toronto, ON, Canada; IBM, Rochester, MN

Abstracts that were granted an exception in accordance with ASCO's Conflict of Interest Policy are designated with a caret symbol (^).

Abstract Disclosures


Background: We have reported on efforts to develop IBM Watson Oncology, a cognitive computing application designed to inform clinical decision making. The initial prototype demonstrated results in lung and breast cancers, and addressed some complexities of natural language processing (Bach, Proc ASCO 2013). Here, we report our experience with a pilot testing program with staff at Memorial Sloan Kettering’s (MSK) regional network who are not part of the development and training team. Methods: Medical oncologists from MSK’s regional network were invited to evaluate the application. These physicians manage patients with many primary types of cancer. We gave login credentials for the internet-based application and asked each doctor to use the program designed to assist in choosing initial therapies for breast and colorectal cancers. Members of the development team, including physicians, analysts, engineers, and usability experts interviewed these pilot users to gather feedback. Data from each user’s selection of treatment options was collected, including assessments of accuracy and quality. Users could report issues directly through an embedded tool. Results: To date, six users have independently logged into the pilot, created and submitted cases, and evaluated the application’s output and functionality. Users recognized value in selecting appropriate treatment options, but noted opportunities to enhance the usability and utility of the pilot for physicians using the tool as part of their normal routines. Users commented that it took too long to enter a case because some of the approximately 20 clinical attributes were irrelevant and felt data should have been imported from existing data sources. Users also requested the device provide more information on how treatments were scored and ranked. Conclusions: IBM Watson Oncology’s development team has created an initial pilot application. We plan to improve the application’s usability and utility by enhancing integration with electronic medical record systems, expanding users’ ability to iteratively add new clinical attributes, and providing additional information on the factors influencing the system’s output. Testing by MSK medical oncologists and other organizations continues.