Disease diagnosis is a tricky business. “Pathologists do an incredible job given the enormity of their task. They are in many cases looking for a needle in a very large haystack, hundreds of times a day,” explained Simon Thomas, a UQ Masters of Bioinformatics student and commercial skin cancer pathology laboratory scientist.
However, when analysing images, it’s not surprising that people can see different things. Mr Thomas described blind experiments in which pathologists were seen to give two different diagnoses for the same case. “Inter-observer variability is a well-known problem in disease diagnosis,” he said.
In his research, Mr Thomas aims to address this variability in skin cancer diagnosis by automating analysis of histology slides of skin cancer biopsies as an assistive technology to the pathologist.
He is working on creating machine learning methods to firstly classify and distinguish those images that have cancers in, and the type of skin cancer; and secondly to also use machine learning to segment and select the regions in the imaging that the cancer is within. These would then be useful in providing advice to the pathologist in deciding whether further surgery or excision may be required.
Convolution Neural Networks is the style of machine learning he is using. These are a relatively recent innovation in machine learning, are based on the structure of the visual cortex, and have recently been very successful in a number of machine vision problems, such as image classification.
For the work, Mr Thomas has imaged 300 histology slides of skin cancer biopsies, namely images of tissue that are representative of a cancer class. Skin cancer has three classes: Basal Cell Carcinoma (BCC), the most common skin cancer; Squamous Cell Carcinoma (SCC), the second most common skin cancer; and Intraepithelial Carcinoma (IEC), which is common enough and can be a precursor to SCC.
The next step is "training" the neural network, or “model” as Mr Thomas calls it, to classify images. This involves showing the network images to classify that you already know the results of, and then altering the network to improve how good it is at the task. Once trained, you should be able to fully trust the network to classify images for you.
To do this work, Mr Thomas is harnessing the image-processing power of UQ’s new supercomputer, Wiener, containing the most powerful Graphics Processing Units (GPUs) ever made, and the Tensorflow machine learning framework installed on it.
“Wiener makes training of a model extremely easy and efficient. You can get results within minutes rather than weeks thanks to the GPU architecture. I can train a model using 30,000 images in about 15 minutes. Deep learning is very much more an art than a science, so being able to train and test your models quickly, greatly improves the time in which you arrive at the best model. Further to that, there are multiple nodes on Wiener, and so it is possible to train several models simultaneously.
“The truth of the matter is that research in this field is really only possible because of the ability to use GPUs to train the networks to classify the imaging. In the absence of meaningful compute power, these problems are intractable”.
“Tensorflow is the basic framework for building artificial neural networks. In practice, I am using another wrapper on top of Tensorflow called Keras, which makes the construction of networks very fast and flexible. Wiener is now set up to work with both of these frameworks.”
Dr Nick Hamilton, Mr Thomas’ research supervisor, and an RCC/QCIF eResearch Analyst in UQ’s Institute of Molecular Bioscience (IMB), said Mr Thomas’ research is “really exciting” and he is creating a “valuable data set”.
“Though preliminary, Simon's classification results are very promising and his current trained network is as good or slightly better than a human doing the classification. We are confident this can be significantly improved now that he has established the pipeline on Wiener,” said Dr Hamilton.
“Machine learning is an ideal problem for GPU supercomputing, and networks can be trained orders of magnitude faster using GPUs than CPUs [Core Processing Units]. So, using Wiener is critical to the project.”
Mr Thomas hopes to continue his research in this area as a UQ PhD student, with Dr Hamilton remaining as his supervisor.
“The PhD project will extend this work, aiming to improve the amount of useful information that can be extracted from images, which can be combined and presented as supportive evidence towards a diagnosis,” said Mr Thomas.
“The ultimate goal is to support pathologists in providing objective and consistent diagnoses, drawing their attention to areas of likely concern and reducing the number of false negatives.”
The skin cancer pathology lab, where Mr Thomas works, is very keen on the project, and is hopeful a tool they can use will come out of it.
The longer-term aim is to go beyond skin cancer and provide a diagnosis aid for all pathologists.
Update: October 2019
Simon Thomas wins travel grant to US pathology conference
UQ bioinformatics PhD student Simon Thomas received a travel grant from a major US pathology conference to present his skin cancer machine learning work there this month.
At the Digital Pathology Association’s conference in Orlando, Florida, 6–8 October, Simon showcased his work from the first 10 months of his PhD. “This work extends and improves upon the methods I developed during my Masters,” he said.
Simon was one of three travel award recipients for “Pathology Visions 2019”, and the only one from outside the US.
Recipients were partly chosen based on their understanding of the impact of digital pathology and whole slide imaging on the specialty of pathology.
This year, Simon has also shared his work to machine learning groups at UQ and at a Translational Research Institute seminar hosted by the Australian Skin and Skin Cancer Research Centre (ASSC). He has also showcased a poster of his work at three other national medical research events.
In his research, Simon is currently focused on interpretability methods.
“In the push to integrate machine learning in many sectors, there is a need for further research into interpretability methods. Instead of using ‘black boxes’, which has become a synonym for Deep Learning systems, we want to have (and in the case of healthcare in particular, are ethically mandated to have) systems which we understand. What is meant by ‘understand’ here is an open question, but generally it can be thought of as developing systems which provide us with an explanation of their internal model of the world,” said Simon.
“There are many researchers working on this multifaceted problem. In the case of classic image classification this could be having a network not just classify dogs and cats correctly but demonstrate ‘dog-ness’ and how it differs from ‘cat-ness’, and if those concepts bear any relation to our own concepts. If the network representations converge on our own representations, e.g. floppy ears for dogs, then in that sense, we understand (some of) the information a network uses to make decisions.
“Along similar lines, my approach has been to not just classify cancer images, but characterise the whole tissue the same way a human would, so that our representations converge. The result of doing this means that we can classify cancers with near perfect accuracy but also perform many other additional tasks, such as orientating a specimen, or automatically providing surgical margin clearance measurements, or drawing attention to specific parts of the tissue that may be of interest.
“These methods can augment the ability of a pathologist which will likely improve their efficiency and accuracy.
“Of course, all the heavy lifting was done with [HPC] Wiener.
“Having established many potential uses of these techniques, the essence of my PhD project is how can we adapt these methods to the scale and variability found in a real-world histopathology laboratory. This is an enormous challenge.”
Update: November 2020
HPC Wiener used in research for skin cancer paper
UQ bioinformatics PhD student Simon Thomas* had a paper published this month in the Medical Image Analysis journal.
Simon is using HPC Wiener in his research using machine learning to automate analysis of histology slides of skin cancer biopsies as an assistive technology to the pathologist.
RCC’s assistance to this work was acknowledged in the paper, which was published online on 21 November 2020 and will feature in the February 2021 issue of the journal.
The paper’s authors include Simon, James Lefevre (Institute for Molecular Bioscience), Dr Glenn Baxter (MyLab Pathology in Salisbury, Queensland) and RCC eResearch Analyst Dr Nick Hamilton, who is Simon’s supervisor at the IMB.
“That Simon's work has now appeared in a very high-impact international journal is a testament to the strength of his research, as well as the resources made available to researchers at UQ to support such research,” said Dr Nick.
“Machine learning and vision has grown in leaps and bounds in the last five years. However, most methods suffer from being ‘black boxes’ in that there is no way to know how the machine came to the conclusion that it did. But when life and death decisions are being made it’s essential to understand how the conclusion was arrived at.
“Simon's work aims to create machine learning methods to see the structure of the skin in cancer pathology imaging in the same way that a pathologist does, with some 12 different components such as epidermis, hair follicles and sweat glands. Hence a skin cancer is ‘seen’ in the context of the skin, much as a pathologist would see it. This can then be used to find cancers automatically or perform common pathologist tasks such as margin clearance to cancer.
“Training Simon’s machine learning models involved thousands of images and thousands of hours of computation. This was only possible because of the exceptional HPC resources at UQ such as the Wiener GPU supercomputer.”
*In 2018, Simon completed his Masters of Bioinformatics and last year became a UQ PhD candidate.
