Intraoperative augmented reality assistance for percutaneous nephrolithotomy—what evidence is emerging?

Introduction

Percutaneous nephrolithotomy (PCNL) is often considered the gold standard for the management of large nephrolithiasis greater than 2 cm, that would otherwise be unable to be treated (1). This process involves puncturing the affected kidney with an introducer trochar, thereby permitting the passage of instruments to clear the stones. This puncture is often achieved under fluoroscopic guidance and is widely considered the most difficult step of the procedure. Multiple punctures can be required to achieve satisfactory entry into the collecting system of the kidney. This can increase the risk of post operative complications, including bleeding, pain, damage to bowel and surrounding structures and damage to the kidney itself. The procedure is also difficult to teach, on account of two-dimensional information being fed back to the operator, when the entire pursuit is a three-dimensional affair. Therefore, there is scope for a system to be developed which can improve the ease of access into the collecting system of the kidney during PCNL.

The concept of augmented reality (AR)

AR has gained significant interest in its potential to act as an intraoperative adjunct. It is important to distinguish between AR and virtual reality, in that AR involves projection of data onto a physical object (thereby “augmenting” reality) whereas virtual reality refers to a fully simulated environment. Currently, there are two major platforms through which AR is displayed. One method relies on two-dimensional information displayed on a tablet or similar device. The second method relies on the use of holographic data projection in real time using an Optical See-Through Head-Mounted Display (OST-HMD). These devices allow information to be projected directly onto a transparent screen mounted directly before the user’s eyes. Therefore, it stands to reason that this technology can be used as an intraoperative adjunct to augment a surgeon’s intraoperative experience.

AR in the surgical literature

AR has been employed as an intraoperative adjunct in several medical fields. The current literature describes its use in the fields of orthopaedics, general surgery and neurosurgery (1,2). In urology, AR has been described in laparoscopic partial nephrectomy and robotic prostatectomy, whereby the operating surgeon has been able to use an OST-HMD to display AR information in real time upon the patient while operating. Therefore, this technology could be employed to replace the conventional fluoroscopic guidance that is used to guide collecting system entry.

Current experiences with AR in PCNL

The feasibility of AR in PCNL has been explored in the literature; however studies are largely limited to ex-vivo reports on animal models, such as that by Müller et al. They describe AR in assisting with PCNL entry into a porcine kidney model, demonstrating that collecting system punctures were more successful with AR assistance than ultrasound or fluoroscopy (3).

Similarly, Rassweiler et al. outline entry into the collecting system using an iPad-system augmenting AR data over intraoperative real-time imaging (4,5). By positioning the iPad over the patient, they were able to superimpose AR models of each patient’s kidney onto the intraoperative video captured by the iPad, thereby augmenting the surgical procedure. A matched pair analysis of 22 patients who underwent AR-enhanced collecting system punctures compared to 22 matched patients who had conventional PCNL demonstrated that puncture success depended significantly on the accuracy of the AR program. Their series showed no significant difference between both methods.

The most promising AR technology of note in the literature involve using an OST-HMD, as described by Porpiglia et al. (6). Their system consists of AR-augmented access into the kidney via the Microsoft HoloLens platform acting as an OST-HMD. A series comparing 10 patients who underwent an AR-enhanced PCNL and a retrospective series of 10 matched patients were selected for matched pair analysis. Porpiglia et al. successfully created high quality AR models of each patient’s renal system from pre-operative CT imaging. Emphasis was placed on highlighting important structures such as renal vasculature, the kidney itself, and underlying stone features. All 10 patients had successful puncture of the inferior calyx, and matched pair analysis showed a large reduction in reduced median radiation exposure time. Of note, there was a greater rate of successful first attempt at renal puncture in the AR cohort.

The negative aspects of AR

While AR has great scope to act as an intraoperative adjunct in multiple surgical fields, there are some recognized drawbacks that must be considered. The current literature describes displaying AR on two-dimensional devices such as an iPad. While this has been described to be a useful adjunct to assist with puncturing the collecting system in the literature, it makes use of fiducial markers applied onto bony landmarks, which are then used as a calibration mechanism to superimpose the AR models intraoperatively. This system requires the precise application of fiducial markers to ensure anatomical accuracy, which can differ from the time of the patient’s initial cross-sectional imaging and the time of surgical intervention. OST-HMD-based systems solve this problem by providing scalable models of the collecting system which can be intraoperatively superimposed in real time based on bony landmarks. However, such a system is markedly more expensive. Furthermore, both systems are unable to account for minute respiratory deformations that can be encountered intraoperatively as the patient is ventilated. This is an important consideration that must be factored when attempting to puncture into the patient’s collecting system. This error can be mitigated by holding the patient in full inspiration intraoperatively when attempting to puncture the collecting system, to closely mimic the conditions in which a patient would have had their pre-operative cross-sectional imaging. It is also worth noting that the literature also suggests that enhanced navigation in the form of AR can narrow a surgeon’s focus, reducing attention which can lead to an increased complication risk (7). This is certainly an important consideration, especially for trainees and surgeons inexperienced in AR augmentation.

The future of AR in PCNL

AR technology is an exciting adjunct to surgery and appears to be of assistance in aiding puncture at time of PCNL. While there are several techniques that incorporate AR models as intraoperative adjuncts, OST-HMD based systems are the most intuitive and integrate well with existing workflow. However, given the relative infancy of this technology and the AR process itself, there is a significant paucity of literature around the feasibility of AR-based systems. The studies discussed in this article all have small sample sizes and there are no studies with large sample sizes comparing AR-guided PCNL entry against conventional fluoroscopy. As such, further investigation is required, to not only determine the optimal method of displaying AR information for the benefit of the operating surgeon, but also to compare outcomes of AR-guided PCNL against conventional fluoroscopic-guided PCNL. At the time of authoring this article, a trial of an OST-HMD system based upon the Microsoft HoloLens platform is underway at the Toowoomba Base Hospital in Queensland, Australia (ACTRN12622000593730).

Acknowledgments

Funding: None.

Footnote

Provenance and Peer Review: This article was a standard submission to the journal. The article did not undergo external peer review.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://amj.amegroups.com/article/view/10.21037/amj-22-49/coif). DD serves as an unpaid editorial board member of AME Medical Journal from July 2022 to June 2024. All authors are co-recipients of AUD $23,952.40 grant from the Toowoomba Hospital Foundation as part of the “augmented reality in percutaneous nephrolithotomy” trial currently being run at the Toowoomba Base Hospital, Toowoomba, Queensland, Australia, 4350. DD also receives consulting fees for Private Urology practice and is an advisor for Cook Medical, Boston Scientific and Bard Medical. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Morris DS, Wei JT, Taub DA, et al. Temporal trends in the use of percutaneous nephrolithotomy. J Urol 2006;175:1731-6. [Crossref] [PubMed]
2. Condino S, Montemurro N, Cattari N, et al. Evaluation of a Wearable AR Platform for Guiding Complex Craniotomies in Neurosurgery. Ann Biomed Eng 2021;49:2590-605. [Crossref] [PubMed]
3. Müller M, Rassweiler MC, Klein J, et al. Mobile augmented reality for computer-assisted percutaneous nephrolithotomy. Int J Comput Assist Radiol Surg 2013;8:663-75. [Crossref] [PubMed]
4. Rassweiler-Seyfried MC, Rassweiler JJ, Weiss C, et al. iPad-assisted percutaneous nephrolithotomy (PCNL): a matched pair analysis compared to standard PCNL. World J Urol 2020;38:447-53. [Crossref] [PubMed]
5. Rassweiler JJ, Müller M, Fangerau M, et al. iPad-assisted percutaneous access to the kidney using marker-based navigation: initial clinical experience. Eur Urol 2012;61:628-31. [Crossref] [PubMed]
6. Porpiglia F, Checcucci E, Amparore D, et al. Percutaneous Kidney Puncture with Three-dimensional Mixed-reality Hologram Guidance: From Preoperative Planning to Intraoperative Navigation. Eur Urol 2022;81:588-97. [Crossref] [PubMed]
7. Dixon BJ, Daly MJ, Chan H, et al. Surgeons blinded by enhanced navigation: the effect of augmented reality on attention. Surg Endosc 2013;27:454-61. [Crossref] [PubMed]