High tibial osteotomy (HTO) is a knee preservation procedure used to treat osteoarthritis of the knee. Identifying the hinge point, surgical point, and Fujisawa point in the patient’s knee X-ray before surgery is a critical task. The aim of this study was to propose an artificial intelligence-based method to effectively help surgeons select the location of these landmark points, which provides important reference for subsequent surgery.
We proposed PFDNet (parallel fusion decoding network), a novel convolutional neural network for key-point estimation of knee X-rays. PFDNet employs Res2Net for feature extraction in the network encoding phase and two partial decoders connected in parallel in the network decoding phase to finely aggregate the multiscale feature information produced by Res2Net. A total of 1842 knee X-ray images were trained, validated and predicted by PFDNet to determine whether the network could accurately detect key-points in the HTO surgical plan.
At the hinge point, surgical point, and Fujisawa point, the average error and standard deviation from the calibration value in the PFDNet test results were 2.06 ± 1.165 mm, 2.713 ± 1.457 mm, and 2.015 ± 1.304 mm, respectively. This method exhibits superior performance compared with four convolutional neural network models that are also based on encoding and decoding frameworks: U-Net, ResUnet, SegNet, and FCN.
The hinge point, surgical point, and Fujisawa point can be clearly selected by PFDNet from knee X-ray images and is locked to the millimeter level. The results show that the proposed artificial intelligence-based strategy can be instrumental in preoperative HTO planning.
To read this article in full you will need to make a payment
Purchase one-time access:Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
One-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:Subscribe to The Knee
Already a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
- Imagenet classification with deep convolutional neural networks.Commun ACM. 2017; 60: 84-90https://doi.org/10.1145/3065386
- Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learning-based approach.Sci Rep. 2018; 8: 1-10https://doi.org/10.1038/s41598-018-20132-7
- Convolutional networks for biomedical image segmentation.in: International Conference on Medical Image Computing and Computer-Assisted Intervention. 2015: 234-241https://doi.org/10.1007/978-3-319-24574-4_28
- Automatic brain tumor detec- tion and segmentation using u-net based fully convolutional networks.in: Annual Conference on Medical Image Understanding and Analysis. Springer, 2017: 506-517https://doi.org/10.1007/978-3-319-60964-5_44
- Covid-19: Automatic detection from X-ray images utilizing transfer learning with convolutional neural networks.Phys Eng Sci Med. 2020; 43: 635-640https://doi.org/10.1007/s13246-020-00865-4
- The effect of high tibial osteotomy on osteoarthritis of the knee. An arthroscopic study of 54 knee joints.Orthop Clin North Am. 1979; 10: 585-608https://doi.org/10.1016/S0030-5898(20)30753-7
- Comparative study of medial opening-wedge high tibial osteotomy using 2 different implants.Arthroscopy. 2013; 29: 1063-1071https://doi.org/10.1016/j.arthro. 2013.02.020
- Introducing prospective national registration of knee osteotomies. A report from the first year in Sweden.Int Orthop. 2015; 39: 1283-1288https://doi.org/10.1007/s00264-014-2621-6
- Surgical options for patients with osteoarthritis of the knee.Nat Rev Rheumatol. 2009; 5: 309-316https://doi.org/10.1038/nrrheum.2009.88
- Detailed planning algorithm for high- tibial osteotomy.in: Osteotomies around the Knee. AO Publishing, 2008: 40-41https://doi.org/10.1055/b-0034-9885
- High tibial osteotomy: A systematic review and current concept.Arch Bone Joint Surg. 2016; 4: 204https://doi.org/10.22038/abjs.2016.7149
- DeepPose: Human pose estimation via deep neural networks.in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2014: 1653-1660https://doi.org/10.1109/cvpr.2014.214
- Detecting anatomical landmarks from limited medical imaging data using two-stage task-oriented deep neural networks.IEEE Trans Image Process. 2017; 26: 4753-4764https://doi.org/10.1109/TIP.2017.2721106
- Joint training of a convolutional network and a graphical model for human pose estimation.Adv Neural Inf Process Syst. 2014; 27https://doi.org/10.48550/arXiv.1406.2984
Lin M, Chen Q, Yan S. Network in network, arXiv preprint arXiv:1312.4400. https://doi.org/10.48550/arXiv.1312.4400 (2013).
- Deep residual learning for image recog- nition.in: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778https://doi.org/10.1109/cvpr.016.90
- Res2net: A new multi-scale backbone architecture.IEEE Trans Pattern Anal Mach Intell. 2019; 43: 652-662https://doi.org/10.1109/TPAMI.2019.2938758
- Cascaded partial decoder for fast and accurate salient object detection.in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 3907-3916https://doi.org/10.1109/CVPR.2019.00403
- Receptive field block net for accurate and fast object detection.in: Proceedings of the European conference on computer vision (ECCV). 2018: 385-400https://doi.org/10.1007/978-3-030-01252-6_24
- Rethinking the inception architecture for computer vision.in: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 2818-2826https://doi.org/10.1109/CVPR.2016.308
- Road extraction by deep residual u-net.IEEE Geosci Remote Sens Lett. 2018; 15: 749-753https://doi.org/10.1109/LGRS.2018.2802944
- Segnet: A deep convolutional encoder–decoder architecture for image segmentation.IEEE Trans Pattern Anal Mach Intell. 2017; 39: 2481-2495https://doi.org/10.1109/TPAMI.2016.2644615
- Fully convolutional networks for semantic segmentation.IEEE Trans Pattern Anal Mach Intell. 2016; 39: 640-651https://doi.org/10.1109/TPAMI.2016.2572683
- Integrating spatial configuration into heatmap regression based CNNs for landmark localization.Med Image Anal. 2019; 54: 207-219https://doi.org/10.1016/j.media. 2019.03.007
- Artificial intelligence for automatic measurement of sagittal vertical axis using resunet framework.J Clin Med. 2019; 8: 1826https://doi.org/10.3390/jcm8111826
- Faster R-CNN: Towards real-time object detection with region proposal networks.IEEE Trans Pattern Anal Mach Intell. 2017; 39: 1137-1149https://doi.org/10.1109/tpami.2016.2577031
- Cephanet: An improved faster R-CNN for cephalometric landmark detection.in: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE, 2019: 868-871https://doi.org/10.1109/ISBI.2019.8759437
- Deep learning-based regression and classification for automatic landmark localization in medical images.IEEE Trans Med Imag. 2020; 39: 4011-4022https://doi.org/10.1109/TMI.2020.3009002
- Automated cephalometric landmark detection with confidence regions using Bayesian convolutional neural networks.BMC Oral Health. 2020; 20: 1-10https://doi.org/10.1186/s12903-020-01256-7
Published online: December 16, 2022
Accepted: November 30, 2022
Received in revised form: November 8, 2022
Received: June 13, 2022
© 2022 Elsevier B.V. All rights reserved.