Prokop-Piotrkowska M et al.

Methods of Bone Age Assessment

REVIEW DO­I: 10.4274/jcrpe.galenos.2020.2020.0091

J Clin Res Pediatr Endocrinol 2021;13(3):251-262

Traditional and New Methods of Bone Age Assessment-An Overview

Monika Prokop-Piotrkowska1, Kamila Marszałek-Dziuba1, Elżbieta Moszczyńska1, Mieczysław Szalecki2,
Elżbieta Jurkiewicz3

1Children’s Memorial Health Institute, Department of Endocrinology and Diabetology, Warsaw, Poland
2Jan Kochanowski University, Collegium Medicum, Kielce, Poland
3Children’s Memorial Health Institute, Department of Diagnostic Imaging, Warsaw, Poland

Abstract
Bone age is one of biological indicators of maturity used in clinical practice and it is a very important parameter of a child’s assessment,
especially in paediatric endocrinology. The most widely used method of bone age assessment is by performing a hand and wrist radiograph
and its analysis with Greulich-Pyle or Tanner-Whitehouse atlases, although it has been about 60 years since they were published. Due to
the progress in the area of Computer-Aided Diagnosis and application of artificial intelligence in medicine, lately, numerous programs
for automatic bone age assessment have been created. Most of them have been verified in clinical studies in comparison to traditional
methods, showing good precision while eliminating inter- and intra-rater variability and significantly reducing the time of assessment.
Additionally, there are available methods for assessment of bone age which avoid X-ray exposure, using modalities such as ultrasound
or magnetic resonance imaging.
Keywords: Maturation, children, radiographs, deep learning, neural networks

Introduction Age at menarche is a solid biological indicator of maturity, but

it is a one-off event and relates to only half of the population
Maturation Indicators (3). Dentists, mainly orthodontists, use dental age judged
The processes of growth and maturation in children are using the Demirjian or Willems scale in daily practice but
usually correlated, but they cannot be treated as one this practice has not been established as a reliable tool for
process as they may not be linear and may proceed at other clinicians (3,4,5). Sexual characteristics, such as that
different paces. Due to numerous disturbances, such as made by assessment of position on the Tanner scale, are
growth hormone (GH) deficiency, deficiency of thyroid useful only in the adolescent period and are very subjective.
hormones or delayed puberty, but also sometimes in The only biological indicator of maturity, which is available
healthy children, the chronological age (CA) doesn’t match from birth to adulthood, is bone age (BA) (3).
the biological age. This is because they are regulated by
various factors, which include genes and nutrition, but Bone Age
also include many hormones, including GH, insulin-like In paediatric endocrinology, BA is an important tool used in
growth factor-1, sex hormones and adrenal steroids such as the clinical assessment of patients, mainly those suffering
cortisol, dehydroepiandrosterone, and testosterone (1,2). In from growth and puberty disorders. Many parameters
paediatric endocrinology, it is especially important to assess correlate better with BA than with CA including height
the child’s growth and puberty in relation to biological age, velocity, menarche, muscle mass and bone mineral mass
rather than CA. Thus, clinicians have been looking for a (6). Delayed BA is typical for GH deficiency, constitutional
good marker of maturation rate in children for decades (3). delay of growth, hypothyroidism, malnutrition and chronic

Address for Correspondence: Monika Prokop-Piotrkowska MD, Children’s Memorial Health Institute, Conflict of interest: None declared
Department of Endocrinology and Diabetology, Warsaw, Poland Received: 08.05.2020
Phone:+48 608 523 869 E-mail: [email protected] ORCID: orcid.org/0000-0003-3323-6784 Accepted: 15.10.2020

©Copyright 2021 by Turkish Pediatric Endocrinology and Diabetes Society

The Journal of Clinical Research in Pediatric Endocrinology published by Galenos Publishing House.

251
Prokop-Piotrkowska M et al. J Clin Res Pediatr Endocrinol
Methods of Bone Age Assessment 2021;13(3):251-262

illness (6,7). On the other hand, BA is advanced in many Greulich-Pyle Atlas

conditions that include precocious puberty and congenital ‘The Radiographic Atlas of Skeletal Development of the
adrenal hyperplasia, when there is a prolonged elevation Hand and Wrist’ by Greulich and Pyle (17) (GP) has been
of sex steroid levels (6,7,8). BA may be also marginally widely recognized and is used in many centers currently.
advanced in cases of overweight children, children with This atlas was created based on radiographs of hands of
tall stature or premature adrenarche (1,6,8). In genetic paediatric patients referred to endocrinologists William
overgrowth syndromes, for example Sotos syndrome, Walter Greulich and Sarah Idell Pyle by paediatricians
Beckwith-Wiedemann syndrome and Marshall-Smith between the years 1931-1942. These patients were
syndrome, BA is usually significantly advanced (6). In all Caucasian children from a generally upper middle class
cases it is important to remember that advancement or background, living in Cleveland, Ohio, United States (19,20).
delay of BA in relation to CA is a slow process, thus BA may This atlas consists of separate reference images for boys
not be altered in the case of examinations performed shortly and girls aged 0-18 (boys) or 0-19 years (girls) in various
after the first manifestations of a disorder and should be intervals (3 months-1 year). Images are accompanied by an
assessed in a temporal manner (7). explanation of the gradual age-related changes in the bones
What is more, BA is used in forensic and legal medicine at a given age and separate BAs calculated for each bone.
to estimate CA, for example in asylum seekers or Due to the natural variability of the BA of different bones
unaccompanied minors without documents. In such cases in one individual, in some bones, it is often more or less
an adequate assessment of age using precise methods is advanced than the standard it is intended to represent. For
crucial. The consequences of incorrect assessment of a example, a radiograph representing the age of 3 years 6
child as an adult may result in more restricted access to month (42 months) includes a 36-month first metacarpal
education, medical care or other forms of support provided and a 54-month lunate (17). BA is calculated by comparing
the non-dominant wrist radiographs of the subject with the
for children (9).
nearest matching reference radiographs provided in the
This article considers different methods of BA assessment atlas. Thus this method is termed a holistic method. Figure
from the perspective of a paediatrician or paediatric 1 presents GP atlas.
endocrinologist (Table 1).

Traditional Methods
Although there have been attempts to assess BA by
examinations of specific bones, such as the clavicle or iliac
bone (Risser sign) (10,11,12,13,14,15), in paediatrics and
paediatric endocrinology, the established way to obtain
BA is by performing a radiograph of the hand and wrist
of the non-dominant hand. Assessment of development
of the bones can be performed in the traditional, manual
way or using one of the automated methods. The manual
method involves a comparison of obtained radiograph with
radiographs in atlases. The manual methods can be divided
into two groups depending on the type of atlas – holistic or
analytic.
The first atlases were published shortly after the discovery of
X-rays in 1895. In 1898, John Poland published the first one:
“skiagraphic atlas showing the development of bones of the
wrist and hand” (16). In his atlas, he depicted skiagraphs
(positive reprints) of hand radiographs of 19 British children,
aged between 1 and 17 years, with an attached description
of each radiograph (16). However, the two most important
publications in this field were issued in 1959 by Greulich
and Pyle (17) and in 1962 by Tanner, Whitehouse and Healy
(18). Figure 1. Greulich-Pyle atlas

252
J Clin Res Pediatr Endocrinol Prokop-Piotrkowska M et al.
2021;13(3):251-262 Methods of Bone Age Assessment

GP is the most popular method among clinicians and the1950s and 1960s of British children coming from average
radiologists, as the assessment by GP is relatively quick socio-economic class (18). It was later updated in 1983 to
and easy to learn. Although widely used, this method has Tanner-Whitehouse 2 (TW2) and in 2001 the latest updated
significant drawbacks. BA assessment (BAA) using GP shows version was published - Tanner-Whitehouse 3 (TW3). These
high inter- and intra-observer variability. In addition given updates have attempted to adjust for the secular trends that
the reference population used in GP, this method may not be influence the relationship between the total bone maturity
an appropriate, universal tool for use in various populations. score and BA (30). In several countries standardized TW
BAA by GP is very subjective and the standard error on a single methods have been created which change the relationship
determination in inter-observer studies ranges from 0.45 to between the total maturity score and BA to make it suitable
0.83 years (21,22,23,24,25). There is no standardization for different ethnic groups (31,32,33).
in how the bones are weighted. Depending on a rater, in TW2 is an analytic or scoring method and it is based on the
clinical practice one may assign different weight to different maturity levels of 20 regions of interest (ROI) in different
bones, some raters may ignore the carpals and others may bones of the hand and wrist. The level of development of
assign even half weight to the carpals during the assessment. each ROI is labeled as a given stage, which is then converted
Raters using the carpals reduce their importance at higher to a numerical score. A total maturity score is calculated by
maturity but again not in a standardized manner (24). adding the scores of the ROIs and it is matched with the age
It has been reported that currently boys and girls develop of boys and girls separately.
secondary sex characteristics earlier than decades ago in
The TW method is considered to be more objective than the
United States (26,27). Thus current use of the GP atlas, even
holistic GP method and to also exhibit higher reproducibility
in a similar population to the original source, may not be as
than GP. Bull et al (21) reported that the intra-observer
precise as when it was created.
variation was greater using GP than TW (95% confidence
What is more, it has been proven that correlation of BA with interval, -2.46 to 2.18 vs -1.48 to 1.43, respectively).
CA, and consequently the applicability of GP, depends on However, assessment using the TW method is more time-
ethnic origin (28,29). According to a recent meta-analysis consuming. In a study performed by King et al (34) the
it has been proven that in African females, in comparison average time required for TW assessment was calculated
to GP standards, BA is significantly advanced. Conversely, as 7.9 min. vs. 1.4 min. in the case of GP assessment. In
in Asian males, BA is significantly delayed between 6 and this study the intra-observer variation between GP and TW
9 years of age and significantly advanced at 17 years (28). assessment was also found to be insignificant (the average
This should be taken into consideration while assessing BA spread of results was 0.74 years for TW and 0.96 years for
in these populations using the GP atlas. the GP). It should be noted that the sample size assessed by
There is an online version of GP uploaded by Brazilian King et al (34) was considerably smaller than that assessed
Instituto Mineiro de Endocrinologia (28). by Bull et al (21). A comparison of GP and TW methods is
presented in Table 1 (Table 2).
Tanner-Whitehouse Atlas
The second most popular tool for BA assessment is the Other Atlases
Tanner-Whitehouse atlas (TW). The first version of TW was The FELS method was developed in 1988 using 13,823
created in 1962 based on 2600 radiographs collected in serial radiographs of the left hand-wrist of boys and girls in
Table 1. Bone age assessment methods
Manual Automatic
Radiograph - Greulich-Pyle Atlas (17) - CASAS (55),
- Tanner-Whitehouse Atlas (30) - BoneXpert (71,72)
- FELS Method (36) - AI methods (97-109)
- Gilsanz and Ratib Atlas (37)
MRI - Pediatric Hand MR Scanner (45,46) - Method of Štern et al (51)
- Method of Tomei et al (48)
- Method of Hojreh et al (49)
USG - Femoral head cartilage thickness (44) - BonAge (40)
- Risser’s stage (45)
MRI: magnetic resonance imaging, USG: ultrasonography

253
Prokop-Piotrkowska M et al. J Clin Res Pediatr Endocrinol
Methods of Bone Age Assessment 2021;13(3):251-262

Table 2. Comparison of Greulich-Pyle and Tanner-Whitehouse methods

Atlas Greulich-Pyle Tanner-Whitehouse
Advantages - Widely recognised
- BAA relatively quick - Latest version from 2001
- Easy to learn - Higher reproducibility than Greulich-Pyle
Disadvantages - High intra- and inter-rater variability - BAA time consuming
- Not applicable to some populations
- One version since 1959
BAA: bone age assessment

the Fels Longitudinal Study performed by William Cameron the BA on the basis of this examination (19,40,41,42,43).
Chumlea, Alex F. Roche and David Thissen from two BonAge® measures the ossifying cartilage structures of the
universities in Kansas and Ohio, US (35). It is based upon wrist as an ultrasonic wave passes through the subject’s
maturity indicators that represent radiographic features distal radius and ulnar epiphysis. According to the producer,
that occur during the maturation of every child (35). The BonAge® provides on-the-spot, easy-to-read, immediate
set of maturity indicators is analysed with a computer results, without exposing children and adolescents to
program that provides the BA and the standard error for ionizing X-ray radiation, and moreover, it is objective and
that assessment (35). However, the FELS method has not safe (40). The time of the examination is approximately five
gained wide recognition. minutes although this can prove problematic in the smallest
children (41).
In 2005 a digital atlas created by Vicente Gilsanz and
Osman Ratib (GR) was published. It consists of artificially Several studies have been performed to assess the precision
created, idealised images of hands and wrists, specific for of this instrument. Mentzel et al (41) and Shimura et al (42)
age and sex. These images were produced by an analysis concluded that the results of BonAge® examinations correlate
of the size, shape, morphology and density of ossification closely with BA evaluated conventionally using the GP or
centres of 522 hand radiographs from healthy Caucasian TW2 method. However, in a more recent study performed
children from Los Angeles, US (50% girls and 50% boys). by Khan et al (43) on a bigger number of patients it was
Each image includes typical characteristics of development shown that BonAge® tended to over read delayed BA and
for each of the ossification centres (36). The images are of under read advanced BA and the authors concluded that
better quality and precision in comparison to GP. Another ultrasonographic assessment should not yet be considered
advantage is the regular spacing of the images at 6-monthly a valid replacement for radiographic BAA.
intervals from the ages of 2 to 6 years and yearly intervals There has also been a report of ultrasonographic assessment
from the age of 7 to 17 years (37). In one study the GR atlas of the thickness of anterior femoral head cartilage, which
was compared to GP and it was concluded that they were correlates strongly with the child’s CA and BA, standing
comparable in terms of precision. Yet again, however, the height and body weight, according to the authors of the
study was performed on a small number of examinations study (44). Ultrasonic examination of ossification of the
(38). iliac crest apophysis, (Risser’s sign), was also studied and it
presented with high accuracy, specificity and sensitivity in
Ultrasound Assessment
comparison to hand X-ray examination and GP assessment
Other imaging modalities, which have developed (45).
considerably over the years, now offer some advantages
Although the majority of the authors of these studies
over the ubiquitous radiograph for assessment of BA. One of
conclude that USG methods investigated are of good
these is ultrasound (USG), the major advantage of which is
accuracy in comparison to hand X-ray, USG-based BAA
that it does not expose the patient to any ionizing radiation,
is rarely used in daily practice. This may be because the
important when patients receive sequential assessment of
examination needs to be performed by a trained specialist
BA. Some studies have been performed to establish different
or there is a need for a specific device. In both cases, it
methods of BAA, including by performing USG (39).
takes more time to perform than an X-ray. Taking into
A result of one of these trials is BonAge® (Sunlight Medical Ltd, consideration that most studies investigating the utility
Tel Aviv, Israel) which consists of a device that performs an of USG in BAA were performed on relatively small groups
ultrasonographic examination and software that calculates of patients, the clinical utility of USG examination is as

254
J Clin Res Pediatr Endocrinol Prokop-Piotrkowska M et al.
2021;13(3):251-262 Methods of Bone Age Assessment

yet unproven. Isolation of the forearm allows for minimal BAA with MRI was feasible and showed good inter-observer
radiation exposure and the radiation during hand X-ray is reproducibility (48).
very low (0.0005 mSv). However, in the future, USG may be
In 2017 the results of another study were published
an advantageous method that may allow total elimination of
regarding the use of MRI in BAA. Hojreh et al (49) performed
children’s exposure to ionizing radiation during BAA.
hand MRI and X-ray examinations in 50 healthy volunteers
and 10 patients, all of whom were adolescents (aged 15±2
Magnetic Resonance Imaging Assessment
years and 13.5±2.6 years, respectively) and assessed both
The first research in the field of BAA using magnetic examinations according to GP criteria. This study concluded
resonance imaging (MRI) was performed in 2007 to find a
that the correlation between estimated patients’ ages on
tool suitable to establish the age of male football players
radiographs assessed by GP and MRI was high with the
without unnecessary radiation exposure (29). Since in
average estimated age difference between the MRIs and
some Asian and African countries registration at birth is
radiographs being −0.05/−0.175 years. However larger,
not compulsory, age determination is crucial to prevent
multicenter studies are necessary to confirm the usefulness
participation in the incorrect age group (29).
of this method. There have also been attempts to automate
In 2012 Terada et al (46) reported a technique for BAA the BAA using MRI instead of radiography (50,51). The
based on MRI examination. BA was determined using an comparison of RTG, USG and MRI methods is presented in
open, compact, newly designed MR imager optimized for Table 3.
evaluation of a child’s hand and wrist and it was scored by
two raters using the TW system adapted for the Japanese Automated Techniques
population. Evaluation of this method was performed on a Due to the problems associated with BAA when using
group of 93 healthy Japanese children and a strong positive traditional methods, such as inter- and intra-observer
correlation with BA and CA was demonstrated. What is variability and the fact that it is time-consuming, a need
more, the intra-and inter-rater reproducibility rates were emerged for new, objective tools that would provide
significantly high (46). Another study from the same authors immediate results. As Computer-Aided Diagnosis (CAD) has
was performed in 2014 to improve the performance of this emerged and has started to be used in clinical practice, one
method (47). This was conducted on a group of 88 healthy obvious procedure, which would be suitable for adaptation
children with three raters assessing BA and it confirmed to CAD was BAA, and BA was one of the first radiologic
the reliability and validity of this method (47). However, a examinations to be automated. This is not recent, however.
disadvantage of MRI is that it requires a relatively long time The first trials of CAD in BAA date back to 1989 when a
to be performed (2 min and 44 sec), therefore it may not be
semi-automated system called HANDX was introduced by
suitable for the youngest children, due to body movement.
Michael and Nelson (52). More recently, work on a system
Another study was performed by Tomei et al (48) and this which is based on assessment of phalangeal regions of
was published in 2014. They performed hand and wrist interest (PROI) was published by Pietka et al (53) in 1991.
MRIs on 179 healthy children aged 11-16 years old and In this method, the PROI were detected and the lengths of
analyzed the correlation with CA. It was concluded that the distal, middle, and proximal phalanx were measured
Table 3. Comparison of radiographic, ultrasonographic and magnetic resonance imaging-based methods
Method Radiograph Ultrasonography Magnetic resonance imaging
Advantages - The most frequently used - No X-ray exposure - No X-ray exposure
- Many recognised atlases - Accuracy validated in studies
- Easy to perform - There are attempts to automate
- Quick BAA using MRI
- Accessible
- Doesn’t require a radiologist to
perform, only to assess
- Automated methods available
Disadvantages - X-ray exposure - Presence of radiologist required to - Not easily accessible
perform - Relatively time consuming
- Time consuming (quicker than USG)
- Only few studies on its accuracy
BAA: bone age assessment, USG: ultrasonography, MRI: magnetic resonance imaging

255
Prokop-Piotrkowska M et al. J Clin Res Pediatr Endocrinol
Methods of Bone Age Assessment 2021;13(3):251-262

automatically. BA was estimated using the standard (100 patients) and with various other endocrinological
phalangeal length table, presented earlier by Garn et al (54). disturbances (514 patients) (75,76,77,78,79). Moreover,
there was a study conducted with 1100 healthy American
CASAS children from four different ethnic groups (Caucasian,
However, the first system to be used by different authors in African American, Asian and Hispanic) (22) and another
studies was CASAS - a computerized image analysis system on 515 eutrophic, overweight and obese children from
for estimating TW2 BA (55). This semi-automated system Brazil (80). Research into the validity of BoneXpert has also
was introduced by Tanner and Gibbons in 1994 and it used been performed in Asian populations, including a study on
the 13 bones of TW RUS system (radius, ulna and short 397 healthy children from Shanghai, China (81), in a large
bones) for BAA. These bones had to be located manually population of 6026 healthy children from five different
on the screen by a rater (correct positioning was assured by cities in China (82) and among Japanese children, using 185
computer templates of each bone stage) and then automatic radiographs from 22 healthy children and 284 radiographs
scoring was performed. Tanner and Gibbons (55) concluded from 22 patients diagnosed with GH deficiency (83).
that CASAS was more reliable and valid than manual TW
What is more, studies have confirmed the validity of BAA
RUS rating (56). Although other researchers have also
via BoneXpert in groups of children suffering from different
reported that CASAS was useful and reliable (57,58), this
disorders, including juvenile idiopathic arthritis (84), in
system has not been widely adopted. The major drawback
severely disabled children (85) and, as previously noted,
was that it took more time to estimate BA with CASAS than
children with short stature (76), precocious puberty (77)
a manual TW assessment. In addition, difficulties with BAA
in cases of abnormally shaped bones restricted the use of and congenital adrenal hyperplasia (78). All these studies
CASAS in some pathological conditions. conclude that BoneXpert is a suitable tool to perform BAA,
it is faster than traditional methods and eliminates rater
More recently there have been numerous approaches to variability. However, it should be noted that one of the
BAA automation (58-71) and the most important ones are authors of most of these studies is a person connected to
described below. the commercial activity of Visiana company, the producer
of BoneXpert.
BoneXpert
This automated tool for BAA was created in 2008 by the
Visiana company, based in Holte, Denmark (72,73,74). This
computer program analyses BA automatically, in several
steps. The first step is the definition of borders and intensity
of the radiologic image of 13 points of interest of the same
13 bones used in the TW RUS system, that is the radius,
ulna and 11 short bones. During this first step the system
also defines if the picture is complete and of appropriate
technical quality. In the next step, BA is assessed for each of
the 13 bones separately. The last step is the transformation
of the summary BA according to GP and TW criteria (72,73).
Figure 2 presents BAA by BoneXpert. BAA is available for
ages 2.5-19 years for boys and 2-18 years for girls (version
2.4.7.6.) (75). The data set used for the creation of this
program consisted of 1678 hand radiographs of healthy
Danish children and children from Belgium diagnosed with
a range of disorders, such as Turner syndrome (73).
To date several papers have been published that verify
the reliability and precision of BAA using BoneXpert in
comparison to GP in different populations (Table 4). In
European populations, studies have been conducted among
healthy children from the Netherlands (405 patients), German
children with short stature (1,097 patients), precocious or
early puberty (116 patients), congenital adrenal hyperplasia Figure 2. Bone age assessment by BoneXpert

256
J Clin Res Pediatr Endocrinol Prokop-Piotrkowska M et al.
2021;13(3):251-262 Methods of Bone Age Assessment

Table 4. Studies assessing the validity of BoneXpert vs. the Greulich-Pyle method
Study Population Validity
claimed
Author Year Size Origin Health status
Van Rijn et al (75) 2009 405 Netherlands Healthy Yes
Martin et al (76) 2008 1097 Germany Short stature Yes
Martin et al (77) 2011 116 Germany Precocious or early puberty Yes
Martin et al (78) 2013 100 Germany Congenital adrenal hyperplasia Yes
Booz et al (79) 2020 514 Germany Various endocrinological disturbances Yes
Thodberg and 2010 1100 American (4 ethnic groups) Healthy Yes
Sävendahl (22)
Artioli et al (80) 2019 515 Brasil Healthy, overweight and obese Yes
Zhang et al (81) 2016 397 Shanghai Healthy Yes
Zhang et al (82) 2013 6026 China Healthy Yes
Martin et al (83) 2010 44 Japan Healthy, deficiency of growth hormone Yes
Anink et al (84) 2014 69 Netherlands Juvenile idiopathic arthritis Yes
Mergler et al (85) 2016 95 Netherlands Severely disabled Yes

BoneXpert has several critical limitations. BA is not identified methods was weaker (89). The authors of these studies noted
directly, the prediction depends on the relationship between that a significant advantage of using BHI, in comparison to
CA, which is an input to the system, and BA (62). The system DXA or pQCT, was that radiation exposure was lower and in
is brittle and will reject radiographs when there is excessive low-risk peripheral areas. Also, BHI has already been used
noise, in one study it rejected 4.5% of individual bones (81). in research studies of BA in patients with juvenile idiopathic
Finally, until recently BoneXpert did not take the carpal
arthritis (89). There is an extension to BoneXpert, known
bones into consideration, although in younger children they
as digital X-ray radiogrammetry (DXR). DXR measures the
contain discriminative features. This has been changed in
the latest version - BoneXpert 3.0 released in September cortical bone thickness in the shafts of the metacarpals and
2019 - which now does include carpal bones in the analysis. has been shown to be effective in the assessment of hand
bone loss caused by rheumatoid arthritis (90).
An additional feature that BoneXpert offers is measurement
of a parameter called the Bone Health Index (BHI) (86), Another advantage of BoneXpert is a prediction of the final
which is a unique parameter. BHI is a measurement of height of a child (91,92), which is a vital element of clinical
bone mass counted as a function of cortical thickness of assessment of a child with short stature. Methods in current
three central metacarpals and their width and length. The routine use take into consideration BAA using traditional
program also automatically calculates standard deviation methods – GP or TW. The variability of these assessments
(SD) values for BHI, based on cohort data of Caucasian is the main reason for the variability of predicted final
children (86). There are several research studies on the height. When BAA derived from BoneXpert is used, it is
comparison of BHI values and traditional methods of bone
possible to predict final height in an objective, precise way.
mass measurement. In one study BHI was compared to dual-
This program takes into consideration sex, CA, height and
energy X-ray-absorption (DXA) and peripheral quantitative
BA of a child in order to predict their final height. One can
computed tomography (pQCT) in a cohort of paediatric
also add the height of parents and height at menarche to
patients from paediatric endocrine or paediatric oncology
outpatient clinics and it was concluded that BHI values obtain even more reliable outcome. It is also compulsory
showed a strong positive correlation with DXA readings to classify the child into one of nine population groups,
and total bone mineral density, as assessed via pQCT, also five within the Caucasian ethnicity, Asian Chinese, Asian
positively correlated with the BHI (87,88). In another study American, Hispanic and African American. The result of
on a group of patients with juvenile idiopathic arthritis, BHI these calculations is accompanied by an SD value and the
measured by BoneXpert was correlated to measurements true height values will be within the indicated range with
of bone mineral density by DXA, however, the correlation 68% probability (93). This method’s accuracy has been
of Z-scores of bone mineral density measured by the two validated in a clinical study (91).

257
Prokop-Piotrkowska M et al. J Clin Res Pediatr Endocrinol
Methods of Bone Age Assessment 2021;13(3):251-262

Artificial Intelligence and Machine Learning left-hand radiographs from evenly distributed, normally
New possibilities of automating BAA emerged with the developed children of Caucasian, Asian, African-American
use of artificial intelligence (AI) and machine learning, and Hispanic origin, both male and female, with an age
especially the specific type of machine learning known as range from 1 to 18 years. Spampinato et al (113) conclude
deep learning. The most popular use a convolutional neural that the best performance was observed with BoNet, which
network (CNN), which has already found application in areas was an original, new CNN trained from scratch specifically
such as detection of patterns of interstitial lung disease on to assess hand radiographs (114).
CT imaging (94) or segmenting the vascular network of the Another study in this area deserving attention, as it is
human eyes on fundus photographs (95). In recent years especially thorough and methods used have been precisely
there has been tremendous progress in this field and there described, concerns a system called the Fully Automated
have been numerous publications reporting the automation Deep Learning System for BAA, which was created in 2017
of BAA using CNN (96-108). by a group of researchers from Massachusetts General
In 2017 Radiological Society of North America (RSNA) Hospital, Harvard Medical School. They used a pre-trained,
conducted a challenge to assess BA from paediatric hand fine-tuned CNN to create a new tool for BAA, using a large
radiographs (RSNA Pediatric Bone Age Machine Learning number of hand radiographs that included 4278 for females
Challenge 2017), as part of efforts to spur the creation and 4047 for males but excluded children aged 0-4 years
of AI tools for radiology (109,110). The goal of the RSNA (115). This system calculates BA and provides a result as a
2017 Machine Learning Challenge was to develop an number with representative picture and presents four more
algorithm which can most accurately determine BA using pictures of BA +1, +2, -1, -2 years. Thus the radiologist
a validation set of paediatric hand radiographs. The results can verify the result and compare it with the closest ones.
were evaluated by determining the mean difference and the It achieved an accuracy of 57.32% and 61.4% for the
mean absolute difference (MAD) between the performance female and male cohorts on held-out test images. Female
of each system and the mean of all reviewers’ estimates. test radiographs were assigned a BAA within 1 year 90.39%
The company 16 Bit were placed first in the competition of the time and within 2 years 98.11% of the time. Male
with a MAD of 4.265 months and concordance correlation test radiographs were assigned 94.18% within 1 year and
coefficient of 0.991 (111). The training data set available for 99.00% within 2 years. It should be noted that this system
competitors contained 12612 images from two American does not reject malformed images (115). These authors also
hospitals with a minimum age of 1 month, maximum age compared the BAA performance of a cohort of paediatric
of 19 years and mean (SD) age of 10 years and 7 months (3 radiologists with and without the assistance of their tool for
years 6 months) (111). Their Paediatric Bone Age Calculator automatic BAA (116). They concluded that AI improves the
is freely available on the website 16Bit.ai, although it is radiologist’s performance for BAA by increasing accuracy
provided with the rider that the application is strictly for and decreasing variability and root mean squared error. The
demonstration purposes and should not be used for clinical best results were achieved when radiological assessment
decision making (111). However, this tool has already was assisted by AI and this was better than using AI alone, a
been validated by a group of Canadian researchers, who radiologist alone, or a pooled cohort of experts (116).
compared its results to BAA using the GP atlas in a group
A comparison of chosen AI methods and BoneXpert is
of 213 male and 213 female patients and found that the
presented in Table 5. Due to the small number of radiographs
differences between BA assessed by these two methods
in training and validating data sets, all the systems based on
was not statistically significant (median difference was
CNNs used data augmentation (increasing the number of
0.33 years) and concluded that the tool created by 16 Bit is
radiographs by rotating the pictures, adding noise, etc.). In
suitable for clinical use (112).
some studies authors tested more than one type of CNN.
Another attempt to automate BAA using CNN was described In these studies the CNN with the best performance is
in 2016 by Spampinato et al (113). They compared presented in the table.
performance of several approaches, ranging from existing,
off-the-shelf CNN, through existing pre-trained CNN (with
Conclusion
general imagery) and fine-tuned programs to custom,
trained from scratch only on BA radiographs (113). All For clinicians, especially paediatric endocrinologists, it is
of these CNNs were tested on the same, public data set, very important to assess BA as precisely as possible to be
the Digital Hand Atlas Database System, provided in 2007 able to make the right diagnosis and monitor closely the
by Gertych et al (114). This atlas includes 1391 digitized, development of a child, the progress of a disease or effects

258
J Clin Res Pediatr Endocrinol Prokop-Piotrkowska M et al.
2021;13(3):251-262 Methods of Bone Age Assessment

Table 5. Comparison of artificial intelligence methods and BoneXpert

Name of tool/ BoneXpert (72) Spampinato Bilbily and Cicero Lee et al (115) Van Steenkiste Liu et al (102)
author et al (113) (111) et al (107)
Year of creation/ 2008/2019 2016 2017 2017 2018 2019
last update
Method Conventional CNN BoNet CNN CNN CNN CNN
(non-deep) (pre-trained (pre-trained (pre-trained (pre-trained
Machine Learning Inception V3) GoogLeNet) VGGNet) VGGNet)
Input Radiograph, race, Radiograph, Radiograph and Radiograph Radiograph and Radiograph and
CA and gender race and gender and gender gender gender
gender
Data set (no. of 1 678* 1 391 12 611 8 325 12 611 1 391
radiographs) (Digital Hand (RSNA Challenge) (RSNA Challenge) (Digital Hand
Atlas) Atlas)
Age range (years) 2.5-19 for boys 0-18 1-19 5-18 1-19 0-18
2-18 for girls
Reported 4.5 (4th place in 9.6 4.265 (1st place in 11.16 (females) 6.8 8.28
accuracy RSNA) challenge) RSNA) challenge) / 9.84
(MAD in months) (males)**
*Validation on numerous groups of patients healthy and with various conditions and of various ethnic origin (Table 2).
**Result reported in RMSE (root mean square error) instead of MAD.
MAD: mean absolute difference, CNN: convolutional neural network, CA: chronological age, RSNA: Radiological Society of North America

of treatment. The traditional methods used to date have Financial Disclosure: The authors declared that this study
very significant drawbacks. These drawbacks include being received no financial support.
highly time consuming, having a high inter- and intra-rater
variability, making comparison of chronologically sequential
References
examinations of one patient difficult and the need to
1. Reinehr T, de Sousa G, Wabitsch M. Relationships of IGF-I and
possess a physical copy of the atlas. The new automated androgens to skeletal maturation in obese children and adolescents. J
BAA techniques provide instant results, eliminate inter- Pediatr Endocrinol Metab 2006;19:1133-1140.
and intra-rater variability and all only need access to the 2. Phillip M, Moran O, Lazar L. Growth without growth hormone. J Pediatr
software. Much research in this field is currently underway Endocrinol Metab 2002;15(Suppl 5):1267-1272.
and the results are very promising. Most of the programs 3. Cox LA. The biology of bone maturation and ageing. Acta Paediatr
described herein have been validated in clinical studies, in Suppl 1997;423:107-108.
comparison to traditional BAA and they show very good 4. Demirjian A, Goldstein H, Tanner JM. A new system of dental age
precision while possessing the benefits of automated BAA assessment. Hum Biol 1973;45:211-227.

systems. There are already some widely available options 5. Sehrawat JS, Singh M. Willems method of dental age estimation in
children: A systematic review and meta-analysis. J Forensic Leg Med
for clinical use, including BoneXpert and the Paediatric 2017;52:122-129. Epub 2017 Aug 25
Bone Age Calculator from 16Bit.ai. It is to be expected that
6. Martin DD, Wit JM, Hochberg Z, Sävendahl L, van Rijn RR, Fricke
these automated tools will continue to gain acceptability O, Cameron N, Caliebe J, Hertel T, Kiepe D, Albertsson-Wikland K,
and widespread usage, making the traditional atlas-based Thodberg HH, Binder G, Ranke MB. The use of bone age in clinical
BAA a thing of the past. practice - part 1. Horm Res Paediatr 2011;76:1-9. Epub 2011 Jun 21
7. Spadoni GL, Cianfarani S. Bone age assessment in the workup of
Ethics children with endocrine disorders. Horm Res Paediatr 2010;73:2-5.
8. Kim SE, Jang JW, Ahn MB, Kim SH, Cho WK, Cho KS, Park SH, Jung
Peer-review: Externally peer-reviewed.
MH, Suh BK. The association between skeletal maturation and adrenal
androgen levels in obese children and adolescents. Ann Pediatr
Authorship Contributions Endocrinol Metab 2017;22:108-114. Epub 2017 Jun 28
Concept: Elżbieta Jurkiewicz, Mieczysław Szalecki, Elżbieta 9. Menjívar C, Perreira KM. Undocumented and unaccompanied: children
Moszczyńska, Monika Prokop-Piotrkowska, Design: Elżbieta of migration in the European Union and the United States. J Ethn Migr
Stud 2019;45:197-217. Epub 2017 Dec 21
Jurkiewicz, Mieczysław Szalecki, Elżbieta Moszczyńska,
10. Kaur G, Khandelwal N, Jasuja OP. Computed tomographic studies
Monika Prokop-Piotrkowska, Literature Search: Monika
on ossification status of medial epiphysis of clavicle: Effect of
Prokop-Piotrkowska, Kamila Marszałek-Dziuba, Writing: slice thickness and dose distribution. J Indian Acad Forensic Med
Monika Prokop-Piotrkowska, Kamila Marszałek-Dziuba. 2010;32:298-302.

259
Prokop-Piotrkowska M et al. J Clin Res Pediatr Endocrinol
Methods of Bone Age Assessment 2021;13(3):251-262

11. Schmidt S, Mühler M, Schmeling A, Reisinger W, Schulz R. Magnetic 30. Assessment of skeletal maturity and prediction of adult height (TW3
resonance imaging of the clavicular ossification. Int J Legal Med method). Tanner JM, Healy M, Goldstein H, Cameron N. 3rd ed.
2007;121:321-324. Epub 2007 Apr 17 London, WB Saunders, Harcourt Publishers Ltd, 2001.
12. Hillewig E, De Tobel J, Cuche O, Vandemaele P, Piette M, Verstraete K. 31. Beunen G, Lefevre J, Ostyn M, Renson R, Simons J, Van Gerven D.
Magnetic resonance imaging of the medial extremity of the clavicle Skeletal maturity in Belgian youths assessed by the Tanner-Whitehouse
in forensic bone age determination: a new four-minute approach. Eur method (TW2). Ann Hum Biol 1990;17:355-376.
Radiol 2011;21:757-767. Epub 2010 Oct 3
32. Murata M. Japanese specific bone age standard on the TW2. Clin
13. Bitan FD, Veliskakis KP, Campbell BC. Differences in the Risser grading Pediatr Endocrinol 1993;(Suppl 3):35-41.
systems in the United States and France. Clin Orthop Relat Res
33. Tanner J, Oshman D, Bahhage F, Healy M. Tanner-Whitehouse bone age
2005;190-195.
reference values for North American children. J Pediatr 1997;131:34-
14. Wittschieber D, Vieth V, Domnick C, Pfeiffer H, Schmeling A. The 40. Erratum in: J Pediatr 2012;161:1180.
iliac crest in forensic age diagnostics: evaluation of the apophyseal
34. King DG, Steventon DM, O’Sullivan MP, Cook AM, Hornsby VP,
ossification in conventional radiography. Int J Legal Med 2013;127:473-
Jefferson IG, King PR. Reproducibility of bone ages when performed
479. Epub 2012 Oct 2
by radiology registrars: an audit of Tanner and Whitehouse II versus
15. Kreitner KF, Schweden FJ, Riepert T, Nafe B, Thelen M. Bone age Greulich and Pyle methods. Br J Radiol 1994;67:848-851.
determination based on the study of the medial extremity of the
clavicle. Eur Radiol 1998;8:1116-1122. 35. Chumela WC, Roche AF, Thissen D. The FELS method of assessing the
skeletal maturity of the hand-wrist. Am J Hum Biol 1989;1:175-183.
16. Skiagraphic atlas showing the development of bones of the wrist and
hand. Poland J. London Smith, Elder Co, 1898. 36. Hand Bone Age: A digital Atlas of Skeletal Maturity. Gilsanz V, Ratib O.
New York, Springer, 2005.
17. Greulich WW, Pyle S. Radiographic atlas of skeletal development of the
hand and wrist. Stanford, Stanford University Press, 1959. 37. Adler BH, Us I. Vicente Gilsanz, Osman Ratib: Bone age atlas. Pediatr
Radiol 2005;35:1035.
18. Tanner JM, Whitehouse RH, Healy M. A new system for estimating
skeletal maturity from the hand and wrist with standards derived from 38. Kaplowitz P, Srinivasan S, He J, McCarter R, Hayeri MR, Sze R.
a study of 2600 healthy british children. Paris, Centre International de Comparison of bone age readings by pediatric endocrinologists and
L’enfance, 1962. pediatric radiologists using two bone age atlases. Pediatr Radiol
2011;41:690-693. Epub 2010 Dec 16.
19. Satoh M. Bone age: assessment methods and clinical applications. Clin
Pediatr Endocrinol 2015;24:143-152. Epub 2015 Oct 24 39. Bilgili Y, Hizel S, Kara SA, Sanli C, Erdal HH, Altinok D. Accuracy of
skeletal age assessment in children from birth to 6 years of age with
20. Mughal AM, Hassan N, Ahmed A. Bone age assessment methods: A
the ultrasonographic version of the Greulich-Pyle atlas. J Ultrasound
critical review. Pakistan J Med Sci 2014;30:211-215.
Med 2003;22:683-690.
21. Bull RK, Edwards PD, Kemp PM, Fry S, Hughes IA. Bone age
40. Radiation-free Solution Measuring for Bone Age. Last Accessed date:
assessment: a large scale comparison of the Greulich and Pyle, and
29.07.2021. Available form: https://www.beammed.com/wp-content/
Tanner and Whitehouse (TW2) methods. Arch Dis Child 1999;81:172-
uploads/2017/05/boneage.pdf
173.
41. Mentzel HJ, Vilser C, Eulenstein M, Schwartz T, Vogt S, Böttcher J,
22. Thodberg HH, Sävendahl L. Validation and reference values of
Yaniv I, Tsoref L, Kauf E, Kaiser WA. Assessment of skeletal age at
automated bone age determination for four ethnicities. Acad Radiol
the wrist in children with a new ultrasound device. Pediatric Radiology
2010;17:1425-1432. Epub 2010 Aug 6
2005;35:429-433.
23. Johnson GF, Dorst JP, Kuhn JP, Roche AF, Dávila GH. Reliability of
skeletal age assessments. Am J Roentgenol Radium Ther Nucl Med 42. Shimura N, Koyama S, Arisaka O, Imataka M, Sato K, Matsuura M.
1973;118:320-327. Assessment of Measurement of Children’s Bone Age Ultrasonically
with Sunlight BonAge. Clinical Pediatric Endocrinology 2005;14(Suppl
24. Roche AF, Rohmann G, French NY, Davila H. Effect of training on 24):17-20.
replicability of assessments of skeletal maturity (Greulich-Pyle). Am J
Roentgenol Radium Ther Nucl Med 1970;108:511-515. 43. Khan KM, Miller BS, Hoggard E, Somani A, Sarafoglou K. Application
of ultrasound for bone age estimation in clinical practice. J Pediatr
25. Kim SY, Oh YJ, Shin JY, Rhie YJ, Lee KH. Comparison of the greulich- 2009;154:243-247. Epub 2008 Sep 27
pyle and tanner whitehouse (TW3) methods in bone age assessment. J
Korean Soc Pediatr Endocrinol 2008;13:50-55. 44. Castriota-Scanderbeg A, De Micheli V. Ultrasound of femoral head
cartilage: a new method of assessing bone age. Skeletal Radiol
26. Euling SY, Herman-Giddens ME, Lee PA, Selevan SG, Juul A, Sørensen
1995;24:197-200.
TI, Dunkel L, Himes JH, Teilmann G, Swan SH. Examination of US
puberty-timing data from 1940 to 1994 for secular trends: panel 45. Wagner UA, Diedrich V, Schmitt O. Determination of skeletal maturity
findings. Pediatrics 2008;121(Suppl 3):S172-91. by ultrasound: a preliminary report. Skeletal Radiol 1995;24:417-420.
27. Herman-Giddens ME, Steffes J, Harris D, Slora E, Hussey M, Dowshen 46. Terada Y, Kono S, Tamada D, Uchiumi T, Kose K, Miyagi R, Yamabe E,
SA, Wasserman R, Serwint JR, Smitherman L, Reiter EO. Secondary Yoshioka H. Skeletal age assessment in children using an open compact
sexual characteristics in boys: data from the Pediatric Research in MRI system. Magn Reson Med 2013;69:1697-1702. Epub 2012 Jul 31
Office Settings Network. Pediatrics 2012;130:e1058-68. Epub 2012 47. Terada Y, Kono S, Uchiumi T, Kose K, Miyagi R, Yamabe E, Fujinaga
Oct 20 Y, Yoshioka H. Improved reliability in skeletal age assessment using
28. Alshamrani K, Messina F, Offiah AC. Is the Greulich and Pyle atlas a pediatric hand MR scanner with a 0.3T permanent magnet. Magn
applicable to all ethnicities? A systematic review and meta-analysis. Reson Med Sci 2014;13:215-219. Epub 2014 Jul 2
Eur Radiol 2019;29:2910-2923. 48. Tomei E, Sartori A, Nissman D, Al Ansari N, Battisti S, Rubini A, Stagnitti
29. Dvorak J, George J, Junge A, Hodler J. Age determination by magnetic A, Martino M, Marini M, Barbato E, Semelka RC. Value of MRI of the
resonance imaging of the wrist in adolescent male football players. Br hand and the wrist in evaluation of bone age: preliminary results. J
J Sports Med 2007;41:45-52. Epub 2006 Oct 4 Magn Reson Imaging 2014;39:1198-1205.

260
J Clin Res Pediatr Endocrinol Prokop-Piotrkowska M et al.
2021;13(3):251-262 Methods of Bone Age Assessment

49. Hojreh A, Gamper J, Schmook MT, Weber M, Prayer D, Herold CJ, 68. Hsieh CW, Jong TL, Tiu CM. Bone age estimation based on phalanx
Noebauer-Huhmann IM. Hand MRI and the Greulich-Pyle atlas in information with fuzzy constrain of carpals. Med Biol Eng Comput
skeletal age estimation in adolescents. Skeletal Radiol 2018;47:963- 2007;45:283-295. Epub 2007 Jan 23
971. Epub 2018 Jan 25 69. Liu J, Qi J, Liu Z, Ning Q, Luo X. Automatic bone age assessment based
50. Ebner T, Stern D, Donner R, Bischof H, Urschler M. Towards automatic on intelligent algorithms and comparison with TW3 method. Comput
bone age estimation from MRI: localization of 3D anatomical Med Imaging Graph 2008;32:678-684.
landmarks. Med Image Comput Comput Assist Interv 2014;17:421-
70. Tristan-Vega A, Arribas JI. A radius and ulna TW3 bone age assessment
428.
system. IEEE Trans Biomed Eng 2008;55:1463-1476.
51. Štern D, Payer C, Urschler M. Automated age estimation from MRI
71. Hsieh CW, Liu TC, Jong TL, Tiu CM. A fuzzy-based growth model with
volumes of the hand. Med Image Anal 2019;58:101538. Epub 2019
principle component analysis selection for carpal bone-age assessment.
Jul 31
Med Biol Eng Comput 2010;48:579-588. Epub 2010 Apr 20
52. Michael DJ, Nelson AC. HANDX: a model-based system for automatic
72. Thodberg HH, Kreiborg S, Juul A, Pedersen KD. The BoneXpert method
segmentation of bones from digital hand radiographs. IEEE Trans Med
for automated determination of skeletal maturity. IEEE Trans Med
Imaging 1989;8:64-69.
Imaging 2009;28:52-66.
53. Pietka E, McNitt-Gray MF, Kuo ML, Huang HK. Computer-assisted
73. Thodberg HH. An Automated Method for Determination of Bone Age. J
phalangeal analysis in skeletal age assessment. IEEE Trans Med
Clin Endocrinol Metab 2009;94:2239-2244.
Imaging 1991;10:616-620.
74. Thodberg HH, van Rijn RR, Jenni OG, Martin DD. Automated
54. Garn SM, Hertzog KP, Poznanski AK, Nagy JM. Metacarpophalangeal
determination of bone age from hand X-rays at the end of puberty and
length in the evaluation of skeletal malformation. Radiology
its applicability for age estimation. Int J Legal Med 2017;131:771-780.
1972;105:375-8.
Epub 2016 Oct 18
55. Tanner JM, Gibbons RD. A computerized image analysis system for
75. van Rijn RR, Lequin MH, Thodberg HH. Automatic determination of
estimating Tanner-Whitehouse 2 bone age. Horm Res 1994;42:282-
Greulich and Pyle bone age in healthy Dutch children. Pediatr Radiol
287.
2009;39:591-597. Epub 2009 Jan 6
56. Tanner JM, Oshman D, Lindgren G, Grunbaum JA, Elsouki R, Labarthe
76. Martin DD, Deusch D, Schweizer R, Binder G, Thodberg HH, Ranke MB.
D. Reliability and validity of computer-assisted estimates of Tanner-
Clinical application of automated Greulich-Pyle bone age determination
Whitehouse skeletal maturity (CASAS): comparison with the manual
in children with short stature. Pediatr Radiol 2009;39:598-607. Epub
method. Horm Res 1994;42:288-294.
2009 Mar 31
57. Frisch H, Riedl S, Waldhör T. Computer-aided estimation of skeletal age
77. Martin DD, Meister K, Schweizer R, Ranke MB, Thodberg HH, Binder
and comparison with bone age evaluations by the method of Greulich-
G. Validation of automatic bone age rating in children with precocious
Pyle and Tanner-Whitehouse. Pediatr Radiol 1996;26:226-231.
and early puberty. J Pediatr Endocrinol Metab 2011;24:1009-1014.
58. Van Teunenbroek A, De Waal W, Roks A, Chinafo P, Fokker M, Mulder
78. Martin DD, Heil K, Heckmann C, Zierl A, Schaefer J, Ranke MB, Binder
P, De Muinck Keizer-Schrama S, Drop S. Computer-aided skeletal age
G. Validation of automatic bone age determination in children with
scores in healthy children, girls with Turner syndrome, and in children
congenital adrenal hyperplasia. Pediatr Radiol 2013;43:1615-1621.
with constitutionally tall stature. Pediatr Res 1996;39:360-367.
Epub 2013 Oct 5
59. Hill K, Pynsent PB. A fully automated bone-ageing system. Acta
79. Booz C, Yel I, Wichmann JL, Boettger S, Al Kamali A, Albrecht MH,
Paediatr Suppl 1994;406:81-83.
Martin SS, Lenga L, Huizinga NA, D’Angelo T, Cavallaro M, Vogl TJ,
60. Rucci M, Coppini G, Nicoletti I, Cheli D, Valli G. Automatic analysis of Bodelle B. Artificial intelligence in bone age assessment: accuracy
hand radiographs for the assessment of skeletal age: a subsymbolic and efficiency of a novel fully automated algorithm compared to the
approach. Comput Biomed Res 1995;28:239-256. Greulich-Pyle method. Eur Radiol Exp 2020;28;4:6.
61. Somkantha K, Theera-Umpon N, Auephanwiriyakul S. Bone age assessment 80. Artioli TO, Alvares MA, Carvalho Macedo VS, Silva TS, Avritchir R,
in young children using automatic carpal bone feature extraction and Kochi C, Longui CA. Bone age determination in eutrophic, overweight
support vector regression. J Digit Imaging 2011;24:1044-1058. and obese Brazilian children and adolescents: a comparison between
62. Seok J, Hyun B, Kasa-Vubu J, Girard A. Automated Classification System computerized BoneXpert and Greulich-Pyle methods. Pediatr Radiol
for Bone Age X-ray Images. IEEE Int Conf Syst Man Cybern 2012;208- 2019;49:1185-1191. Epub 2019 May 31
213. 81. Zhang J, Lin F, Ding X. Maturation disparity between hand-wrist
63. Cao F, Huang HK, Pietka E, Gilsanz V. Digital hand atlas and web-based bones in a chinese sample of normal children: An analysis based on
bone age assessment: system design and implementation. Comput automatic bonexpert and manual greulich and pyle atlas assessment.
Med Imaging Graph 2000;24:297-307. Korean J Radiol 2016;17:435-442. Epub 2016 Apr 14
64. Gross GW, Boone JM, Bishop DM. Pediatric skeletal age: determination 82. Zhang SY, Liu G, Ma CG, Han YS, Shen XZ, Xu RL, Thodberg HH.
with neural networks. Radiology 1995;195:689-695. Automated determination of bone age in a modern chinese population.
ISRN Radiol 2013;2013:874570.
65. Sato K, Ashizawa K, Anzo M, Otsuki F, Kaneko S, Tanaka T, Tsukagoshi
K, Nimura A, Matsuok H, Matsuo N, Mitani H, Murata M. Setting up 83. Martin DD, Sato K, Sato M, Thodberg HH, Tanaka T. Validation of a
an automated system for evaluation of bone age. Endocr J 1999;(46 new method for automated determination of bone age in Japanese
Suppl):S97-S100. children. Horm Res Paediatr 2010;73:398-404. Epub 2010 Apr 14
66. Mahmoodi S, Sharif BS, Chester EG, Owen JP, Lee R. Skeletal growth 84. Anink J, Nusman CM, van Suijlekom-Smit LW, van Rijn RR, Maas M, van
estimation using radiographic image processing and analysis. IEEE Rossum MA. Automated determination of bone age and bone mineral
Trans Inf Technol Biomed 2000;4:292-297. density in patients with juvenile idiopathic arthritis: a feasibility study.
67. Zhang A, Gertych A, J LB. Automatic bone age assessment for young Arthritis Res Ther 2014;16:424.
children from newborn to 7-year-old using carpal bones. Comput Med 85. Mergler S, de Man SA, Boot AM, Heus KGCBB de, Huijbers WAR, van
Imaging Graph.2007;31:299-310. Rijn RR, Penning C, Evenhuis HM. Automated radiogrammetry is a

261
Prokop-Piotrkowska M et al. J Clin Res Pediatr Endocrinol
Methods of Bone Age Assessment 2021;13(3):251-262

feasible method for measuring bone quality and bone maturation in 101. Mutasa S, Chang PD, Ruzal-Shapiro C, Ayyala R. MABAL: a Novel deep-
severely disabled children. Pediatr Radiol 2016;46:1017-1022. Epub learning architecture for machine-assisted bone age labeling. J Digit
2016 Mar 30 Imaging 2018;31:513-519.
86. Thodberg HH, van Rijn RR, Tanaka T, Martin DD, Kreiborg S. A paediatric 102. Liu Y, Zhang C, Cheng J, Chen X, Wang ZJ. A multi-scale data fusion
bone index derived by automated radiogrammetry. Osteoporos Int framework for bone age assessment with convolutional neural
2010;21:1391-1400. Epub 2009 Nov 24 networks. Comput Biol Med 2019;108:161-173.
87. Schündeln MM, Marschke L, Bauer JJ, Hauffa PK, Schweiger B, Führer- 103. Ren X, Li T, Yang X, Wang S, Ahmad S, Xiang L, Stone SR, Li L, Zhan
Sakel D, Lahner H, Poeppel TD, Kiewert C, Hauffa BP, Grasemann C. A
Y, Shen D, Wang Q. Regression Convolutional Neural Network for
Piece of the Puzzle: The Bone Health Index of the BoneXpert Software
Automated Pediatric Bone Age Assessment From Hand Radiograph.
Reflects Cortical Bone Mineral Density in Pediatric and Adolescent
IEEE J Biomed Health Inform 2019;23:2030-2038. Epub 2018 Oct 19
Patients. PLoS One 2016;11:e0151936.
104. Spampinato C, Palazzo S, Giordano D, Aldinucci M, Leonardi R. Deep
88. Nusman CM, Anink J, Van Rossum MAJ, Van Rijn RR, Maas M, Van
Suijlekom-Smit LWA. Bone health assessment of patients with juvenile learning for automated skeletal bone age assessment in X-ray images.
idiopathic arthritis: A comparison between DXA and BoneXpert. Med Image Anal 2017;36:41-51. Epub 2016 Oct 29
Pediatr Radiol 2014;44(Suppl 1):S321-S322. 105. Tong C, Liang B, Li J, Zheng Z. A Deep Automated Skeletal Bone Age
89. Twilt M, Pradsgaard D, Spannow AH, Horlyck A, Heuck C, Herlin T. Assessment Model with Heterogeneous Features Learning. J Med Syst
Joint cartilage thickness and automated determination of bone age and 2018;42:249.
bone health in juvenile idiopathic arthritis. Pediatr Rheumatol Online 106. Kim JR, Shim WH, Yoon HM, Hong SH, Lee JS, Cho YA, Kim S.
J 2017;15:63. Computerized bone age estimation using deep learning based
90. Pfeil A, Thodberg HH, Renz DM, Reinhardt L, Oelzner P, Wolf G, program: evaluation of the accuracy and efficiency. AJR Am J
Böttcher J. Metacarpal bone loss in patients with rheumatoid arthritis Roentgenol 2017;209:1374-1380.
estimated by a new Digital X-ray Radiogrammetry method - Initial
107. Van Steenkiste T, Ruyssinck J, Janssens O, Vandersmissen B,
results. BMC Musculoskelet Disord 2017;18:1-10.
Vandecasteele F, Devolder P, Achten E, Van Hoecke S, Deschrijver D,
91. Thodberg HH, Jenni OG, Caflisch J, Ranke MB, Martin DD. Prediction Dhaene T. Automated assessment of bone age using deep learning
of adult height based on automated determination of bone age. J Clin and gaussian process regression. Annu Int Conf IEEE Eng Med Biol
Endocrinol Metab 2009;94:4868-4874. Epub 2009 Nov 19
Soc 2018;2018:674-677.
92. Satoh M. Bone age: assessment methods and clinical applications. Clin
108. Larson DB, Chen MC, Lungren MP, Stence NV, Langlotz CP.
Pediatr Endocrinol 2015;24:143-152. Epub 2015 Oct 24
Performance of a deep-learning neural network model in assessing
93. Adult Height Predictor. Accessed on: 29.07.2021. Available from: skeletal maturity on. Radiology 2018;287:313-322.
https://www.bonexpert.com/documentation/adult-height-predictor
109. Halabi SS, Prevedello LM, Kalpathy-cramer J, Mamonov AB. The
94. Anthimopoulos M, Christodoulidis S, Ebner L, Christe A, Mougiakakou RSNA pediatric bone age machine learning Challenge. Radiology
S. Lung Pattern Classification for Interstitial Lung Diseases Using
2018;290:498-503.
a Deep Convolutional Neural Network. IEEE Trans Med Imaging
2016;35:1207-1216. Epub 2016 Feb 29 110. Siegel EL. What Can We Learn from the RSNA Pediatric Bone Age
Machine Learning Challenge ? Radiology 2018;290:504-505.
95. Liskowski P, Krawiec K. Segmenting retinal blood vessels with deep
neural networks. IEEE Trans Med Imaging 2016;35:2369-2380. Epub 111. Predicting Skeletal Age. Available from: https://www.16bit.ai/bone-age
2016 Mar 24
112. Gerges M, Eng H, Chhina H, Cooper A. Modernization of bone age
96. Lee JH, Kim KG. Applying deep learning in medical images: the case of assessment: comparing the accuracy and reliability of an artificial
bone age estimation. Healthc Inform Res 2018;24:86-92. Epub 2018 intelligence algorithm and shorthand bone age to Greulich and Pyle.
Jan 31 Skeletal Radiol 2020;49:1449-1457.
97. Hao PY, Chokuwa S, Xie XH, Wu FL, Wu J, Bai C. Skeletal bone age 113. Spampinato C, Palazzo S, Giordano D, Aldinucci M, Leonardi R. Deep
assessments for young children based on regression convolutional learning for automated skeletal bone age assessment in X-ray images.
neural networks. Math Biosci Eng 2019;16:6454-6466.
Med Image Anal 2017;36:41-51.
98. Duc T, Lee J, Shin J. Incorporated region detection and classification
114. Gertych A, Zhang A, Sayre J, Pospiech-Kurkowska S, Huang HK. Bone
using deep convolutional networks for bone age assessment. Artif
Age Assessment of Children using a Digital Hand Atlas. Comput Med
Intell Med 2019;97:1-8.
Imaging Graph 2007;31:322-331.
99. Wang F, Gu X, Chen S, Liu Y, Shen Q, Pan H, Shi L, Jin Z. Artificial
intelligence system can achieve comparable results to experts for 115. Lee H, Tajmir S, Lee J, Zissen M, Yeshiwas BA, Alkasab TK, Choy G, Do
bone age assessment of Chinese children with abnormal growth and S. Fully Automated Deep Learning System for Bone Age Assessment.
development. PeerJ. 2020;8:e8854. J Digit Imaging 2017;30:427-441.

100. Wang F, Gu X, Chen S, Liu Y, Shen Q, Pan H, Shi L, Jin Z. Artificial 116. Tajmir SH, Lee H, Shailam R, Gale HI, Nguyen JC, Westra SJ, Lim R,
intelligence system can achieve comparable results to experts for Yune S, Gee MS, Do S. Artificial intelligence-assisted interpretation of
bone age assessment of Chinese children with abnormal growth and bone age radiographs improves accuracy and decreases variability.
development. PeerJ 2020;8:e8854. Skeletal Radiol 2019;48:275-283. Epub 2018 Aug 1

262