|Year : 2018 | Volume
| Issue : 2 | Page : 86-91
Effect of photobiomodulation on maxillary decrowding and root resorption: A randomized clinical trial
Nour Al Okla1, Danah M.A. Bader1, Laith Makki2
1 Private Orthodontic Practice, Dubai, UAE
2 Department of Orthodontics, European University College, Dubai, UAE
|Date of Web Publication||18-Jun-2018|
European University College, Dubai Healthcare City, Ibn Sina Building, No 27, Block D, 3rd Floor, PO Box 53382, Dubai
Source of Support: None, Conflict of Interest: None
Purpose: The effects of low-level laser therapy (LLLT) with light-emitting diode (LED) delivery (Biolux OrthoPulse® device) were tested for no differences from sham-controlled conventional orthodontics in maxillary anterior alignment treatment efficiency and maxillary central incisor root resorption after 6 months of treatment. Materials and Methods: Two prospective clinical trial samples were matched for pretreatment irregularity index with (n = 14) and without (n = 12) photobiomodulation therapy (850 nm wavelength, 0.065 J/cm2, 5 min per-arch-per-day) and examined every 2 weeks for reduction of irregularity index to <1 mm. The sham control sample was provided with LED devices that did not deliver infrared light. Standardized periapical radiographs of maxillary central incisors were compared at initial and 6 months of treatment. Results: Photobiomodulation resolved maxillary anterior crowding with 35.2% greater efficiency (41.0 vs. 63.3 days, P= 0.028) at nearly double the tooth movement rate-per-week (1.02 vs. 62 mm/week, P= 0.045). Mean maxillary central incisor root lengths were significantly shorter at the 6-month treatment interval after LLLT (19.63 vs. 20.85 mm, P= 0.021). Conclusions: LED photobiomodulation therapy at 850 nm wavelength resulted in 1.7X more rapid maxillary anterior alignment.
Keywords: Accelerated orthodontics, low-level laser therapy, photobiomodulation, root resorption
|How to cite this article:|
Al Okla N, Bader DM, Makki L. Effect of photobiomodulation on maxillary decrowding and root resorption: A randomized clinical trial. APOS Trends Orthod 2018;8:86-91
|How to cite this URL:|
Al Okla N, Bader DM, Makki L. Effect of photobiomodulation on maxillary decrowding and root resorption: A randomized clinical trial. APOS Trends Orthod [serial online] 2018 [cited 2019 Jan 19];8:86-91. Available from: http://www.apospublications.com/text.asp?2018/8/2/86/234673
| Introduction|| |
One of the common deterrents to orthodontic treatment is the length of time for the active treatment phase. Authors have reported that orthodontic therapy would take an average of 20–30 months to complete the treatment planned. During the past two decades, many biologically based treatment strategies have been advocated to reduce active orthodontic treatment time; photobiomodulation has emerged second to the surgical category. Surgical intervention techniques such as corticotomy have been shown to be safe clinically with three to four times more rapid active treatment time. However, surgical procedures requiring a mucogingival flap carry the risk of postsurgical morbidity and the invasive nature of these techniques have not been widely accepted.
There are multiple investigations on the effect of low-level laser therapy (LLLT) on orthodontic tooth movement, but only a few studies with a focus on the amount of time to resolve dental crowding, i.e., irregularity index ≤1 mm. Kau et al. reported maxillary and mandibular anterior alignment rate differences in a multicenter study with the extraoral LLLT test groups using 850 nm, 60 mW, continuous wave, and varied J/cm 2 daily or weekly. The rates of tooth movement in the alignment phase were 1.12 mm/week collectively for those in the photobiomodulation treatment group compared to 0.49 mm in the control group.
Four publications reported dental crowding resolution in days; three articles ,, reported 22.2%–25.6% reduction in amount of time to eliminate dental arch crowding, and one article  reported 54% more rapid decrowding [Table 1].
|Table 1: Publications reporting the amount of time for dental decrowding including publication author and year, which dental arch was monitored, low-level laser application specifics, amount of time for crowding resolution (days), and percent difference between test and control groups|
Click here to view
Orthodontically induced root resorption (OIRR) detected at the root apex is a clinical reality in approximately 90% of patients. Fortunately, the degree of apical root resorption is typically mild, rarely compromises dental function or the longevity of the dentition, but occasionally (~0.5%) present severe lesions affecting more than one-third of the root length. Despite the high prevalence of apical root resorption among orthodontic patients, there is no effective therapy found to date for preventing the formation of root resorption lacunae.
A few studies have tested the effect of LLLT on OIRR in animals. Some authors reported that light emitting diode (LED) therapy inhibited root resportion  or had a significant reparative effect on OIRR, but there is no consensus as to whether LLLT prevents OIRR. Others have stated that patients using LLLT had no significant effect on root resorption when compared with patients in the control group.
The present study investigated orthodontic treatment efficacy and root resorption with and without photobiomodulation therapy using the OrthoPulse ®, Biolux device. Purposes of the study were to compare (1) therapy time for the maxillary anterior alignment phase of orthodontic treatment and (2) severity of orthodontically induced apical root resorption at the 6-month interval after initiating therapy. The null hypotheses tested were no significant differences with and without photobiomodulation therapy in the following: (1) time and/or rate of alignment phase and (2) amount of apical root resorption following 6 months of therapy.
| Materials and Methods|| |
Ethical approval was obtained from the EUC Institutional Review Board of European University College (EUC), Dubai Healthcare City, United Arab Emirates.
This prospective, randomized, double-blind clinical trial included 38 subjects recruited from the clinics of the Master Degree in Orthodontic Program at EUC. Subjects were randomly divided into two groups: The treatment group used a LED device for photobiomodulation (n = 17), and the control group used a placebo (sham) device (n = 21). Patients requiring comprehensive orthodontic treatment who met the following inclusion criteria were asked to participate in the study: (1) presence of permanent dentition, (2) comprehensive orthodontic treatment with fixed appliances, (3) Class I or Class II malocclusion, (4) nonextraction, (5) irregularity index >4 mm in the maxillary and mandibular arches, (6) no adjunct treatment such as extra or intraoral appliances (except TPA, HG used for anchorage), (7) good oral hygiene, and (8) age range 12–40 years.
The study exclusion criteria included the following: (1) pregnant females, (2) patients enrolled in other clinical studies, (3) use of a nonsteroidal anti-inflammatory drug use during the study (acetaminophen accepted), (4) use or history of bisphosphonates, (5) smoker and user of chewing tobacco, (6) presence of any periodontally compromised teeth, (7) presence of unerupted or partially erupted maxillary teeth, (8) malaligned teeth unable to engage in an initial archwire because of degree of malalignment, (9) spacing in the maxillary arch, and (10) noncompliance.
The treatment protocol and research methodology were explained to the patients upon their enrolment; all patient-subjects signed informed consent before participating in the study.
Full mouth scaling and polishing was completed before beginning orthodontic treatment, and oral hygiene instructions were given. Routine initial orthodontic treatment records were taken including photographs, full arch study casts, panoramic X-ray, lateral cephalogram, and periapical radiograph for the maxillary incisors taken using the paralleling technique.
All patient-subjects in the study underwent comprehensive orthodontic treatment. Orthodontic appliance bonding was carried out with 0.022” slot Master Series MBT prescription brackets (American Orthodontics, USA) or 0.022 slot Clarity Advanced Ceramic MBT prescription brackets (3M Unitek, USA). No bracket repositioning was allowed after initial bonding of maxillary incisors to maintain the bracket position for measurements of root length. Wire sequence for all patients was as follows: (1) 0.016 HANT with steel ties, (2) 0.019 × 0.025 HANT, and (3) 0.019 × 0.025 stainless steel.
Every patient enrolled in the treatment and control groups received a Low-Level Laser Therapy (OrthoPulse ®) device kit. OrthoPulse ® is manufactured by Biolux Research Ltd. (Vancouver, BC, Canada) and is approved for use by the United States Federal Drug Administration for use with either fixed orthodontic appliances or clear aligners. Patients enrolled in the experimental group received wireless OrthoPulse® devices [Figure 1]a. Patients in the sham control group received wired Biolux devices [Figure 1]b; the sham devices were adjusted by the manufacturer to not emit any light as a placebo. All kits were comprised of an OrthoPulse ® controller unit, an intra-oral appliance, a DC power adaptor, and a carrying case.
|Figure 1: (a) Treatment group: Wireless OrthoPulse® comprised wireless mouthpiece case, power supply, and patient usage. The wireless experimental device emitted infrared incoherent light-emitting diode light of 850 nm wavelength. (b) Control group: Wired OrthoPulse® device comprised controller and electrical power supply, mouthpiece, and patient usage. The wired control device was a placebo and did not emit any infrared light|
Click here to view
The active OrthoPulse ® device used in the study emitted light generated by a low-energy laser or LED in the near infrared range. Application specifics for the intraoral device used in the study included 850 nm wavelength, continuous wave, 0.065 J/cm 2, and 5 min per-arch-per-day.
It was anticipated that patients might occasionally miss a light therapy session using the OrthoPulse ® device. In such cases, patients were instructed to make up the missed session the following day by doing one 5 min light therapy session per arch in the morning and another 5 min per arch light therapy session in the evening. If any issues occurred with the device that may have inhibited the patient from receiving complete treatment, the patient was asked to contact the investigator immediately, and if needed, to come to the clinic before his/her next 2-week scheduled appointment so the device would be replaced.
Verbal and written instructions were provided to all patient-subjects on how to use the device at home. All patients were instructed to use the device for 5 min per arch every day, i.e., 10 min total per day, until the end of the orthodontic treatment. Immediately after orthodontic treatment appliance bonding and orthodontic archwire insertion, all patients were asked to demonstrate competency in using the photobiomodulation device, i.e., the first light therapy session was conducted at chairside (T0).
Patients were scheduled every 2 weeks (±3 days) thereafter for intraoral photographs of the maxillary anterior teeth to measure the rate of maxillary anterior teeth alignment until an irregularity index score of ≤1 mm was achieved (T1). The main purpose of the 2-week visits was to capture the time-point of resolution of maxillary anterior crowding (≤1 mm irregularity index) as close as possible to the actual resolution time point. At this time, impressions were taken using alginate impression material and were poured in gypsum stone; the impressions were poured immediately to avoid dimensional changes of the impression material. All study casts were identified with the patient's ID and date of impression. The amount of crowding of the maxillary anterior dentition was assessed on gypsum stone study casts using the irregularity index described by Little. The irregularity index, defined as the sum of adjacent anatomical contact point displacement in the six anterior teeth, measured to the nearest 0.1 mm on the stone study casts using a fine-tip digital caliper (Tresna Instrument Point Digital Caliper, SC02, #23487).
Initial periapical digital radiographs of the maxillary central incisors were taken immediately after bonding during the first appointment and served as a baseline for measuring root length. Periapical digital radiographs were then repeated for the maxillary central incisors after 6 months ± 2 weeks. The resolution for all periapical radiographs was standardized at 96 dpi. Widest width of the initial central incisor was measured on 3D-scanned study cast (STL) images with (3 Shape HQ, Holmens Kanal 7 DK-1060 Copenhagen, Denmark) software and applied to the initial and 6-month periapical images to eliminate magnification error and convert pixels to millimeters. Each maxillary central incisor root length was measured on the digital radiographs individually as the length of the line extending from the gingival edge of the orthodontic bracket to the root apex using ImageJ software version 1.50i; all central incisor brackets were continuous from initial to 6 months without bracket repositioning. The difference in tooth length, i.e., apical root resorption, was calculated by subtracting the measurement taken at the 6-month treatment progress interval from the pretreatment measurement taken at initial fixed appliance bonding appointment.
After the 6-month study interval, orthodontic treatment continued with visits scheduled every 6 weeks, ±5 days; photobiomodulation therapy continued as initiated. On the day of debonding, complete postorthodontic records were taken and the patient-subject exited the study and asked to complete an exit form.
Data were collected and stored (Excel, Microsoft, Seattle, WA, USA) and later transformed for use for analysis (SPSS software v. 15.0.1, IBM, Armonk, NY, USA). Descriptive statistics were computed for pretreatment age, irregularity index, treatment times, and root length. Intergroup differences were compared using the independent t-test and the paired t-test was used for intragroup differences. The 0.05 probability level of significance was used for all testing purposes. Intraoperator reliability testing was conducted by repeating irregularity index and root length measurements on five individuals from each of the two subgroups weekly for 5 weeks. Paired t-tests revealed no differences in the means and reliability was judged as satisfactory.
| Results|| |
Thirty-eight patient-subjects were initially recruited which decreased to 26 subjects by the 6-month phase of the study: Seven subjects were noncompliant with device usage and five subjects did not attend the 2-week appointment schedule before resolution of maxillary crowding, i.e., ≤1 mm irregularity index.
Maxillary anterior alignment
There was no statistical difference in mean maxillary anterior pretreatment irregularity index when comparing 14 experimental and 12 control subjects (5.7 and 5.0 mm, P > 0.05). Experimental group age was significantly greater initially (16.7 vs. 13.2 years, P= 0.032). For experimental subjects, maxillary anterior alignment was significantly more rapid (41.0 vs. 63.3 days, P= 0.028) and weekly rate of tooth movement was significantly greater (1.02 vs. 62 mm/week, P= 0.045) [Table 2].
|Table 2: Descriptive statistics for patients completing initial maxillary anterior alignment including pretreatment age, pretreatment irregularity index, maxillary alignment rate (days), and increment (mm/week) of maxillary anterior alignment change|
Click here to view
Apical root resorption
Periapical radiographs taken at pretreatment and at 6 months were judged diagnostic for 23 patient-subjects, i.e., 12 experimental and 11 control. Paired t-tests revealed no right side-left side intragroup differences (P > 0.05) between the two central incisors at the two study periods; therefore, data from the central incisors were pooled together. Mean root length at the 6-month time interval was significantly shorter for the experimental group (19.63 mm) compared to the control (20.85 mm, P= 0.021) group [Table 3].
|Table 3: Results of inter-group statistical testing for root length measurements of maxillary central incisors after pooling right-left sides|
Click here to view
| Discussion|| |
The main finding of the present study was that the photobiomodulation combined with orthodontic therapy is an effective method for accelerating orthodontic tooth movement in dental crowding cases. Alignment of the maxillary anterior dental segment was 35.2% more rapid when orthodontics was combined with LLLT (41.0 vs. 63.3 days, P= 0.28). In comparison to previous LLLT investigations evaluating amount of time to resolve dental crowding, the present study result was higher than three previous LLLT studies,,, i.e., 22.2%, 25.5%, and 25.6%, but lower than a fourth previous study, i.e., 54%.
Moreover, the rate of tooth alignment was 0.97 mm/week for patients in the test group of the present study compared to 0.54 mm/week in the control group, an alignment rate difference of 44.3%. Kau et al. reported a higher alignment tooth movement rate of 56.3% using LLLT in both maxillary and mandibular alignment; however, there were three test groups using different application specifics in that study.
When teeth move very quickly during orthodontic treatment, there is a natural concern about changes in root morphology. Earlier studies have shown that there is a high frequency of apical root resorption in upper incisors that undergo orthodontic treatment., Maxillary incisor root resorption was assessed in the present study because there is a consensus in the scholarly literature that maxillary incisors experience the greatest amount of orthodontics apical root resorption with 25% of incisors undergoing >2 mm of root resorption. Using the 6-month stage of orthodontic treatment to assess the amount of apical root loss was in accordance with Levander et al., who reported great variation in root resorption after 6–9 months of treatment; furthermore, severe resorption after treatment was not found in any teeth without resorption at the 6–9-month treatment interval.
The amount of maxillary central incisor root resorption was evaluated in 23 patients; 12 using the LED device and 11 using the sham device. Periapical radiographs were taken immediately after bonding; the bracket was used as a reference point and repeated for all patients after 6 months. Periapical films were taken in a standardized manner using the paralleling technique to eliminate differences in projection and magnification. Radiographic images were reconstructed using Regeemy software, and the measurements were done on digital periapical radiographs. The technique was judged “reliable” following intraoperator testing, but the technique is not without confounding factors.
Root length averaged 0.95 mm shorter for photobiomodulation subjects compared to control but the difference was not statistically significant (P = 0.097). However, there was a significant difference (P = 0.021) at posttreatment with the photobiomodulation group averaging 1.25 mm shorter central incisor roots. Comparing increment of root length change revealed no differences, however (0.45 vs. 0.19, P= 0.334). It is noteworthy to point out that if pretreatment root lengths were slightly more similar, it is unlikely that a root length difference would have been found at the 6-month study interval.
In the present study, intragroup testing demonstrated significant (P < 0.05) apical root shortening from pretreatment to 6 months of therapy averaging 0.45 mm in the test group and averaging 0.19 mm in the control group. These findings were consistent with Smale et al., who found a mean average root resorption for the four maxillary incisors was 0.53 mm following 6 months of active orthodontic treatment. From a clinical perspective using 0.5 mm as a “clinically significance” guideline, 11 of 23 or 47.8% of the LLLT test group subjects 6 of 21 or 28.6% of the orthodontic treatment group without LLLT had root shortening ≥0.5 mm.
Results of the present study are also consistent with Nimeri et al., who tested the effect of photobiomodulation on twenty patients using cone-beam computed tomography images between pretreatment and an ill-defined completion-of-alignment-phase time point (the mean amount of time needed for the alignment phase was not stated). The study concluded that photobiomodulation did not cause root resorption greater than the normal range that is commonly detected in orthodontic treatment.
One important but difficult issue for LLLT use is to define the optimal dose or energy density in orthodontic treatment. Many investigators have discovered that the biostimulation of LLLT followed a dose dependency.,,, Ge reported that the highest cellular activity was observed at the dose of 1.0 J/cm 2, and that energy densities lower or higher than 1.0 J/cm 2 would reduce the bio-stimulatory effect showing no statistical difference between experiment group and control groups. The OrthoPulse® device used in the current study used a wavelength of 850 nm and a power output of 65 mW/cm 2 (±13), which is equivalent to 0.065 J/cm 2 which falls well below the recommended dosage for treatment efficacy. Previous studies have used different wavelengths ranging from 600 to 1000 nm with various power outputs, which may explain the differences in findings. An optimal dose remains undetermined.
Confounding factors for root resorption not accounted for in the present study include root dilacerations, pointed teeth, increased tooth length, and overjet. Well-designed clinical trials with greater sample sizes are needed, and the optimal dose or density and frequency of exposure of LLLT should be defined to maximize the efficacy of this promising treatment.
| Conclusions|| |
Orthodontic treatment efficacy for resolving maxillary anterior alignment as well as maxillary central incisor root resorption at the 6-month treatment interval was compared in samples matched for pretreatment irregularity index with (n = 14) and without (n = 12) photobiomodulation therapy. Both null hypotheses were rejected as total treatment time and/or rate of maxillary alignment as well as maxillary central incisor root length at the post 6-month treatment interval differed significantly with and without LLLT therapy.
- Resolution of maxillary anterior crowding, as determined by irregularity index, was more efficient using photobiomodulation therapy (41.0 vs. 63.3 days, P= 0.028), i.e., 35.2% greater efficiency
- The weekly rate of tooth movement change during the maxillary alignment phase was significantly greater in the LLLT group (1.02 vs. 62 mm/week, P= 0.045)
- Maxillary central incisor root length at the 6-month treatment interval was significantly shorter in the photobiomodulation group (19.63 vs. 20.85 mm, P= 0.021).
Based on the conditions of the present study, LED photobiomodulation therapy resulted in a significant reduction in treatment time for maxillary anterior alignment.
The authors wish to thank Biolux Research LTD for use of OrthoPulse ® devices loaned during the period of the study.
Financial support and sponsorship
This study was self-funded by the authors and their institution; OrthoPulse ® photobiomodulation devices were loaned from Biolux Research LTD, Vancouver, Canada.
Conflicts of interest
There are no conflicts of interest.
| References|| |
Ge MK, He WL, Chen J, Wen C, Yin X, Hu ZA, et al.
Efficacy of low-level laser therapy for accelerating tooth movement during orthodontic treatment: A systematic review and meta-analysis. Lasers Med Sci 2015;30:1609-18.
Nimeri G, Kau CH, Abou-Kheir NS, Corona R. Acceleration of tooth movement during orthodontic treatment – A frontier in orthodontics. Prog Orthod 2013;14:42.
Wilcko WM, Ferguson DJ, Bouquot JE, Wilcko MT. Rapid orthodontic decrowding with alveolar augmentation: Case report. World J Orthod 2003;4:197-205.
Kumar S, Yadav S, Bhomick D, Prahbhat M, Sreedhar A. Periodontally accelerated osteogenic orthodontics (PAOO): Perio-ortho interrelationship. Int J Period Implant 2016;1:38-43.
Kau CH, Kantarci A, Shaughnessy T, Vachiramon A, Santiwong P, de la Fuente A, et al.
Photobiomodulation accelerates orthodontic alignment in the early phase of treatment. Prog Orthod 2013;14:30.
Shaughnessy T, Kantarci A, Kau CH, Skrenes D, Skrenes S, Ma D, et al.
Intraoral photobiomodulation-induced orthodontic tooth alignment: A preliminary study. BMC Oral Health 2016;16:3.
AlSayed Hasan MM, Sultan K, Hamadah O. Low-level laser therapy effectiveness in accelerating orthodontic tooth movement: A randomized controlled clinical trial. Angle Orthod 2017;87:499-504.
Nahas AZ, Samara SA, Rastegar-Lari TA. Decrowding of lower anterior segment with and without photobiomodulation: A single center, randomized clinical trial. Lasers Med Sci 2017;32:129-35.
Caccianiga G, Paiusco A, Perillo L, Nucera R, Pinsino A, Maddalone M, et al.
Does low-level laser therapy enhance the efficiency of orthodontic dental alignment? Results from a randomized pilot study. Photomed Laser Surg 2017;35:421-6.
Fonseca PD, de Lima FM, Higashi DT, Koyama DF, Toginho Filho Dde O, Dias IF, et al.
Effects of light emitting diode (LED) therapy at 940 nm on inflammatory root resorption in rats. Lasers Med Sci 2013;28:49-55.
Walker S. Root resorption during orthodontic treatment. Evid Based Dent 2010;11:88.
Ekizer A, Uysal T, Güray E, Akkuş D. Effect of LED-mediated-photobiomodulation therapy on orthodontic tooth movement and root resorption in rats. Lasers Med Sci 2015;30:779-85.
Nimeri G, Kau CH, Corona R, Shelly J. The effect of photobiomodulation on root resorption during orthodontic treatment. Clin Cosmet Investig Dent 2014;6:1-8.
Langland OE, Sippy FH. A study of radiographic longitudinal distortion of anterior teeth using the paralleling technique. Oral Surg Oral Med Oral Pathol 1966;22:737-49.
Little RM. The irregularity index: A quantitative score of mandibular anterior alignment. Am J Orthod 1975;68:554-63.
DeShields RW. A study of root resorption in treated class II, division I malocclusions. Angle Orthod 1969;39:231-45.
Sameshima GT, Sinclair PM. Predicting and preventing root resorption: Part I. Diagnostic factors. Am J Orthod Dentofacial Orthop 2001;119:505-10.
Levander E, Malmgren O. Evaluation of the risk of root resorption during orthodontic treatment: A study of upper incisors. Eur J Orthod 1988;10:30-8.
Reukers E, Sanderink G, Kuijpers-Jagtman AM, van't Hof M. Assessment of apical root resorption using digital reconstruction. Dentomaxillofac Radiol 1998;27:25-9.
Smale I, Artun J, Behbehani F, Doppel D, van't Hof M, Kuijpers-Jagtman AM, et al.
Apical root resorption 6 months after initiation of fixed orthodontic appliance therapy. Am J Orthod Dentofacial Orthop 2005;128:57-67.
Chan E, Darendeliler MA. Physical properties of root cementum: Part 7. Extent of root resorption under areas of compression and tension. Am J Orthod Dentofacial Orthop 2006;129:504-10.
do Nascimento RX, Callera F. Low-level laser therapy at different energy densities (0.1-2.0 J/cm2) and its effects on the capacity of human long-term cryopreserved peripheral blood progenitor cells for the growth of colony-forming units. Photomed Laser Surg 2006;24:601-4.
Goulart CS, Nouer PR, Mouramartins L, Garbin IU, de Fátima Zanirato Lizarelli R. Photoradiation and orthodontic movement: Experimental study with canines. Photomed Laser Surg 2006;24:192-6.
Schindl A, Merwald H, Schindl L, Kaun C, Wojta J. Direct stimulatory effect of low-intensity 670 nm laser irradiation on human endothelial cell proliferation. Br J Dermatol 2003;148:334-6.
Seifi M, Shafeei HA, Daneshdoost S, Mir M. Effects of two types of low-level laser wave lengths (850 and 630 nm) on the orthodontic tooth movements in rabbits. Lasers Med Sci 2007;22:261-4.
[Table 1], [Table 2], [Table 3]