Multiple testing procedures for identifying desirable dose combinations in bifactorial designs

mibe000026 Original Article Multiple testing procedures for identifying desirable dose combinations in bifactorial designs Multiple Testprozeduren zur Identifikation sinnvoller Dosiskombinationen in bifaktoriellen Plänen Buchheister Buchheister Bettina B

Institute for Medical Statistics, Informatics and Epidemiology, University of Cologne, 50924 Cologne, GermanyInstitute for Medical Statistics, Informatics and Epidemiology, University of Cologne, Cologne, Germany

Bettina.buchheister@medizin.uni-koeln.de author Lehmacher Lehmacher Walter W

Walter.Lehmacher@Uni-Koeln.de author German Medical Science

Düsseldorf, Köln

610 drug combination Min test bifactorial design AVE- and MAX test closed testing procedure linear contrast test experimentwise error rate Medikamenten-Kombination Min-Test bifaktorielle Pläne AVE- und MAX-Test Abschlusstest-Prozedur Linearer Kontrast-Test experimentweises Signifikanzniveau 20060629 engl 1860-9171 2 2 GMS Medizinische Informatik, Biometrie und Epidemiologie GMS Med Inform Biom Epidemiol 07 Die von Hung, Chi und Lipicky vorgeschlagenen AVE- und MAX-Tests ermöglichen die Analyse in einem bifaktoriellen Plan, ob Kombinationen von zwei Medikamenten mit verschiedenen Dosen existieren, die die wünschenswerte Eigenschaft der Überlegenheit zu beiden Einzelkomponenten besitzen. Die Tests sind jedoch globale Tests und können keine speziellen Dosis-Kombinationen identifizieren, die effektiver sind als ihre beiden Einzelkomponenten. In dieser Arbeit werden multiple Testprozeduren vorgeschlagen, die auf linearen Kontrast-Tests und dem Abschlusstest-Prinzip beruhen. Ein Vergleich mit simultanen Min-Tests nach Laska und Meisner wird angestellt sowie das Verhalten dieser Ansätze anhand von Simulationsstudien untersucht. Hung, Chi, and Lipicky proposed the AVE and MAX tests to analyse in a bifactorial design whether combinations of two drugs at several doses fulfil the desirable property of superiority to both their single drug components. These are global tests and do not identify the special combinations which are more effective than their respective single components. Here multiple testing procedures based on linear contrast tests and on the closed testing principle will be presented. They will be compared with simultaneous Min tests of Laska and Meisner. The performance of these approaches is investigated by simulation studies. Introduction According to the guidelines of the U. S. Food and Drug Administration (FDA, CFR 300.50), one of the requirements for approving the use of a drug combination is that each component must make a contribution to the claimed effects. Analogously the guideline for combination drugs of the European Agency for the Evaluation of Medical Products (EMEA, CPMP/EWP/240/95) requires that the benefit/risk assessment of the fixed combination is equal or exceeds the one of each of its substances taken alone. That means, the combination needs to be simultaneously more effective than its single components. This property of superiority may be tested by using the Min test of Laska, Meisner , in (2x2) factorial design trials where each component of the combination is chosen at some fixed dose level based on prior information. Due to unknown potential interactions of the components the preselection of the dose combination is often difficult. Therefore multi level factorial designs involving simultaneous multiple dose combinations are demanded. In case of more than one combination drug, however, there is a multiple testing problem and two questions are posed: (1) Globally: Is there any combination which fulfils the property of superiority to their single components? - Here one global hypothesis is tested. (2) Locally: Which specific combinations fulfil this property? - In this case test procedures controlling the experimentwise error rate α are required. That means the probability of at least one wrong inference should be controlled by the error rate α. Neither dose response analyses nor general analysis of variance with interactions are suited for answering these two questions. Approaches which solely compare the effect of the combinations with the effects of their components are interesting. Hung, Chi, Lipicky and Hung developed two global test procedures which protect the overall type I error rate. For the local problem they only recommended to use a adjusted simultaneous Min tests according to the Hochberg procedure. In this article new procedures based on the closed testing principle will be presented. They are formed of different families of elementary hypotheses and allow for multiple testing hypotheses in a step down manner. Specific linear contrast tests were used. Furthermore, a local maximum test based on the global MAX test from Hung et al. will be developed and a modification of simultaneous Min tests will be suggested. All procedures control the experimentwise error rate α. The performance of the proposed test procedures is investigated by simulation studies and suggestions for practical applications will be formulated. Notation Consider a two factorial design with I dose levels of drug A and J dose levels of drug B. Let µij, i=0, …, I and j=0, …, J, denote the true mean responses of the dose combination (i, j), whereby high values of µij's indicate benefit. (i, 0) and (0, j) denote the single drug components (Table 1 ). Let µij be estimated by the group mean , where Xijk, k=1, …, nij, is the observed effect of the k-th subject in the (i, j)-th dose combination group. nij is the sample size of the (i, j)-th dose combination group. Assuming variance homogeneity, the pooled estimator of σ2 is given by There are 2 • IJ marginal hypotheses, which compare each combination with one of its single component: versus versus . Examining the claimed combination superiority, IJ local combination hypotheses can be formulated as union hypotheses of the two marginal hypotheses: versus These IJ local combination hypotheses should be tested controlling the experimentwise error rate α. If the global testing problem is considered the global hypothesis is . Previous approaches Up to now, two approaches have been published. In case of only one combination drug Laska and Meisner , suggest the Min test. For the general (I+1) x (J+1) design, two global tests are proposed by Hung, Chi, Lipicky . Laska-Meisner Min test In the simple case (I = J = 1) only one combination drug is observed and the hypothesis of interest is the following union hypothesis: versus HAB will be rejected if both marginal hypotheses HA and HB are rejected at level α, using appropriate test statistics. This so-called Min test is a test for the simple combination drug problem with experimentwise level α. Under rather mild conditions Laska and Meisner , showed, that this test is the uniformly most powerful within the class of monotone level α tests. A generalization for testing union hypotheses with more than two hypotheses is possible. By the extended Min test procedure a union hypothesis is rejected if each partial hypothesis can be rejected at level α. Global tests from Hung, Chi and Lipicky In the general (I + 1) x (J + 1) case, a multiple testing problem arises. Two global tests are presented by Hung, Chi, Lipicky . Their test statistics are based on the minimum gains over all dose combinations . The tested global hypothesis versus is equivalent to H0. The "AVErage" global test statistic TAVE is defined by the average of the observed minimum gains, and the "MAXimum" global test statistic TMAX is the maximum of the observed minimum gains. That is, where S is the pooled estimator of σ and the observed mean effect of the combination (i, j). Both tests are one-sided level α tests, requiring a balanced design and normally distributed data with homogeneous variances. The distributions of TAVE and TMAX are derived by Hung et al. . A more precisely and extended table of critical values of the tests as that given by Hung et al. is presented in Table 2 . The tests, however, are not concerned with the multiplicity of the testing problem of the IJ local combination union hypotheses. Note that these two global tests are developed for balanced designs, but two modified global tests in case of unequal sample sizes are provided . New approaches In practice, one might be often interested in the local question: Which dose combination(s) have the property of superiority to their respective components? Therefore, multiple procedures to find desirable combinations which control the experimentwise error rate are required. Closed testing procedure of IJ local combination hypotheses Consider the IJ local combination hypotheses as elementary hypotheses. Constructing a closed system of hypotheses, the global hypothesis is the intersection of all IJ local combination hypotheses H0. Therefore each global test for H0 is a competitor to the AVE and MAX global tests. The family of all intersections of the IJ local combination hypotheses can be tested at the α level by using a step down procedure. Two examples of closed system of hypothesis with local combination hypotheses are given in Figure 1 and Figure 2 . In latter only the hierarchy of hypothesis is presented; arrows are omitted for sake of clearness. Notice, that all hypotheses are intersection union hypotheses. But most of the general used level α test procedures are constructed for union intersection hypotheses. Thus each intersection union hypothesis must first be transformed into union intersection hypothesis by the rules of elementary set theory algebra. Afterwards generalized Min tests may be used. Specific level α tests for the intersection hypotheses based on linear contrast tests will be specified later on. Closed testing procedure of 2 • IJ marginal hypotheses Consider the 2 • IJ marginal hypotheses and , which compare the effect of the combination with the effect of one of its components, as elementary hypotheses. Then, a system of hypotheses closed under intersection can be constructed (e. g. Figure 3 ). This system of hypotheses contains hypotheses and is substantially larger than the system of hypotheses constructed by the IJ elementary local combination hypotheses (cf. Figure 3 ). However, it contains only intersection hypotheses without unions which are easier to test. The family of all intersections of the 2 • IJ marginal hypotheses will be tested by a step down procedure. Subsequently a local combination hypothesis can be rejected by the Min test principle, if both of its marginal hypotheses and are rejected by the step down procedure using level α tests. The global hypothesis of this closed system of hypotheses is the intersection of all marginal hypotheses and differs from H0: . Accordingly, a global test for is not a competitor to the AVE and MAX global tests. This test procedure allows one only to answer the local question. Two simultaneous closed testing procedures for each drug Consider two simultaneous closed testing procedures generated by the IJ marginal hypotheses and the IJ marginal hypotheses (e. g. Figure 4 ). This procedure includes the advantages of both approaches mentioned above: Both systems of hypotheses are as small as in the first approach, and there are only intersection hypotheses without unions as in the second approach. But the disadvantage is that an α adjustment is required in order to control the overall error rate α. The two families of hypotheses will be tested separately for drug A and for drug B by step down procedures at level α/2. Finally Min tests can be applied to test the local combination hypotheses. As in the approach before both global hypotheses and and their union or intersection differ from H0. This approach does not test the global question. Simultaneous Min tests and a modification A common procedure to answer the local question by controlling the overall error rate are simultaneous Min tests. Each local combination hypotheses will be simultaneous tested using the Min test from Laska, Meisner at an adjusted level α* ≤ α . Several α adjustments are described in the literature (e. g. Bonferroni , or , ). Simultaneous Min tests also belong to the class of closed testing principle. An α adjustment by Holm is e. g. a closed testing procedure using the Bonferroni inequality at each step. Lehmacher and Lehmacher, Wassmer, Reitmeir propose a modification which is a short cut version of a closed testing procedure. Their suggested approach is a two step procedure where both the global and the local combination hypotheses are tested. A combination drug fulfils the property of superiority over its components if the global hypothesis can be rejected at level α and the corresponding local combination hypothesis can be rejected using the modified Bonferroni-Holm procedure with modified levels α/(IJ-1), α/(IJ-1), α/(IJ-2), …, α/2, α. Special linear contrast tests When testing superiority of combination drugs in a multi level two factorial design not all pairwise comparisons of treatments will be considered, but the 2 • IJ comparisons of combination drug with their components. Therefore, in the above described closed testing procedure partition hypotheses will be tested where two or more disjunctive hypotheses are intersected (e. g. . Usual test statistics like, e. g., F-tests do not apply. In order to control the experimentwise error rate, partition hypotheses can be tested by multiple tests with α adjustment. Another possibility is to use a special linear contrast test which could be less conservative and more applicable. Thus, each hypothesis in the closed test procedures will be tested by a specific linear contrast statistic. In case of testing an intersection of union hypotheses a transformation in an union of intersection hypotheses is required. The test statistics of the partition hypotheses will be constructed by averaging the corresponding marginal hypotheses. That is, the suitable contrasts cij will be calculated as the sum of the differences between the effect of the combinations and the effect of their single components. The test statistic is given by where Index set π1 and π2 {(i, j) |i=1,…, I; j=1,…,J} and |π1| and |π2| denote the number of elements in π1 and π2, respectively. S is the pooled estimator of σ for all treatment groups with cij 0. In case of normally distributed data with homogeneous variances T is t-distributed with degrees of freedom. Local MAX test The MAX test from Hung, Chi, Lipicky is a global test. Here an extension to the local question, the local MAX test will be developed. The test statistic TMAX of the global MAX test is based on the combination drug with the maximum observed minimum gain over its components (in the following called "MAX-combination"). Rejecting the global hypothesis (TMAX > ) at least the MAX-combination fulfils the property of superiority to their respective components. But indeed there could be other combinations which fulfil this property too. The idea of the local MAX test is to test all local combination hypotheses against the critical value . Thus, each combination drug whose local test statistic is greater or equal to fulfils the property of combination superiority. The local MAX test is a step up procedure (cf. ) based on the ordered local test statistics T(1) ≤ … ≤ T(IJ) = TMAX and a fixed critical value C: step 1: test the local combination hypothesis by the critical value C: 2. step Stop! Reject all local combination hypotheses step 2: test the local combination hypothesis by the critical value C: 3. step Stop! Retain and reject all , i = 2,..., IJ step n: test the local combination hypothesis by the critical value C: (n = 3,...,IJ) n + 1. step Stop! Retain and reject all . It is obvious that the local MAX test controls the experimentwise error rate α when the critical value C is . Consider any true hypothesis , then error rate = P (Reject ) = 1 - P (Retain ) = 1 - P (T(1) < C, …, T(i) < C). Under this error rate is clearly maximum and from this follows that α = 1 - P (T(1) < C, …, TMAX < C) and consequently C = . Simulation studies of statistical power All approaches described above control the experimentwise error rate α. The power of the following methods for the two posed questions is compared by simulation. The global hypothesis H0 is tested by: • Average (AVE) and maximum (MAX) test, • Linear contrast test for the transformed global hypothesis H0 (GCo) and • Multiple test of Simes using Min tests (SIM). The IJ local combination hypotheses are tested by: • Simultaneous Min tests according to the Hochberg procedure (simMin), • Simultaneous Min tests according to the modified Holm procedure after rejection of the global hypothesis (MinAVE, MinMAX, MinGCo, MinSIM), • Closed test procedure of IJ local combination hypotheses (CTPloc), • Closed test procedures of 2 • IJ marginal hypotheses (CTPmar), • Two simultaneous closed test procedures at level α/2 (TwoCTP) and • Local MAX test (loMAX). The simulations are based on normally distributed data with different means, homogeneous variances σ = 1, balanced design nij = 30 and significance level α = 0.05. All approaches are very conservative. There is no procedure which is uniformly more powerful than the other ones. Nevertheless depending on the kind of design and a priori informations, suggestions for practical applications can be formulated. In this paper the results of simulation are mainly qualitatively described. Some quantitative results for a (2x3) as well as for a (3x3)-design (Table 3 ) are presented as an example. Detailed power analyses are given by Buchheister . In case of the global question the results clearly show that the global contrast test (GCo) may substitute the global AVE test. GCo is the most powerful test in situations where as yet the global AVE test was more powerful than the global MAX test (see Figure 5 ). An example of comparison of power simultaneous in different situations for (2x3)-design is given in Figure 6 . There the black areas describe situations where the global contrast test is more powerful than the global MAX test and in grey areas is power(GKo) < power(MAX). The global contrast test (GCo) is recommended when all combination drugs fulfil the property of superiority or only few combinations do not and the others are similarly strong effective than their components. Otherwise the global MAX test of Hung, Chi, Lipicky is suggested. In case of very large multi level two factorial design, when the tables of critical values for the global MAX test (cf. Table 2 or ) do not suffice, for practical reasons the multiple test of Simes (SIM) will be recommended. The loss of power is negligible. The results of the simulations, concerning the local question are quite difficult to summarize. Comparing the three new closed testing procedures among each other shows that the closed testing procedures of IJ local combination hypotheses (CTPloc) are often the most powerful one. But in large multi level designs this closed testing procedure is hardly used in practice because of many transformations of intersection union hypotheses in union intersection hypotheses. In contrast the practical application of the closed testing procedures CTPmar and twoCTP is much easier. Therefore the closed testing procedure of 2 • IJ marginal hypotheses is preferable, except that there is no interest in the global hypothesis. The loss of power in relevant situations is negligible too. The approach of two simultaneous closed testing procedures at level α/2 is very conservative in small designs. However, it becomes interesting in larger designs because of its manageable systems of hypotheses. Depending on the number of dose combinations which fulfil the property of superiority and the size of their effectiveness, different most powerful procedures can be specified: 1) In case that all or nearly all combinations are similar more effective than their components the three new closed testing procedures have the largest power. In this situation the closed testing procedure of the 2 • J marginal hypotheses is suggested for small designs and two simultaneous closed testing procedure at level α/2 are suggested in case of large designs. 2) If the sizes of effectiveness of the dose combinations differ a lot, but there are only few combinations which are not simultaneously more effective than all of their single components, simultaneous Min tests like simMin, MinMAX or MinSIM are more powerful. The power of these three procedures is similar. 3) Otherwise, if only few dose combinations fulfil the property of superiority the local maximum test has the largest power. Using the tabulated level α critical value (cf. Table 2 or ) it is easy to apply. These suggestions are simplified and summarized in Table 4 and an example is given in Figure 7 . If no a priori informations about the effect of the combinations are known it will be difficult to assume that all dose combinations are similarly effective to their components, especially in larger multi level two factorial designs. Therefore concerning simplicity, robustness and power, the new global linear contrast test with subsequent α adjusted simultaneous Min tests according to the modified Holm procedures (MinGCo) are a good compromise and thus the procedure of choice in this situation. The loss of power by using a simultaneous Min test in case of similarly strong effects is small. Discussion When more than one dose combination should be tested if they are simultaneous more effective than all of their single components, a multiple testing problem arises. Different test procedures concerning the global or the local approach can be used. Due to the complexity of the problem, an optimal procedure cannot be recommended in general (cf. ). It is not really worse retaining the use of the hitherto most used global tests of Hung et al . Beside a very easy quick extension of the global MAX test to the local question is the here called local MAX test. Anyway there are some advantages of the closed testing procedures. With the closed testing procedures there is less restriction to the data in contrast to the two global tests from Hung et al . which requires normally distributed data with homogeneous variances. Each intersection hypothesis may be tested by suitable level α tests regarding the nature of the data. Because of the numerous partition hypotheses in the closed testing procedures for testing the property of combination superiority it is recommended to use special contrast tests. In case of variance heterogeneity Welch-type modifications of the tests can be easily applied. In case of binary data Gauss tests can be used. Note also that applying closed testing procedures no placebo dose combination must be included in the study. This can be important when there are ethical or medical problems to administer a placebo. In view of the complexity and the multiplicity of the problem (cf. the application in ) a sequential design as presented by Lehmacher, Kieser, Hothorn could be more advantageous, because during the conduct of the study drug combinations can be skipped. This paper is focussed on the multiple identification of desirable dose combinations. There are multiple decision procedures and related simultaneous confidence intervals are not available. Another related topic is the identification of minimum effective doses; multiple testing procedures for this problem are proposed by Hellmich and Lehmacher . Hung HMJ Chi GYH Lipicky RJ Testing for the existence of a desirable dose combination 1993 Biometrics 85-94 Hung HMJ, Chi GYH, Lipicky RJ. Testing for the existence of a desirable dose combination. Biometrics. 1993;49:85-94. Laska EM Meisner MJ Testing whether an identified treatment is best: The combination problem 1986 Proceedings of the Biopharmaceutical Section of the American Statistical Association 163-70 Laska EM, Meisner MJ. Testing whether an identified treatment is best: The combination problem. Proceedings of the Biopharmaceutical Section of the American Statistical Association. 1986;163-70. Laska EM Meisner MJ Testing whether an identified treatment is best 1989 Biometrics 1139-51 Laska EM, Meisner MJ. Testing whether an identified treatment is best. Biometrics. 1989;45:1139-51. Hung HMJ Testing for existence of desirable dose combination. 1994 Biometrics 307-8 Hung HMJ. Testing for existence of desirable dose combination (Correspondence). Biometrics. 1994;50:307-8. Hochberg Y A sharper Bonferroni procedure for multiple tests of significance 1988 Biometrika 800-2 Hochberg Y. A sharper Bonferroni procedure for multiple tests of significance. Biometrika. 1988;75:800-2. Hung HMJ Evaluation of a combination drug with multiple doses in unbalanced factorial design clinical trials 2000 Statist Med 2079-87 Hung HMJ. Evaluation of a combination drug with multiple doses in unbalanced factorial design clinical trials. Statist Med. 2000;19:2079-87. Holm S A simple sequentially rejective multiple test procedure 1979 Scand J Statist 65-70 Holm S. A simple sequentially rejective multiple test procedure. Scand J Statist. 1979;6:65-70. Hommel G A stagewise rejective multiple test procedure based on a modified Bonferroni test 1988 Biometrika 383-6 Hommel G. A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika. 1988;75:383-6. Hommel G A comparison of two Bonferroni procedures 1989 Biometrika 624-5 Hommel G. A comparison of two Bonferroni procedures. Biometrika. 1989;76:624-5. Lehmacher W 1987 Verlaufskurven und Crossover Lehmacher W. Verlaufskurven und Crossover. Heidelberg: Springer; 1987. Lehmacher W Wassmer G Reitmeir P Procedures for two-sample comparisons with multiple endpoints controlling the experimentwise error rate 1991 Biometrics 511-21 Lehmacher W, Wassmer G, Reitmeir P. Procedures for two-sample comparisons with multiple endpoints controlling the experimentwise error rate. Biometrics. 1991;47:511-21. Tamhane AC Hochberg Y Dunnett CW Multiple test procedures for dose finding 1996 Biometrics 21-37 Tamhane AC, Hochberg Y, Dunnett CW. Multiple test procedures for dose finding. Biometrics. 1996;52:21-37. Simes RJ An improved Bonferroni procedure for multiple tests of significance 1986 Biometrika 751-4 Simes RJ. An improved Bonferroni procedure for multiple tests of significance. Biometrika. 1986;73:751-4. Buchheister B 2001 Statistische Methoden zum Nachweis der Effektivität von Kombinationspräparaten Buchheister B. Statistische Methoden zum Nachweis der Effektivität von Kombinationspräparaten [Dissertation]. Köln: Medizinische Fakultät der Universität zu Köln; 2001. Letzel H Blümner E Bivariate Dosis-Wirkungs-Beziehungen für ein Kombinationsantihypertensivum: Biometrische Erfahrungen mit einem komplexen Studienmodell 1997 Medizinische Informatik, Biometrie und Epidemiologie, 41. Jahrestagung der GMDS, Bonn 382-6 Letzel H, Blümner E. Bivariate Dosis-Wirkungs-Beziehungen für ein Kombinationsantihypertensivum: Biometrische Erfahrungen mit einem komplexen Studienmodell. In: Baur MP et al.: Medizinische Informatik, Biometrie und Epidemiologie, 41. Jahrestagung der GMDS, Bonn. München: Urban und Vogel; 1997. p. 382-6. Lehmacher W Kieser M Hothorn L Sequential and multiple testing for dose-response analysis 2000 Drug Inf J 591-7 Lehmacher W, Kieser M, Hothorn L. Sequential and multiple testing for dose-response analysis. Drug Inf J. 2000;34:591-7. Hellmich M Lehmacher W Closure Procedures for Monotone Bi-Factorial Dose-Response Designs 2005 Biometrics 270-7 Hellmich M, Lehmacher W. Closure Procedures for Monotone Bi-Factorial Dose-Response Designs. Biometrics. 2005;61:270-7. 22

2 2 Figure 2: Closed system of hypotheses with local combination hypotheses, (3x3)-design

3 3 Figure 3: Closed system of hypotheses with marginal hypotheses, (2x3)-design

4 4 Figure 4: Closed systems of hypotheses for each drug with marginal hypotheses drugs, (3x3)-design

6 6 Figure 6: Difference of power of global MAX test (MAX) and global contrast test (GKo) in (2x3)-design of Table 3

5 5 Figure 5: Power of global tests in (3x3)-design of Table 3 when all combinations are similar more effective than their components (δ 11 =δ 12 =δ 21 =δ 22 =δ)

7 7 Figure 7: Power of local tests in (3x3)-design of Table 3 with (δ 11 =0, δ 12 =0.5δ, δ 21 =0, δ 22 =δ)

1 1 Figure 1: Closed system of hypotheses with local combination hypotheses, (2x3)-design 7

1 1 Equation 1

2 2 Equation 2

3 3 Equation 3

4 4 Equation 4

5 5 Equation 5

6 6 Equation 6

7 7 Equation 7

8 8 Equation 8

10 10 Equation 10

11 11 Equation 11

12 12 Equation 12

13 13 Equation 13

14 14 Equation 14

16 16 Equation 16

17 17 Equation 17

18 18 Equation 18

19 19 Equation 19

20 20 Equation 20

21 21 Equation 21

22 22 Equation 22

26 26 Equation 26

27 27 Equation 27

28 28 Equation 28

29 29 Equation 101

30 30 Equation 30

31 31 Equation 31

32 32 Equation 32

33 33 Equation 33

34 34 Equation 34

35 35 Equation 35

36 36 Equation 36

37 37 Equation 37

38 38 Equation 38

39 39 Equation 111

40 40 Equation 40

42 42 Equation 42

45 45 Equation 45

46 46 Equation 46

23 23 Equation 23

24 24 Equation 24

25 25 Equation 25

9 9 Equation 9

41 41 Equation 41

43 43 Equation 43

44 44 Equation 44

15 15 Equation 15 46 0