A Comparison of Techniques for Creating Permuted Blocked Randomization Lists

Two of the most important elements to the integrity of a controlled clinical study are the patient randomization and the treatment blinding. In clinical trials, we are most often comparing groups defined by treatments (e.g., active vs. placebo, or dose levels). Effective randomization helps to balance the known and unknown prognostic factors across comparative groups. Treatment group imbalances within important prognostic variables cast doubt on the primary results at best. More importantly, they can reduce the probability of a positive statistical outcome. For instance, if the outcome variable under study is influenced by age, such as mortality, a chance allocation of one treatment group to older subjects would likely cause an increase in mortality in that group that could bias the treatment effect in the study.

Statistically, the effect of imbalance can often be adjusted for in a model, if the analysis method is parametric. However, if the covariate adjusted model is not pre-specified, the FDA will not count it as the primary analysis. Often, there is a set of unknown prognostic factors. Many of them are due to different levels of experience and expertise treating the disease under study at the enrolling investigational centers. For example, we may expect better outcomes among patients treated at a high-level academic medical center than at a rural hospital in a developing country. A common technique for mitigating the influence of these unknown prognostic factors on the treatment effect has been to ensure equal treatment group ratios of patients across enrolling centers. Indeed, the ICH E9 Guidance¹ states, “It is advisable to have a separate random scheme for each center, i.e., to stratify by center or to allocate several blocks to each center.” In addition, blocking a randomization list also prevents bias coming from temporal trends. For example, if the ratio of a two-treatment-groups study was 1:1, we would desire a 1:1 ratio at each enrolling center. We will refer to this example repeatedly in the next few paragraphs.

By far, the most common way to proactively balance the known and unknown prognostic factors is to stratify the randomization using a permuted block randomization list.² At clinical trial specialty services provider Bracket, its experience with approximately 1,500 controlled clinical studies confirms that permuted block randomization was used more than 80% of the time. 

The first step in creating a permuted blocked randomization list is to determine the block size. In general, the best treatment group balance is achieved with the smallest block size. For example, a study with two treatment groups allocated in a 1:1 ratio could use a block size of 2. However, this block size carries the greatest risk of predicting the next treatment assignment. For instance, a block size of 2 with a 1:1 ratio has two permutations (A:B, B:A). If the block size is known, then it is certain that the second study subject will receive a different treatment than the first. In this case, if there are any reactions to one of the treatments, such as injection site pain, redness, fever, etc., then the next treatment assignment can be readily inferred. Hence, to avoid the potential of unblinding, a block size larger than the sum of the treatment group ratio is used. Typically, two times the sum of the treatment group ratio is used. In the previous example, a block size of 4 would contain six unique permutations of treatment order, which makes prediction of the next randomization assignment much more difficult.

The next step would be to select the block permutations. If stratification is employed, the list of blocks would be divided equally among the levels of the stratification factor. A different method of assigning the sampled blocks to strata would be to dynamically assign blocks within each stratum as needed. When the first subject is ready to randomize with stratum specific characteristics, a block is allocated to that stratum. The subject is assigned to the first treatment in the block and the remaining slots are assigned as subjects continue to randomize within that stratum. As randomizations continue and no more slots are available in the previously assigned block, a new block is assigned. In this manner, blocks from the stratified randomization list are assigned to strata dynamically. Implementation of this method requires a central randomization system, often referred to as randomization and trial supply management (RTSM), interactive response technologies (IRT), and Interactive Voice/Web Response System (IVRS/IWRS).

To minimize the ability to guess the next treatment assignment (selection bias), study statisticians desire a relatively equal representation of permutations in their randomization lists, especially within strata. Selection bias was originally discussed at a fairly technical level by Blackwell and Hodges in 1957.³ To illustrate the idea of selection bias, imagine a study with two treatment groups (A, B) where the patients are randomly assigned to one of the groups using a permuted blocked list stratified by enrolling center with a block size of 2. There would be two permutations within this list [A, B] and [B, A]. Suppose, by chance, a stratum was allocated all of one permutation so that the list for that center looked like this: [A, B, A, B, A, B, A, B, …]. Because of the perfectly alternating pattern of treatment allocation, there would be a high probability of guessing the next treatment, thus, a high selection bias (especially in an open-label study). One way to reduce the selection bias would be to increase the block size to four. However, in many studies, maintaining a high level of treatment group balance requires a block size of 2.

We like the idea of producing balance beyond that simply produced by permuted blocks, and think it is an attractive feature that the possibility of having the same sequences repeated is reduced. At the same time, we realize that this is a controversial suggestion since others may disagree that imposing further balance is desirable. In particular, the 2003 Committee for Proprietary Medicinal Products (CPMP) guidance document warns sponsor companies to avoid the use of dynamic randomization, which is designed to increase treatment group balance by imposing constraints to the randomization.⁴ Nevertheless, in the remainder of this manuscript we explore ways to reduce severe departures and same sequences of blocks.

The other method to reduce selection bias is to ensure all the permutations are represented in a relatively uniform distribution within each stratum. With current permuted block methods, a probability exists that a significant departure from a uniform distribution of permutations can occur. In fact, one will often see that a particular permutation appears consecutively for a few blocks. This event is not rare for a randomization list, especially for a list with only a few block patterns. In our example, a list of 24 blocks per stratum, the probability of having three consecutive blocks with the same design is 0.3350 per stratum. The probability of having four consecutive blocks with the same design is 0.0885 per stratum. In a more extreme example, the probability of having 10 consecutive blocks with the same design is 0.00000138434 per stratum. While the probabilities might not seem high, since these probabilities are additive, they increase as the number of blocks and/or strata.

An informal method to avoid extreme departures is to generate a permuted block list with blocks pre-assigned to strata, and apply an acceptance test. A new list would be generated for any rejected list and the acceptance criteria would be applied to the new list. This process repeats until the list passes the acceptance criteria. A shortfall of this method is that at the end of randomization, typically only a subset of blocks is used within each stratum. However, it is unknown how many blocks will be used, so the acceptance criterion is applied to all the blocks, not just the blocks that were used. Our experience has shown that lists with many strata are easily rejected using this criterion.

The novel method proposed in this paper to achieve uniform block distribution would be to sample the permutations within stratum without replacement. “With replacement” indicates that samples are independent. “Without replacement” indicates that samples are not independent. To further illustrate using a block size of 4, we would sample from the six possible block permutations without replacement until no more are available, then sample six more, and repeat until we have reached the desired number of blocks within each stratum (see diagram below). This method produces a uniform distribution of permutations within each stratum.

Each individual method of randomization list creation had the above combination of inputs in their simulations.

Therefore, an acceptance test is not required. This method can also be applied to dynamic block assignment and would produce an equivalent distribution of permutations. To our knowledge, the properties of this method of uniform block distribution have not been studied. We conducted a simulation study that compares the properties of the three methods of assigning blocks to strata described above: Stratified permuted blocks sampled with and without replacement, and dynamic block assignment sampled without replacement.

Prior to the simulations, we theorized that the more restrictions the method imposed on the list, the lower the probability for extreme block pattern distributions, and strings of repeated block patterns. In other words, we expected that the uniform permuted block distribution method would be superior at equal block pattern representation, followed by the pre-assigned permuted blocks method with rejection method, then the pre-assigned permuted blocks, and, finally, the dynamic block assignment method. In our example, the appearance of three of the same block patterns in a group of six where optimally they would all be unique almost doubles the selection bias for that grouping. When we examined the frequency of three of the same block patterns occurring in a group of six, our results show that this novel method we propose can reduce the overall selection bias by almost a third.

The randomization method studied is a stratified permuted block permutation. We studied the four methods of allocating blocks to strata described in the previous section, i.e.:

• Dynamic site assignment – blocks are assigned with the proper treatment ratio within each block based on enrollment at a particular location.

• Pre-assigned permuted blocks – blocks are pre-assigned at random to the stratum.

• Pre-assigned permuted blocks with rejection – a set number of blocks in the randomization list is assigned to a strata before study start after the list has passed a Fisher’s exact test to the block patterns within each stratum.

• Uniform permuted block distribution – the method presented in this paper.

We chose to study permuted blocks with two treatments (for the purposes of this study, “active” and “placebo”), as this is a very common study design we encounter. For each of the four block assignment methods, there were three block ratios examined in the simulation. The ratios were: 2A: 1P; 3A:1P; 1A:1P. Three block sizes were used in the simulation. The block sizes were: 3, 4, and a combination of a block size of 4 and 2. The block sizes were allocated at random. We chose three sample sizes to examine (n=100, n=200 and n=500), as these are very common sample sizes that we encounter. Because the majority of randomization lists are stratified by investigational site, we chose to stratify the randomization by site in this analysis.

For the above illustration, there were 12 combinations of ratios dependent on sample size and block size for each method. In all, there were 48 combinations of simulation study parameters examined.

In the simulation experiment, the four methods were studied by a Monte Carlo simulation. Results were reported for each method.

As mentioned previously, the blocks were distributed based on the order of enrollment. The location of the enrollment site became a stratification factor. For example, when Patient 1 enrolled at a site, Site A, Site A occupied the first block in the list. When Patient 2 enrolled into the study at another site, Site B, Site B occupied the second block in the list. Enrollments at each site progressed through the assigned block until that block was filled with patients before having another block in the list assigned. The same method of block assignment was applied throughout the sample simulation until all patients were assigned with a treatment.

At the end of the sample simulation, this method produced:

• A list of sites where patients had enrolled

• The corresponding blocks and treatments assigned from those sites

Since the result from the simulation had sites assigned listed, those assigned sites became a stratification factor when the study concluded.

Since the blocks were assigned at random, each block permutation assignment to a stratum was independent from the previous block assignment. Therefore, it was possible for a single block permutation to be assigned consecutively two, three, or more times.

In the pre-assigned permutated block method, each site is a stratification and the blocks were pre-assigned to these sites so that all patients at a particular site would receive treatments in the order they enrolled into the study. Pre-assigned permutated blocks with rejection adds an additional step- the list is rejected if it had a significant disparity of block permutations when compared against the ideal block permutation distribution.

Based on the treatment ratio within each block and its block size, we determined the applicable block permutations for each study. For instance, for a study with a treatment ratio of 2A:2P and a combination of block sizes of 2 and 4, the block permutations were as follows: AAPP, PPAA, APAP, PAPA, APPA, PAAP, AP and PA The uniform permuted block distribution method ensured that each block permutation showed up an equal number of times in the list before repeating any block permutation. The entire randomization list should have an equal representation of block distribution, given the number of slots in a randomization list.

Several characteristics were evaluated including how frequently each method departed from an acceptable block distribution, and how far each departed from it. We analyzed the performance of block distribution of each method by slicing block permutation uniqueness into deciles, counts of triple and multiple block permutation evaluations, and an analysis of block distribution using Fisher’s Exact Test. The purpose of using three distinct methods-as opposed to statistical tests-offered different approaches to examining block permutation arrangement and the uniqueness of block permutations per strata.

The results show that there is very little difference between methods 1-3. However, the uniform permuted block distribution method, as expected, produces a much higher percentage of block permutation uniqueness within the strata.

Among methods 1-3, Pre-assigned permuted blocks with rejection method generally produces lists with more even distributions of block permutations that are less volatile. Out of the four methods, the dynamic Site assignment method yields the most volatile block permutation distribution. In other words, the permutations in each site have a wider variation in terms of block permutation compilation compared to the other methods.

In the pre-assigned permuted blocks with rejection method (our informal method of avoiding extreme departure), the standard deviations were smaller, indicating a smaller probability of having an undesirable distribution of block permutations in each site.

The uniform permuted block distribution method yielded the least spread in terms of p-value (a standard deviation of 0) and the highest p-values (p-value of 1). Combining the previous results, uniform permuted block distribution method ensured an equal representation of block permutations on a pre-populated randomization list. It also guaranteed an equal representation of block permutations for each stratification if a trial stopped at any point before enrollment was completed.

In the past, study statisticians have been concerned about an unequal representation of block permutations. Before our simulations, we theorized a novel process, uniform permuted block distribution method, which would sample block permutations without replacement, thereby reducing the probability of overrepresentation (or underrepresentation) of block permutations. We compared the four methods of randomization list creation: dynamic site assignment method, pre-assigned permutated blocks method, pre-assigned permuted blocks with rejection method, and our new method, uniform permuted block distribution method. We analyzed the methods by measuring uniqueness, repeated block permutations, and Fisher’s Exact Test. After performing analyses, we concluded that the uniform permuted block distribution method could reduce occurrences of a significant departure from the ideal number of block permutations per stratification.

We wish to note that, in theory, the uniform permuted block distribution method can be implemented in a dynamic site assignment permutation. The challenge only being the complex programming involved to create the uniform block permutations as new blocks are assigned dynamically to sites as needed.

Carol Shum, is SAS Programmer, Gilead Sciences; Scott Hamilton, PhD, is Principal Biostatistician, Bracket; Kim Hung Lo, is Director, Biostatistics, Janssen Pharmaceuticals

References

1. ICH-E9. (1998). International Conference on Harmonization, E9 document, Guidance on Statistical Principles for Clinical Trials. Federal Register, 63: 49583-49598.

2. N.W. Scott, G. C. McPherson, C. R. Ramsay, et.al. (2002). The method of minimization for allocation to clinical trials: a review. Controlled Clinical Trials, 23, 662-674.

3. D. Blackwell and J. L. Hodges (1957). Permutation for the Control of Selection Bias. Annals of Mathematical Statistics, 28: 449-460.

4. CPMP, C. f. (2003). Points to Consider on Adjustment for Baseline Covariates. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003639.pdf

Additional Readings

W. N. Kernan, C. M Viscoli, R. W. Makuch, et.al. (1999). Stratified Randomization for Clinical Trials. Journal of Clinical Epidemiology, 52: 19-26.

McEntegart, D. (2008). Blocked Randomization. In R. S. D'Agostino, Wiley Encyclopedia of Clinical Trials (p. DOI:10.1002/9780471462422.eoct301). Hoboken: John Wiley & Sons, Inc.