Totally Unclustered BWT Images of Any Length over Non-Binary Alphabets
Abstract
We prove that for every integer and for every alphabet of size , there exist words of length whose Burrows–Wheeler Transform (BWT) is totally unclustered, i.e., it consists of exactly runs with no two consecutive equal symbols. These words represent the worst-case behavior of the clustering effect of the BWT. We also establish a lower bound on their number. This contrasts with the binary case, where the existence of infinitely many totally unclustered BWT images is still an open problem, related to Artin’s conjecture on primitive roots.
Keywords and phrases:
Burrows–Wheeler Transform, BWT-runs, Repetitiveness Measure, Clustering Effect, Generalized de Bruijn WordsFunding:
Gabriele Fici: Supported by MIUR project PRIN 2022 APML – 20229BCXNW.Copyright and License:
2012 ACM Subject Classification:
Mathematics of computing Combinatorics ; Theory of computation Data structures design and analysisEditors:
Philip Bille and Nicola PrezzaSeries and Publisher:
Leibniz International Proceedings in Informatics, Schloss Dagstuhl – Leibniz-Zentrum für Informatik
1 Introduction
The Burrows–Wheeler Transform (BWT) is a transformation on words introduced in 1994 [4]. When applied to a text (also called a word in the context of this paper), it produces a permutation of the characters obtained by concatenating the last character of each cyclic rotation of the text, after sorting all such rotations in lexicographic order. Since two rotations of the same word yield the same BWT, this transform can be naturally viewed as a mapping from the set of necklaces over a finite ordered alphabet (i.e., equivalence classes of words under rotation) to the set of words of . The BWT is a widely used tool in data compression and indexing, because it is very likely to produce long runs of identical consecutive symbols (BWT-runs) when the input text is highly repetitive. This effect is usually called the clustering effect of the BWT. From a theoretical perspective, counting the number of BWT-runs provides a natural measure of repetitiveness of the input text and a suitable parameter for evaluating the performance of compressed indexing structures for highly repetitive data. This measure is also sensitive even to simple combinatorial operations, which can drastically change the number of BWT-runs [14, 1, 11, 15, 12]. The relation between the number of BWT-runs and other measures of repetitiveness, in particular those based on dictionary compressors, has been investigated [27, 19]. Moreover, combinatorial properties of texts that are maximally compressible through the BWT, namely those having the minimum number of BWT-runs, have also been studied [25, 7, 20, 10]. From an application perspective, the remarkable clustering effect of the BWT has been recognized as the key to its compression power [28], and it is effectively exploited in both text compression and compressed indexing when combined with run-length encoding techniques [26, 8, 13].
However, it is interesting to note that, while the BWT of a text typically exhibits some degree of clustering, there exist texts for which no such clustering occurs. In this work, we focus precisely on this phenomenon, which so far has been the least explored: the existence of words whose BWT is totally unclustered, i.e., such that no two consecutive characters are equal. These totally unclustered BWT images correspond to a worst-case scenario for compression and indexing methods based on run-length BWT, which achieve space and time efficiency when the number of BWT runs is much smaller than the text length.
This problem was previously studied by Mantaci et al. [24] in the case of binary alphabets, revealing a connection with the generators of the multiplicative group of integers modulo , as stated in the following result:
Theorem 1 ([24]).
There exists a word of length with a totally unclustered BWT if and only if is an odd prime and generates the cyclic group .
However, the question of whether infinitely many binary words with totally unclustered BWT exist is left open in [24]. This problem is tightly connected to the still-open Artin’s conjecture on the existence of primitive roots modulo infinitely many prime numbers.
Here, we move beyond the binary setting and prove that the existence of totally unclustered BWT images is guaranteed for every length as soon as the alphabet contains at least three letters. In particular, we prove the following:
Theorem 2.
For every , and for every , there exists a word of length over such that every letter of appears at least once and is totally unclustered.
Our proof is constructive, as we explicitly show how to build words of any length whose BWT is totally unclustered. More in detail, our approach builds upon the structure of generalized de Bruijn words introduced in [9]. The key point is that generalized de Bruijn words have a BWT consisting of consecutive blocks, each of which is a permutation of the alphabet. As proved in [9], generalized de Bruijn words are precisely the Hamiltonian cycles of the generalized de Bruijn graphs, introduced in the early 80s independently by Imase and Itoh [18], and by Reddy, Pradhan, and Kuhl [30] (see also [6]) in the context of network design.
In particular, we prove the following result:
Theorem 3.
For every integer , and , there exists a generalized de Bruijn word of length over with totally unclustered BWT.
Here, we provide a new algorithmic strategy that exploits combinatorial properties of generalized de Bruijn graphs, making it possible to remove equal letters between consecutive blocks of the BWT, thereby obtaining words of length whose BWT images are totally unclustered. We then show how to extend this procedure to every length, not just multiples of the size of the alphabet. To the best of our knowledge, this is the first work exhibiting words over fixed-size alphabets with BWT-runs for all .
A de Bruijn word of order over is a word of length that contains every string of length exactly once as a circular factor. Since generalized de Bruijn words reduce to ordinary ones whenever is a power of , Theorem 3 guarantees the existence of ordinary de Bruijn words with totally unclustered BWTs for such . Moreover, we prove that for such words, the ratio between the number of runs in the BWT and the number of runs in the lexicographically smallest rotation is exactly , when . It has been proved in [24, Theorem 3.3] that this ratio, also called clustering ratio, is always at most . In this sense, our result shows that the BWT always increases the number of runs of the lexicographically smallest rotation of the de Bruijn word to which it is applied. This was previously known only for binary alphabets, although it has been proven that the clustering ratio is never equal to for binary de Bruijn words, when [24, Theorem 5.2].
Beyond the existence of words with totally unclustered BWT, we also investigate counting aspects: we provide a lower bound of on the number of words of length , over any alphabet of size at least , with totally unclustered BWT. We conclude by examining a special case connected to Artin’s conjecture, showing that certain highly structured words of length are BWT images precisely when is prime and is a primitive root modulo .
2 Preliminaries
We begin by introducing some preliminary definitions. For a thorough introduction, we refer the reader to [5] and [22].
Words and Necklaces
Let , with , denote the integer alphabet of size , whose elements are called letters. A word over the alphabet is a concatenation of elements of . The length of a word is denoted by . For a letter , denotes the number of occurrences of in . The vector is the Parikh vector of . For a word , the -th power of is the word obtained by concatenating copies of . A word is primitive if, for any non-empty word and integer , implies and . For any integer , by alphabet permutation we mean any string of length satisfying for all . In other words, it is any word whose Parikh vector is . We call a concatenation of one or more alphabet permutations over an alphabet-permutation power.
Example 4.
The word is an alphabet-permutation power over .
Let be a word of length . Two words and are conjugates if and for some words and , i.e., if is a cyclic rotation of . A necklace (resp. aperiodic necklace) is the conjugacy class of words (resp. of primitive words). Observe that a necklace is aperiodic if and only if , i.e., all the conjugates of are distinct.
Example 5.
Consider the words and . The necklace is aperiodic, while is not.
Burrows–Wheeler Transform and Standard Permutation
The Burrows–Wheeler table (BWT table) of a word is the square table whose rows are the cyclic rotations of , taken in ascending lexicographic order. Let us denote by and , respectively, the first and the last column of the BWT table of , read from top to bottom. We have that , where is the Parikh vector of ; , instead, is the Burrows–Wheeler Transform (BWT) of , denoted . From its definition, it follows that for two words it holds that if and only if and are conjugates. Since the action of the BWT on a necklace is the same as its action on any conjugate of , we use the notation or indifferently. The BWT is therefore an injective map from the set of necklaces to the set of words. A word is a BWT image if it is the BWT of some necklace.
| 0 | 0 | 1 | 1 | 0 | 2 | 1 | 0 | 0 | 1 | 1 | 2 | 0 | 2 | 1 | 2 | 2 | 2 |
| 0 | 0 | 1 | 1 | 2 | 0 | 2 | 1 | 2 | 2 | 2 | 0 | 0 | 1 | 1 | 0 | 2 | 1 |
| 0 | 1 | 1 | 0 | 2 | 1 | 0 | 0 | 1 | 1 | 2 | 0 | 2 | 1 | 2 | 2 | 2 | 0 |
| 0 | 1 | 1 | 2 | 0 | 2 | 1 | 2 | 2 | 2 | 0 | 0 | 1 | 1 | 0 | 2 | 1 | 0 |
| 0 | 2 | 1 | 0 | 0 | 1 | 1 | 2 | 0 | 2 | 1 | 2 | 2 | 2 | 0 | 0 | 1 | 1 |
| 0 | 2 | 1 | 2 | 2 | 2 | 0 | 0 | 1 | 1 | 0 | 2 | 1 | 0 | 0 | 1 | 1 | 2 |
| 1 | 0 | 0 | 1 | 1 | 2 | 0 | 2 | 1 | 2 | 2 | 2 | 0 | 0 | 1 | 1 | 0 | 2 |
| 1 | 0 | 2 | 1 | 0 | 0 | 1 | 1 | 2 | 0 | 2 | 1 | 2 | 2 | 2 | 0 | 0 | 1 |
| 1 | 1 | 0 | 2 | 1 | 0 | 0 | 1 | 1 | 2 | 0 | 2 | 1 | 2 | 2 | 2 | 0 | 0 |
| 1 | 1 | 2 | 0 | 2 | 1 | 2 | 2 | 2 | 0 | 0 | 1 | 1 | 0 | 2 | 1 | 0 | 0 |
| 1 | 2 | 0 | 2 | 1 | 2 | 2 | 2 | 0 | 0 | 1 | 1 | 0 | 2 | 1 | 0 | 0 | 1 |
| 1 | 2 | 2 | 2 | 0 | 0 | 1 | 1 | 0 | 2 | 1 | 0 | 0 | 1 | 1 | 2 | 0 | 2 |
| 2 | 0 | 0 | 1 | 1 | 0 | 2 | 1 | 0 | 0 | 1 | 1 | 2 | 0 | 2 | 1 | 2 | 2 |
| 2 | 0 | 2 | 1 | 2 | 2 | 2 | 0 | 0 | 1 | 1 | 0 | 2 | 1 | 0 | 0 | 1 | 1 |
| 2 | 1 | 0 | 0 | 1 | 1 | 2 | 0 | 2 | 1 | 2 | 2 | 2 | 0 | 0 | 1 | 1 | 0 |
| 2 | 1 | 2 | 2 | 2 | 0 | 0 | 1 | 1 | 0 | 2 | 1 | 0 | 0 | 1 | 1 | 2 | 0 |
| 2 | 2 | 0 | 0 | 1 | 1 | 0 | 2 | 1 | 0 | 0 | 1 | 1 | 2 | 0 | 2 | 1 | 2 |
| 2 | 2 | 2 | 0 | 0 | 1 | 1 | 0 | 2 | 1 | 0 | 0 | 1 | 1 | 2 | 0 | 2 | 1 |
Example 6.
Let us consider the necklace , where of length . The BWT table of is depicted in Fig. 1, from which it follows that .
The run-length encoding of a word , denoted , is the sequence , where , and for every . If for a necklace , we write . The clustering ratio of a necklace is the ratio between the number of equal-letter runs in the BWT of and its lexicographically smallest rotation in , i.e., 111Note that . This follows from the fact that any rotation can split at most one circular run of the necklace into two linear runs.. It has been proved that for any word, the clustering ratio is at most [24, Theorem 3.3]. A word is totally unclustered if . We say that a word (resp. a necklace ) has a totally unclustered BWT if is totally unclustered.
Example 7.
Consider the necklace . Its Burrows–Wheeler Transform is the word . Observe that is totally unclustered. Moreover, since is the smallest word in lexicographical order in , the clustering ratio is .
A permutation on is determined by the images of its elements and can be represented in two-line notation as . It can also be defined in terms of its cycle decomposition as a sequence of tuples partitioning where, for all , , with and the smallest integer for which . Such a decomposition is unique up to reordering of the cycles [31]. The standard permutation of a word , , is the permutation such that if and only if or and . In other words, in two-line notation, orders distinct letters of lexicographically, and equal letters by occurrence order, starting from . In the context of BWT (and its extensions to a multiset of words [23]), the standard permutation of the output word is also called the -mapping. The inverse standard permutation of a word , written in two-line notation, can be obtained by listing in left-to-right order the positions of in , then the positions of , and so on. The inverse standard permutation of a word produced as output by the BWT (or by its extensions to a multiset of words) is also called -mapping.
The following lemma is a direct consequence of the definition of BWT and [23, Theorem 13].
Lemma 8.
An -length word is a of some aperiodic necklace if and only if (or equivalently, ) is a cycle of length .
The following examples illustrate the two cases in which the standard permutation is not a single cycle: words that are not BWT images of any necklace, and words that are BWT images of periodic necklaces.
Example 9.
Let us consider the word used in Example 4. The standard permutation and its cycle decomposition are as follows:
The inverse standard permutation and its cycle decomposition of are as follows:
By using Lemma 8, is not the image of any aperiodic necklace under the BWT.
Example 10.
Let us consider the periodic necklace , where . The reader can verify that and . The standard permutation and its cycle decomposition are as follows:
Generalized De Bruijn Graphs and Words
Let be a finite directed graph, where is the set of vertices and is the set of edges. Each edge is directed from its source vertex to its target vertex . We write , and for every . A path in the graph is any sequence of edges verifying that for all ; the path is further called cycle if .
Given a finite directed graph , the directed line graph (or edge graph) of is a graph having as vertices the edges of , and as edges the set
A directed graph is Hamiltonian if it has a Hamiltonian cycle, i.e., one that traverses each vertex exactly once, while it is Eulerian if it has an Eulerian cycle, i.e., one that traverses each edge exactly once. As is well known, a directed graph is Eulerian if and only if for all vertices . If a graph is Eulerian, its line graph is Hamiltonian.
For all integers , the de Bruijn graph is the directed graph where each vertex is a string of length over and there exists an edge for each and . As is well known, de Bruijn graphs are both Eulerian and Hamiltonian, and one has .
The generalized de Bruijn graph is a graph having as vertices , and for every vertex there is an edge from to for every . Observe that for every , it holds222Up to renaming integer labels with their base- representations in digits. that . Generalized de Bruijn graphs have been introduced independently by Imase and Itoh [18], and by Reddy, Pradhan, and Kuhl [30] (see also [6]). They are Eulerian for every and . The line graph of is [21]. More precisely, one has the following:
Lemma 11 ([21]).
Given integers, the graph is isomorphic to the line graph of , where the vertex corresponding to an edge is associated to the edge with label .
The Eulerian cycles of correspond to the Hamiltonian cycles of .
A de Bruijn word of order on is a necklace of length such that every string in occurs exactly once as a circular factor. Since each of the distinct words of length occurs as a prefix of consecutive rows of the Burrows–Wheeler table of a de Bruijn word, the Burrows–Wheeler transform of a de Bruijn word is an alphabet-permutation power (cf. [16]).
De Bruijn words were generalized in several different ways (see, e.g., [3, 2, 29]). We use here the definition from [9]: a necklace of length over the alphabet is a generalized de Bruijn word if is an alphabet-permutation power. Generalized de Bruijn words are aperiodic necklaces.
Example 12.
Consider the word from Example 6. The necklace is a generalized de Bruijn word. Indeed, is an alphabet-permutation power.
Observe that not all alphabet-permutation powers are BWT images, as witnessed by the word in Example 9.
The following theorem establishes a direct correspondence between generalized de Bruijn words and Hamiltonian cycles in generalized de Bruijn graphs.
Theorem 13 ([9]).
A necklace of length over is a generalized de Bruijn word if and only if is a Hamiltonian cycle of . When for some integer , generalized de Bruijn words of length over coincide with ordinary de Bruijn words of order on .
The next example shows a generalized de Bruijn word whose BWT is a totally unclustered alphabet-permutation power, also providing an instance of the statement of Theorem 13.
Example 14.
The necklace , where is a generalized de Bruijn word of length , since is an alphabet-permutation power. Moreover, , so has a totally unclustered BWT. Note that the standard permutation is a cycle of length :
The inverse permutation is:
3 Main Result
In this section, we establish the existence of necklaces (and, hence, of classes of words) of any length with totally unclustered BWT over alphabets of size at least three. Recall from [25, Proposition 2] that if a necklace has totally unclustered BWT, then it must be aperiodic. Our approach is based on generalized de Bruijn words and their representation as Hamiltonian cycles in generalized de Bruijn graphs. We first show how to construct such necklaces of length that is a multiple of the alphabet size, and then how to extend the construction to arbitrary lengths.
3.1 Constructing totally unclustered BWT images from generalized de Bruijn words
We begin by proving that for every integer , and , there exists a generalized de Bruijn word of length over with totally unclustered BWT. The key insight is that adjacent equal letters between consecutive alphabet-permutation blocks can be systematically eliminated by rerouting Hamiltonian cycles in the corresponding de Bruijn graph. This yields generalized de Bruijn words with totally unclustered BWT.
Let be a generalized de Bruijn word, and let us write . By definition, is an alphabet-permutation power, hence a sequence of length- blocks that are permutations of . Clearly, a repeated letter can occur only at the boundary between two consecutive such blocks. In other words, has a totally unclustered BWT if and only if for every . On the contrary, if , we say that has a tie at block . Ties for can also be characterized by the inverse standard permutation of , as shown in the following lemma.
Lemma 15.
Let be a generalized de Bruijn word of length over , , and . The word has a tie at block if and only if for some , one has:
Proof.
Let . Consider the permutation . It can be understood as follows: writing the multiset of letters of in lexicographic order, where equal letters are sorted by occurrence order in , the position in of the -th letter is . In particular, since the Parikh vector of is , the positions of 0 in are , the positions of 1 are , and so on, until the last letter, which occurs at positions . In particular, there is a tie in at block if and only if, for some the th letter of occurs at positions and , and in that case they are respectively the -th and the -th occurrences of this letter. In other words, we have and , as claimed.
The following lemma provides conditions under which Hamiltonian cycles can be rerouted to avoid a specified edge. We will use it to break ties in BWT images of generalized de Bruijn words by rerouting the corresponding Hamiltonian cycles so that they avoid one of the edges responsible for the tie.
Lemma 16.
Let be a directed Hamiltonian graph, and such that:
-
1.
both and have cardinality at least ;
-
2.
every pair is an edge of ;
-
3.
every edge starting at a vertex in ends at a vertex in .
Then, if is a Hamiltonian cycle for visiting an edge , it can be rerouted into a Hamiltonian cycle such that: (i) does not visit , and (ii) if , visits if and only if visits .
Proof.
Let be a directed Hamiltonian graph, and two subsets of satisfying the conditions of the statement, and a Hamiltonian cycle visiting . Since is a cycle, we can write it as a sequence of vertices starting with and ending with . Sets and have cardinality at least , and every vertex from must be followed by a vertex from . Hence, one can write where each starts with a vertex from and ends with a vertex from , for every . Since each is an edge, the sequence is still a Hamiltonian cycle for , which does not visit . Furthermore, and differ only on edges in , and the statement follows.
Remark 17.
Note that Lemma 16 does not require having .
Remark 18.
If is the line graph of an Eulerian graph , Lemma 16 has a simpler interpretation: the set (resp. ) corresponds to the set of all incoming (resp. all outgoing) edges for a fixed vertex having indegree and outdegree at least , and the edge to be avoided corresponds to a sequence of two edges and . In that case, the cycle corresponds to an Eulerian cycle on , which by definition must visit at least times, and can be written as a concatenation of cycles starting and ending at . In that case, the rerouting performed corresponds to a permutation of the order in which those cycles are traversed. We presented the Hamiltonian version of the result because the correspondence of Theorem 13 is more direct from the Hamiltonian viewpoint; however, all related statements can be reformulated in terms of Eulerian graphs.
The following lemma shows that the rightmost tie in the BWT of a generalized de Bruijn word can always be removed.
Lemma 19.
Let be a generalized de Bruijn word of length over , with , and . Assume that is the rightmost tie for . Then, there exists such that: (i) for some generalized de Bruijn word of length over ; (ii) for every ; (iii) has no tie at block for any .
Proof.
Suppose is the rightmost tie of . From the hypothesis, it follows that , and from Lemma 15, there is a fixed such that and , where depends on the letter .
By Lemma 8, the permutation is a cycle, and by Theorem 13, this cycle corresponds to the consecutive vertex labels of a Hamiltonian cycle of . Consider the sets , . Note that the edge is visited by the cycle corresponding to and that it is an edge from to . Those sets both have cardinality , and by construction of , they satisfy the conditions of Lemma 16. We can then obtain a Hamiltonian cycle of , corresponding to a permutation , with , hence has no tie at , and for every (importantly, we still have , so the conditions of Lemma 15 cannot hold for some ), and the words and differ only at block (namely, at positions ). Since was the rightmost tie in , we deduce that has no tie at block .
We can now prove our main result:
Theorem 3. [Restated, see original statement.]
For every integer , and , there exists a generalized de Bruijn word of length over with totally unclustered BWT.
Proof.
By Theorem 13, generalized de Bruijn words are in bijection with Hamiltonian cycles in , or equivalently, with Eulerian cycles in . Since is Eulerian for every , there is at least one generalized de Bruijn word of length over . If has no tie, then is a generalized de Bruijn with totally unclustered BWT. Otherwise, let be the rightmost tie for . By Lemma 19, there exists , for some generalized de Bruijn word , such that either has no tie, and therefore is totally unclustered, or its rightmost tie is at block , and recursively apply at most times the same approach until we eventually obtain a word with no ties, which is then the BWT of some generalized de Bruijn word that has a totally unclustered BWT.
Example 20.
Let us consider the necklace , where . One has , hence is a generalized de Bruijn word of length over with . Since has a tie at block , does not have a totally unclustered BWT. One has , and , and . We have , , and we write , with , , . Then, also corresponds to a cycle on , namely , and , with , . The word still has one tie – observe, however, that the tie is now at block , and that there are no ties at later blocks.
One has , and . We have , , and we write with , . Then, corresponds to a cycle of , namely , and , with , where . The necklace is a de Bruijn word with totally unclustered BWT and its clustering ratio is .
As a direct consequence of Theorem 3, we obtain the following result for ordinary de Bruijn words.
Corollary 21.
For every , and every , there exists a de Bruijn word over of order (and hence length ) with totally unclustered BWT.
Corollary 21 shows a clear separation from the binary case, where de Bruijn words with totally unclustered BWT do not exist [24, Theorem 5.2].
Moreover, in the case of de Bruijn words with totally unclustered BWT, the clustering ratio depends only on , as shown in the following proposition.
Proposition 22.
Let be a de Bruijn word over of order with totally unclustered BWT, for some and . If is the lexicographically smallest rotation in , then , and .
Proof.
Recall that is a de Bruijn word of length . Clearly, . Since is a de Bruijn word, each string from occurs exactly times as a circular factor of , and contains strings having two distinct letters. Finally, since is minimal among rotations of , its first and last letters must be distinct, hence exactly one of the circular occurrences of a pair of distinct letters is not an occurrence in . Wrapping up, we obtain . Now, since has totally unclustered BWT, which means that , we conclude that .
Remark 23.
Words that have a totally unclustered are examples of words for which the cannot be compressed by run-length encoding. Additionally, not only can a word have a totally unclustered , but also the clustering ratio can be maximized, as witnessed by the word , whose is , thus is a generalized de Bruijn word with totally unclustered and the clustering ratio is . However, we do not know if this can happen for infinitely many lengths.
In the case of ordinary de Bruijn words over alphabets of size at least , Proposition 22 tells us that the clustering ratio cannot achieve the maximum value . However, it implies the existence of an infinite family of ternary words of increasing length whose BWT are totally unclustered and for which the clustering ratio equals , thus highlighting the unclustering effect of the BWT on this family of ternary words.
3.2 Extending to all lengths
After establishing the existence of totally unclustered BWT images for generalized de Bruijn words of length , we extend the construction to arbitrary word lengths in the ternary case. In fact, we show how to insert or delete a single letter while preserving the cyclic structure of the standard permutation.
We start by introducing some notation. For , let and be two sets defined as follows: if is a ternary generalized de Bruijn word of length , then (resp. ) if and only if (resp. if ).
Lemma 24.
Let . The sets and are non-empty and in bijection.
Proof.
Let be a ternary generalized de Bruijn word of length , and . Let be its inverse standard permutation, corresponding to a Hamiltonian cycle of . Let and , and write for the edge and . Notice also that the edge is visited by no Hamiltonian cycle, since it is a self-loop. Now, must visit one edge among and : in particular, it visits if and only if , and if and only if . In both cases, since and satisfy the conditions of Lemma 16, we can obtain a Hamiltonian cycle visiting (resp. ) if and only if visits (resp. ), hence a word which is the image of a de Bruijn word and such that (resp. ) if and only if (resp. ). Since no other block is affected, we obtain an involution on the set of BWT images of generalized de Bruijn words, mapping to .
Lemma 25.
Let . The set (resp. ) contains at least one totally unclustered word.
Proof.
From Lemma 24, we know that and are both nonempty. Without loss of generality, let . We apply Lemma 19 iteratively to , following the same procedure as in the proof of Theorem 3. At each step, we eliminate the rightmost tie (at some block ) until the word becomes totally unclustered. Crucially, this process does not affect the final block of the word, so the position remains unchanged. Hence, we obtain a totally unclustered word .
Lemma 26.
Let be a totally unclustered alphabet-permutation power that is the BWT image of some necklace , for some . If (i.e., ), then is the BWT image of some necklace . If (i.e., ), then is the BWT image of some necklace .
Proof.
Let us assume , and therefore . Observe that the word of length is obtained by deleting from the 2 at position . As the last block of is a permutation of , we have , so remains totally unclustered.
We now track the effect on the standard permutation. For all , the relative order remains the same, so . Since the removed 2 was the last of its kind, we had . The symbol at position in is now at in , so . All other values are preserved. Thus, is obtained by deleting from the cycle of , which yields a -cycle. Therefore, for some necklace of length , and it is totally unclustered.
Now, assume that , and therefore . We define as the word of length obtained by inserting a 2 between positions and in . This insertion is not adjacent to the existing 2 at position , so it increases the number of runs by , and remains totally unclustered.
Let us verify that , the standard permutation of , is a cycle. Since the insertion occurs after the last 2 of , all positions before preserve their relative order: for all , we have . The inserted 2 at position is now the new last 2, so . The former position (now shifted to ) keeps its image: . All subsequent values are unchanged. Thus, is obtained by inserting just after in the cycle of . Since is a -cycle, this operation yields a -cycle, so for some necklace of length , and is totally unclustered.
Theorem 27.
For every integer , there exists a necklace of length over the alphabet having totally unclustered BWT.
Example 28.
Let as in Example 20. Note that . It has length , and removing ties yields . Removing the last 2 in gives , a totally unclustered word of length , which is the BWT of , whose clustering ratio is .
On the other hand, consider the word , which is totally unclustered. Hence, inserting a 2 in penultimate position gives , a totally unclustered word of length , which is the BWT of , whose clustering ratio is .
Any ternary word is a special case of a word over , with , where only the letters 0, 1, and 2 occur. Hence, the result of Theorem 27 can be easily extended to any alphabet of size larger than . It is natural to wonder whether the same result can be achieved if we require that for all . We show the following:
Theorem 2. [Restated, see original statement.]
For every , and for every , there exists a word of length over such that every letter of appears at least once and is totally unclustered.
Proof.
Let be any word having runs that is a BWT image of some aperiodic necklace , which exists by Theorem 27. We can then define a word as follows:
By construction, one can observe that , i.e., is the BWT image of some aperiodic necklace containing at least one occurrence of each character in . Moreover, the word is obtained by replacing some one-letter runs in with characters that occur only once in , thus preserving the property of being totally unclustered, and the thesis follows.
Example 29.
| 0 | 0 | 1 | 0 | 2 | 1 | 1 | 2 | 2 |
| 0 | 1 | 0 | 2 | 1 | 1 | 2 | 2 | 0 |
| 0 | 2 | 1 | 1 | 2 | 2 | 0 | 0 | 1 |
| 1 | 0 | 2 | 1 | 1 | 2 | 2 | 0 | 0 |
| 1 | 1 | 2 | 2 | 0 | 0 | 1 | 0 | 2 |
| 1 | 2 | 2 | 0 | 0 | 1 | 0 | 2 | 1 |
| 2 | 0 | 0 | 1 | 0 | 2 | 1 | 1 | 2 |
| 2 | 1 | 1 | 2 | 2 | 0 | 0 | 1 | 0 |
| 2 | 2 | 0 | 0 | 1 | 0 | 2 | 1 | 1 |
| 0 | 0 | 1 | 0 | 4 | 1 | 2 | 5 | 3 |
| 0 | 1 | 0 | 4 | 1 | 2 | 5 | 3 | 0 |
| 0 | 4 | 1 | 2 | 5 | 3 | 0 | 0 | 1 |
| 1 | 0 | 4 | 1 | 2 | 5 | 3 | 0 | 0 |
| 1 | 2 | 5 | 3 | 0 | 0 | 1 | 0 | 4 |
| 2 | 5 | 3 | 0 | 0 | 1 | 0 | 4 | 1 |
| 3 | 0 | 0 | 1 | 0 | 4 | 1 | 2 | 5 |
| 4 | 1 | 2 | 5 | 3 | 0 | 0 | 1 | 0 |
| 5 | 3 | 0 | 0 | 1 | 0 | 4 | 1 | 2 |
4 Lower Bound
For an alphabet of size at least , we can obtain a lower bound on the number of words having totally unclustered BWT, using the generalized Euler’s totient function.
Let be a prime number. The generalized Euler’s totient function counts the number of polynomials over of degree smaller than and coprime with . It can be computed using the formula
| (1) |
for , where is the largest power of dividing , and is the multiplicative order of modulo .
Theorem 30.
For every integers , writing with , there exist at least necklaces of length over having a totally unclustered .
Proof.
The number of generalized de Bruijn words of length over is [9].
By repeatedly applying Lemma 19 to any such word, we eventually obtain a totally unclustered BWT. Each application corresponds to choosing a subset of blocks to untie. Since there are possible subsets, every totally unclustered word can arise as the image of at most generalized de Bruijn words.
Applying this argument to the whole family of generalized de Bruijn words yields at least totally unclustered words of length . Applying it instead to the sets and , we obtain at least totally unclustered words in each of them, since Lemma 24 gives
Finally, by Lemma 26, each totally unclustered word in (resp. ) is injectively mapped to a necklace of length (resp. ) with a totally unclustered BWT. Hence, there are at least such necklaces of length and the same number of length .
5 Special Case related to Artin’s Conjecture
We showed that necklaces with totally unclustered BWT exist for every length when the alphabet has size at least , contrasting with the binary case (Theorem 1). In particular, this case is related to a famous conjecture of Emil Artin (see [17] for a more detailed presentation).
An integer is a primitive root modulo if generates multiplicatively the group , namely if .
Conjecture 31.
Let be an integer that is not a square number and not . Then, is a primitive root modulo for infinitely many primes .
This happens because, in the binary case, a word that is a BWT image and is totally unclustered is necessarily of the form . Reciprocally, the word is a BWT image if and only if is an odd prime and generates the cyclic group , as stated in Theorem 1. This is no longer the case if the alphabet contains at least letters, since more words have maximal . However, one can still find a trace of this phenomenon in the larger alphabet case:
Theorem 32.
Let , and be the word obtained by concatenating each letter of in decreasing lexicographic order. For every integer , the word is a BWT image if and only if is a prime and is a primitive root modulo .
Proof.
The proof is essentially the same as in the binary case [24]. By relabeling and considering the standard permutation on the set , one can express , hence . This is a single cycle if and only if . Since and are coprime, the left-hand set is contained in , and since , the three sets are equal. In particular, , which holds if and only if is prime. Thus is prime and is a primitive root modulo .
Remark 33.
Note that, when is a BWT image, the necklace such that is, by definition, a generalized de Bruijn word (see also Example 14 for the case and ). However, it is never an ordinary de Bruijn word (equivalently, an integer such that for some integer never satisfies the condition of Theorem 32). This was shown in [24] for the case . For , observe that , so . This means that the order of must divide . But for every , so cannot have order , and cannot have as a primitive root.
6 Conclusions and Future Work
In this work, we proved that for every alphabet of size at least three and for every length , a necklace (and hence a class of words) whose BWT is totally unclustered can be constructed. This shows that the worst-case behavior of the BWT with respect to clustering is not only possible but also unavoidable for all lengths once the alphabet has at least three symbols. Our approach is based on generalized de Bruijn words and their interpretation as Hamiltonian cycles in generalized de Bruijn graphs, from which we also derived a lower bound on the number of such words.
These results open new research directions. From a combinatorial perspective, it would be interesting to characterize all necklaces with totally unclustered BWT (in particular, those that are not generalized de Bruijn words), and to derive tighter lower bounds on their number.
Another natural research direction is the study of how other compression-based repetitiveness measures behave on these extremal cases.
Finally, although the existence of infinitely many binary words with totally unclustered BWT is still unproven, it has been shown that if a binary word has unclustered BWT, then the clustering ratio can only take the values or , where is the lexicographically smallest rotation of [24, Propositions 4.5 and 4.7]. For arbitrary alphabets, following Example 7 and Remark 23, we leave open the problem of characterizing words that have both a totally unclustered BWT and a clustering ratio of .
References
- [1] Tooru Akagi, Mitsuru Funakoshi, and Shunsuke Inenaga. Sensitivity of string compressors and repetitiveness measures. Inf. Comput., 291:104999, 2023. doi:10.1016/J.IC.2022.104999.
- [2] Yu Hin Au. Generalized de Bruijn words for primitive words and powers. Discrete Mathematics, 338(12):2320–2331, 2015. doi:10.1016/j.disc.2015.05.025.
- [3] Lenore Blum, Manuel Blum, and Michael Shub. Comparison of Two Pseudo-Random Number Generators. In David Chaum, Ronald L. Rivest, and Alan T. Sherman, editors, Advances in Cryptology, pages 61–78, Boston, MA, 1983. Springer US. doi:10.1007/978-1-4757-0602-4_6.
- [4] Michael Burrows and David J. Wheeler. A Block-sorting Lossless Data Compression Algorithm. SRS Research Report, 124, 1994.
- [5] Maxime Crochemore, Christophe Hancart, and Thierry Lecroq. Algorithms on strings. Cambridge University Press, 2007.
- [6] Ding-Zhu Du and Frank K. Hwang. Generalized de Bruijn digraphs. Networks, 18(1):27–38, 1988. doi:10.1002/NET.3230180105.
- [7] Sébastien Ferenczi and Luca Q. Zamboni. Clustering Words and Interval Exchanges. Journal of Integer Sequences, 16(2):Article 13.2.1, 2013. URL: https://cs.uwaterloo.ca/journals/JIS/VOL16/Zamboni/zamboni2.pdf.
- [8] Paolo Ferragina and Giovanni Manzini. Indexing compressed text. J. ACM, 52(4):552–581, 2005. doi:10.1145/1082036.1082039.
- [9] Gabriele Fici and Estéban Gabory. Generalized De Bruijn Words, Invertible Necklaces, and the Burrows-Wheeler Transform. In Paweł Gawrychowski, Filip Mazowiecki, and Michał Skrzypczak, editors, 50th International Symposium on Mathematical Foundations of Computer Science (MFCS 2025), volume 345 of Leibniz International Proceedings in Informatics (LIPIcs), pages 48:1–48:18, Dagstuhl, Germany, 2025. Schloss Dagstuhl – Leibniz-Zentrum für Informatik. doi:10.4230/LIPIcs.MFCS.2025.48.
- [10] Gabriele Fici, Sabrina Mantaci, Antonio Restivo, Giuseppe Romana, Giovanna Rosone, and Marinella Sciortino. BWT and combinatorics on words. In The Expanding World of Compressed Data, volume 131 of OASIcs, pages 1:1–1:23. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 2025. doi:10.4230/OASIcs.MANZINI.1.
- [11] Gabriele Fici, Giuseppe Romana, Marinella Sciortino, and Cristian Urbina. On the Impact of Morphisms on BWT-Runs. In CPM 2023, volume 259 of LIPIcs, pages 10:1–10:18. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 2023. doi:10.4230/LIPIcs.CPM.2023.10.
- [12] Gabriele Fici, Giuseppe Romana, Marinella Sciortino, and Cristian Urbina. Morphisms and BWT-Run Sensitivity. In MFCS, volume 345 of LIPIcs, pages 49:1–49:18. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 2025. doi:10.4230/LIPIcs.MFCS.2025.49.
- [13] Travis Gagie, Gonzalo Navarro, and Nicola Prezza. Fully functional suffix trees and optimal text searching in BWT-runs bounded space. J. ACM, 67(1):2:1–2:54, 2020. doi:10.1145/3375890.
- [14] Sara Giuliani, Shunsuke Inenaga, Zsuzsanna Lipták, Nicola Prezza, Marinella Sciortino, and Anna Toffanello. Novel Results on the Number of Runs of the Burrows-Wheeler-Transform. In SOFSEM, volume 12607 of Lecture Notes in Computer Science, pages 249–262. Springer, 2021. doi:10.1007/978-3-030-67731-2_18.
- [15] Sara Giuliani, Shunsuke Inenaga, Zsuzsanna Lipták, Giuseppe Romana, Marinella Sciortino, and Cristian Urbina. Bit Catastrophes for the Burrows-Wheeler Transform. Theory Comput. Syst., 69(2):19, 2025. doi:10.1007/s00224-024-10212-9.
- [16] Peter M. Higgins. Burrows-Wheeler transformations and de Bruijn words. Theor. Comput. Sci., 457:128–136, 2012. doi:10.1016/J.TCS.2012.07.019.
- [17] Christopher Hooley. On Artin’s conjecture. Journal für die reine und angewandte Mathematik, 1967(225):209–220, 1967. doi:10.1515/crll.1967.225.209.
- [18] Makoto Imase and Masaki Itoh. Design to Minimize Diameter on Building-Block Network. IEEE Trans. Computers, 30(6):439–442, 1981. doi:10.1109/TC.1981.1675809.
- [19] Dominik Kempa and Tomasz Kociumaka. Resolution of the Burrows-Wheeler transform conjecture. Commun. ACM, 65(6):91–98, 2022. doi:10.1145/3531445.
- [20] Mélodie Lapointe and Christophe Reutenauer. Characterizations of perfectly clustering words. Electron. J. Comb., 32(3), 2025. doi:10.37236/12851.
- [21] Xueliang Li and Fuji Zhang. On the numbers of spanning trees and Eulerian tours in generalized de Bruijn graphs. Discret. Math., 94(3):189–197, 1991. doi:10.1016/0012-365X(91)90024-V.
- [22] M. Lothaire. Combinatorics on Words. Advanced Book Program. Addison-Wesley, Advanced Book Program, World Science Division, 1983.
- [23] Sabrina Mantaci, Antonio Restivo, Giovanna Rosone, and Marinella Sciortino. An extension of the Burrows-Wheeler Transform. Theor. Comput. Sci., 387(3):298–312, 2007. doi:10.1016/J.TCS.2007.07.014.
- [24] Sabrina Mantaci, Antonio Restivo, Giovanna Rosone, Marinella Sciortino, and Luca Versari. Measuring the clustering effect of BWT via RLE. Theoretical Computer Science, 698:79–87, October 2017. doi:10.1016/j.tcs.2017.07.015.
- [25] Sabrina Mantaci, Antonio Restivo, and Marinella Sciortino. Burrows-Wheeler transform and Sturmian words. Information Processing Letters, 86:241–246, 2003. doi:10.1016/S0020-0190(02)00512-4.
- [26] Giovanni Manzini. An analysis of the Burrows-Wheeler transform. J. ACM, 48(3):407–430, 2001. doi:10.1145/382780.382782.
- [27] Gonzalo Navarro. Indexing Highly Repetitive String Collections, Part I: Repetitiveness Measures. ACM Comput. Surv., 54(2):29:1–29:31, 2022. doi:10.1145/3434399.
- [28] Gonzalo Navarro. Technical perspective: The compression power of the BWT. Commun. ACM, 65(6):90, 2022. doi:10.1145/3531443.
- [29] Abhinav Nellore and Rachel A. Ward. Arbitrary-Length Analogs to de Bruijn Sequences. In Hideo Bannai and Jan Holub, editors, 33rd Annual Symposium on Combinatorial Pattern Matching, CPM 2022, June 27-29, 2022, Prague, Czech Republic, volume 223 of LIPIcs, pages 9:1–9:20. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 2022. doi:10.4230/LIPIcs.CPM.2022.9.
- [30] S.M. Reddy, D.K. Pradhan, and J.G. Kuhl. Directed graphs with minimum diameter and maximal connectivity. Technical report, School of Engineering, Oakland University, July 1980.
- [31] Bruce Sagan. The symmetric group: representations, combinatorial algorithms, and symmetric functions, volume 203. Springer Science & Business Media, 2001.
