Generalized De Bruijn Words, Invertible Necklaces, and the Burrows–Wheeler Transform
Abstract
We define generalized de Bruijn words as those words having a Burrows–Wheeler transform that is a concatenation of permutations of the alphabet. We show that generalized de Bruijn words are in 1-to-1 correspondence with Hamiltonian cycles in the generalized de Bruijn graphs, introduced in the early ’80s in the context of network design. When the size of the alphabet is a prime , we define invertible necklaces as those whose BWT-matrix is non-singular. We show that invertible necklaces of length correspond to normal bases of the finite field , and that they form an Abelian group isomorphic to the Reutenauer group . Using known results in abstract algebra, we can make a bridge between generalized de Bruijn words and invertible necklaces. In particular, we highlight a correspondence between binary de Bruijn words of order , binary necklaces of length having an odd number of ’s, invertible BWT matrices of size , and normal bases of the finite field .
Keywords and phrases:
Burrows–Wheeler Transform, Generalized de Bruijn Word, Generalized de Bruijn Graph, Circulant Matrix, Invertible Necklace, Sandpile Group, Reutenauer GroupFunding:
Gabriele Fici: Supported by MIUR project PRIN 2022 APML – 20229BCXNW.Copyright and License:
2012 ACM Subject Classification:
Mathematics of computing Combinatorics on wordsEditors:
Paweł Gawrychowski, Filip Mazowiecki, and Michał SkrzypczakSeries and Publisher:
Leibniz International Proceedings in Informatics, Schloss Dagstuhl – Leibniz-Zentrum für Informatik
1 Introduction
The Burrows–Wheeler matrix of a word is the matrix whose rows are the conjugates (rotations) of in ascending lexicographic order. Its last column is called the Burrows–Wheeler transform of , and has the property of being easier to compress when the input word is a repetitive text. For this reason, it is largely used in textual data compression, in particular in bioinformatics [21, 19, 18]. But the Burrows–Wheeler transform also has many interesting combinatorial properties (see [31, 12] for a survey). Since the Burrows–Wheeler transform is the same for any conjugate of , it can be viewed as a map from the set of necklaces (conjugacy classes of words) to the set of words.
A de Bruijn word of order over an alphabet is a necklace of length such that each of the distinct words of length occurs exactly once in it, as a cyclic factor. Since each of the distinct words of length occurs as a prefix of consecutive rows of the Burrows–Wheeler matrix of a de Bruijn word, the Burrows–Wheeler transform of a de Bruijn word is characterized (among words that are images under the Burrows–Wheeler transform) by the fact that it is a concatenation of alphabet-permutations (de Bruijn words of order ). This motivates us to define a generalized de Bruijn word as one whose Burrows–Wheeler transform is a concatenation of (any number of) alphabet-permutations.
As is well known, de Bruijn words are in -to- correspondence with Hamiltonian cycles in de Bruijn graphs. We show that, analogously, generalized de Bruijn words are in -to- correspondence with Hamiltonian cycles in generalized de Bruijn graphs, introduced in the early ’s independently by Imase and Itoh [16], and by Reddy, Pradhan, and Kuhl [29] (see also [10]) in the context of network design. The generalized de Bruijn graph has vertices and for every vertex there is an edge from to for every .
In particular, we provide a new and simple interpretation of the well-known correspondence between de Bruijn words and Hamiltonian cycles in de Bruijn graphs in terms of the inverse standard permutation of the Burrows–Wheeler transform, and we show that this interpretation extends to the generalized case.
The other class of words we introduce is that of invertible necklaces. We call an aperiodic necklace over an alphabet of prime size invertible if its Burrows–Wheeler matrix is nonsingular, i.e., has determinant nonzero modulo . We show that each invertible necklace of length , as a set of vectors over the base field , corresponds to a normal base of the field . Invertible necklaces can also be represented by conjugacy classes of their circulant matrices, and they form an Abelian group, called the Reutenauer group [11]. The set of invertible necklaces of a given length can therefore be endowed with a multiplication operation that makes it isomorphic to the -th Reutenauer group.
Using a known result in abstract algebra [7], we show that every aperiodic necklace of length with non-zero weight modulo (the weight of a word is the sum of its digits) is invertible if and only if is either a power of or a -rooted prime. However, it is an open problem to decide whether there are infinitely many lengths , different from a power of , for which every aperiodic necklace of length with non-zero weight modulo has an invertible Burrows–Wheeler matrix. This seems to be a challenging problem, as it is related to Artin’s conjecture on primitive roots.
In the last part of the paper, we show a connection between generalized de Bruijn words and invertible necklaces. Chan, Hollmann and Pasechnik [5] proved that for every prime and any , one has , where denotes the sandpile group of the graph . Since the structure of the sandpile group of an Eulerian graph is determined by the invariant factors of its Laplacian matrix, Chan et al. gave the precise structure of sandpile groups of generalized de Bruijn graphs, and hence of Reutenauer groups.
In particular, when and , we show that there is a bijection between de Bruijn words of order and binary necklaces of length with an odd number of ’s.
Our contributions
We introduce two new classes of circular words: generalized de Bruijn words and invertible necklaces, both of which are defined in terms of the Burrows–Wheeler transform. We show that generalized de Bruijn words correspond to those whose Burrows–Wheeler transform has an inverse standard permutation that labels a Hamiltonian cycle in a generalized de Bruijn graph (Theorem 9). This result allows us to derive a formula for counting the number of generalized de Bruijn words using the BEST theorem (Theorem 12).
Next, we focus on alphabets of prime size . We define invertible necklaces as those whose Burrows–Wheeler matrix is non-singular. We demonstrate that the conjugacy classes of invertible necklaces form an Abelian group, isomorphic to the Reutenauer group of invertible circulant matrices. This leads to a characterization of the equivalence of various properties of invertible necklaces (Theorem 16).
In the final section, we connect generalized de Bruijn words and invertible necklaces by using a previously known isomorphism between the sandpile groups of generalized de Bruijn graphs and Reutenauer groups. We further speculate on the implications of this isomorphism, and end by showing a bijection between binary de Bruijn words, invertible necklaces, and normal bases of finite fields (Theorem 26).
Related work
Several generalizations of de Bruijn words (sequences) have been proposed in the literature [4, 1, 28], all based on the number of occurrences of factors (substrings) of some kind.
The relation between the BWT and de Bruijn words has been studied in several papers. In [14], the author extended the BWT to multisets of necklaces and characterized de Bruijn sets – generalizations of de Bruijn words – by their BWT image, leading to a BWT-based characterization of de Bruijn words when the multiset has a single element. In [24], this correspondence is used to design an efficient algorithm for constructing arbitrary de Bruijn words.
Recently, the algebraic structures generated by invertible BWT matrices have also been explored. In [35, 30], the authors showed that BWT matrices of Christoffel words are closed under multiplication and described the multiplicative group of invertible BWT matrices of Christoffel words. Christoffel words are the unbordered factors of aperiodic infinite words of minimal complexity (Sturmian words) [3]. The conjugacy classes of Christoffel words are precisely the binary aperiodic necklaces whose BWT has the minimal possible number of equal-letter runs [27].
Paper organization
Our paper is structured as follows. In Section 2, we introduce the necessary preliminaries on words, necklaces, and the Burrows–Wheeler transform. Section 3 defines generalized de Bruijn words and establishes their graph-theoretic characterization using generalized de Bruijn graphs, which extends the classical correspondence between de Bruijn words and de Bruijn graphs. In Section 4, we introduce the concept of an invertible necklace and relate it to circulant matrices and Reutenauer groups. Section 5 connects these concepts, using known group isomorphisms to show a bijection between generalized de Bruijn words and invertible necklaces in the context of prime-sized alphabets. Finally, in Section 6, we summarize our findings and propose open problems and future directions.
2 Preliminaries
We begin by introducing some preliminary definitions related to strings and words. For a thorough introduction, we refer the reader to [9], and [25].
Let , , be a sorted set of letters.
A word over the alphabet is a concatenation of elements of . The length of a word is denoted by . For a letter , denotes the number of occurrences of in . The vector is the Parikh vector of .
Let be a word of length . The weight of is . When , the shift of is the word .
For a word , the -th power of is the word obtained by concatenating copies of .
Definition 1.
We call an alphabet-permutation a word over that contains each letter of exactly once, and an alphabet-permutation power a concatenation of one or more alphabet-permutations over .
Example 2.
The word is an alphabet-permutation power over .
Notice that an alphabet-permutation power of length has a balanced Parikh vector, i.e., a Parikh vector of the form .
In the binary case, a word is an alphabet-permutation power if and only if , where is a binary word of length and is the Thue–Morse morphism, the substitution that maps to and to .
Two words and are conjugates if and for some words and . The conjugacy class of a word can be obtained by repeatedly applying the shift operator, and contains distinct elements if and only if is primitive, i.e., for any nonempty word and integer , implies . A necklace (resp. aperiodic necklace) is a conjugacy class of words (resp. of primitive words). Necklaces are also called circular words. Notice that, given a word , the weight of is invariant under shift, hence we can define the weight of the necklace as the weight of any of its representatives.
For example, is an aperiodic necklace, while is not aperiodic.
The number of aperiodic necklaces of length over is given by
where is the Möbius function, defined by: , if is the product of distinct primes or otherwise, i.e., if is divisible by the square of a prime number.
The number of necklaces of length over is given by
where is the Euler’s totient function. Recall that the Euler’s totient function counts the number of positive integers less than or equal to and coprime with , i.e., the order of the multiplicative group , and can be computed using the formula
for , and , where the product is taken over the distinct primes dividing .
The Burrows–Wheeler matrix (BWT matrix) of a necklace is the matrix whose rows are the shifts of in ascending111This is the standard convention in the literature. Of course, one can also choose the descending lexicographic order, the properties are symmetric. lexicographic order. Let us denote by and , respectively, the first and the last column of the BWT matrix of . We have that , where is the Parikh vector of ; , instead, is the Burrows–Wheeler Transform (BWT) of . The BWT is therefore a map from the set of necklaces to the set of words. As is well known, it is an injective map (we describe below how to invert it). A word is a BWT image if it is the BWT of some necklace.
The standard permutation of a word , , is the permutation of such that if and only if or and . In other words, in one-line notation, orders distinct letters of lexicographically, and equal letters by occurrence order, starting from . When is the BWT image of some necklace, the standard permutation is also called -mapping. As is well known, a word is a BWT image of an aperiodic necklace if and only if is a length- cycle.
The inverse standard permutation of a word , written in one-line notation, can be obtained by listing in left-to-right order the positions of in , then the positions of , and so on. The inverse standard permutation of a BWT image is also called -mapping.
An aperiodic necklace can be uniquely reconstructed from its BWT, using either of the standard permutation or its inverse. To do this, it is convenient to write these permutations in cycle form. Let be the BWT of , and be the standard permutation of in cycle form, with . Then, is the first row of the BWT matrix of , i.e., the lexicographically least element of the necklace . This is also equal to , where is the inverse standard permutation of in cycle form, with .
The following remark will be used in later sections:
Remark 3.
In the case of a balanced Parikh vector , then, one can reverse the BWT from its inverse standard permutation in cycle form , , by mapping to for .
For example, the BWT of is the word , whose inverse standard permutation in cycle form is . Mapping: to ; to ; to , one obtains the word , the first row of the BWT matrix of .
Necklaces that are not aperiodic can also be reconstructed from their BWT, as a consequence of the following result:
Proposition 4 ([27, Proposition 2]).
For any , is the BWT of an aperiodic necklace if and only if is the BWT of .
Let be a finite directed graph, which may have loops and multiple edges (in this case, it is also called a multigraph in the literature). Each edge is directed from its source vertex to its target vertex .
The directed line graph (or edge graph) of has as vertices the edges of , and as edges the set
An oriented spanning tree of is a subgraph containing all of the vertices of , having no directed cycles, in which one vertex, the root, has outdegree , and every other vertex has outdegree . The number of oriented spanning trees of is sometimes called the complexity of .
A directed graph is Hamiltonian if it has a Hamiltonian cycle, i.e., one that traverses each node exactly once, while it is Eulerian if it has an Eulerian cycle, i.e., one that traverses each edge exactly once. As is well known, a directed graph is Eulerian if and only if for all vertices . If a graph is Eulerian, its line graph is Hamiltonian.
3 Generalized de Bruijn graphs and generalized de Bruijn words
We start by briefly discussing (ordinary) de Bruijn graphs and de Bruijn words, and relating them to the inverse standard permutation of the Burrows–Wheeler transform.
Recall that the de Bruijn graph of order is the directed graph whose vertices are the words of length over and there is an edge from to , where is a letter, if and only if for some letter . As is well known, de Bruijn graphs are Eulerian and Hamiltonian. One has .
A de Bruijn word of order over is a necklace over containing as a circular factor each of the words of length over exactly once. De Bruijn words correspond to Hamiltonian cycles in , as they can be obtained by concatenating the first letter of the labels of the vertices it traverses.
Now, since the vertex labels of are the base- representations of the integers in on digits (padded with leading zeroes), the de Bruijn graph of order over can be equivalently defined as the one having vertex set and containing an edge from to for every . We let denote this equivalent representation of the de Bruijn graph of order . A Hamiltonian cycle in is then a cyclic permutation of . The relation between a de Bruijn word of order over , i.e., a Hamiltonian cycle in , and the corresponding Hamiltonian cycle in , is given in the following lemma.
Lemma 5.
Let be the necklace encoding a Hamiltonian cycle in , and let be the corresponding Hamiltonian cycle in , namely, the one such that is the -ary expansion on digits of for each . Then is the inverse standard permutation of the Burrows–Wheeler transform of , in cycle form. In particular, one has for every .
Proof.
For a given integer in , the first digit of in its base- representation on digits is . The claim then follows directly from Remark 3 and the fact that de Bruijn words have a balanced Parikh vector.
Example 6.
In , the de Bruijn word corresponds to the Hamiltonian cycle obtained by visiting the nodes , , , , , , , ; and to the Hamiltonian cycle in . Indeed, one has , and , in cycle notation. By applying the mapping , one retrieves .
We now introduce generalized de Bruijn words. In [14], Higgins showed that a word of length over is a (ordinary) de Bruijn word if and only if its BWT is an alphabet-permutation power. This motivates us to introduce the following definition:
Definition 7.
A necklace of length over is a generalized de Bruijn word if its BWT is an alphabet-permutation power over .
For example, is a generalized de Bruijn word since its BWT is .
Notice that, in the literature, there already exist several other definitions of generalized de Bruijn words (see, e.g., [4, 1, 28]).
The generalized de Bruijn graph has vertices and for every vertex there is an edge from to for every . Generalized de Bruijn graphs have been introduced independently by Imase and Itoh [16], and by Reddy, Pradhan, and Kuhl [29] (see also [10]). They are Eulerian and Hamiltonian. By definition, a generalized de Bruijn graph is an ordinary de Bruijn graph when for some . As an example, the generalized de Bruijn graph is displayed in Fig. 1.
The line graph of is [22]. The Eulerian cycles of correspond to the Hamiltonian cycles of .
In Theorem 9, we give a characterization of generalized de Bruijn words in terms of generalized de Bruijn graphs, which generalizes Lemma 5.
Lemma 8.
Let be a word of length over , with Parikh vector . The following statements are equivalent:
-
1.
The word is an alphabet-permutation power;
-
2.
For each , , one has ;
-
3.
For each , there is an edge from to in .
Proof.
We first observe that . By definition, the edges of the de Bruijn graph are of the form for . In particular, for any with and , we have . Thus, lies in this interval if and only if there is an edge from to in . Next, we show that (1) implies (2). We start with the following general remark: for a word and an integer , one has if is the position of the th letter in , according to the order defined by , which is the lexicographic order with equal letters sorted by occurrence position. Suppose is an alphabet-permutation power, so consists of consecutive blocks of length , each of which is a permutation of . In particular, each letter appears exactly times in total. Let and . The index corresponds to the th occurrence of the letter in . Since is a concatenation of blocks, and each block is a permutation of , the th occurrence of letter must appear within the th block of , i.e., within positions . Therefore, , as required. This proves that .
It remains to show that (2) implies (1). Assume that for all and , we have . Fix any . Then the values for all lie in . Since is a permutation, these values must be distinct. Therefore, .
Let us now examine the content of these positions in . The value is the -th letter in this collection. But is the position of the th occurrence of the letter , hence . It follows that , which means that the substring is a permutation of . Since this holds for every , the word is a concatenation of permutations of , i.e., an alphabet-permutation power. This completes the proof.
As a consequence, we have:
Theorem 9.
A necklace is the label of a Hamiltonian cycle of a generalized de Bruijn graph if and only if it is the inverse standard permutation, in cycle form, of the BWT of a generalized de Bruijn word of length over .
Proof.
The fact that every Hamiltonian cycle of corresponds to the inverse standard permutation of some length word over letters follows from condition (2) of Lemma 8 and from the fact that a permutation is the standard permutation of a word over letters if and only if there are at most indices such that [8, Theorem 1]. We use the fact that generalized de Bruijn words are aperiodic necklaces: indeed, if is a power (with ), then its BWT can be derived from the BWT of by repeating each letter times (Proposition 4). Consequently, the BWT of a periodic necklace cannot be an alphabet-permutation power and, conversely, a periodic necklace cannot have a BWT that is an alphabet-permutation power. We now conclude from Lemma 8, and from the fact that a word is a BWT image of an aperiodic necklace if and only if its (inverse) standard permutation is a cycle.
Therefore, the generalized de Bruijn words of length over the alphabet can be constructed from the Hamiltonian cycles of the generalized de Bruijn graph by mapping for every .
Example 10.
The Hamiltonian cycles in are: , , , and . Applying the mapping , one obtains the ternary generalized de Bruijn words of length : , , , and .
Thanks to the characterization given in Theorem 9, we can obtain a formula for counting generalized de Bruijn words. We make use of the BEST theorem for directed graphs, together with the fact that the number of Hamiltonian cycles in is equal to the number of Eulerian cycles in .
Theorem 11 (BEST theorem [34]).
The number of distinct Eulerian cycles in a Eulerian directed graph is given by
| (1) |
where is the outdegree of .
By definition, the outdegree of every node in the generalized de Bruijn graph is , therefore . So, we have the following result.
Theorem 12.
For every , the number of generalized de Bruijn words of length over is
| (2) |
| 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 1 | 2 | 3 | 4 | 7 | 16 | 21 | 48 | 93 | |
| 4 | 24 | 64 | 512 | 1728 | 13312 | 32768 | 373248 | 1310720 | 10903552 |
The number of spanning trees can be computed by means of a determinant, as an application of the matrix-tree theorem below. Recall that the Laplacian matrix of a directed graph is the matrix , where and are the degree matrix and the adjacency matrix of , respectively. It can be defined by
where is the number of edges directed from to , and is the outdegree of , so that is the outdegree of minus the number of its self loops.
Theorem 13 (Matrix-Tree Theorem for Eulerian graphs [17]).
Let be an Eulerian graph. The number is equal to the determinant of the matrix obtained from the Laplacian matrix of after removing the last row and the last column.
The matrix obtained by removing the last row and column from is sometimes called the reduced Laplacian matrix of .
We now discuss the relations between generalized de Bruijn words and sandpile groups, algebraic objects capturing important properties of directed Eulerian graphs, which can be defined in terms of the reduced Laplacian matrix of the graph. We start by giving some standard results and definitions from [33]. Recall that any integer matrix can be written in a canonical form , where both matrices , are integer matrices and is an integer diagonal matrix with the property that if are the nonzero entries of the main diagonal of , then for every . The matrix is called the Smith Normal Form of . The are called the invariant factors of the matrix .
Let be a finite strongly connected directed graph, that is, for any there are directed paths in from to and from to . Then associated to any vertex of is an Abelian group , the sandpile group (also known as critical group, Picard group, Jacobian, and group of components, see [33] for the detailed definition). When is Eulerian, the groups and are isomorphic for any , and we let denote the sandpile group of , and one has:
| (3) |
where are the invariant factors of , or equivalently, are the invariant factors of .
By the matrix-tree theorem, the order of the sandpile group is equal to , the determinant of the reduced Laplacian matrix of .
Example 14.
Consider the generalized de Bruijn graph (Fig. 1). Its Laplacian matrix is
and its Smith Normal Form is
Hence, by (3), we have . The matrix has determinant , so . Thus, by Theorem 12, there are ternary generalized de Bruijn words of length – the same as the number of distinct Eulerian cycles in and Hamiltonian cycles in .
As another example, the Laplacian matrix of is
whose Smith Normal Form is
We have , and in fact there are binary generalized de Bruijn words of length , namely , , , and .
For ordinary de Bruijn graphs over a binary alphabet, the general structure of the sandpile group was described by Levine [20]:
| (4) |
In the case of generalized de Bruijn graphs, Chan, Hollmann and Pasechnik gave the structure of for arbitrary and [5, Theorem 3.1]. We refer the reader to [5, 6] for further details.
In the next sections, we focus on the case where the alphabet size is a prime number , allowing us to connect generalized de Bruijn words to necklaces having a BWT matrix that is invertible over .
4 Invertible necklaces and Reutenauer groups
In this section, we define invertible necklaces and highlight their connections with abstract algebra.
In [35] (see also the related paper [30]), the authors used BWT matrix multiplication to obtain a group structure on important classes of binary words (namely, Christoffel words and balanced words) that have an invertible BWT matrix. Following this idea, we define invertible necklaces:
Definition 15.
Let be a prime. A necklace over the alphabet is called invertible if its BWT matrix is non-singular, i.e., has nonzero determinant modulo .
As observed in [35], the product of two BWT matrices is not, in general, a BWT matrix; but the author suggests replacing BWT matrices by circulant matrices. As we will see, using circulant matrices, we can define the product between any two invertible necklaces.
Let be a word over . The circulant matrix of is the square matrix whose th row is . For example, the circulant matrix of is .
One can immediately observe that, since the circulant matrix of a word can be obtained by permuting the rows of its BWT matrix, a necklace is invertible if and only if any (and then, every) of its associated circulant matrices is invertible.
Recall that for every prime and any positive integer , there is a unique finite field with elements, denoted , and its multiplicative group is cyclic. The set of invertible -circulant matrices over forms, with respect to matrix multiplication, an Abelian group . We consider the group , where is the permutation matrix of the cycle . This group is called the Reutenauer group in [11]. Intuitively, an element of the Reutenauer group corresponds to an equivalence class of invertible circulant matrices, where two circulant matrices are equivalent if and only if they are the circulant matrices of two conjugated words. Thus, is in natural bijection with the set of invertible necklaces of length over , and one can write for the class of circulant matrices associated with a given invertible necklace of length over . In particular, this induces a group structure on the set of such invertible necklaces, isomorphic to , namely with respect to the multiplication defined by where is such that . This multiplication does not depend on the choice of representatives for each necklace (equivalently, for each matrix). The trace of a matrix is defined as the sum of its diagonal coefficients. For a word , one can observe that has trace . Since is invariant under rotation, one can define the trace of a class of circulant matrices as .
As shown in [11], the Reutenauer group acts on the set of all aperiodic necklaces as follows: Let be an aperiodic necklace of length over . For any , we define as the necklace of . In particular, this operation does not depend on the choice of a representative. For example, since .
The orbit of an aperiodic necklace is therefore .
We now recall some classical results and definitions in abstract algebra (see, for example, [23] for a more detailed presentation), which have a strong connection with circulant matrices and invertible necklaces.
It is well known that the number of irreducible polynomials over of degree is equal to the number of aperiodic necklaces of length over , but there is no known canonical bijection between the two sets. The minimal polynomial of a nonzero element of is the (unique) monic polynomial with the least degree for which is a root. This polynomial has as roots all the distinct elements of the form , where is the least integer such that , i.e., the distinct roots of are the orbit of under the application of the Frobenius automorphism . Thus, has degree and can be factored as
The sum of all roots of is called the trace of .
But is also a vector space over . An element of is called normal if forms a vector space basis for over , called a normal basis.
The minimal polynomial of a normal element of over is called a normal polynomial. Normal polynomials have non-zero trace (modulo ).
If is a normal element of , every element of can be written as a linear combination of elements in the normal basis . That is, we can write all the elements of as -length words, or -length vectors. More precisely, if , for every , we can represent with the word . The application of the Frobenius automorphism to an element of the field corresponds to the shift of the corresponding word, and an orbit to a conjugacy class, i.e., to a necklace. This necklace is aperiodic if and only if the orbit is maximal, i.e., has size .
| orbit | orbit as vectors (words) | normal |
|---|---|---|
| no | ||
| yes | ||
| no | ||
| no | ||
| yes | ||
| no |
Fixing a normal element and writing elements of in the induced normal basis, the orbit of corresponds to the necklace . There may be other orbits constituted by normal elements. For example, the orbit corresponding to the necklace is another orbit of normal elements in .
The number of normal elements of is counted by , the generalized Euler’s totient function. Equivalently, counts the number of polynomials over of degree smaller than and coprime with . It can be computed using the formula
for , where is the largest power of dividing , and is the multiplicative order of modulo . The first mention of this formula seems to come from [13], and a more recent study can be found in [15].
The first few values of the sequences and are presented in Table 3.
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 8 | 15 | 24 | 49 | 128 | 189 | 480 | 1023 | 1536 | |
| 1 | 4 | 18 | 32 | 160 | 324 | 1456 | 2048 | 13122 | 25600 | 117128 | 209952 |
Let be a prime, and consider words in as vectors in the vector space .
An -circulant matrix over is in if and only if it is invertible (or non-singular), if and only if its determinant is non-zero modulo ; or, equivalently, if its rows form a normal basis of .
Since every element of a normal basis can be chosen as the first row of a corresponding invertible circulant matrix, the order of is , and the order of (hence, the number of invertible necklaces) is .
We therefore arrive at the following characterization:
Theorem 16.
Let be a necklace over , prime. The following are equivalent:
-
1.
The necklace is invertible;
-
2.
Any word in has an invertible circulant matrix over ;
-
3.
belongs to the Reutenauer group ;
-
4.
Any word in is a vector corresponding to a normal element of ;
-
5.
Every word of length over can be written in a unique way as a sum (modulo ) of words in .
Example 17.
Let and . We have three aperiodic binary necklaces, namely , , and . Out of them, the first and the last are invertible, while the second is not, since . Indeed, in , so the elements are not linearly independent as vectors, and therefore do not form a basis of .
The Reutenauer group is therefore constituted by the classes of matrices and .
So, the set of aperiodic necklaces of length over with invertible BWT matrix forms an abelian group isomorphic to the Reutenauer group , but the multiplication has to be carried out with the circulant matrices rather than with BWT matrices. This was also speculated in a remark we recently found at the end of [35] (see also the related paper [30]).
Lemma 18.
Let be a necklace over and let be the minimal polynomial of , seen as a vector. Then, has trace , where . If is invertible, then .
Proof.
Since the roots of are, as vectors, shifts of , the trace of is the vector . Furthermore, if is normal, all elements of are invertible, and the trace must be nonzero.
In particular, binary invertible necklaces have an odd number of ’s. However, note that, in general, having a non-zero trace is not a sufficient condition for an element to be normal.
Recall that a prime is called -rooted if is a generator of the cyclic group . For example, is -rooted since , while is not -rooted since .
In [7], the authors obtained the following result:
Theorem 19 ([7]).
Let be a positive integer. Every monic irreducible polynomial of degree over with nonzero trace is normal if and only if is either a power of or a -rooted prime.
The latter result, together with Lemma 18, immediately implies the following:
Corollary 20.
Let be a positive integer. Every aperiodic necklace of length over with non-zero weight modulo is invertible if and only if is either a power of or a -rooted prime.
For example, the sequence of -rooted primes is A001122 in OEIS [32]. Therefore, by Corollary 20, this sequence precisely corresponds to the lengths (excluding powers of ) for which all binary necklaces of non-zero weight modulo are invertible. This latter is also the sequence of numbers of the form such that is a BWT image (see [2, 26]), i.e., lengths for which one can have a BWT with the largest possible number of runs (which could be considered as the worst case of the BWT). Notice that words having BWT of the form are, by definition, generalized de Bruijn words.
However, we do not know whether there exist infinitely many , different from a power of , for which every aperiodic necklace of length over with non-zero weight modulo is invertible. This seems to be a difficult problem, since it is related to the famous conjecture of Emil Artin stating that a given integer that is neither a square number nor is a primitive root modulo infinitely many primes .
5 Generalized de Bruijn words and invertible necklaces
In this section, we further explore the connection between the content of Sections 3 and 4. Specifically, we show that over an alphabet whose size is a prime number, generalized de Bruijn words and invertible necklaces are linked not only through their relationship with the Burrows–Wheeler transform, but also via an isomorphism of abelian groups.
First, recall that every finite abelian group is isomorphic to the direct sum of cyclic groups, i.e., . One has, for example, and (remember that the order of is ).
Chan, Hollmann and Pasechnik [5] proved that the structure of Reutenauer groups can be found by means of an isomorphism with the sandpile groups of generalized de Bruijn graphs, as reported in the next theorem.
Theorem 21 ([5]).
Let be the sandpile group of the generalized de Bruijn graph , prime. Then
where is the -th Reutenauer group over .
Thus, for prime, , and therefore .
As a consequence of Theorem 21, we arrive at a remarkable fact: the structure of the Reutenauer groups is entirely determined by the structure of the generalized de Bruijn graphs. An illustrative example is provided in Table 4. We point out that the (reduced) Laplacian matrix of a generalized de Bruijn graph (as well as its Smith Normal Form and invariant factors) can be derived directly from the arithmetic definition of the graph, without the need to construct it explicitly.
We thus obtain the following formula for the number of generalized de Bruijn words over an alphabet of prime size:
Theorem 22.
Let be a prime. For every , the number of generalized de Bruijn words of length over is
| (5) |
where is the number of invertible necklaces of length over .
Proof.
Remark 23.
When is a power of a prime (which corresponds to the case of ordinary de Bruijn words), the precise structure of the sandpile group (and hence of the Reutenauer group) can be easily given in terms of and . Indeed, from the results in [5], one can derive:
Remark 24.
Let , , i.e., . Since , from (5) we get
i.e., the number of (ordinary) de Bruijn words of order over .
In the special case , we have, by Theorem 21, that . As a consequence, we have the following:
Theorem 25.
For every , the following sets are in bijection:
-
The set of binary generalized de Bruijn words of length ;
-
The set of binary invertible necklaces of length ;
-
The set of (unordered) normal bases of .
In particular, when for some , we have a bijection between the set of binary de Bruijn words of length and the set of binary necklaces of length having an odd number of ’s. This follows from Corollary 20. Indeed, notice that we do not need to add the hypothesis that the necklaces are aperiodic, since their lengths are a power of .
Theorem 26.
For every , the following sets are in bijection:
-
The set of binary de Bruijn words of order ;
-
The set of binary necklaces of length having an odd number of ’s;
-
The set of (unordered) normal bases of .
Remark 27.
Any constructive bijection would allow one to derive algorithms for interchanging between these objects. One could, for example, use such a bijection to generate every binary de Bruijn word with uniform probability, a problem that is still open – in [24] the authors presented an efficient algorithm to generate every binary de Bruijn word with positive probability. Moreover, this could be made efficient in practice since the sum in is nothing else than the XOR.
6 Conclusions and Open Problems
As emphasized in the introduction, the central motivation of our work is to present a new formalism, based on combinatorics on words, that bridges various algebraic structures. This approach builds on known results relating sandpile groups of (generalized) de Bruijn graphs and groups of invertible matrices over finite fields (Reutenauer groups). The new classes of words we introduce in this paper – generalized de Bruijn words on the one hand, and conjugacy classes of words (necklaces) with invertible Burrows–Wheeler matrix on the other hand – offer a fresh reinterpretation of these connections in terms of words. The key advantage of the new framework is its foundation in the Burrows–Wheeler transform, which not only clarifies links to longstanding open problems in elementary number theory but also opens new avenues for further research.
We expect that different research communities, including those focused on combinatorics on words, commutative algebra, combinatorial algorithms, finite fields, and cryptography, will build upon the new constructions introduced in this paper. For instance, it would be interesting to develop combinatorial characterizations of the newly introduced classes of words. Another compelling direction is to gain a deeper understanding of the group operation on invertible necklaces and investigate its relationship to the one recently introduced by Zamboni in the context of Christoffel words [35], and further explored in [30].
References
- [1] Yu Hin Au. Generalized de Bruijn words for primitive words and powers. Discret. Math., 338(12):2320–2331, 2015. doi:10.1016/J.DISC.2015.05.025.
- [2] D. J. Aulicino and Morris Goldfeld. A new relation between primitive roots and permutations. The American Mathematical Monthly, 76(6):664–666, 1969. doi:10.1080/00029890.1969.12000297.
- [3] Jean Berstel, Aaron Lauve, Christophe Reutenauer, and Franco Saliola. Combinatorics on Words: Christoffel Words and Repetition in Words, volume 27 of CRM monograph series. American Mathematical Society, 2008.
- [4] Lenore Blum, Manuel Blum, and Michael Shub. Comparison of Two Pseudo-Random Number Generators. In David Chaum, Ronald L. Rivest, and Alan T. Sherman, editors, Advances in Cryptology, pages 61–78, Boston, MA, 1983. Springer US.
- [5] Swee Hong Chan, Henk D. L. Hollmann, and Dmitrii V. Pasechnik. Critical groups of generalized de Bruijn and Kautz graphs and circulant matrices over finite fields. In Jaroslav Nešetřil and Marco Pellegrini, editors, The Seventh European Conference on Combinatorics, Graph Theory and Applications, pages 97–104, Pisa, 2013. Scuola Normale Superiore.
- [6] Swee Hong Chan, Henk D.L. Hollmann, and Dmitrii V. Pasechnik. Sandpile groups of generalized de Bruijn and Kautz graphs and circulant matrices over finite fields. Journal of Algebra, 421:268–295, 2015. Special issue in memory of Ákos Seress. doi:10.1016/j.jalgebra.2014.08.029.
- [7] Yaotsu Chang, T. K. Truong, and I. S. Reed. Normal bases over . Journal of Algebra, 241:89–101, 2001.
- [8] Maxime Crochemore, Jacques Désarménien, and Dominique Perrin. A note on the burrows - wheeler transformation. Theor. Comput. Sci., 332(1-3):567–572, 2005. doi:10.1016/J.TCS.2004.11.014.
- [9] Maxime Crochemore, Christophe Hancart, and Thierry Lecroq. Algorithms on strings. Cambridge University Press, 2007.
- [10] Ding-Zhu Du and Frank K. Hwang. Generalized de Bruijn digraphs. Networks, 18(1):27–38, 1988. doi:10.1002/NET.3230180105.
- [11] S. V. Duzhin and D. V. Pasechnik. Groups acting on necklaces and sandpile groups. Journal of Mathematical Sciences, 200(6):690–697, 2014. doi:10.1007/s10958-014-1960-6.
- [12] Gabriele Fici, Sabrina Mantaci, Antonio Restivo, Giuseppe Romana, Giovanna Rosone, and Marinella Sciortino. BWT and Combinatorics on Words. In Paolo Ferragina, Travis Gagie, and Gonzalo Navarro, editors, The Expanding World of Compressed Data: A Festschrift for Giovanni Manzini’s 60th Birthday, volume 131 of OASIcs. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 2025.
- [13] K. Hensel. Ueber die darstellung der zahlen eines gattungsbereiches für einen beliebigen primdivisor. Journal für die reine und angewandte Mathematik, 1888(103):230–237, 1888. doi:10.1515/crll.1888.103.230.
- [14] Peter M. Higgins. Burrows-Wheeler transformations and de Bruijn words. Theor. Comput. Sci., 457:128–136, 2012. doi:10.1016/J.TCS.2012.07.019.
- [15] Trevor Hyde. Normal elements in finite fields, 2018. arXiv:1809.02155.
- [16] Makoto Imase and Masaki Itoh. Design to minimize diameter on building-block network. IEEE Trans. Computers, 30(6):439–442, 1981. doi:10.1109/TC.1981.1675809.
- [17] G. Kirchhoff. On the solution of the equations obtained from the investigation of the linear distribution of galvanic currents. IRE Transactions on Circuit Theory, 5(1):4–7, 1958. doi:10.1109/TCT.1958.1086426.
- [18] Ben Langmead and Steven L. Salzberg. Fast gapped-read alignment with Bowtie 2. Nat. Methods, 9(4):357–359, 2012.
- [19] Ben Langmead, Cole Trapnell, Mihai Pop, and Steven L Salzberg. Ultrafast and memory-efficient alignment of short DNA sequences to the human genome. Genome Biol., 10(3):R25, 2009.
- [20] Lionel Levine. Sandpile groups and spanning trees of directed line graphs. J. Comb. Theory A, 118(2):350–364, 2011. doi:10.1016/J.JCTA.2010.04.001.
- [21] Heng Li and Richard Durbin. Fast and accurate long-read alignment with Burrows-Wheeler transform. Bioinformatics, 26(5):589–595, 2010. doi:10.1093/BIOINFORMATICS/BTP698.
- [22] Xueliang Li and Fuji Zhang. On the numbers of spanning trees and Eulerian tours in generalized de Bruijn graphs. Discret. Math., 94(3):189–197, 1991. doi:10.1016/0012-365X(91)90024-V.
- [23] Rudolf Lidl and Harald Niederreiter. Finite Fields. Encyclopedia of Mathematics and its Applications. Cambridge University Press, 2 edition, 1996.
- [24] Zsuzsanna Lipták and Luca Parmigiani. A BWT-Based Algorithm for Random de Bruijn Sequence Construction. In José A. Soto and Andreas Wiese, editors, LATIN 2024: Theoretical Informatics - 16th Latin American Symposium, Puerto Varas, Chile, March 18-22, 2024, Proceedings, Part I, volume 14578 of Lecture Notes in Computer Science, pages 130–145. Springer, 2024. doi:10.1007/978-3-031-55598-5_9.
- [25] M. Lothaire. Combinatorics on Words. Advanced Book Program. Addison-Wesley, Advanced Book Program, World Science Division, 1983.
- [26] Sabrina Mantaci, Antonio Restivo, Giovanna Rosone, Marinella Sciortino, and Luca Versari. Measuring the clustering effect of BWT via RLE. Theor. Comput. Sci., 698:79–87, 2017. doi:10.1016/J.TCS.2017.07.015.
- [27] Sabrina Mantaci, Antonio Restivo, and Marinella Sciortino. Burrows-Wheeler transform and Sturmian words. Inf. Process. Lett., 86(5):241–246, 2003. doi:10.1016/S0020-0190(02)00512-4.
- [28] Abhinav Nellore and Rachel A. Ward. Arbitrary-Length Analogs to de Bruijn Sequences. In Hideo Bannai and Jan Holub, editors, 33rd Annual Symposium on Combinatorial Pattern Matching, CPM 2022, June 27-29, 2022, Prague, Czech Republic, volume 223 of LIPIcs, pages 9:1–9:20. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 2022. doi:10.4230/LIPICS.CPM.2022.9.
- [29] S.M. Reddy, D.K. Pradhan, and J.G. Kuhl. Directed graphs with minimum diameter and maximal connectivity. Technical report, School of Engineering, Oakland University, July 1980.
- [30] Christophe Reutenauer and Jeffrey O. Shallit. Christoffel Matrices and Sturmian Determinants. CoRR, abs/2409.09824, 2024. doi:10.48550/arXiv.2409.09824.
- [31] Giovanna Rosone and Marinella Sciortino. The Burrows-Wheeler Transform between Data Compression and Combinatorics on Words. In Paola Bonizzoni, Vasco Brattka, and Benedikt Löwe, editors, The Nature of Computation. Logic, Algorithms, Applications - 9th Conference on Computability in Europe, CiE 2013, Milan, Italy, July 1-5, 2013. Proceedings, volume 7921 of Lecture Notes in Computer Science, pages 353–364. Springer, 2013. doi:10.1007/978-3-642-39053-1_42.
- [32] N. J. A. Sloane. The On-Line Encyclopedia of Integer Sequences. Available electronically at http://oeis.org.
- [33] Richard P. Stanley. Smith normal form in combinatorics. J. Comb. Theory A, 144:476–495, 2016. doi:10.1016/J.JCTA.2016.06.013.
- [34] T. van Aardenne-Ehrenfest and N.G. de Bruijn. Circuits and Trees in Oriented Linear Graphs, pages 149–163. Birkhäuser Boston, Boston, MA, 1987. doi:10.1007/978-0-8176-4842-8_12.
- [35] Luca Q. Zamboni. On Christoffel words and their lexicographic array. CoRR, abs/2409.07974, 2024. doi:10.48550/arXiv.2409.07974.
