TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 348, Number 6, June 1996

RELATIVELY FREE INVARIANT ALGEBRAS OF FINITE REFLECTION GROUPS

MÁTYÁS DOMOKOS

Abstract. Let G be a finite subgroup of Gl_{n}(K) (K is a field of characteristic 0 and n ≥ 2) acting by linear substitution on a relatively free algebra K⟨x_{1}, …, x_{n}⟩/I of a variety of unitary associative algebras. The algebra of invariants is relatively free if and only if G is a pseudo-reflection group and I contains the polynomial [[x_{2}, x_{1}], x_{1}].

1. Introduction

Throughout the paper K is a field of characteristic 0 and K⟨x_{1}, …, x_{n}⟩ is the unitary free associative K−algebra of rank n. The general linear group Gl_{n} = Gl_{n}(K) acts on the free algebra by linear substitution. More explicitly, if g = (g_{ij}) ∈ Gl_{n}, then

g · x_{j} = ∑_{i=1}^{n} g_{ij}x_{i},

and for any f(x_{1}, …, x_{n}) ∈ K⟨x_{1}, …, x_{n}⟩

g · f = f(g · x_{1}, …, g · x_{n}).

An ideal I of the unitary free associative algebra K⟨x_{1}, x_{2}, …⟩ of countable rank is called a T-ideal if I is invariant under any K−algebra endomorphism of the free algebra, that is, f(x_{1}, …, x_{n}) ∈ I implies f(u_{1}, …, u_{n}) ∈ I for any u_{1}, …, u_{n} ∈ K⟨x_{1}, x_{2}, …⟩.

I_{n} = I ∩ K⟨x_{1}, …, x_{n}⟩ is invariant under the action of Gl_{n}, so

F_{n}(I) = K⟨x_{1}, …, x_{n}⟩/I_{n}

inherits the Gl_{n}−structure. F_{n}(I) is a relatively free algebra of rank n of the variety of unitary associative algebras satisfying all the identities f = 0 with f ∈ I. Denote by y_{i} the image of x_{i} under the natural homomorphism

K⟨x_{1}, …, x_{n}⟩ → K⟨x_{1}, …, x_{n}⟩/I_{n}.
\begin{matrix}c \\ ◯ \end{matrix}1996 American Mathematical Society

Received by the editors September 7, 1994.

1991 Mathematics Subject Classification. Primary 16W20; Secondary 16R10.

Key words and phrases. Relatively free algebra, reflection group, invariants, Hilbert series.

This research was partially supported by Széchenyi István Scholarship Foundation and by Hungarian National Foundation for Scientific Research grant no. T4265.

Let G be a finite subgroup of Gl_{n} and denote by

F_{n}(I)^{G} = {f ∈ F_{n}(I) | g · f = f ∀g ∈ G}

the algebra of invariants of G. We refer to the article of Formanek [8] for a survey of the results on the algebra of invariants of a finite linear group acting on a relatively free algebra. In the special case of the variety of all unitary commutative algebras, that is, when I is generated by the commutator [x_{1}, x_{2}] = x_{1}x_{2}−x_{2}x_{1}, the relatively free algebra

F_{n}(I) = K[x_{1}, …, x_{n}]

is the n variable commutative polynomial algebra, and the study of F_{n}(I)^{G} is the topic of classical invariant theory.

An element g ∈ Gl_{n}(K) is called a pseudo-reflection, if it fixes an n − 1 dimensional subspace of K^{n}. If g is of finite order then g is a pseudo-reflection if and only if g has the eigenvalue 1 with multiplicity n−1. A finite subgroup G < Gl_{n} is called a pseudo-reflection group if it is generated by pseudo-reflections. Our starting point is the following famous result (Shephard-Todd [13], Chevalley [2]):

Theorem 1.1. K[x_{1}, …, x_{n}]^{G} is a polynomial algebra if and only if G is a pseudoreflection group. Moreover, if G is a pseudo-reflection group, then there exist n algebraically independent homogeneous forms which generate K[x_{1}, …, x_{n}]^{G}. The degrees d_{1} ≤ … ≤ d_{n} of these forms are uniquely determined by G, and are called the degrees of G. We have the equality |G| = ∏_{i=1}^{n} d_{i}, and the number of pseudoreflections in G is ∑_{i=1}^{n}(d_{i} − 1).

We shall extend the notion of algebraic independence to any variety of unitary associative K−algebras. We fix a T-ideal I, and denote by V(I) the variety defined by I. Let R be an algebra in V(I). We say that the elements f_{1}, …, f_{m} ∈ R are algebraically independent, if h(f_{1}, …, f_{m}) = 0 for some h ∈ K⟨x_{1}, …, x_{m}⟩ implies that h ∈ I. Note that this notion of algebraic independence depends on the T-ideal I and belongs to the variety V(I).

Guralnick proved in [9] that if n ≥ 2, k ≥ 2 and I is the T-ideal of the k × k matrix algebra over K, then F_{n}(I)^{G} is not relatively free for any finite group G. As it was pointed out in [8, p. 105], this result implies easily that F_{n}(I)^{G} is not relatively free if I is contained in the T-ideal of the 2 × 2 matrix algebra. In this paper we give a complete answer to the question of determining when F_{n}(I)^{G} is relatively free. The answer was conjectured by Drensky [7].

The case n = 1 is trivial, because then F_{1}(I) ≅ K[x_{1}] ≅ K⟨x_{1}⟩ and G is a cyclic group acting by scalar multiplication, so the algebra of invariants is K[x^{m}] for some positive integer m.

Theorem 1.2. Let G < Gl_{n}(K) (n ≥ 2) be a finite group and let I be a T-ideal. Then F_{n}(I)^{G} is generated by algebraically independent elements if and only if G is a pseudo-reflection group and I contains the polynomial [[x_{2}, x_{1}], x_{1}].

2. Preliminaries

The algebra K⟨x_{1}, …, x_{n}⟩^{G} is always free and almost never finitely generated by [4] and [10]. However, working in a proper subvariety of the variety of all unitary associative K−algebras the notion of transcendence degree makes sense.

Proposition 2.1. Fix a non-zero T-ideal I and consider the variety V(I). If R is an algebra in the variety V(I) generated by n elements, then it does not contain more than n algebraically independent elements.

Proof. R is a homomorphic image of F_{n}(I) and obviously the preimages of algebraically independent elements of R are also algebraically independent, so it suffices to prove the proposition for the relatively free algebra F_{n}(I). Assume that f_{1}, …, f_{m} ∈ F_{n}(I) are algebraically independent. Let J ⊆ K⟨x_{1}, x_{2}, …⟩ be the radical of I, that is,

J = {f ∈ K⟨x_{1}, x_{2}, …⟩ | f^{N} ∈ I for some N}.

It is well known (see for example [12, Theorems 1.5.32, 2.4.7 and 3.2.6]) that J is the set of identities of the k ×k matrix algebra for some k ≥ 1. We have the natural onto homomorphism

K⟨x_{1}, …, x_{n}⟩/I_{n} → K⟨x_{1}, …, x_{n}⟩/J_{n}.

Denote by h_{1}, …, h_{m} the images of f_{1}, …, f_{m}. Suppose that for some p ∈ K⟨x_{1}, …, x_{m}⟩ we have p(h_{1}, …, h_{m}) = 0. Then there exists an integer N such that p^{N}(f_{1}, …, f_{m}) = 0 in F_{n}(I), and the algebraic independence of the f_{i}s implies that p^{N}(x_{1}, …, x_{m}) ∈ I_{m}. Hence p(x_{1}, …, x_{m}) ∈ J_{m} by the definition of J_{m}, and this shows the algebraic independence of h_{1}, …, h_{m} in F_{n}(J).

The center of F_{m}(J) is a commutative domain and its transcendence degree is d = (m − 1)k^{2} + 1 (see for example [8, p. 105]). Let

c_{1} = c_{1}(y_{1}, …, y_{m}), …, c_{d} = c_{d}(y_{1}, …, y_{m})

be a transcendence basis of the center of F_{m}(J) (so the c_{i}(x_{1}, …, x_{m}) are central polynomials for the k × k matrix algebra). The algebraic independence of h_{1}, …, h_{m} in F_{n}(J) together with the algebraic independence (in the ordinary sense) of c_{1}, …, c_{d} in the center of F_{m}(J) implies that c_{i}(h_{1}, …, h_{m}) (i = 1, …, d) are algebraically independent in the center of F_{n}(J). But this center has transcendence degree (n − 1)k^{2} + 1, implying that m ≤ n. □

We use the following crucial argument from [8] or [9]. Any non-zero T-ideal I is contained in the T-ideal generated by [x_{1}, x_{2}]. So we have the natural onto homomorphism

F_{n}(I) → K[x_{1}, …, x_{n}]

which commutes with the action of G, and by the complete reducibility of this action the homomorphism

F_{n}(I)^{G} → K[x_{1}, …, x_{n}]^{G}

is also onto. Therefore the image of a generating set of F_{n}(I)^{G} is a generating set of K[x_{1}, …, x_{n}]^{G}, hence F_{n}(I)^{G} can not be generated by less than n elements. Thus if F_{n}(I)^{G} is generated by algebraically independent elements, then by Proposition 2.1 it is generated by n elements.

Assume that f_{1}, …, f_{n} generate F_{n}(I)^{G}. Their images in K[x_{1}, …, x_{n}] generate K[x_{1}, …, x_{n}]^{G}, hence G must be a pseudo-reflection group. We may assume by [9, Lemma 2] that f_{1}, …, f_{n} are homogeneous, therefore their degrees are the degrees of G. Moreover, the proof of [9, Lemma 2] shows that if h_{1}, …, h_{n} ∈ F_{n}(I)^{G} such that their images generate K[x_{1}, …, x_{n}]^{G}, then F_{n}(I)^{G} = K⟨h_{1}, …, h_{n}⟩.

By the above discussion Theorem 1.2 splits into the next two statements:

Theorem 2.2. Let G be a pseudo-reflection group and let I be a T-ideal containing [[x_{2}, x_{1}], x_{1}]. If f_{1}, …, f_{n} are homogeneous elements in F_{n}(I)^{G} such that their images generate K[x_{1}, …, x_{n}]^{G}, then f_{1}, …, f_{n} are algebraically independent in F_{n}(I) and they generate F_{n}(I)^{G}. In particular, F_{n}(I)^{G} ≅ F_{n}(I).

Theorem 2.3. If the T-ideal I does not contain the polynomial [[x_{2}, x_{1}], x_{1}], then F_{n}(I)^{G} can not be generated by n elements for any n ≥ 2 and finite group G < Gl_{n}(K).

We recall some basic facts about the T-ideals of the unitary free associative K−algebra. Let B denote the subalgebra of K⟨x_{1}, x_{2}, …⟩ generated by all the commutators [x_{i_{1}}, …, x_{i_{r}}] with r ≥ 2, where [x_{1}, …, x_{r}] is defined inductively by [x_{1}, …, x_{r}] = [[x_{1}, …, x_{r−1}], x_{r}] for r ≥ 3. The elements of B are called proper polynomials, and they can be characterised as the polynomials with zero partial derivatives. Clearly, B_{n} = B∩K⟨x_{1}, …, x_{n}⟩ is a Gl_{n}−submodule of K⟨x_{1}, …, x_{n}⟩. Drensky showed (see [5, Theorem 2.2]) that any T-ideal I of the free unitary algebra is generated by the proper polynomials that it contains, and as Gl_{n}−modules

(2.1)  F_{n}(I) ≅ K[x_{1}, …, x_{n}] ⊗ (B_{n}/B_{n} ∩ I).

Both I_{n} and B_{n} are multigraded subalgebras with respect to the usual multigrading on K⟨x_{1}, …, x_{n}⟩, so F_{n}(I) and B_{n}/B_{n} ∩ I inherit the multigrading.

For any α = (α_{1}, …, α_{n}) and multigraded algebra R we put

R^{(α)} = {f ∈ R | f is multihomogeneous of multidegree α},

and for any ℕ−graded algebra R and non-negative integer d

R^{(d)} = {f ∈ R | f is homogeneous of total degree d}.

Obviously B_{n}^{(0)} = K and B_{n}^{(1)} = ∅. If n ≥ 2, then B_{n}^{(2)} is an irreducible Gl_{n}−module generated by [x_{1}, x_{2}], and B_{n}^{(3)} is an irreducible Gl_{n}−module generated by [x_{2}, x_{1}, x_{1}]. Therefore there exists a unique maximal T-ideal M among the T-ideals not containing [x_{2}, x_{1}, x_{1}], M is generated by the proper polynomials of degree greater than 3 (in particular, M has no elements of degree less than 4). If I is a T-ideal not containing [x_{2}, x_{1}, x_{1}], then F_{n}(M)^{G} is a homomorphic image of F_{n}(I)^{G}, thus Theorem 2.3 is a consequence of the following more special statement:

Proposition 2.4. Let M be the T-ideal generated by the proper polynomials of degree greater than 3, and let G < Gl_{n} (n ≥ 2) be a finite pseudo-reflection group. Then F_{n}(M)^{G} can not be generated by n elements.

The Hilbert series of a multigraded algebra R is an n variable formal power series defined by

H(R; t_{1}, …, t_{n}) = ∑_{α=(α_{1},...,α_{n})} dim_{K}(R^{(α)})t_{1}^{α_{1}} … t_{n}^{α_{n}},

and the Hilbert series of an ℕ−graded algebra R is the formal power series

H(R; t) = ∑_{d=0}^{∞} dim_{K}(R^{(d)})t^{d}.

Let G be a finite subgroup of Gl_{n}(K). For any element g ∈ G denote by ω_{1}(g), …, ω_{n}(d) the eigenvalues of g in the algebraic closure of K. F_{n}(I)^{G} is an ℕ−graded algebra, and we have the noncommutative Molien-Weyl formula (see [8, Theorem 7]):

(2.2)  H(F_{n}(I)^{G}; t) = \frac{1}{|G|} ∑_{g∈G} H(F_{n}(I); ω_{1}(g)t, …, ω_{n}(g)t).

3. Proof of Theorem 2.2

Denote by J the T-ideal generated by [x_{2}, x_{1}, x_{1}], and let

s_{d}(x_{1}, …, x_{d}) = ∑_{π∈Sym(d)}(−1)^{π}x_{π(1)} … x_{π(d)}

be the standard polynomial. It is well known (see [6, 3.2.1. Theorem]) that B_{n}^{(d)}/B_{n}^{(d)} ∩ J = 0, if d is odd or d > n, and it is an irreducible Gl_{n}−module generated by s_{d}(y_{1}, …, y_{d}) if d is even and d ≤ n. Let J(m) be the T-ideal generated by [x_{2}, x_{1}, x_{1}] and s_{2m}(x_{1}, …, x_{2m}), so the only T-ideals containing [x_{2}, x_{1}, x_{1}] are J and J(m) (m = 1, 2, …). We note that

K⟨x_{1}, …, x_{n}⟩/J(m)_{n} = K⟨x_{1}, …, x_{n}⟩/J_{n} if 2m > n.

Let f_{1}, …, f_{n} ∈ F_{n}(J(m))^{G} be homogeneous invariants such that their images in K[x_{1}, …, x_{n}] generate K[x_{1}, …, x_{n}]^{G}. We prove by induction on m that f_{1}, …, f_{n} are algebraically independent in F_{n}(J(m)). In the case m = 1 there is nothing to prove. Suppose that the statement is true for F_{n}(J(k)), where k ≤ m. We may assume that 2m ≤ n. The algebra F_{n}(J(m + 1)) has a linear basis (see [6])

{y_{1}^{α_{1}} … y_{n}^{α_{n}}s_{2k}(y_{i_{1}}, …, y_{i_{2k}})|α_{1}, …, α_{n} ∈ ℕ, 0 ≤ k ≤ m, 1 ≤ i_{1} < … < i_{2k} ≤ n}.

Suppose that h(f_{1}, …, f_{n}) = 0 in F_{n}(J(m + 1)) for some h ∈ K⟨x_{1}, …, x_{n}⟩ with h ∉ J(m + 1)_{n}. Then we may assume that h is of the form

h = ∑_{α=(α_{1},...,α_{n})} ∑_{k=0}^{m} ∑_{1≤i_{1}<...<i_{2k}≤n} a_{α,k,i}x_{1}^{α_{1}} … x_{n}^{α_{n}}s_{2k}(x_{i_{1}}, …, x_{i_{2k}}),

where at least one of the coefficients a_{α,k,i} is non-zero. Let k_{0} be the minimal k such that a_{α,i,k} ≠ 0 for some α, i. If k_{0} ≤ m − 1, then by the induction hypothesis

∑_{α,i} a_{α,k_{0},i}f_{1}^{α_{1}} … f_{n}^{α_{n}}s_{2k_{0}}(f_{i_{1}}, …, f_{i_{2k_{0}}} ) ∉ J_{n}(k_{0} + 1)/J_{n}(m + 1).

On the other hand

∑_{α,k>k_{0},i} a_{α,k,i}f_{1}^{α_{1}} … f_{n}^{α_{n}}s_{2k}(f_{i_{1}}, …, f_{i_{2k}}) ≡ 0 (modulo J_{n}(k_{0} + 1)/J_{n}(m + 1)),

contradicting that h(f_{1}, …, f_{n}) = 0 in F_{n}(J(m + 1)). Thus k_{0} = m and we have

∑_{α,i} a_{α,i}f_{1}^{α_{1}} … f_{n}^{α_{n}}s_{2m}(f_{i_{1}}, …, f_{i_{2m}}) = 0

in F_{n}(J(m + 1)).

Define the maps

\frac{∂}{∂y_{i}} : F_{n}(J(m + 1)) → F_{n}(J(m + 1)) (i = 1, …, n)

in the following way. If f = f(y_{1}, …, y_{n}) ∈ F_{n}(J(m + 1)) is multihomogeneous of multidegree α = (α_{1}, …, α_{n}), then let \frac{∂}{∂y_{i}} f be the multihomogeneous component of multidegree (α_{1}, …, α_{i} − 1, …, α_{n}) of f(y_{1} + 1, …, y_{n} + 1) (\frac{∂}{∂y_{i}} f = 0 if α_{i} = 0). Now extend \frac{∂}{∂y_{i}} to F_{n}(J(m + 1)) by linearity. Clearly, \frac{∂}{∂y_{i}} is a derivation.

Lemma 3.1. For any f, g ∈ F_{n}(J(m + 1)) we have the equality

[f, g] ≡ ∑_{1≤i,j≤n} \frac{∂f}{∂y_{i}} \frac{∂g}{∂y_{j}} [y_{i}, y_{j}] (modulo C^{2}),

where

C = F_{n}(J(m + 1))[F_{n}(J(m + 1)), F_{n}(J(m + 1))]F_{n}(J(m + 1))

is the commutator ideal of F_{n}(J(m + 1)).

Proof. By the multilinearity of the derivations and [ , ] it suffices to prove the lemma when

f = y_{i_{1}} … y_{i_{k}} and g = y_{j_{1}} … y_{j_{l}}

are monomials. We have

(y_{i_{1}} … y_{i_{k}})(y_{j_{1}} … y_{j_{l}})

= y_{i_{1}} … y_{i_{k−1}}y_{j_{1}}y_{i_{k}}y_{j_{2}} … y_{j_{l}} + y_{i_{1}} … y_{i_{k−1}}[y_{i_{k}}, y_{j_{1}}]y_{j_{2}} … y_{j_{l}} = (y_{i_{1}} … y_{i_{k−1}})(y_{j_{2}} … y_{j_{l}})[y_{i_{k}}, y_{j_{1}}] + y_{i_{1}} … y_{i_{k−1}}y_{j_{1}}y_{i_{k}}y_{j_{2}} … y_{j_{l}}

(the second equality follows from the fact that any commutator lies in the center of F_{n}(J(m + 1))). Now exchange y_{i_{k−1}} and y_{j_{1}} in the second term of the above sum, that is, replace this term by

y_{i_{1}} … y_{i_{k−2}}y_{j_{1}}y_{i_{k−1}}y_{i_{k}}y_{j_{2}} … y_{j_{l}} + (y_{i_{1}} … y_{i_{k−2}}y_{j_{k}})(y_{j_{2}} … y_{j_{l}})[y_{i_{k−1}}, y_{j_{1}}].

Continuing this process we obtain

(y_{i_{1}} … y_{i_{k}})(y_{j_{1}} … y_{j_{l}})

= y_{j_{1}}(y_{i_{1}} … y_{i_{k}})(y_{j_{2}} … y_{j_{l}}) + ∑_{r=1}^{k}(y_{i_{1}} … ŷ_{i_{r}} … y_{i_{k}})(y_{j_{2}} … y_{j_{l}})[y_{i_{r}}, y_{j_{1}}],

where the sign ŷ_{i_{r}} means that we delete y_{i_{r}} in the word y_{i_{1}} … y_{i_{k}}. On applying the same process to the word (y_{i_{1}} … y_{i_{k}})(y_{j_{2}} … y_{j_{l}}) we obtain that

(y_{i_{1}} … y_{i_{k}})(y_{j_{1}} … y_{j_{l}}) = ∑_{r=1}^{k}(y_{i_{1}} … ŷ_{i_{r}} … y_{i_{k}})(y_{j_{2}} … y_{j_{l}})[y_{i_{r}}, y_{j_{1}}]

+ ∑_{r=1}^{k} y_{j_{1}}(y_{i_{1}} … ŷ_{i_{r}} … y_{i_{k}})(y_{j_{3}} … y_{j_{l}})[y_{i_{r}}, y_{j_{2}}]

+ y_{j_{1}}y_{j_{2}}(y_{i_{1}} … y_{i_{k}})(y_{j_{3}} … y_{j_{l}})

≡^{(mod C^{2})} ∑_{r=1}^{k}(y_{i_{1}} … ŷ_{i_{r}} … y_{i_{k}})(ŷ_{j_{1}}y_{j_{2}} … y_{j_{l}})[y_{i_{r}}, y_{j_{1}}]

+ ∑_{r=1}^{k}(y_{i_{1}} … ŷ_{i_{r}} … y_{i_{k}})(y_{j_{1}} ŷ_{j_{2}}y_{j_{3}} … y_{j_{l}})[y_{i_{r}}, y_{j_{2}}]

+ y_{j_{1}}y_{j_{2}}(y_{i_{1}} … y_{i_{k}})(y_{j_{3}} … y_{j_{l}})

= ∑_{i=1}^{n}(\frac{∂}{∂y_{i}} (y_{i_{1}} … y_{i_{k}}))(ŷ_{j_{1}}y_{j_{2}} … y_{j_{l}})[y_{i}, y_{j_{1}}]

+ ∑_{i=1}^{n}(\frac{∂}{∂y_{i}} (y_{i_{1}} … y_{i_{k}}))(y_{j_{1}} ŷ_{j_{2}} … y_{j_{l}})[y_{i}, y_{j_{2}}]

+ y_{j_{1}}y_{j_{2}}(y_{i_{1}} … y_{i_{k}})(y_{j_{3}} … y_{j_{l}}).

Repeating this algorithm finally we get

(y_{i_{1}} … y_{i_{k}})(y_{j_{1}} … y_{j_{l}})

≡^{(mod C^{2})} (y_{j_{1}} … y_{j_{l}})(y_{i_{1}} … y_{i_{k}}) + ∑_{i=1}^{n} ∑_{r=1}^{l}(\frac{∂}{∂y_{i}} (y_{i_{1}} … y_{i_{k}}))(y_{j_{1}} … ŷ_{j_{r}} … y_{j_{l}})[y_{i}, y_{j_{r}}]

= (y_{j_{1}} … y_{j_{l}})(y_{i_{1}} … y_{i_{k}}) + ∑_{i=1}^{n} ∑_{j=1}^{n}(\frac{∂}{∂y_{i}} (y_{i_{1}} … y_{i_{k}}))(\frac{∂}{∂y_{j}} (y_{j_{1}} … y_{j_{l}}))[y_{i}, y_{j}],

which explicitly shows the claim. □

Now we use Lemma 3.1 to rewrite A = s_{2m}(f_{1}, …, f_{2m}). By the definition of s_{2m}

A = \frac{1}{2^{m}} ∑_{π∈Sym(2m)}(−1)^{π}[f_{π(1)}, f_{π(2)}] … [f_{π(2m−1)}, f_{π(2m)}].

We have C^{m+1} = 0 in F_{n}(J(m+1)), because the commutators are central elements of F_{n}(J(m + 1)) and any proper polynomial of degree greater than 2m + 1 is contained in J(m + 1). Therefore in the right hand side of the above equality we may replace [f_{π(2k−1)}, f_{π(2k)}] by something that is congruent with it modulo C^{2}.

2224 M. DOMOKOS

So by Lemma 3.1 we have

A = \frac{1}{2^{m}} ∑_{π∈Sym(2m)}(−1)^{π} ∑_{(i_{1},...,i_{2m})} \frac{∂f_{π(1)}}{∂y_{i_{1}}} \frac{∂f_{π(2)}}{∂y_{i_{2}}} [y_{i_{1}}, y_{i_{2}}]

… \frac{∂f_{π(2m−1)}}{∂y_{i_{2m−1}}} \frac{∂f_{π(2m)}}{∂y_{i_{2m}}} [y_{i_{2m−1}}, y_{i_{2m}}]

= \frac{1}{2^{m}} ∑_{π∈Sym(2m)}(−1)^{π} ∑_{(i_{1},...,i_{2m})} \frac{∂f_{π(1)}}{∂y_{i_{1}}} … \frac{∂f_{π(2m)}}{∂y_{i_{2m}}} [y_{i_{1}}, y_{i_{2}}] … [y_{i_{2m−1}}, y_{i_{2m}}].

The polynomial [x_{1}, x_{2}][x_{1}, x_{3}] is contained in J, hence if the i_{1}, …, i_{2m} are not pairwise different, then the corresponding term is zero in the above expression, implying that

A = \frac{1}{2^{m}} ∑_{π∈Sym(2m)}(−1)^{π} ∑_{1≤i_{1}<...<i_{2m}≤n} ∑_{ρ∈Sym(2m)} \frac{∂f_{π(1)}}{∂y_{i_{ρ(1)}}}

… \frac{∂f_{π(2m)}}{∂y_{i_{ρ(2m)}}} [y_{i_{ρ(1)}}, y_{i_{ρ(2)}}] … [y_{i_{ρ(2m−1)}}, y_{i_{ρ(2m)}}]

= ∑_{1≤i_{1}<...<i_{2m}≤n} \frac{1}{2^{m}} ∑_{ρ∈Sym(2m)}(−1)^{ρ}[y_{i_{ρ(1)}}, y_{i_{ρ(2)}}] … [y_{i_{ρ(2m−1)}}, y_{i_{ρ(2m)}}]

× ∑_{π∈Sym(2m)}(−1)^{π}(−1)^{ρ} \frac{∂f_{π(1)}}{∂y_{i_{ρ(1)}}} … \frac{∂f_{π(2m)}}{∂y_{i_{ρ(2m)}}} .

Again by C^{m+1} = 0 we may permute \frac{∂f_{π(1)}}{∂y_{i_{ρ(1)}}} , …, \frac{∂f_{π(2m)}}{∂y_{i_{ρ(2m)}}} among each other in the above expression, and we get

A = ∑_{1≤i_{1}<...<i_{2m}≤n} ∑_{σ∈Sym(2m)}(−1)^{σ} \frac{∂f_{1}}{∂y_{i_{σ(1)}}} … \frac{∂f_{2m}}{∂y_{i_{σ(2m)}}} )s_{2m}(y_{i_{1}}, …, y_{i_{2m}}).

For any 1 ≤ i_{1} < … < i_{2m} ≤ n and 1 ≤ j_{1} < … < j_{2m} ≤ n we put

f_{j_{1},...,j_{2m}}^{i_{1},...,i_{2m}} = ∑_{σ∈Sym(2m)}(−1)^{σ} \frac{∂f_{i_{1}}}{∂y_{j_{σ(1)}}} … \frac{∂f_{i_{2m}}}{∂y_{j_{σ(2m)}}} .

Consider the natural homomorphism

ψ : F_{n}(J(m + 1)) → K[x_{1}, …, x_{n}].

The image of any f ∈ F_{n}(J(m + 1)) has a normal form

ψ(f) = ∑_{α=(α_{1},...,α_{n})} b_{α}x_{1}^{α_{1}} … x_{n}^{α_{n}}.

Now let ϕ be the map F_{n}(J(m + 1)) → F_{n}(J(m + 1)) defined by

ϕ(f) = ∑_{α} b_{α}y_{1}^{α_{1}} … y_{n}^{α_{n}}.

Obviously, we have ϕ(f) ≡ f (mod C), hence C^{m+1} = 0 implies that

fs_{2m}(y_{i_{1}}, …, y_{i_{2m}}) = ϕ(f)s_{2m}(y_{i_{1}}, …, y_{i_{2m}}).

Summarizing, we have

0 = h(f_{1}, …, f_{n})

= ∑_{α=(α_{1},...,α_{n})} ∑_{1≤i_{1}<...<i_{2m}≤n} a_{α,i}f_{1}^{α_{1}} … f_{n}^{α_{n}}s_{2m}(f_{i_{1}}, …, f_{i_{2m}})

= ∑_{1≤j_{1}<...<j_{2m}≤n} ϕ(∑_{α,i} a_{α,i}f_{1}^{α_{1}} … f_{n}^{α_{n}}f_{j_{1},...,j_{2m}}^{i_{1},...,i_{2m}})s_{2m}(y_{j_{1}}, …, y_{j_{2m}}).

Now since the elements y_{1}^{α_{1}} … y_{n}^{α_{n}}s_{2m}(y_{i_{1}}, …, y_{i_{2m}}) are linearly independent in F_{n}(J(m + 1)),

ϕ(∑_{α,i} a_{α,i}f_{1}^{α_{1}} … f_{n}^{α_{n}}f_{j_{1},...,j_{2m}}^{i_{1},...,i_{2m}})

must be zero for all j_{1}, …, j_{2m}, implying that

ψ(∑_{α,i} a_{α,i}f_{1}^{α_{1}} … f_{n}^{α_{n}}f_{j_{1},...,j_{2m}}^{i_{1},...,i_{2m}}) = 0,

because the monomials y_{1}^{β_{1}} … y_{n}^{β_{n}} are linearly independent in F_{n}(J(m + 1)). Introduce the symbols dx_{1}, …, dx_{n}, and let

E = K⟨dx_{1}, …, dx_{n} | dx_{i}dx_{j} = −dx_{j}dx_{i}, 1 ≤ i, j ≤ n⟩

be the Grassmann algebra of an n dimensional linear space. Consider the map

d : K[x_{1}, …, x_{n}] → K[x_{1}, …, x_{n}] ⊗ E

defined by

df = d(f) = ∑_{i=1}^{n} \frac{∂f}{∂x_{i}} dx_{i}.

The commutative polynomials h_{i} = ψ(f_{i}) (i = 1, …, n) form an algebraically independent generating set of K[x_{1}, …, x_{n}]^{G}. Direct computation shows that

∑_{α,i} a_{α,i}h_{1}^{α_{1}} … h_{n}^{α_{n}}dh_{i_{1}} … dh_{i_{2m}}

= ∑_{1≤j_{1}<...<j_{2m}≤n} ψ(∑_{α,i} a_{α,i}f_{1}^{α_{1}} … f_{n}^{α_{n}}f_{j_{1},...,j_{2m}}^{i_{1},...,i_{2m}})dx_{j_{1}} … dx_{j_{n}},

and by our hypothesis the latter is zero. Now [14, Theorem] implies that

{h_{1}^{α_{1}} … h_{n}^{α_{n}}dh_{i_{1}} … dh_{i_{2m}} | α_{1}, …, α_{n} ∈ ℕ, 1 ≤ i_{1} < … < i_{2m} ≤ n}

is a linearly independent subset of K[x_{1}, …, x_{n}] ⊗ E, hence a_{α,i} = 0 for every α, i, contradicting the hypothesis that f_{1}, …, f_{n} ∈ F_{n}(J(m + 1)) are algebraically dependent.

Remark. The above argument did not use the fact that f_{1}, …, f_{n} are invariants. So we have proved for all m = 1, 2, … that if f_{1}, …, f_{n} ∈ F_{n}(J(m)) are algebraically independent (in the ordinary sense) modulo the commutator ideal, then they are algebraically independent in F_{n}(J(m)).

We have to show that f_{1}, …, f_{n} generate F_{n}(J(m))^{G}. They are homogeneous of degree d_{1}, …, d_{n}, and as we pointed out earlier these are the degrees of G. We conclude from the formula (2.1) and the Gl_{n}−structure of B_{n}/B_{n} ∩ J(m) that the Hilbert series of F_{n}(J(m)) is

H(F_{n}(J(m)); t_{1}, …, t_{n}) = \frac{∑_{i=0}^{m−1} σ_{2i}(t_{1}, …, t_{n})}{∏_{j=1}^{n}(1 − t_{j})},

where σ_{k} is the kth elementary symmetric function (σ_{0} = 1 and σ_{k} = 0 if k > n). We have the isomorphism

K⟨f_{1}, …, f_{n}⟩ ≅ F_{n}(J(m)),

hence the ℕ−graded Hilbert series of the graded subalgebra K⟨f_{1}, …, f_{n}⟩ of F_{n}(J(m))^{G} is

H(F_{n}(J(m)); t^{d_{1}}, …, t^{d_{n}}) = \frac{∑_{i=0}^{m−1} σ_{2i}(t^{d_{1}}, …, t^{d_{n}})}{∏_{j=1}^{n}(1 − t^{d_{j}})}.

Using the noncommutative Molien-Weyl formula (2.2)

H(F_{n}(J(m))^{G}; t) = \frac{1}{|G|} ∑_{g∈G} \frac{∑_{i=0}^{m−1} σ_{2i}(ω_{1}(g)t, …, ω_{n}(g)t)}{∏_{j=1}^{n}(1 − ω_{j}(g)t)}.

Solomon’s formula

(3.1)  \frac{1}{|G|} ∑_{g∈G} \frac{σ_{p}(ω_{1}(g), …, ω_{n}(g))}{∏_{j=1}^{n}(1 − ω_{j}(g)t)} = \frac{σ_{p}(t^{d_{1}−1}, …, t^{d_{n}−1})}{∏_{j=1}^{n}(1 − t^{d_{j}})}

(see [14, formula (5)]) says that F_{n}(J(m))^{G} has the same Hilbert series as its subalgebra K⟨f_{1}, …, f_{n}⟩, therefore the two algebras must coincide. This finishes the proof of Theorem 2.2.

Remark. The polynomial [x_{2}, x_{1}, x_{1}] appears in the theory of PI-algebras as a generator of the T-ideal of identities of the infinite dimensional Grassmann algebra (cf. [11]). It is interesting to note that both parts of the above proof use Solomon’s results from [14] on a pseudo-reflection group acting on the tensor product of K[x_{1}, …, x_{n}] and the Grassmann algebra of an n dimensional vector space.

4. Proof of Proposition 2.4

Let G be a pseudo-reflection group, and suppose that F_{n}(M)^{G} is generated by n elements (where M is the T-ideal generated by all the proper polynomials of degree greater than 3). As we mentioned in Section 2, then F_{n}(M)^{G} is generated by its homogeneous elements f_{1}, …, f_{n} if and only if their images in K[x_{1}, …, x_{n}] generate K[x_{1}, …, x_{n}]^{G}. In the sequel we assume that f_{1}, …, f_{n} ∈ F_{n}(M)^{G} have this property. By (2.1) the Hilbert series of F_{n}(M) is

H(F_{n}(M); t_{1}, …, t_{n}) = \frac{1 + S_{(1,1)}(t_{1}, …, t_{n}) + S_{(2,1)}(t_{1}, …, t_{n})}{∏_{i=1}^{n}(1 − t_{i})},

where S_{λ}(t_{1}, …, t_{n}) is the Schur function corresponding to the partition λ = (λ_{1}, …, λ_{n}). On expressing the Schur functions S_{(1,1)} and S_{(2,1)} by the elementary symmetric polynomials we obtain

H(F_{n}(M); t_{1}, …, t_{n}) = \frac{1 + σ_{2}(t_{1}, …, t_{n}) − σ_{3}(t_{1}, …, t_{n}) + σ_{1}σ_{2}(t_{1}, …, t_{n})}{∏_{i=1}^{n}(1 − t^{d_{i}})}.

Since F_{n}(M)^{G} is generated by f_{1}, …, f_{n}, it is an ℕ−graded homomorphic image of F_{n}(M), where we give the degrees d_{1}, …, d_{n} to the generators of F_{n}(M). Hence in the formal power series

D(t) = H(F_{n}(M); t^{d_{1}}, …, t^{d_{n}}) − H(F_{n}(M)^{G}; t)

each coefficient is a non-negative integer. By (2.2) and (3.1) we have

D(t) = \frac{F (t)}{∏_{i=1}^{n}(1 − t^{d_{i}})} − H_{1}(t),

where

F (t) = (t^{d_{1}} + … + t^{d_{n}}) ∑_{1≤i<j≤n} t^{d_{i}+d_{j}}

and

(4.1)  H_{1}(t) = \frac{1}{|G|} ∑_{g∈G} \frac{(ω_{1}(g)t + … + ω_{n}(g)t)(∑_{1≤i<j≤n} ω_{i}(g)ω_{j}(g)t^{2})}{∏_{i=1}^{n}(1 − ω_{i}(g)t)}.

Steinberg proved (see [1, p. 127]) that

∑_{1≤i<j≤n} ω_{i}ω_{j} : G → ℂ

is an irreducible character of G. (Actually, ∑ ω_{i}ω_{j} is the character of G < Gl(V) acting on the second exterior power of V, and we do not need here the irreducibility.) Therefore

(ω̄_{1} + … + ω̄_{n}) ∑_{1≤i<j≤n} ω̄_{i}ω̄_{j}

is a character of degree n\binom{n}{2} of G, and we may decompose it as ∑_{i=1}^{r} m_{i}χ_{i}, where χ_{1}, …, χ_{r} are pairwise different irreducible characters of G and m_{1}, …, m_{r} are positive integers. Let I be the ideal of K[x_{1}, …, x_{n}] generated by the invariants of G of strictly positive degree. Chevalley showed in [2] that the representation of G on K[x_{1}, …, x_{n}]/I is equivalent to the regular representation. Following [15] we associate with any irreducible character χ of G the polynomial

p_{χ}(t) = ∑_{i=0}^{∞} a_{i}(χ)t^{i},

where a_{i}(χ) is the multiplicity of χ in the ith homogeneous component of K[x_{1}, …, x_{n}]/I. (Note that what we call p_{χ}(t) is pχ̄(t) in [15], because there G acts on K[x_{1}, …, x_{n}] via the adjoint representation.) It turns out that a_{i}(χ) = 0 if i > ∑_{j=1}^{n}(d_{j} − 1) [15, Lemma 2.9]. The coefficient of t^{d} in the formal power series

\frac{1}{|G|} ∑_{g∈G} \frac{(ω_{1}(g) + … + ω_{n}(g)) ∑_{1≤i<j≤n} ω_{i}(g)ω_{j}(g)}{∏_{i=1}^{n}(1 − ω_{i}(g)t)}

is the scalar product of the character ∑_{i=1}^{r} m_{i}χ_{i} and the character of G acting on the dth homogeneous component of K[x_{1}, …, x_{n}]. So by [15, 2.6 Proposition and 2.9 Lemma] we have

(4.2)  H_{1}(t) = \frac{t^{3} ∑_{i=1}^{r} m_{i}p_{χ_{i}}(t)}{∏_{i=1}^{n}(1 − t^{d_{i}})} = \frac{G(t)}{∏_{i=1}^{n}(1 − t^{d_{i}})},

where G(t) is of the form

G(t) = ∑_{i=1}^{n\binom{n}{2}} t^{e_{i}}

with 0 ≤ e_{1} ≤ … ≤ e_{n\binom{n}{2}} ≤ ∑_{i=1}^{n}(d_{i} − 1) + 3.

Now we reduce to the case of irreducible groups as it was done by Guralnick in [9]. Assume that n ≥ 2 is minimal such that F_{n}(M)^{G} is generated by n elements for some pseudo-reflection group G.

1. The same argument as in [9] shows that G is not Abelian.

[9, Lemma 3] remains valid with the same proof for any relatively free algebra instead of the generic matrix algebra. Therefore the minimality of n implies that G is irreducible, and we may assume that G is a complex unitary pseudo-reflection group. We shall use the classification of these groups given in [13].

2. n > 2.

Suppose that n = 2, and let χ = ω_{1} + ω_{2} denote the character of the given representation of G as a subgroup of Gl_{2}(ℂ). We showed above that the degree of G(t) (see (4.2)) is at most d_{1} + d_{2} + 1. Though this bound would suffice for our purpose, for sake of completeness we derive some more precise information on the polynomial G(t) :

H_{1}(t) = \frac{1}{|G|} ∑_{g∈G} \frac{χ(g)ω_{1}(g)ω_{2}(g)t^{3}}{(1 − ω_{1}(g)t)(1 − ω_{2}(g)t)}

= \frac{1}{|G|} \frac{t^{3}}{t^{2}} ∑_{g∈G} \frac{χ(g)}{(1 − ω_{1}(g^{−1})t^{−1})(1 − ω_{2}(g^{−1})t^{−1})}

= \frac{t}{|G|} ∑_{g∈G} \frac{χ̄(g)}{(1 − ω_{1}(g)t^{−1})(1 − ω_{2}(g)t^{−1})}

= t\frac{p_{χ}(t^{−1})}{(1 − t^{−d_{1}})(1 − t^{−d_{2}})}

by a similar argument we used before. The character of the representation of G on the 1st homogeneous component of K[x_{1}, …, x_{n}]/I is χ, so p_{χ}(t) = t + ∑_{i=2}^{d_{1}+d_{2}−2} a_{i}(χ)t^{i}, hence

H_{1}(t) = t\frac{t^{−1} + ∑_{i=2}^{d_{1}+d_{2}−2} a_{i}(χ)t^{−i}}{(1 − t^{−d_{1}})(1 − t^{−d_{2}})} = \frac{t^{d_{1}+d_{2}} + ∑_{j=3}^{d_{1}+d_{2}−1} a_{d_{1}+d_{2}+1−j}(χ)t^{j}}{(1 − t^{d_{1}})(1 − t^{d_{2}})} .

On the other hand, the minimal degree term of \frac{F (t)}{∏_{i=1}^{n}(1−t^{d_{i}})} is t^{2d_{1}+d_{2}}, thus the

coefficient of t^{d_{1}+d_{2}} in D(t) = \frac{F (t)−G(t)}{(1−t^{d_{1}} )(1−t^{d_{2}} )} is −1. This contradicts the assumption that D(t) has non-negative coefficients.

3. G is not one of the groups of no. 23, 28, 30, 35, 36, 37 in the table [13, Table VII]. (The argument in 5 below applies also for these groups. However, first we eliminate them by exhibiting some invariants. This argument shows the role of the polynomial [x_{2}, x_{1}, x_{1}] explicitly.)

Suppose that G is one of these groups. It is well known (see for example [16, 4.2.15. Lemma]) that G can be defined over the reals if and only if G has an invariant of degree 2. So we can suppose that G < Gl_{n}(ℝ) is an orthogonal group, and then a straightforward calculation shows that

x_{1}^{2} + … + x_{n}^{2} ∈ K⟨x_{1}, …, x_{n}⟩^{G}.

Thus we can choose f_{1}, …, f_{n} such that

f_{1} = y_{1}^{2} + … + y_{n}^{2}.

Consider the invariant

f = ∑_{g∈G} g · (y_{1}(y_{1}^{2} + … + y_{n}^{2})y_{1}).

Comparing the condition K⟨f_{1}, …, f_{n}⟩ = F_{n}(M)^{G} and the degrees of the groups under consideration (cf. [13, Table VII]) we conclude that the only invariants of G of degree 4 are the scalar multiples of f_{1}^{2}, therefore f = af_{1}^{2} in F_{n}(M) for some a ∈ ℝ. Now we have

f = ∑_{g∈G}(g_{11}y_{1} + … + g_{n1}y_{n})(y_{1}^{2} + … + y_{n}^{2})(g_{11}y_{1} + … + g_{n1}y_{n}),

and the polynomial

h = ∑_{g∈G}(g_{11}x_{1} + … + g_{n1}x_{n})(x_{1}^{2} + … + x_{n}^{2})(g_{11}x_{1} + … + g_{n1}x_{n}) − a(x_{1}^{2} + … + x_{n}^{2})^{2}

is contained in M. The homogeneous component of h of degree 4 in x_{i} is

(∑_{g∈G} g_{i1}^{2} − a)x_{i}^{4}, (i = 1, …, n),

and it is contained in M, implying that a = ∑_{g∈G} g_{i1}^{2}, i = 1, …, n. Since for any g ∈ G there exists an i with g_{i1} ≠ 0, we have a ≠ 0. The multihomogeneous component of h of multidegree (2, 2)

∑_{g∈G} g_{11}^{2}x_{1}x_{2}^{2}x_{1} + ∑_{g∈G} g_{21}^{2}x_{2}x_{1}^{2}x_{2} −ax_{1}^{2}x_{2}^{2} −ax_{2}^{2}x_{1}^{2} = a(x_{1}x_{2}^{2}x_{1} +x_{2}x_{1}^{2}x_{2} −x_{1}^{2}x_{2}^{2} −x_{2}^{2}x_{1}^{2})

is contained in M. Make the substitution x_{1} → x_{1} + 1, x_{2} → x_{2} + 1 in the above polynomial, then we get

2ax_{1}x_{2}x_{1} + other monomials.

Hence M contains a polynomial of degree 3, which contradicts the definition of M. 4. G is not G(m, p, n) (the groups no. 2 in [13, Table VII]). Recall the definition of G(m, p, n). Let n ≥ 3 (the case n = 2 was handled in

2.), m ≥ 2, p be positive integers such that p divides m. Let A(m, p, n) < Gl_{n}(ℂ) be the group of diagonal matrices whose diagonal entries are mth roots of unity and the determinant is an \frac{m}{p}th root of unity. Consider Sym(n) as the group of permutation matrices in Gl_{n}(ℂ). Clearly, Sym(n) normalizes A(m, p, n), and G(m, p, n) is defined as the semidirect product

G(m, p, n) = A(m, p, n) ⋊ Sym(n).

Consider the polynomials

h_{i} = ∑_{π∈Sym(n)} π · σ_{i}(x_{1}^{m}, …, x_{n}^{m}), i = 1, …, n − 1;

h_{n} = ∑_{π∈Sym(n)} π · (x_{1} … x_{n})^{m/p}.

The above semidirect decomposition of G(m, p, n) shows that

h_{1}, …, h_{n} ∈ K⟨x_{1}, …, x_{n}⟩^{G},

and it is well known that their images in K[x_{1}, …, x_{n}] form a basic set of invariants, with degrees m, 2m, 3m, …, (n−1)m, \frac{m}{p}n. So we may suppose that f_{1}, …, f_{n} are the images of h_{1}, …, h_{n} under the natural homomorphism K⟨x_{1}, …, x_{n}⟩ → F_{n}(M). Consider the invariant

f = ∑_{g∈G} g · (y_{1}^{m−1}f_{1}y_{1}).

Any g ∈ G can be written in the form g = aπ for some a ∈ A(m, p, n) and π ∈ Sym(n). So we have g(y_{1}) = θy_{π(1)} for some permutation π and mth root of unity θ, and

f = (n − 1)!m^{n}/p ∑_{i=1}^{n} y_{i}^{m−1}(y_{1}^{m} + … + y_{n}^{m})y_{i}.

By our hypothesis f is contained in K⟨f_{1}, …, f_{n}⟩, and since deg(f_{3}), …, deg(f_{n}) > deg(f) = 2m, f can be expressed with f_{1}, f_{2} and f_{n}. More precisely, there exists a polynomial

h = ∑_{i=1}^{n} x_{i}^{m−1}(x_{1}^{m} + … + x_{n}^{m})x_{i} − (c_{1}h_{1}^{2} + c_{2}h_{2} + ∑_{j} a_{j}h_{n}b_{j}) ∈ M,

where c_{1}, c_{2} ∈ ℂ, and a_{j}, b_{j} ∈ K⟨x_{1}, …, x_{n}⟩. Consider the multihomogeneous components of h. The coefficient of x_{1}^{2m} is 1 − c_{1}, implying that c_{1} = 1. Since n ≥ 3, any monomial of h_{n} contains the variable x_{3}, therefore the sum of the coefficients of the monomials of multidegree (m, m) is 2 − 2c_{1} − kc_{2} for some positive integer k, showing that c_{2} = 0. Thus the polynomial

x_{1}^{m−1}x_{2}^{m}x_{1} + x_{2}^{m−1}x_{1}^{m}x_{2} − x_{1}^{m}x_{2}^{m} − x_{2}^{m}x_{1}^{m}

is contained in M. After the substitution x_{1} → x_{1} + 1, x_{2} → x_{2} + 1 we obtain a polynomial in which the coefficient of x_{1}x_{2}x_{1} is (m − 1)m. Again this contradicts the fact that M has no elements of degree 3.

5. G is not one of the groups no. 24, 26, 27, 29, 31, 32, 33, 34 or no. 1 with n ≥ 4 in [13, Table VII].

Let a, b be functions ℕ → ℂ, and let c be a function ℕ → ℕ. We say that a(n) = b(n) + O(c(n)), if |a(n) − b(n)| ≤ Lc(n) for some constant L. We need the following lemma (cf. [16, 2.5.9. Lemma]):

Lemma 4.1. Let H(t) ∈ ℂ[[t]] be a formal power series of the form

H(t) = \frac{A(t)}{∏_{i=1}^{n}(1 − t^{d_{i}})} = ∑_{d=0}^{∞} c_{d}t^{d},

where A(t) ∈ ℂ[t].

(i) (1 − t)^{n}H(t)|_{t=1} = \frac{A(1)}{∏_{i=1}^{n} d_{i}} .

(ii) (1 − t)^{n−1}H(t) − \frac{A(1)}{∏_{i=1}^{n} d_{i}} \frac{1}{1 − t}|_{t=1} = \frac{\frac{1}{2}A(1) ∑_{i=1}^{n}(d_{i} − 1) − A′(1)}{∏_{i=1}^{n} d_{i}}.

(iii) Suppose that n ≥ 3 and all sets of n − 1 of the d_{i} have greatest common divisor 1. Then we have

c_{d} = \frac{A(1)}{(n − 1)! ∏_{i=1}^{n} d_{i}} d^{n−1} + \frac{\frac{1}{2}A(1) ∑_{i=1}^{n} d_{i} − A′(1)}{(n − 2)! ∏_{i=1}^{n} d_{i}}d^{n−2} + O(d^{n−3}).

(In [16] n ≥ 4 is required, but the proof works also for n = 3.)

Now we investigate the power series D(t) = \frac{F (t)−G(t)}{∏_{i=1}^{n}(1−t^{d_{i}})}. Since F (1) = G(1) =

n\binom{n}{2}, by Lemma 4.1 (i) D(t) has no pole \frac{1}{(1−t)^{n}.} By Lemma 4.1 (ii) the coefficient

of the pole \frac{1}{(1−t)^{n−1}} in the Laurent series of \frac{F (t)}{∏_{i=1}^{n}(1−t^{d_{i}})} is

\frac{\frac{1}{2}n\binom{n}{2} ∑_{i=1}^{n}(d_{i} − 1) − \frac{3}{2}n(n − 1) ∑_{i=1}^{n} d_{i}}{∏_{i=1}^{n} d_{i}} = \frac{n(n − 1)}{4|G|}((n − 6) ∑_{i=1}^{n}(d_{i} − 1) − 6n).

By (4.1) the coefficient of the pole \frac{1}{(1−t)^{n−1}} in the Laurent series of H_{1}(t) is

\frac{1}{|G|} ∑_{g∈R} \frac{(1 + … + 1 + ω(g))(\binom{n−1}{2} + (n − 1)ω(g))}{1 − ω(g)},

where R = {g ∈ G | g is a pseudo-reflection} and ω(g) is the eigenvalue of g different from 1. It is well known (and one can derive from Lemma 4.1 (i)) that

∑_{g∈R} \frac{1}{1 − ω(g)} = \frac{1}{2} ∑_{i=1}^{n}(d_{i} − 1).

Using this formula we get the equalities

∑_{g∈R} \frac{ω(g)}{1 − ω(g)} = − ∑_{g∈R} \frac{1}{1 − ω̄(g)} = −\frac{1}{2} ∑_{i=1}^{n}(d_{i} − 1)

and

∑_{g∈R} \frac{ω(g)^{2}}{1 − ω(g)} = ∑_{g∈R}(−1 − ω(g) + \frac{1}{1 − ω(g)}) = N − \frac{1}{2} ∑_{i=1}^{n}(d_{i} − 1),

where N is the number of the reflecting hyperplanes. Thus we have

\frac{1}{|G|} ∑_{g∈R} \frac{((n − 1) + ω(g))(\binom{n−1}{2} + (n − 1)ω(g))}{1 − ω(g)}

= \frac{n − 1}{4|G|}((n^{2} − 6n + 4) ∑_{i=1}^{n}(d_{i} − 1) + 4N),

and the coefficient of the pole \frac{1}{(1−t)^{n−1}} in D(t) is

\frac{n − 1}{2|G|}(−3n^{2} − 2 ∑_{i=1}^{n}(d_{i} − 1) − 2N) = \frac{G′(1) − F′(1)}{|G|}.

(The last equality holds by Lemma 4.1 (ii).)

Denote by δ the greatest common divisor of d_{1}, …, d_{n}. The assumption F_{n}(M)^{G} = K⟨f_{1}, …, f_{n}⟩ implies that δ divides each of the e_{1}, …, e_{n\binom{n}{2}}, that is, we may write

D(t) = \frac{A(s)}{∏_{i=1}^{n}(1 − s^{d_{i}/δ})} = ∑_{d=0}^{∞} c_{d}s^{d},

where s = t^{δ} and A(s) ∈ ℂ[s]. One can check in [13, Table VII] that for the groups under consideration the numbers d_{1}/δ, …, d_{n}/δ satisfy the condition of Lemma 4.1 (iii), so applying this lemma and the equalities A(1) = 0 and \frac{∂}{∂s}A(s)|_{s=1} = δ(G′(1) − F′(1)) we obtain

c_{d} = \frac{δ(G′(1) − F′(1))}{(n − 2)! ∏_{i=1}^{n} d_{i}}d^{n−2} + O(d^{n−3})

= \frac{−δ(n − 1)}{2(n − 2)!|G|}(3n^{2} + 2 ∑_{i=1}^{n}(d_{i} − 1) + 2N)d^{n−2} + O(d^{n−3}).

This immediately implies that for sufficiently large d the coefficient of t^{d} in the power series D(t) is strictly negative, which is a contradiction.

6. G is not the group no. 25 in [13, Table VII].

This group has 24 pseudo-reflections, all of them are of order 3 (see [3, p. 412, Table]), so the number of reflecting hyperplanes is 12. By the calculations in 5 we have

F′(1) − G′(1) = 3 · 3^{2} + 2 · 24 + 2 · 12.

On the other hand

F′(1) − G′(1) = 9(d_{1} + d_{2} + d_{3}) − (e_{1} + … + e_{9});

therefore

\frac{e_{1} + … + e_{9}}{9} = 16,

implying that e_{1} ≤ 16. But we have

F (t) = t^{2d_{1}+d_{2}} + higher degree terms,

and since 2d_{1} + d_{2} = 21, the coefficient of t^{e_{1}} in D(t) = \frac{F (t)−G(t)}{∏_{i=1}^{3}(1−t^{d_{i}})} is strictly negative. This is a contradiction.

7. G is not Sym(4), that is, the group no. 1 with n = 3 in [13, Table VII].

One can calculate the power series D(t) directly and conclude that it has a negative coefficient.

We have eliminated all the finite irreducible complex unitary reflection groups, so the proof of Proposition 2.4 is complete.

Acknowledgements

I would like to thank the Széchenyi István Scholarship Foundation for the support which made it possible to visit the Penn State University in the first half of 1994. I thank the Department of Mathematics, especially E. Formanek for their hospitality during my visit. I am also grateful to V. Drensky for some earlier conversations on this topic.

References

1. N. Bourbaki, Groupes et Algébres de Lie, Chap. IV, V, VI, Hermann, Paris, 1968. MR 39:1590 2. C. Chevalley, Invariants of finite groups generated by reflections, Amer. J. Math. 77 (1955),

778-782. MR 17:345d

3. A. M. Cohen, Finite complex reflection groups, Ann. Sci. École Norm. Sup. 9 (1976), 379-436. MR 54:10437

4. W. Dicks and E. Formanek, Poincare series and a problem of S. Montgomery, Lin. Mult. Alg. 12 (1982), 21-30. MR 84c:15032

5. V. Drensky, Polynomial identities for 2 × 2 matrices, Acta Appl. Math. 21 (1990), 137-161. MR 91k:16024

6. V. Drensky, Computational techniques for PI-algebras, Topics in Algebra, Banach Center Publications, Vol. 26, Part 1., PWN-Polish Scientific Publishers, Warsaw, 1990, pp. 17-44. MR 93e:16036

7. V. Drensky, Private communication (1993).

8. E. Formanek, Noncommutative invariant theory, Contemporary Mathematics 43, Group Actions on Rings, S. Montgomery ed., Amer. Math. Soc., 1985, pp. 87-119. MR 87d:16046 9. R. M. Guralnick, Invariants of finite linear groups acting on relatively free algebras, Linear

Alg. Appl. 72 (1985), 85-92. MR 87d:16050

10. V. K. Kharchenko, Noncommutative invariants of finite groups and Noetherian varieties, J. Pure Appl. Alg. 31 (1984), 83-90. MR 85j:16052

11. D. Krakowsky and A. Regev, The polynomial identities of the Grassmann algebra, Trans. Amer. Math. Soc. 181 (1973), 429-438. MR 48:4005

12. L. H. Rowen, Polynomial Identities in Ring Theory, Academic Press, New York, 1980. MR 82a:16021

13. G. C. Shephard and J. A. Todd, Finite unitary reflection groups, Canad. J. Math. 6 (1954), 274-304. MR 15:600b

14. L. Solomon, Invariants of finite reflection groups, Nagoya Math. J. 22 (1963), 57-64. MR 27:4872

15. T. A. Springer, Regular elements of finite reflection groups, Inv. Math. 25 (1974), 159-198. MR 50:7371

16. T. A. Springer, Invariant Theory, Lecture Notes in Mathematics 585, Springer-Verlag, Berlin, Heidelberg, New York, 1977. MR 56:5740

Mathematical Institute of the Hungarian Academy of Sciences, P.O.B. 127, H-1364, Budapest, Hungary

E-mail address: domokos@math-inst.hu