# How Large Must the Kernel of Polynomials in Matrices Be?

Assume I have two symmetric matrices X and Y and I tell you the eigenvalues, counted with multiplicity, of each of them. Then I apply a polynomial P (which is also known to you) to those matrices and ask you to guess the size of the kernel of P(X,Y). If your guess is smaller than the actual size, the Queen of Hearts will pay out your guess in gold; otherwise, if your guess is too large, off with your head. Is there any strategy to survive this for sure and to get out as rich as possible? Let’s say, I don’t even tell you the size of the matrices and only give you the relative number of the eigenvalues, like: the first matrix X has 2/3 of its eigenvalues at 0, 1/6 at 1, and 1/6 at 2; the second matrix Y has 3/4 of its eigenvalues at -1, 1/8 at 0 and 1/8 at +1. What is your guess for the size of the kernel of the anti-commutator P(X,Y)=XY+YX? To be on the safe side you can of course always choose zero; this let’s you survive in any case, but it won’t make you rich. Is there a better guess, which still guarantees you keep your head?

My guess is 5/12. If you want to know how I get this and, in particular, how I can be so sure that this is the best safe bet – have a look on my new paper, joint with Octavio Arizmendi, Guillaume Cebron, Sheng Yin, “Universality of free random variables: atoms for non-commutative rational functions“, which we just uploaded to the arXiv.

If you want to think about the problem before having a look at the paper, take X as before, but Y having now 1/2 of its eigenvalues at -1, 3/8 at 0 and 1/8 at +1. Which of the following is your guess for the size of the kernel of the anti-commutator in this case:

• 5/12
• 1/6
• 1/2
• 1/3

# PhD position for a project on “Free Probability Aspects of Neural Networks” at Saarland University

Update: the deadline for applications has been extended to July 15, 2021!

I have funding from the German Science Foundation DFG for a PhD position on the interrelations between free probability, random matrices, and neural networks. Below are more details. See also here for the announcement as a pdf.

Project: Neural networks are, roughly speaking, functions of many parameters in a high-dimensional space and the training of such a network consists in finding the parameters such that the function does what it is supposed to do on the “training inputs”, but also generalizing this in a meaningful way to “real test inputs”. Random matrix and free probability theory are mathematical theories which deal with typical behaviours of functions (which have an underlying matrix structure) in high dimensions and in the limit of large matrix size. Thus it is not surprising that those theories should have something to offer for describing and dealing with neural networks. Accordingly, there have been approaches relying on random matrix and/or free probability theory to investigate questions around deep learning. This interaction between free probability and neural networks is hoped to be bi-directional in the long run. However, the project will not address practical purposes of deep learning; instead we want to take the deep learning challenges as new questions around random matrices and free probability and we aim to develop those theories further on a mathematical level.

Prerequisites: Applicants should have an equivalent of a Master’s degree and a background in at least one of the subjects

• free probability
• random matrices
• neural networks

and an interest in learning the remaining ones and, in particular, in working on their interrelations.

Application: Inquiries and applications should be addressed to Roland Speicher. Your application, in German or in English, should arrive before June 20, 2021. It should contain your curriculum vitae and an abstract of your Master’s thesis. Arrange also for at least one recommendation letter to be sent directly to Roland Speicher, preferably by email. State in your application the name of those you asked for such a letter .

Contact:
Prof. Dr. Roland Speicher
Saarland University
Department of Mathematics
Postfach 15 11 50
66041 Saarbrücken
Germany
speicher@math.uni-sb.de
https://www.uni-saarland.de/lehrstuhl/speicher/

# Announcement of two talks on free probability at the Technion … and of some more talks

update (from Jan 27): the recordings of the talks of Tobias and mine have been uploaded to youtube, here are the direct links:

I will give a colloquium talk at the Math Department of the Technion, Israel on next Monday, January 25 – online, of course. They have the nice option of a pre-colloquium talk, which provides students with some background for the material appearing in the colloquium talk. Tobias agreed to give such a preparation for my talk. So he will give tomorrow (on Thursday, January 21) an introduction to free probability and its relation with random matrices. Surely a great opportunity for everyone to learn (more) about the subject.

My talk will, of course, have such material in the background, but I tried to prepare it in such a way that even without knowing about free probability one should be able to get the main ideas. So, it might help to know what free semi-circulars are, but it is not necessary (and not assumed) for my talk.

Below are the titles and abstracts of our talks; and here is a link to the Technion page with access information:

Colloquium

update: actually, next week seems to be a busy week for talks around free probability; don’t forget that the UC Berkeley Probabilistic Operator Algebra Seminar will be starting again, on Monday, January 25, with a talk of Friedrich Goetze; and then there will also be a talk by Serban Belinschi on “The Christoffel-Darboux kernel in noncommutative probability” at the Probability Seminar at  Warsaw University of Technology, on Tuesday, January 26.

Tobias Mai: What actually is free probability theory? (Thursday, January 21, 2021)

In my talk, I want to answer this question by giving an introduction to the underlying ideas, basic concepts, and fundamental results of free probability theory. In particular, I will highlight the deep connections of this field with random matrix theory.

Roland Speicher: Singularity of matrices in non-commuting variables and free probability (Monday, January 25, 2021)

The Edmonds’ problem asks to decide about the singularity of a given matrix with linear polynomials in commuting variables as entries, or more general to compute the rank of such a matrix over the field of rational functions. This problem has no known deterministic polynomial time algorithm and it relates to fundamental questions in complexity theory.

Recently, there has been much interest in analyzing a non-commutative variant of the Edmonds’ problem, where the entries are linear polynomials in non-commuting variables and the rank is over the field of non-commutative rational functions (aka free skew field). Garg, Gurvits, Oliveira, and Wigderson showed that for this non-commutative Edmonds’ problem there exists a deterministic polynomial time algorithm. This problem has a remarkable number of diverse origins and motivations and I will present in my talk another such manifestation of the problem, arising from the relation with free probability and random matrix theory. In particular, this approach results also in another, quite analytic, algorithm for calculating the non-commutative rank.

This talk is based on joint work with Johannes Hoffmann, Tobias Mai, and Sheng Yin.

# Nothing New on Connes’ Embedding

It’s now almost a year that we have been told that Connes’ embedding conjecture is not a conjecture anymore, but that it’s actually false. In principle, this is great news as it should open totally new playgrounds, with von Neumann algebras never seen before. The only problem is that we still have not seen them. I am sure that many are looking for them but as far as I am aware nobody outside the quantum information community was able to shed more light on the refutation of Connes’ embedding.

As a believer in the power of non-commutative distributions I tried all my arsenal of moments, cumulants, or Cauchy transforms to get a grasp on how such a non-embeddable von Neumann algebra could look like — of course, without any success. But let me say a few more words an some of my thoughts – if only to come up with a bit longer post for the end of the year.

In our non-commutative distribution language, the refutal of Connes’ embedding says that there are operators in a tracial von Neumann algebra whose mixed moments cannot be approximated by moments of matrices with respect to the trace. We have quite a few of distributions in free probability theory, but the main problem in the present context is that all of them usually can be approximated by matrices, and also all available constructions (like taking free products) preserve such approximations (in particular, since we can model free independence via conjugation by unitary random matrices). Very roughly: our constructions of distributions take some input and then produce some distribution — however, if the input is embeddable, then the output will be so, too. Thus I cannot use those constructions directly to make the leap from our known universe to the new ones which should be out there. The only way I see to overcome this obstruction is to look for distributions which create themselves “out of nothing” via such constructions, i.e., for fixed point distributions of those constructions. For such fixed point distributions I see at least no apriori reason to be embeddable.

But is there any way to make this concrete? My naive attempt is to use the transition from moments to cumulants (or, more analyticially, from Cauchy transforms to R-transforms) for this. We know that infinitely divisible distributions (in particular, compound Poisson ones) are given in the form that their free cumulants are essentially the moments of some other distribution. So I am trying to find reasonable fixed points of this mapping, i.e., I am looking for distributions whose cumulants are (up to scaling or shift) the same as their moments. Unfortunately, all concrete such distributions seem to arise via solving the fixed point equation in an iterative way – which is also bad from our embedding point of view, since those iterations also seem to preserve embeddability. So I have to admit complete and utter failure.

Anyhow, if the big dreams are not coming true, one should scale down a bit and see whether anything interesting is left … so let me finally come to something concrete, which might, or might not, have some relevance …

In the case of one variable we are looking on probability measures, and as those can be approximated by discrete measures with uniform weights on the atoms (thus by the distribution of matrices), this situation is not relevant for Connes’ embedding question. However, I wonder whether a fixed point of the moments-to-cumulants mapping in this simple situation has any relevance. The only meaningful mapping in this case seems to be that I take a moment sequence, shift it by 2 and then declare it as a cumulant sequence — necessarily of an infinitely divisible distribution. Working out the fixed point of this mapping gives the following sequence of even moment/cumulants: 1, 1, 3, 14, 84, 596. The Online Encyclopedia of Integer Sequences labels this as A088717 — which gives, though, not much more information than the fixed point equation for the generating power series.

The above moment-cumulant mapping was of course using free cumulants. Doing the same with classical cumulants gives by a not too careful quick calculation the sequence 1, 1, 4, 34, 496, which seems to be https://oeis.org/A002105 — which goes under the name ”reduced tangent numbers”. There are also a couple of links to various papers, which I still have to check …

Okay, I suppose that’s it for now. Any comment on the relevance or meaning of the above numbers, or their probability distributions, would be very welcome – as well, as any news on Connes’ conjecture.

# Another blog on “Free Probability” by Teo Banica

Teo Banica got a bit bored by the lockdown and started to write a series of blogs on various topics, close to his heart and his knowledge – one of them is also one free probability. Check it out here. It’s written in Teo’s personal style, which might seem annoying or provocative to some, but in any case it’s interesting …

Update (September 2020): It seems that Teo got also bored or annoyed of his own blog, so the link above does not work any more … but much of the material has actually been moved to lecture notes and videos. In particular, Teo has the goal of trying to reorganize the quantum group basics, via a series of books. Probably the best to stay updated on this is to check his website or his YouTube channel

# Another online seminar: Wales MPPM Zoom Seminar

At the moment there are many online activities going on …. and here is another one: the Wales Mathematical Physics Zoom Seminar, organized by Edwin BeggsDavid EvansGwion Evans,Rolf GohmTim Porter.

Why do I mention in particular this one; there are at least two reasons. Today there is a talk by Mikael Rordam around the Connes embedding problem, and next week I will give a talk, on my joint work with Tobias Mai and Sheng Yin of the last years around rational functions of random matrices and operators.

If you are interested in any of this, here is the website of the seminar, where you can find more information.

Update: The talks are usually recorded and posted on a youtube channel. There you can find my talk on “Random Matrices and Their Limits”.

# Berkeley’s Probabilistic Operator Algebra Seminar (Hosted by Voiculescu) is Now Online

Due to the current conditions, Voiculescu’s seminar on free probability and operator algebras is now being held online via the platform Zoom. Members of the community worldwide are welcome to join.
Announcements and the Zoom link for each talk will be shared via a mailing list. If you would like to be added to this list, please send an email to: jgarzavargas@berkeley.edu

In this webpage you can find the titles, abstracts and dates for past and future talks: https://math.berkeley.edu/~jgarzav/seminar.html

Sincerely,
Jorge Garza Vargas

# The saga ends …

I have now finished my class on random matrices. The last lecture motivated the notion of (asymptotic) freeness from the point of view of looking on independent GUE random matrices. So you might think that there should now be continuations on free probability and alike coming soon. But actually this part of the story was already written and recorded and if you don’t want to spoil the tension you should watch the series not in its historical but in its logical order:

1. Random Matrices (videos, homepage of class)
2. Free Probability Theory (videos, homepage of class)
3. Non-commutative Distributions (and Operator-Valued free Probability Theory) (videos, homepage of class)

More information, in particular the underlying script (sometimes in a handwritten version, sometimes in a more polished texed version), can be found on the corresponding home page of the lecture series.

May freeness be with you …

# Is there an impact of a negative solution to Connes’ embedding problem on free probability?

There is an exciting new development on Connes’ embedding problem. The recent preprint MIP*=RE by Ji, Natarajan, Vidick, Wright, Yuen claims to have solved the problem to the negative via a negative answer to Tsirelson’s problem via the relation to decision problems on the class MIP* of languages that can be decided by a classical verifier interacting with multiple all powerful quantum provers. I have to say that I don’t really understand what all this is about – but in any case there is quite some excitement about this and there seems to be a good chance that Connes’ problem might have a negative solution. To get some idea about the excitement around this, you might have look on the blogs of Scott Aaronson or of Gil Kalai. At the operator front I have not yet seen much discussion, but it might be that we still have to get over our bafflement.

Anyhow, there is now a realistic chance that there are type II factors which are not embeddable and this raises the question (among many others) what this means for free probability. I was asked this by a couple of people and as I did not have a really satisfying answer I want to think a bit more seriously about this. At the moment my answer is just: Okay, we have our two different approaches to free entropy and a negative solution to Connes embedding problem means that they cannot always agree. This is because we always have for the non-microstates free entropy $\chi^*$ that $\chi^*(x_1+\sqrt\epsilon s_n,\dots,x_n+\sqrt\epsilon s_n)>-\infty$, if $s_1,\dots,s_n$ are free semicircular variables which are free from $x_1,\dots,x_n$. The same property for the microstates free entropy $\chi$, however, would imply that $x_1,\dots,x_n$ have microstates, i.e., the von Neumann algebra generated by $x_1,\dots,x_n$ is embeddable; see these notes of Shlyakhtenko.

But does this mean more then just saying that there are some von Neumann algebras for which we don’t have microstates but for which the non-microstates approach give some more interesting information, or is there more to it? I don’t know, but hopefully I will come back with more thoughts on this soon.

Of course, everybody is invited to share more information or thoughts on this!

# Welcome to the Non-Commutative World!

About two weeks ago I posted with Tobias Mai on the archive the preprint “A Note on the Free and Cyclic Differential Calculus”. Here is what we say in the abstract:

In 2000, Voiculescu proved an algebraic characterization of cyclic gradients of noncommutative polynomials. We extend this remarkable result in two different directions: first, we obtain an analogous characterization of free gradients; second, we lift both of these results to Voiculescu’s fundamental framework of multivariable generalized difference quotient rings. For that purpose, we develop the concept of divergence operators, for both free and cyclic gradients, and study the associated (weak) grading and cyclic symmetrization operators, respectively. One the one hand, this puts a new complexion on the initial polynomial case, and on the other hand, it provides a uniform framework within which also other examples – such as a discrete version of the Ito stochastic integral – can be treated.

At the moment I am not in the mood to say more specifically about this preprint (maybe Tobias or I will do so later), but I want to take the opportunity — in particular as the first anniversary of this blog is also coming closer — to put this in a bigger context and mumble a bit about the bigger picture and our dreams … so actually about what this blog should be all about.

Free probability theory has come a long way. Whereas born in the subject of operator algebras, the realization that is also has to say quite a bit about random matrices paved the way to its use in many (and, in particular, also applied) subjects. Hence there are now also papers in statistics, like this one, or in deep learning, like this one or this one, which use tools from free probability for their problems. The last words on how far the use of free probability goes in those subjects are surely not yet spoken but I am looking forward to see more on this.

This is of course all great and nice for our subject, but on the other hand there is also a bigger picture in the background, where I would hope for some more fundamental uses of free probability.

This goes roughly like this. There is the classical world, where we are dealing with numbers and functions and everything commutes; then there is our non-commutative world, where we are dealing with operators and limits of random matrices and where on the basic level nothing commutes. That’s where quite a bit of maximal non-commutative mathematics has been (and is still being) developed from various points of views:

• free probability deals with a non-commutative notion of independence for non-commuting random variables;
• there is a version of a non-commutative differential calculus which allows to talk about derivatives in non-commutative variables; my paper with Tobias mentioned above is in this context and tries to formalize and put all this a bit further;
• free analysis (or free/non-commutative function theory) aims at a non-commutative version of classical complex analysis, i.e., a theory of analytic functions in non-commuting variables;
• free quantum groups provide the right kind of symmetries for such non-commuting variables.

The nice point is that all those subjects have their own source of motivation but it turns out that there are often relations between them which are non-commutative analogues of classical results.

So, again this is all great and nice, BUT apart from the commutative and our maximal non-commutative world there is actually the, maybe most important, quantum world. This is of course also non-commutative, but only up to some point. There operators don’t commute in general, but commutativity is replaced by some other relations, like the canonical commutation relations, and there are actually still a lot of operators which commute (for example, measurements which are at space-like positions are usually modeled by commuting operators). Because of this commutativity, basic concepts of free probability do not have a direct application there.

Here is a bit more concretely what I mean with that. In free probability we have free analogues of such basic concepts as entropy or Fisher information. There are a lot of nice statements and uses of those concepts and via random matrices they can also be seen as arising as a kind of large N limit of the corresponding classical concepts. However, in the classical world those concepts have usually also a kind of operational meaning by being the answer to fundamental questions. For example, the classical Shannon entropy is the answer to the question how much information one can transmit over classical channels. Now there are quantum channels and one can ask how much information one can transmit over them; again there are answers in terms of an entropy, but this is unfortunately not free entropy, but von Neumann entropy, a more commutative non-commutative cousin of classical entropy. There are just too many tensor products showing up in the quantum world which prevent a direct use of basic free probability concepts. But still, I am dreaming of finding some day operational meanings of free entropy and similar quantities.

Anyhow, I hope to continue to explain in this blog more of the concrete results and problems which we have in free probability and related subjects; but I just wanted to point out that there are also some bigger dreams in the background.