Introduction
7 months ago Elegia published a blog about D-finite functions (part1, part2) in which he showed some examples of problems that can be solved by D-finite functions.
Here I will explain a fast evaluation algorithm of the coefficients of a D-finite function.
Let $$$K$$$ be some field of characteristic $$$0$$$. Let $$$K[ [x_{1},\ldots,x_{d}] ]$$$ be the ring of formal power series on it. Let $$$K[x_{1},\ldots,x_{d}]$$$ be the polynomials of it. Let $$$K(x_{1},\ldots,x_{d})=\mathop{\mathrm{Frac}}(K[x_{1},\ldots,x_{d}])$$$ be the field of rational functions over it. Let $$$ [ x_{1}^{i_{1}}\ldots x_{d}^{i_{d}} ] f$$$ be the coefficient in $$$f$$$ of the monomial $$$x_{1}^{i_{1}}\ldots x_{d}^{i_{d}}$$$.
Definitions
A function $$$f \in K[ [x_{1},\ldots,x_{d}] ]$$$ is called D-finite if for each $$$i$$$ there exists a positive integer $$$n_{i}$$$ and polynomials (not all zeroes) $$$P_{i,0}(x_{1}, \ldots,x_{d}),\ldots,P_{i,n_{i}}(x_{1},\ldots,x_{d})$$$ such that $$$P_{i,0}f+P_{i,1}\frac{df}{dx_{i}}+\ldots+P_{i,n_{i}}\frac{d^{n_{i}}(f)}{(dx_{i})^{n_{i}}}=0$$$.
An infinite table of coefficients $$$a(i_{1},\ldots,i_{d})$$$ is called P-recursive if there exists $$$k \in \mathbb{N}$$$ such that for each $$$j \in \{ 1,\ldots,d \} $$$ there exists a mapping $$$p: \{ 0,\ldots,k \} ^{d} \rightarrow K[x]$$$ such that at least one $$$p_{v} \neq 0$$$ and for all $$$i_{1},\ldots,i_{d} \geq k$$$ $$$\sum\limits_{v \in \{0,\ldots,k\}^{d}}^{} p_{v}(i_{j}) a(i_{1}-v_{1},\ldots,i_{d}-v_{d}) = 0$$$.
In particular, for $$$d=1$$$ it means that there exist such polynomials (not all zeroes) $$$P_{i}$$$: $$$ \forall n \geq k : \sum\limits_{0 \leq i \leq k}^{} P_{i}(n)a(n-i)=0$$$.
Properties:
1 The definition is equivalent to the "linear space of partial derivatives of $$$f$$$ over $$$K(x_{1},\ldots,x_{d})$$$ has finite dimension".
In particular, every rational function is obviously D-finite
2 They form a linear algebra over $$$K(x_{1},\ldots,x_{d})$$$ (we can multiply them by a scalar, add them and multiply them, and the result will be D-finite)
3 Every algebraic over $$$K(x_{1},\ldots,x_{d})$$$ function is D-finite
4 If $$$u_{1},\ldots,u_{d} \in K[ [u_{1},\ldots,u_{d}] ]$$$ are algebraic, and $$$f$$$ is D-finite and composition $$$f(u_{1},\ldots,u_{d})$$$ is well-defined in some sense, then it is D-finite.
5(Lipshitz, 1988) If $$$f$$$ is D-finite, then $$$\sum\limits_{i_{12},i_{3},\ldots,i_{d}}^{} x_{1}^{i_{12}}x_{3}^{i_{3}}\ldots x_{d}^{i_{d}} ([x_{1}^{i_{12}}x_{2}^{i_{12}}x_{3}^{i_{3}}\ldots x_{d}^{i_{d}}] f)$$$ is D-finite. That is, you can contract the function $$$f$$$ to the diagonal coefficients. (You can do a series of contractions to select only coefficients on the diagonal)
6(Lipshitz, 1989) The function $$$f=\sum\limits a(i_{1},\ldots,i_{d}) x_{1}^{i_{1}}\ldots x_{d}^{i_{d}}$$$ is D-finite if and only if its coefficients are P-recursive.
7(Lipshitz, 1988) If $$$f$$$ is D-finite, then for each $$$i_{1}$$$: $$$\sum\limits_{i_{2},i_{3},\ldots,i_{d}}^{} x_{2}^{i_{2}}x_{3}^{i_{3}}\ldots x_{d}^{i_{d}} ([x_{1}^{i_{1}}x_{2}^{i_{2}}x_{3}^{i_{3}}\ldots x_{d}^{i_{d}}] f)$$$ is D-finite. Moreover, the number of polynomials and their degrees in the D-finiteness relations are uniformly bounded.
Note: D-finite functions are not a field
Let's prove 7
Note: Here in the proof we actually have shown that the non-negative (by degree of $$$x_{1}$$$) part of a D-finite function is D-finite.
What functions are D-finite?
Can you guess which of these functions are D-finite (that is, the sequence/table of coefficients is P-recursive)?
(1) $$$a_{n} = [ x^{n+1} ] (1+x+x^{2})^{n}$$$ ($$$f(x)=\sum\limits_{}^{} a_{n}x^{n}$$$)
(2) $$$a_{n}=n!$$$ ($$$f(x)=\sum\limits_{}^{} n!x^{n}$$$)
(3) $$$a_{n,k}=S(n,k)$$$ Stirling numbers of the second kind, number of ways to partition a set of $$$n$$$ objects into $$$k$$$ non-empty subsets.
$$$\sum\limits_{n=0}^{\infty} \sum\limits_{k=0}^{n} S(n,k) \frac{x^n}{n!} y^k = e^{y(e^x-1)}$$$
(It is not so hard to see, that the D-finiteness of the OGF of some sequence/table is equivalent to the D-finiteness of EGF of this sequence/table)
(4) Eulerian numbers $$$a_{n,k}=E(n,k)$$$ is a number of $$$n$$$-element permutations with $$$k$$$ ascents.
$$$\sum\limits_{n=0}^{\infty} \sum\limits_{k=0}^{n} E(n,k)t^{k} \frac{x^n}{n!} = \frac{t-1}{t - e^{(t-1)\,x}} = \left(1-\frac{e^{(t-1)x}-1}{t-1}\right)^{-1}$$$
(5) $$$a_{u,v,w}=\sum\limits_{0 \leq x \leq u, 0\leq y \leq v, 0\leq z \leq w} \frac{(x+y)!(y+z)!(z+x)!}{x!y!z!}$$$.
The D-finite functions with $$$d=1$$$ are just P-recursive sequences. And we can evaluate the first $$$n$$$ terms of a P-recursive sequence in a stupid way in $$$O(n)$$$, or there is an algorithm to evaluate the $$$n$$$-th term (only one) in $$$O(\sqrt{n}\log(n))$$$.
But how to solve problems using this?
Let us assume that some function $$$f$$$ is D-finite, for example $$$f=\frac{P(x,y)}{Q(x,y)}$$$ where $$$P(x,y)$$$ and $$$Q(x,y)$$$ are fixed polynomials. And we want to compute $$$[x^{n}y^{m}] f$$$ (number of variables actually does not matter).
Let's notice that $$$h(u)=[x^{n}y^{u}] f$$$ is D-finite (this is the property 7) that is equivalent to P-recursive, and we can compute $$$h(m)$$$ in $$$O(\sqrt{m}\log(m))$$$ from some first values of $$$h$$$: $$$h(0),\ldots,h(k)$$$ where $$$k$$$ is a constant.
To compute $$$h(l)$$$ we will do the same, we will notice that $$$g(u)=[x^{u}y^{l}] f$$$ is P-recursive, so we can compute $$$g(l)$$$ in $$$O(\sqrt{n}\log(n))$$$.
So we can compute $$$[x^{n}y^{m}] f$$$ in $$$O(\sqrt{n+m}\log(n+m))$$$.
But when I say "Let's compute the coefficient of this P-recursive sequence", how do I know that the P-recursive relation holds?
One can compute it from the proof of property $$$\textbf{7}$$$, but I think this is a lot of pain. So to find P-recursive relation let's compute the first $$$k$$$ coefficients (for some $$$k$$$) and run a Gauss.
Note: We can also notice from the proof of property $$$\textbf{7}$$$, that the coefficients of P-recursion of $$$h(u)$$$ can be expressed as polynomials of $$$u$$$, and then we can try to evaluate them from some interpolation of a rational function, though I have not implemented it yet.
Example of this algorithm:
Assume we have a function $$$f(n,m)=[x^ny^m] e^{\frac{x+y}{(1-x)(1-y)}}$$$. And we want to compute $$$f(10^{7},10^{8}) \pmod{998\,244\,353}$$$.
This function $$$f$$$ is D-finite.
Let's compute its values for $$$0 \leq n \lt 25$$$ and $$$0 \leq m \lt 25$$$. Then we can extend
$$$f(\cdot,0),f(\cdot,1),f(\cdot,2),\ldots,f(\cdot,24)$$$ to $$$f(10^{7},0),f(10^{7},1),\ldots,f(10^{7},24)$$$ by P-recursion. Then we can extend $$$f(10^{7},\cdot)$$$ from the first $$$25$$$ values to $$$f(10^{7},10^{8})$$$ by P-recursion too. It works in $$$O(25\sqrt{10^{7}}\log(10^{7})+\sqrt{10^{8}}\log(10^{8}))$$$ which takes ~3.1 sec.
Note: for $$$f(10^{8},10^{8})$$$ we can compute it from diagonal (property 5).
Implementation:
Here is a code
Code works for all prime moduli. Explanation:
findPrecursionfinds P-recursionevaluatePrecursionevaluates P-recursion from the initial values if possibleevaluatePrecursionfastevaluates the element of a sequence from P-recursion and the initial valuessequenceextenderis trying to extend the sequenceoptimalgetvaluebyidis trying to get the element of an extended sequence by its positionextendtableextends table A, finding A(que[i].first,que[i].second)getcolumnoftableandgetrowoftableget column or row of an extended table
A "small" problem
But there is one caveat in this algorithm. Assume that there is a sequence $$$a_{i}$$$ with P-recursion with relation $$$a_{n}P_{0}(n)+\ldots+a_{n-l}P_{l}(n)=0$$$.
You know $$$a(0),\ldots,a(l-1)$$$ and you want to extend this sequence to $$$a(0),\ldots,a(N)$$$. But what if $$$P_{0}(k)=0$$$ for some $$$l \leq k \leq N$$$? Then we can't extend this sequence. And I don't know what to do with that, but luckily, I suppose that for almost all functions when we can use it, it can't be zero.
Example where it matters
Let's try to compute $$$f(10^{6},10^{6})$$$ with $$$f(n,m)=[x^{n}y^{m}] \frac{1}{1-xy}$$$. We will try to calculate some first columns, and then a row. But even if we managed to somehow guess the first columns, we would be unable to get a row, because our initial values in this row will all be zeroes, and we want to get $$$1$$$ somehow, but we can't.
Examples of the problems
1) Atcoder Regular 202 problem D
2) We can also solve 1747E - List Generation in $$$O(\sqrt{n}\log(n))$$$ per test (compute the coefficient of $$$(1-x)^{2}(1-y)^{2}(1-2x-2y+2xy)^{-2}$$$), but actually, because there is a multitest and the constraints are big this solution is hard to pass (I have managed, but after a lot of pain and with some strange FFT: 330486092).
I think there are some other examples of problems that can be solved by this method.
Conclusion (Warning)
When I got it at first, it seemed to me that "every" function in combinatorics is D-finite, but it is actually wrong (see before). I think that one should use this algorithm only if one has the proof that some function is D-finite, otherwise it can be a waste of time. So I don't recommend using this algorithm as magic.
Thanks to orz for editing this blog!








Auto comment: topic has been updated by turmax (previous revision, new revision, compare).
Auto comment: topic has been updated by turmax (previous revision, new revision, compare).
Hi, what might be considered the prerequisites for this blog? I'd like to read it thoroughly, but it seems somewhat intimidating.
Hi, firstly, if you don't know a lot about generating functions, this blog is very good for you.
Then you can read definition of P-recursive sequence in case $$$d=1$$$ in this blog, then see "Exp in Gennady Korotkevich Contest 5" and "Discussion about ODE and P-recursive sequence" In this blog. This blog can be useful too.
Nice, thanks! I’ll check out the generating functions blog first and go slowly.