черновик:

An "easy" way to solve counting problems (D-finite functions)

Правка ru10, от turmax, 2025-07-25 11:24:16

Let $$$K$$$ be some field of characteristic $$$0$$$. Let $$$K[ [x_{1},...,x_{d}] ]$$$ be the ring of formal power series on it. Let $$$K[x_{1},...,x_{d}]$$$ be the polynomials of it. Let $$$K(x_{1},...,x_{d})=Frac(K[x_{1},...,x_{d}])$$$ be the field of rational functions over it.

Definition

A function $$$f \in K[ [x_{1},...,x_{d}] ]$$$ is called D-finite if for each $$$i$$$ there exists $$$n_{i}$$$ and polynomials (not all zeroes) $$$P_{i,0}(x_{1},...,x_{d}),...,P_{i,n_{i}}(x_{1},...,x_{d})$$$ such that $$$P_{i,0}f+P_{i,1}\frac{df}{dx_{i}}+...+P_{i,n_{i}}\frac{d^{n_{i}}(f)}{(dx_{i})^{n_{i}}}=0$$$.

Properties:

1 The definition is equivalent to the "linear space of derivatives of $$$f$$$ over $$$K(x_{1},...,x_{d})$$$ has finite dimension".

2 They forms a linear algebra over $$$K(x_{1},...,x_{d})$$$ (we can multiply them to scalar, add them and multiply them, and the result will be D-finite)

3 Every algebraic over $$$K(x_{1},...,x_{d})$$$ function is D-finite

4 If $$$u_{1},...,u_{d} \in K[ [u_{1},...,u_{d}] ]$$$ are algebraic, and $$$f$$$ is D-finite and composition $$$f(u_{1},...,u_{d})$$$ is well-defined in some sense, then it is D-finite.

5(Lipshitz, 1988) If $$$f$$$ is D-finite, then $$$\sum\limits_{i_{12},i_{3},...,i_{d}}^{} x_{1}^{i_{12}}x_{3}^{i_{3}}...x_{d}^{i_{d}} ([x_{1}^{i_{12}}x_{2}^{i_{12}}x_{3}^{i_{3}}...x_{d}^{i_{d}}] f)$$$ is D-finite. That is you can contract the function $$$f$$$ to the diagonal coefficients. (You can do a series of contractions to select only coefficients on diagonal)

6(Lipshitz, 1989) The function $$$f=\sum\limits a(i_{1},...,i_{d}) x_{1}^{i_{1}}...x_{d}^{i_{d}}$$$ is D-finite if and only if its coefficients are P-recursive, that is for each $$$j$$$ there exists some recurence $$$\sum\limits_{v}^{} p_{v}(i_{j})a(i_{1}-v_{1},...,i_{d}-v_{d})=0$$$ where $$$v$$$ are some fixed arrays, $$$p_{v}$$$ are some fixed polynomials for each $$$v$$$.

7(Lipshitz, 1988) If $$$f$$$ is D-finite, then for each $$$i_{1}$$$: $$$\sum\limits_{i_{2},i_{3},...,i_{d}}^{} x_{2}^{i_{2}}x_{3}^{i_{3}}...x_{d}^{i_{d}} ([x_{1}^{i_{1}}x_{2}^{i_{2}}x_{3}^{i_{3}}...x_{d}^{i_{d}}] f)$$$ is D-finite. Moreover, the number of polynomials and their degrees in the D-finiteness relations are uniformly bounded.

Let's prove 7

Proof

Note: Here in the proof we actually have shown that the non-negative (by degree of $$$x_{1}$$$) part of a D-finite function is D-finite.

What functions are D-finite?

Can you guess, which of this functions are D-finite?

(1) $$$a_{n}=(1+x+x^{2})^{n}[x^{n+1}]$$$

(2) $$$a_{n}=n!$$$

(3) $$$a_{n,k}=S(n,k)$$$ Stirling numbers of the second kind, number of ways to partition a set of $$$n$$$ objects into $$$k$$$ non-empty subsets.

$$$\sum\limits_{n=0}^{\infty} \sum\limits_{k=0}^{n} S(n,k) \frac{x^n}{n!} y^k = e^{y(e^x-1)}$$$

(4) Eulerian numbers $$$a_{n,k}=E(n,k)$$$ is a number of $$$n$$$-element permutations with $$$k$$$ ascents.

$$$\sum\limits_{n=0}^{\infty} \sum\limits_{k=0}^{n} E(n,k)t^{k} \frac{x^n}{n!} = \frac{t-1}{t - e^{(t-1)\,x}} = \left(1-\frac{e^{(t-1)x}-1}{t-1}\right)^{-1}$$$

(5) $$$a_{u,v,w}=\sum\limits_{0 \leq x \leq u, 0\leq y \leq v, 0\leq z \leq w} \frac{(x+y)!(y+z)!(z+x)!}{x!y!z!}$$$.

Answer

The D-finite functions with $$$d=1$$$ are just P-recursive sequences. And we can evaluate first n terms of a P-recursive sequence in a stupid way in $$$O(n)$$$, or there is an algorithm to evaluate the $$$n$$$-th term (only one) in $$$O(\sqrt{n}log(n))$$$.

Algorithm

But how to solve problems using this?

Let us assume that some function $$$f$$$ is D-finite, for example $$$f=\frac{P(x,y)}{Q(x,y)}$$$ where we know $$$P(x,y)$$$ and $$$Q(x,y)$$$ are fixed polynomials. And we want to compute $$$[x^{n}y^{m}] f$$$ (number of variables actually does not matter).

Let's notice that $$$h(u)=[x^{n}y^{u}] f$$$ is D-finite (this is the property 7) that is equivalent to P-recursive, and we can compute $$$h(m)$$$ in $$$O(\sqrt{m}log(m))$$$ from some first values of $$$h$$$: $$$h(0),...,h(k)$$$ where $$$k$$$ is a constant.

To compute $$$h(l)$$$ we will do the same, we will notice that $$$g(u)=[x^{u}y^{l}] f$$$ is P-recursive, so we can compute $$$g(l)$$$ in $$$O(\sqrt{n}log(n))$$$.

So we can compute $$$[x^{n}y^{m}] f$$$ in $$$O(\sqrt{n+m}log(n+m))$$$.

But when I saying let's compute the coefficient of this P-recursive sequence, how do I know the P-recursive relation?

One can compute it from the proof of property $$$\textbf{7}$$$, but I think this is a lot of pain. So to find P-recursive relation let's compute the some first $$$k$$$ coefficients and run a Gauss.

Note: We can also notice from the proof of property $$$\textbf{7}$$$, that the coefficients of P-recursion of $$$h(u)$$$ can be expressed as a polynomials of $$$u$$$, and then we can try to evaluate them from some interpolation of a rational function, though I have not implemented it yet.

Example of this algorithm:

Let we have a function $$$f(n,m)=[x^ny^m] e^{\frac{x+y}{(1-x)(1-y)}}$$$. And we want to compute $$$f(10^{7},10^{8})$$$ (mod 998244353)

This function $$$f$$$ is D-finite.

Let's compute it values for $$$0 \leq n \lt 25$$$ and $$$0 \leq m \lt 25$$$. Then we can extend

$$$f(\cdot,0),f(\cdot,1),f(\cdot,2),...,f(\cdot,24)$$$ to $$$f(10^{7},0),f(10^{7},1),...,f(10^{7},24)$$$ by P-recursion. Then we can extend $$$f(10^{7},\cdot)$$$ from first $$$25$$$ values to $$$f(10^{7},10^{8})$$$ by P-recursion too. It works in $$$O(25\sqrt{10^{7}}log(10^{7})+\sqrt{10^{8}}log(10^{8}))$$$ which takes ~3.1 sec.

Implementation:

Here is a code

code

Code works for all prime modulos \ findPreccursion finds P-recursion \ evaluatePrecursion evaluates P-recursion from the initial values if possible \ evaluatePrecursionfast evaluates the element of sequence from P-recursion and the initial values \ sequenceextender is trying to extend the sequence \ optimalgetvaluebyid is trying to get the element of an extended sequence by its position \ extendtable extends table A, finding A(que[i].first,que[i].second) \ getcolumnoftable/getrowoftable get column and row of an extended table \
\ \ \ But there is one caveat in this algorithm. Assume that there is a sequence $$$a_{i}$$$ with P-recursion with relation $$$a_{n}P_{0}(n)+...+a_{n-l}P_{l}(n)=0$$$. You know $$$a(0),...,a(l-1)$$$ and you want to extend this sequence to $$$a(0),...,a(N)$$$. But what if $$$P_{0}(k)=0$$$ for some $$$l \leq n \leq N$$$. Then we can't extend this sequence. And I don't know what to do with that, but luckily, I suppose that for almost all function when we can use it, it can't be zero. \ \ \ Example where it matters: \ Let's try to $$$f(10^{6},10^{6})$$$ with $$$f(n,m)=[x^{n}y^{m}] \frac{1}{1-xy}$$$. We will try to calculate some first columns, and then a row. But even if we managed to somehow guess the first columns, we would be unable to get a row, because our initial values in this row will all be zeroes, and we want to get $$$1$$$ somehow, but we can't. \ \ \ Example of the problems: \
\hyperlink{https://atcoder.jp/contests/arc202/tasks/arc202_d}{Atcoder Regular 202 problem D} \ Here if you are solving it in $$$O((H+w+T)^{1.5}$$$ using the reflection principle you have a subproblem calculate the $$$(1+x+x^{2})^{a}[x^{a+k}]$$$ for all $$$0 \leq a \leq T$$$. This can be solved by FFT (and even in such a way, that it will not add the logarithm to the asymptotic of the solution), but this also can be solved using the fact that it is P-recursive (because it is а column shift of a diagonal) and find coefficients with $$$a=k,...,k+C$$$, and then extend it to the whole sequence in a linear time. \ \ We can also solve \hyperlink{https://codeforces.me/contest/1747/problem/E}{1747E} in $$$O(\sqrt{n}log(n))$$$ per test (compute the coefficient of $$$(1-x)^{2}(1-y)^{2}(1-2x-2y+2xy)^{-2}$$$), but actually, because there is a multitest and the constraints are big this solution is hard to pass (I have managed, but after a lot of pain and with some strange FFT \hyperlink{https://codeforces.me/contest/1747/submission/330486092}{submission}) \ \ I think there are some other examples of problems that can be solved by this method. \ \end{document}

История

 
 
 
 
Правки
 
 
  Rev. Язык Кто Когда Δ Комментарий
en23 Английский turmax 2025-07-26 10:35:09 3
en22 Английский turmax 2025-07-25 23:25:18 0 (published)
en21 Английский turmax 2025-07-25 23:23:59 52
en20 Английский turmax 2025-07-25 23:04:03 46
en19 Английский turmax 2025-07-25 22:44:06 22
en18 Английский turmax 2025-07-25 22:42:31 242
en17 Английский turmax 2025-07-25 18:12:33 19
en16 Английский turmax 2025-07-25 18:11:20 66
en15 Английский turmax 2025-07-25 17:59:08 64
en14 Английский turmax 2025-07-25 17:49:40 8
en13 Английский orz 2025-07-25 17:40:40 8
en12 Английский turmax 2025-07-25 17:36:38 2
en11 Английский turmax 2025-07-25 17:33:35 90
en10 Английский orz 2025-07-25 17:28:44 1180
en9 Английский turmax 2025-07-25 16:45:33 23
en8 Английский turmax 2025-07-25 16:43:33 12
en7 Английский turmax 2025-07-25 16:43:10 206
en6 Английский turmax 2025-07-25 16:41:53 20
en5 Английский turmax 2025-07-25 16:38:34 534
en4 Английский turmax 2025-07-25 16:22:13 12
en3 Английский turmax 2025-07-25 16:20:23 987
en2 Английский turmax 2025-07-25 16:08:28 6
ru21 Русский turmax 2025-07-25 15:54:52 40
ru20 Русский turmax 2025-07-25 15:54:03 135
ru19 Русский turmax 2025-07-25 13:45:00 2
ru18 Русский turmax 2025-07-25 13:04:07 1
en1 Английский turmax 2025-07-25 13:00:48 44524 Initial revision for English translation (saved to drafts)
ru17 Русский turmax 2025-07-25 12:55:00 1
ru16 Русский turmax 2025-07-25 12:51:43 104
ru15 Русский turmax 2025-07-25 12:40:48 10
ru14 Русский turmax 2025-07-25 12:03:06 374
ru13 Русский turmax 2025-07-25 11:59:09 570
ru12 Русский turmax 2025-07-25 11:42:55 147
ru11 Русский turmax 2025-07-25 11:33:44 13813
ru10 Русский turmax 2025-07-25 11:24:16 40197
ru9 Русский turmax 2025-07-25 11:15:26 278
ru8 Русский turmax 2025-07-25 11:06:27 117
ru7 Русский turmax 2025-07-25 10:43:01 33
ru6 Русский turmax 2025-07-24 22:38:15 320
ru5 Русский turmax 2025-07-24 22:19:26 110
ru4 Русский turmax 2025-07-24 22:05:25 73
ru3 Русский turmax 2025-07-24 22:01:57 1274
ru2 Русский turmax 2025-07-24 21:50:09 16620
ru1 Русский turmax 2025-07-24 21:36:04 135 Первая редакция (сохранено в черновиках)