Блог пользователя Seferoglu

Автор Seferoglu, история, 12 месяцев назад, По-английски

Good evening CodeForces,today I tried solving https://oj.uz/problem/view/JOI16_selling_rna. My code has a very weird behaviour. To start, compiling in C++20 takes so long that I guess oj.uz marks it as compilation error. The same code gets AC in C++17 (and compiles instantly on my machine). See here:

https://oj.uz/submission/1277541

https://oj.uz/submission/1277540

Another weird behavior I encountered while implementing is if you switch the vector ids; with set ids; in the Node struct, and change the push_back's with insert's, it causes an infinite loop I wasn't able to put my finger on. Also if one compiles the set solution with glibcxx, the following error message is produced (which I'm unable to use to my advantage)

Spoiler

Please help me since I had a similar problem with c++20 before at an important exam (I had compilation TLE on cms on a very simple code which I dont have access to rn)

  • Проголосовать: нравится
  • +45
  • Проголосовать: не нравится

»
12 месяцев назад, скрыть # |
 
Проголосовать: нравится 0 Проголосовать: не нравится

Auto comment: topic has been updated by Seferoglu (previous revision, new revision, compare).

»
12 месяцев назад, скрыть # |
 
Проголосовать: нравится 0 Проголосовать: не нравится

Auto comment: topic has been updated by Seferoglu (previous revision, new revision, compare).

»
12 месяцев назад, скрыть # |
 
Проголосовать: нравится 0 Проголосовать: не нравится

You mean you changed ids to set? There is no nds in the Node struct

»
12 месяцев назад, скрыть # |
 
Проголосовать: нравится 0 Проголосовать: не нравится

Auto comment: topic has been updated by Seferoglu (previous revision, new revision, compare).

»
12 месяцев назад, скрыть # |
← Rev. 2  
Проголосовать: нравится 0 Проголосовать: не нравится

I think the issue is the large global vectors of size N. Replacing these vectors with arrays makes the compile time go from 10s to 2s on my machine

»
12 месяцев назад, скрыть # |
 
Проголосовать: нравится -13 Проголосовать: не нравится

Your code goes in an infinite loop because it has undefined behaviour. The message

Error: attempt to dereference a past-the-end iterator.

says that you are trying to dereference an iterator that does not have a value.
For example, auto temp = set.end(); auto x = *temp;. It is being caught by glibcxx's debug implementation of std::set.
See: std::set::end().

As for the high compilation times, they are certainly not due to the large sizes of the global vectors. The size of a vector does not effect compilation at all. It's quite the opposite, large global arrays effect compilation time since they are stored in the program image. I suspect the reason compilation times improve with arrays is due to the large number of vector operations you use which reduce to primitive array operations. For example, the function call std::vector<Node, std::allocator<Node>>::operator[]() reduces to a primitive array indexing operation.

»
12 месяцев назад, скрыть # |
 
Проголосовать: нравится -8 Проголосовать: не нравится

That's insane :p

Long compilation time issue

I managed to reduce the long compilation issue to this program:


#include <iostream>
#include <string>
#include <vector>
using namespace std;

const int N = 1e5 + 1;
vector<string> strs(N), rev(N);
vector<pair<string, string>> qs(N);

int main() {
    // Fake work
    cout << strs.capacity() << ' ' << rev.capacity() << ' ' << qs.capacity() << '\n';
}

Here are some compilation time benchmarks on my local system with gcc (GCC) 14.2.1 20250405.

g++ -O2 -std=c++17 with mean 291.7 ms
g++ -std=c++20 with mean 7.023 s
g++ -O2 -std=c++20 with mean 7.015 s

An easy fix is to initialize the vectors with length N at runtime, like:

vector<string> strs, rev;
vector<pair<string, string>> qs;
vi ans;

void init() {
    strs.resize(N);
    rev.resize(N);
    qs.resize(N);
    ans.resize(N);
}

int main() {
    init();
    ...
}

The compilation time for your solution goes from 9.314s to 2.056s in C++20, and from 1.377s to 1.275s in C++17, after using the init function.

I tested on Compiler Explorer and for x86-64, gcc 14.2 takes approx 18s, while every version >= 14.3 takes approx 4s. So it is definitely a gcc bug, and the OJ is using a version <= 14.2.


set, in place of vector, causes infinite loop

The offending part is:

        for (auto id : nds[it].ids) {
            if (nds[node].trieroot == -1) {
                nds[node].trieroot = newnode();
            }
            ...
        }

You have two such loops where nds.push_back is called, via newnode(), while iterating over some field of some element of nds. When nds.push_back is called, the internal buffer might be reallocated and moved to some other memory address if the nds.capacity() == nds.size(). This means the nds[it] that you are working might be freed.

Again, here is a minimal reproduction of the issue:

#include <bits/stdc++.h>
using namespace std;

int main() {
    vector<vector<int>> nodes(1); // 1
    // vector<set<int>> nodes(1); // 2

    nodes[0] = {0, 1, 2};
    assert(nodes.capacity() == 1);
    auto* p = addressof(nodes[0]);

    for (auto id : nodes[0]) {
        if (nodes.size() == 1) {
            nodes.push_back({});
            assert(nodes.capacity() == 2);
        }

        auto* q = addressof(nodes[0]);
        cout << p->size() << ' ' << q->size() << ' ' << id << '\n';
        assert(p != q);
    }
}

You can comment out the vector line, and uncomment set one, to see the issue. Memory is invalidated in both the cases but my guess is that the range based for loop over a vector stores the size of the vector at the beginning, but range for loop over a set cannot do that so it must iterate till the iterator equals set::end().

  • »
    »
    12 месяцев назад, скрыть # ^ |
     
    Проголосовать: нравится 0 Проголосовать: не нравится

    Continuing

    Something like this:

    vector<int> v;
    for (size_t i = 0, len = std::size(v); i < len; ++i) {
        auto it = v[i];
        ...
    }
    
    set<int> s;
    for (auto iter = v.begin(); iter != v.end(); ++iter) {
        auto it = *iter;
        ...
    }
    

    So, even if the memory is freed, the no. of iterations is bounded in case of a vector, and the odds are that the memory is not overwritten so all the elements are intact too (must be the case since you got AC). This fails in case of set.


    An easy fix for this issue is to do nds.reserve(<MAX NO. OF NODES YOU EXPECT TO HAVE>); in the init function. You should be able to use a set after this.

  • »
    »
    12 месяцев назад, скрыть # ^ |
     
    Проголосовать: нравится 0 Проголосовать: не нравится

    Tried to compile your minified example with clang (19.1.7), got similar results:

    Times
»
12 месяцев назад, скрыть # |
 
Проголосовать: нравится 0 Проголосовать: не нравится

As for long compile time for the C++20, I've done some research and turns out it's not a bug. Since C++20 there were introduced Constant initialization for all static storage duration variables, global vectors do have a static storage duration. Constant initialization is usually applied at compile time, so it probably increasing compile time for the big global vectors. If you set smaller value to the N, like 1000, Compile time will significantly be decreased.

As for global arrays getting compiled faster, it's obvious because they usually are way faster in initialization part than vectors, and doesn't allocate extra memory for later new elements.

  • »
    »
    12 месяцев назад, скрыть # ^ |
     
    Проголосовать: нравится 0 Проголосовать: не нравится

    Not all static storage duration variables, only those initialised with a constant expression. There can be local statics.

    • »
      »
      »
      12 месяцев назад, скрыть # ^ |
       
      Проголосовать: нравится 0 Проголосовать: не нравится

      Read the link I attached, there is a part introduced in C++20.

      • »
        »
        »
        »
        12 месяцев назад, скрыть # ^ |
         
        Проголосовать: нравится 0 Проголосовать: не нравится

        Right back at you. Read what's under the link contained in the "part introduced in C++20".

        The full-expression of its initialization is a constant expression

        You literally can't constant-init using a non-constant, which can happen with local statics.

        • »
          »
          »
          »
          »
          12 месяцев назад, скрыть # ^ |
          ← Rev. 3  
          Проголосовать: нравится 0 Проголосовать: не нравится

          Well, my bad. But I don't understand why its written like that in cppreference, It's written

          A variable or temporary object(until C++26) with static or thread storage duration is constant-initialized.

          In my viewing it should mean variable with static storage duration must be constant-initialized. But in contrary in under the link contained in that same sentence It says not all static storage duration variables are constant-initialized. Very strange for me.

          • »
            »
            »
            »
            »
            »
            12 месяцев назад, скрыть # ^ |
             
            Проголосовать: нравится +5 Проголосовать: не нравится

            Simple: treat the linked phrase in its entirety, including the link. Writing out long caseworkey definitions all the time is impractical but if you can just refer to an existing definition, that shortens things considerably. It's the same principle as subroutines in programming.

  • »
    »
    12 месяцев назад, скрыть # ^ |
     
    Проголосовать: нравится 0 Проголосовать: не нравится

    Thanks for this. I tried looking it up but did not know what to search for. Seems like gcc versions <= 14.2 had a buggy implementation of constant initialization. Because the compilation time is very unusual and it improves 5x over the next version 14.3.