Time increase with iteration

[Jiayun Liu]

I have to repeatedly call a custom function and I notice that the time for each iteration is roughly the same for the first ~100 iterations but suddenly doubles after ~150 iterations going from 3 ms to eventually 10 ms after 200 iterations. I have attached the code below. What could be the reason for this?

    for (int i = 0; i < 200; i++) {
        std::string filename = std::to_string(i) + ".png";
        cv::Mat img = cv::imread(filename, cv::IMREAD_GRAYSCALE);
        af::array afimg(2048, 2048, img.data, afHost);
        af::array bfimg = afimg.as(f32);

        af::timer::start();
       
        index = bufferInd(0);

        buffer(af::span, af::span, index) = bfimg;

        bufferReshape(index, af::span) = af::moddims(bfimg, 1, rows * cols);

        bufferInd = af::shift(bufferInd, -1, 0, 0, 0);

        af::sync();

        std::cout << "Reading time: " << af::timer::stop() << " s" << std::endl;
       
    } 

[ste...@arrayfire.com]

Can you try adding an af:eval(bufferInd); before the af::sync() to see if that normalizes your time? It's likely the JIT isn't able to find a good tree size for this usecase and eval will prevent the JIT tree from growing too large with multiple iterations. This issue pops up when you are using a JIT operation like af::shift and you use the same variable on the left hand side and the right hand side.