Joint Bilateral Upsampling in Laplacian Pyramid

在 Laplacian pyramid 重建回原影像時，我們嘗試利用 edge-preserved upsampling 的方式重建。

\[G_{i}=upsample(G_{i+1})+L_{i}\]

與 Gaussian upsampling 比較，joint bilateral upsampling 利用 Gaussian pyramid 中前一級較大解析度的影像作為 guide image。可以發現在高光處的擴散光暈有明顯的減少，更接近 ground truth 的結果。

 

 

void ToneMap3::jointUpsampling(const Mat1f &inLow, const Mat1f &inHigh, Mat1f &out) {
    Mat1f lumaMap   = Mat1f::zeros( inHigh.size() );
    Mat1f oneMap  = Mat1f::zeros( inHigh.size() );
    for (int by = 0, sy = 0; sy < inLow.rows && by < lumaMap.rows; by+=2, sy++) {
        for (int bx = 0, sx = 0; sx < inLow.cols && bx < lumaMap.cols; bx+=2, sx++) {
            if (by >= lumaMap.rows || bx > lumaMap.cols) break;
            lumaMap(by, bx)  = inLow(sy, sx);
            oneMap(by, bx)   = 1.0;
        }
    }

    int pad = gaussianKernelDown5.rows / 2;

    Mat1f high2;
    copyMakeBorder(inHigh , high2  , pad, pad, pad, pad, BORDER_REFLECT_101);
    copyMakeBorder(lumaMap, lumaMap, pad, pad, pad, pad, BORDER_REFLECT_101);
    copyMakeBorder(oneMap , oneMap , pad, pad, pad, pad, BORDER_REFLECT_101);

    // Bilateral filtering
    Mat1f weightLuma = Mat1f::zeros( inHigh.size() );
    Mat1f weightOne = Mat1f::zeros( inHigh.size() );
    #pragma omp parallel for
    for (int row = pad; row < lumaMap.rows-pad; ++row) {
        for (int col = pad; col < lumaMap.cols-pad; ++col) {
            // guide image
            const Mat1f guide = high2(Range(row-pad, row+pad+1), Range(col-pad, col+pad+1));

            // Bilateral weighting
            Mat1f weight(guide.rows, guide.cols);

            float c = guide(guide.total() / 2); // center intensity

            for (unsigned int i = 0; i < guide.total(); ++i) {
                float diff = fabs(guide(i) - c);
                weight(i) = exp(-diff*diff / 2) * gaussianKernelDown5(i);
            }
            // normalize weight
            weight /= sum(weight)[0];

            // smooth luma map
            const Mat1f subLuma = lumaMap(Range(row-pad, row+pad+1), Range(col-pad, col+pad+1));
            weightLuma(row-pad, col-pad) = sum(weight.mul(subLuma))[0];

            // smooth one map
            const Mat1f subOne = oneMap(Range(row-pad, row+pad+1), Range(col-pad, col+pad+1));
            weightOne(row-pad, col-pad) = sum(weight.mul(subOne))[0];
        }
    }

    // normalize weighting
    out = weightLuma.mul(1.0 / weightOne);
}

Reference:

Joint Bilateral Upsampling – Microsoft Research