How To Merge Images As Transparent Layers?

I am working on video editor for raspberry pi, and I have a problem with speed of placing image over image. Currently, using imagemagick it takes up to 10 seconds just to place one

Solution 1:

My Raspberry Pi is unavailable at the moment - all I am saying is that there was some smoke involved and I do software, not hardware! As a result, I have only tested this on a Mac. It uses Numba.

First I used your Numpy code on these 2 images:

and

Then I implemented the same thing using Numba. The Numba version runs 5.5x faster on my iMac. As the Raspberry Pi has 4 cores, you could try experimenting with:

@jit(nopython=True,parallel=True)defmethod2(image1,image2,transparency):
   ...

Here is the code:

#!/usr/bin/env python3import numpy as np
from PIL import Image

import numba
from numba import jit

defmethod1(image1,image2,transparency):
   result = image1*transparency+image2*(1-transparency)
   return result

@jit(nopython=True)defmethod2(image1,image2,transparency):
   h, w, c = image1.shape
   for y inrange(h):
      for x inrange(w):
         for z inrange(c):
            image1[y][x][z] = image1[y][x][z] * transparency + (image2[y][x][z]*(1-transparency))
   return image1

i1 = np.array(Image.open('image1.jpg').convert('RGB'))
i2 = np.array(Image.open('image2.jpg').convert('RGB'))

res = method1(i1,i2,0.4)
res = method2(i1,i2,0.4)

Image.fromarray(np.uint8(res)).save('result.png')

The result is:

Other thoughts... I did the composite in-place, overwriting the input image1 to try and save cache space. That may help or hinder - please experiment. I may not have processed the pixels in the optimal order - please experiment.

Solution 2:

Just as another option, I tried in pyvips (full disclosure: I'm the pyvips maintainer, so I'm not very neutral):

#!/usr/bin/python3

import sys
import time
import pyvips

start = time.time()

a = pyvips.Image.new_from_file(sys.argv[1], access="sequential")
b = pyvips.Image.new_from_file(sys.argv[2], access="sequential")
out = a * 0.2 + b * 0.8
out.write_to_file(sys.argv[3])

print("pyvips took {} milliseconds".format(1000 * (time.time() - start)))

pyvips is a "pipeline" image processing library, so that code will execute the load, processing and save all in parallel.

On this two core, four thread i5 laptop using Mark's two test images I see:

$ ./overlay-vips.py blobs.jpg ships.jpg x.jpg
took 39.156198501586914 milliseconds

So 39ms for two jpg loads, processing and one jpg save.

You can time just the blend part by copying the source images and the result to memory, like this:

a = pyvips.Image.new_from_file(sys.argv[1]).copy_memory()
b = pyvips.Image.new_from_file(sys.argv[2]).copy_memory()

start = time.time()
out = (a * 0.2 + b * 0.8).copy_memory()
print("pyvips between memory buffers took {} milliseconds"
        .format(1000 * (time.time() - start)))

I see:

$ ./overlay-vips.py blobs.jpg ships.jpg x.jpg 
pyvips between memory buffers took 15.432596206665039 milliseconds

numpy is about 60ms on this same test.

I tried a slight variant of Mark's nice numba example:

#!/usr/bin/python3import sys
import time
import numpy as np
from PIL import Image

import numba
from numba import jit, prange

@jit(nopython=True, parallel=True)defmethod2(image1, image2, transparency):
   h, w, c = image1.shape
   for y in prange(h):
      for x inrange(w):
         for z inrange(c):
            image1[y][x][z] = image1[y][x][z] * transparency \
                    + (image2[y][x][z] * (1 - transparency))
   return image1

# run once to force a compile
i1 = np.array(Image.open(sys.argv[1]).convert('RGB'))
i2 = np.array(Image.open(sys.argv[2]).convert('RGB'))
res = method2(i1, i2, 0.2)

# run again and time it
i1 = np.array(Image.open(sys.argv[1]).convert('RGB'))
i2 = np.array(Image.open(sys.argv[2]).convert('RGB'))

start = time.time()
res = method2(i1, i2, 0.2)
print("numba took {} milliseconds".format(1000 * (time.time() - start)))

Image.fromarray(np.uint8(res)).save(sys.argv[3])

And I see:

$ ./overlay-numba.py blobs.jpg ships.jpg x.jpg 
numba took 8.110523223876953 milliseconds

So on this laptop, numba is about 2x faster than pyvips.

If you time load and save as well, it's quite a bit slower:

$ ./overlay-numba.py blobs.jpg ships.jpg x.jpg 
numba plus load and save took 272.8157043457031 milliseconds

But that seems unfair, since almost all that time is in PIL load and save.