How Do I Apply Function To Third-dimension Array Effectively With Numpy?

I want to apply arbitrary function to 3d-ndarray as element, which use (3rd-dimensional) array for its arguments and return scalar.As a result, we should get 2d-Matrix. e.g) pseudo

Solution 1:

Given the function implementation, we could vectorize it using NumPy ufuncs that would operate on the entire input array A in one go and thus avoid the math library functions that doesn't support vectorization on arrays. In this process, we would also bring in the very efficient vectorizing tool : NumPy broadcasting. So, we would have an implementation like so -

np.log(A/np.power(np.sum(A,2,keepdims=True),1/3))

Sample run and verification

The function implementation without the lamdba construct and introducing NumPy functions instead of math library functions, would look something like this -

def chromaticity(pixel): 
    geo_mean = np.power(np.sum(pixel),1/3) 
    return np.log(pixel/geo_mean)

Sample run with the iterative implementation -

In [67]: chromaticity(A[0,0,:])
Out[67]: array([-0.59725316,  0.09589402,  0.50135913])

In [68]: chromaticity(A[0,1,:])
Out[68]: array([ 0.48361096,  0.70675451,  0.88907607])

In [69]: chromaticity(A[1,0,:])
Out[69]: array([ 0.88655887,  1.02009026,  1.1378733 ])

In [70]: chromaticity(A[1,1,:])
Out[70]: array([ 1.13708257,  1.23239275,  1.31940413])    

Sample run with the proposed vectorized implementation -

In [72]: np.log(A/np.power(np.sum(A,2,keepdims=True),1/3))
Out[72]: 
array([[[-0.59725316,  0.09589402,  0.50135913],
        [ 0.48361096,  0.70675451,  0.88907607]],

       [[ 0.88655887,  1.02009026,  1.1378733 ],
        [ 1.13708257,  1.23239275,  1.31940413]]])

Runtime test

In [131]: A = np.random.randint(0,255,(512,512,3)) # 512x512 colored image

In [132]: deforg_app(A):
     ...:     out = np.zeros(A.shape)     
     ...:     for i inrange(A.shape[0]):
     ...:         for j inrange(A.shape[1]):
     ...:             out[i,j] = chromaticity(A[i,j])
     ...:     return out
     ...: 

In [133]: %timeit org_app(A)
1 loop, best of 3: 5.99 s per loop

In [134]: %timeit np.apply_along_axis(chromaticity, 2, A) #@hpaulj's soln1 loop, best of 3: 9.68 s per loop

In [135]: %timeit np.log(A/np.power(np.sum(A,2,keepdims=True),1/3))
10 loops, best of 3: 90.8 ms per loop

That's why always try to push in NumPy funcs when vectorizing things with arrays and work on as many elements in one-go as possible!

Solution 2:

apply_along_axis is designed to make this task easy:

In [683]: A=np.arange(1,13).reshape(2,2,3)
In [684]: A
Out[684]: 
array([[[ 1,  2,  3],
        [ 4,  5,  6]],

       [[ 7,  8,  9],
        [10, 11, 12]]])
In [685]: np.apply_along_axis(np.sum, 2, A)
Out[685]: 
array([[ 6, 15],
       [24, 33]])

It, in effect, does

for alli,j:
    out[i,j] = func( A[i,j,:])

taking care of the details. It's not faster than doing that iteration yourself, but it makes it easier.

Another trick is to reshape your input to 2d, perform the simpler 1d iteration, and the reshape the result

 A1 = A.reshape(-1, A.shape[-1])
 for i inrange(A1.shape[0]):
     out[i] = func(A1[i,:])
 out.reshape(A.shape[:2])

To do things faster, you need to dig into the guts of the function, and figure out how to use compile numpy operations on more than one dimension. In the simple case of sum, that function already can work on selected axes.