Filter Tensorflow Array With Specific Condition Over Numpy Array

I have a tensorflow array names tf-array and a numpy array names np_array. I want to find specific rows in tf_array with regards to np-array. tf-array = tf.constant(

Solution 1:

Here is the solution from https://stackoverflow.com/a/56510832/7207392 with necessary modifications. For the sake of simplicity I use np.array for all data. I'm no tensortflow expert, so if translating is not entirely straight forward, you'll have to ask somebody else how to do it.

import numpy as np

def f(a1, a2, n):
    N,M = a1.shape
    a1p = np.concatenate([a1,np.zeros((1,a1.shape[1]),a1.dtype)], axis=0)
    a2 = np.sort(a2, axis=1)
    a2[:,1:][a2[:,1:]==a2[:,:-1]] = N
    y,x = np.where(np.count_nonzero(a1p[a2], axis=1) >= n)
    out = np.zeros_like(a1p)
    out[a2[y],x[:,None]] = a1p[a2[y],x[:,None]]
    returnout[:-1]

a1 = np.array(
    [[9.968594,  8.655439,  0.,        0.       ],
     [0.,        8.3356,    0.,        8.8974   ],
     [0.,        0.,        6.103182,  7.330564 ],
     [6.609862,  0.,        3.0614321, 0.       ],
     [9.497023,  0.,        3.8914037, 0.       ],
     [0.,        8.457685,  8.602337,  0.       ],
     [0.,        0.,        5.826657,  8.283971 ]])

a2 = np.array(
 [[2, 5, 1],
  [1, 6, 4],
  [0, 0, 0],
  [2, 3, 6],
  [4, 2, 4]])

print(f(a1,a2,2))

Output:

[[0.        0.        0.        0.       ][0.        8.3356    0.        8.8974   ][0.        0.        6.103182  7.330564 ][0.        0.        3.0614321 0.       ][0.        0.        3.8914037 0.       ][0.        8.457685  8.602337  0.       ][0.        0.        5.826657  8.283971 ]]

Solution 2:

The one eficient way you can try is to make bit flags for each row what value are there like for (0,3,4) will be 1 <<0 | 1<<3 | 1<<4. You will have array of values with flags.Try if << and | operator work in numpy. Make the same for another array, i guess tf- arrays are just wrapped numpys. After having 2 array of flags, make bitwise "and" over those. Where you condition is true for rows, the result will have at least two non zero bits. Also cound of bits can be done also efficient, google for that.

This hovever wont work with float - you ll need convert those to pretty small ints.

import numpy as np



arr_one =  np.array(
 [[2, 5, 1],
  [1, 6, 4],
  [0, 0, 0],
  [2, 3, 6],
  [4, 2, 4]])

arr_two =  np.array(
 [[2, 0, 7],
  [1, 3, 4],
  [5, 5, 6],
  [1, 3, 6],
  [4, 2, 4]])




print('1 << arr_one.T[0] ' , 1 << arr_one.T[0] )


arr_one_flags = 1 << arr_one.T[0] | 1 << arr_one.T[1] | 1 << arr_one.T[2]

print('arr_one_flags ', arr_one_flags)

arr_two_flags = 1 << arr_two.T[0] | 1 << arr_two.T[1] | 1 << arr_two.T[2]

arr_and = arr_one_flags & arr_two_flags

print('arr_and ', arr_and)



def get_bit_count(value):
    n = 0while value:
        n += 1
        value &= value-1return n

arr_matches = np.array([get_bit_count(x) for x in arr_and])


print('arr_matches ', arr_matches )


arr_two_filtered = arr_two[arr_matches > 1]

print('arr_two_filtered ', arr_two_filtered )