How To Join 2 Lists Of Dicts In Python?

I have 2 lists like this: l1 = [{'a': 1, 'b': 2, 'c': 3, 'd': 4}, {'a': 5, 'b': 6, 'c': 7, 'd': 8}] l2 = [{'a': 5, 'b': 6, 'e': 100}, {'a': 1, 'b': 2, 'e': 101}] and I want to ob

Solution 1:

You should accumulate the results in a dictionary. You should use the values of 'a' and 'b' to form a key of this dictionary

Here, I have used a defaultdict to accumulate the entries

l1 = [{'a': 1, 'b': 2, 'c': 3, 'd': 4}, {'a': 5, 'b': 6, 'c': 7, 'd': 8}]
l2 = [{'a': 5, 'b': 6, 'e': 100}, {'a': 1, 'b': 2, 'e': 101}]

from collections import defaultdict
D = defaultdict(dict)
for lst in l1, l2:
    for item in lst:
        key = item['a'], item['b']
        D[key].update(item)

l3 = D.values()
print l3

output:

[{'a': 1, 'c': 3, 'b': 2, 'e': 101, 'd': 4}, {'a': 5, 'c': 7, 'b': 6, 'e': 100, 'd': 8}]

Solution 2:

Simple list operations would do the thing for you as well:

l1 = [{'a': 1, 'b': 2, 'c': 3, 'd': 4}, {'a': 5, 'b': 6, 'c': 7, 'd': 8}]
l2 = [{'a': 5, 'b': 6, 'e': 100}, {'a': 1, 'b': 2, 'e': 101}]
l3 = []

for i in range(len(l1)):
    for j in range(len(l2)):
        if l1[i]['a'] == l2[j]['a'] and l1[i]['b'] == l2[j]['b']:
            l3.append(dict(l1[i]))
            l3[i].update(l2[j])

Solution 3:

My approach is to sort the the combined list by the key, which is keys a + b. After that, for each group of dictionaries with similar key, combine them:

from itertools import groupby

defab_key(dic):
    return dic['a'], dic['b']

defcombine_lists_of_dicts(list_of_dic1, list_of_dic2, keyfunc):
    for key, dic_of_same_key in groupby(sorted(list_of_dic1 + list_of_dic2, key=keyfunc), keyfunc):
        combined_dic = {}
        for dic in dic_of_same_key:
            combined_dic.update(dic)
        yield combined_dic

l1 = [{'a': 1, 'b': 2, 'c': 3, 'd': 4}, {'a': 5, 'b': 6, 'c': 7, 'd': 8}]
l2 = [{'a': 5, 'b': 6, 'e': 100}, {'a': 1, 'b': 2, 'e': 101}]

for dic in combine_lists_of_dicts(l1, l2, ab_key):
    print dic

Discussion

• The function ab_key returns a tuple of value for key a and b, used for sorting a groupping
• The groupby function groups all the dictionaries with similar keys together
• This solution is less efficient than that of John La Rooy, but should work fine for small lists

Solution 4:

One can achieve a nice and quick solution using pandas.

l1 = [{'a': 1, 'b': 2, 'c': 3, 'd': 4}, {'a': 5, 'b': 6, 'c': 7, 'd': 8}]
l2 = [{'a': 5, 'b': 6, 'e': 100}, {'a': 1, 'b': 2, 'e': 101}]

import pandas as pd
pd.DataFrame(l1).merge(pd.DataFrame(l2), on=['a','b']).to_dict('records')