I'm trying do a very specific thing with dictionaries. Given a dictionary, I want to write a function that returns the same dictionary, but with no nested fields. I want to keep all nested values, but I want them to be at the top level with some distinct key - this key, should be a string of every key accessed before the value, joined with an underscore. For example, stock["ratios"]["portfolio_percent"], should become stock["ratios_portfolio_percent"]. And so on with all the other nested values. Except I also don't want redundant keys. I don't want the key to be ratios_portfolio_percent if it can be portfolio_percent without conflicts (with other keys). So the function should recursively look through a dictionary and find the shortest key possible for all the nested fields while joining all previous, necessary keys. My problem is not that I can't write this function - I already have, and it works okay (feel free to suggest optimizations). def serialize(dictionary): field_list = [] def fields_recursion(value, trail=[]): for field in value: new_trail = trail new_value = value[field] if type(new_value) == dict: new_trail = new_trail + [field] fields_recursion(new_value, new_trail) else: field_list.append({"field": field, "trail": new_trail}) fields_recursion(dictionary) duplicates_found = [] field_count = {} for item in field_list: fid = item["field"] if fid not in field_count: field_count[fid] = [] field_count[fid].append(item) for fid, items in field_count.items(): if len(items) > 1: duplicates_found.extend(items) while len(duplicates_found) > 0: duplicate = duplicates_found.pop(0) field = duplicate["field"] trail = duplicate["trail"] trailing = trail[-1] new_trail = trail[:-1] new_field = f"{trailing}_{field}" for i, f in enumerate(field_list): if f == duplicate: field_list = {"field": new_field, "trail": new_trail} serialize(dictionary) # Look below for a link to an example dictionary (Use this JSON (or this Python compatible dictionary) as an example of what this should work with. The problem (not really an issue, more just confusing) lies with following two lines. trailing = trail[-1] new_trail = trail[:-1] If instead of taking the last item in trail with a -1 index, I .pop the trail array, the entire program breaks. To replicate this behavior, replace the above two lines with the following. trailing = trail.pop(-1) new_trail = trail Although the code should be identical, if you pop the trail array, for some reason, it pops the last item of the trail array for every matching item in field_list. If you run the code (with the example JSON provided above), and break at the pop line, the first break should be when field is "action", and the last item in trail is "value". Normally, this should replace the corresponding item in field_list with a field of "value_action" and a trail that is one shorter (with the last element, "value", removed). Instead, it goes through every item in field_list and pops "value" from every trail array. So instead of one item in the array changing, seven do, which causes following iterations of the duplicates loop to fail because trail becomes too short (you cannot access -1 if the array is empty). Is there any reasonable explanation for why this works the way it is? My guess is that this has something to do with how loose a language Python is. Continue reading...
