[Python] How to decode a tfrecord based on the available meta data

Discussão em 'Python' iniciado por Stack, Agosto 1, 2025.

  1. Stack

    Stack Membro Participativo

    I have a three tfrecords (train,test valid) from the a deepmind github repo (https://github.com/google-deepmind/deepmind-research/tree/master/meshgraphnets) that I am trying to decode (at the minute just to interrogate the data but long term to train a GNN) however I can't work out how to decode the tfrecord.

    The json below is the meta data that comes with the data that explains the features' shape, dtype etc. but can't work out how to use that to retrieve the data i need.

    {
    "simulator": "comsol",
    "dt": 0.01,
    "collision_radius": null,
    "features": {
    "cells": {
    "type": "static",
    "shape": [
    1,
    -1,
    3
    ],
    "dtype": "int32"
    },
    "mesh_pos": {
    "type": "static",
    "shape": [
    1,
    -1,
    2
    ],
    "dtype": "float32"
    },
    "node_type": {
    "type": "static",
    "shape": [
    1,
    -1,
    1
    ],
    "dtype": "int32"
    },
    "velocity": {
    "type": "dynamic",
    "shape": [
    600,
    -1,
    2
    ],
    "dtype": "float32"
    },
    "pressure": {
    "type": "dynamic",
    "shape": [
    600,
    -1,
    1
    ],
    "dtype": "float32"
    }
    },
    "field_names": [
    "cells",
    "mesh_pos",
    "node_type",
    "velocity",
    "pressure"
    ],
    "trajectory_length": 600
    }


    I can get as far as retrieving binary strings for each of the features using the code below however I can't figure out how to decode the binary into something usable.

    from tensorflow.io import FixedLenFeature, VarLenFeature

    def extract_fn(data_record):
    features = {
    "cells": FixedLenFeature([], tf.string),
    "mesh_pos": FixedLenFeature([], tf.string),
    "node_type":FixedLenFeature([], tf.string),
    "velocity":FixedLenFeature([], tf.string),
    "pressure":FixedLenFeature([], tf.string),

    }
    sample = tf.io.parse_single_example(data_record, features)
    #sample = tf.cast(sample["image/encoded"], tf.float32)
    return sample

    filename = "/content/CylinderData/cylinder_flow/train.tfrecord"
    dataset = tf.data.TFRecordDataset(filename)
    #print(f'Number of records: {sum(1 for _ in dataset)}')
    #dataset = dataset.map(extract_fn)
    #iterator = dataset.make_one_shot_iterator()
    #next_element = iterator.get_next()
    for i in dataset:
    example = extract_fn(i)
    print(example)
    break

    Continue reading...

Compartilhe esta Página