I'm looking for a way to store the internal state of an sha256 hash in python and then bring it back later so that I can continue hashing the stream of content where the code left off previously. I suppose this might be possible via pickle but that raises a security risk because I want to store the state externally. Is there any way to get at the internal bits/bytes of a hashlib.sha256() and then later create a new one from those same internal bytes. IE: this would work as if the later created one was made via the copy() method on the first. A little more context I'm uploading chunks of data to AWS S3 which will ultimately form a multipart upload. They're sent in a strict order and will initially be uploaded as S3 objects in their own right. When all have been uploaded they'll then be rolled into one object using upload_part_copy. At the end of the process I want to know the final SHA256 hash of the combined object. I won't know the full file content until all parts are uploaded, so there's no way to know the SHA256 hash ahead of time. There's a requirement to make this process resumable. What I DON'T want to do is download 500GB of content just to re-compute the hashes. My idea is that I could maintain a hashlib.sha256() object while I'm uploading then I might be able to extract the internal state of the hash object each time I upload a chunk of data and attach that as metadata to the chunk. If I need to resume then all I need to do is fetch the metadata on the last chunk and restore the hash object. Continue reading...
