[Python] How do I make a custom field output different datatypes depending on file asking for...

I'm building a molecule properties displaying website using django and sqlite. The database takes a numpy array and stores it as a BLOB. Currently, it just outputs the numpy array back out when asked for it. I'd like it to check what kind of file is asking the data, and output a numpy array if python is asking for it ( for instance for a data integrity test ), and a json object if requested by a javascript file ( like for molecule visualization ).

I'm not even sure if it's possible, so I guess my question is twofold. One, is it possible? Two, if it's not, as long as the data is read only, are there any major problems that might come up if I let inputs be as numpy arrays, and outputs be as json.

The custom field is defined as so:

The packages used:

import io
import json
import numpy as np
from django.db.models import BinaryField
from django.core.exceptions import ValidationError
from django.utils.translation import gettext_lazy as _


For the current numpy only implementation:

# Stores numpy array as BLOB then returns BLOB as np array
class NpArrayField(BinaryField):
"""
Turns numpy array into raw binary data to store in database. When take from database, it outputs the numpy array.
"""
def from_db_value(self, value, expression, connection):

if value is None:
return None
buffer = io.BytesIO(value)
return np.load(buffer, allow_pickle=True) # returns numpy array


def to_python(self, value):

if value is None or isinstance(value, np.ndarray):
return value

# If it's bytes from DB or form
if isinstance(value, (bytes, bytearray)):
buffer = io.BytesIO(value)
return np.load(buffer, allow_pickle=True)

raise ValidationError(_("NpArrayField value must be a numpy.ndarray or a byte/bytearray"))


def get_prep_value(self, value):

if value is None:
return None

if not isinstance(value, np.ndarray):
raise ValidationError(_("NpArrayField value must be a numpy.ndarray"))

buffer = io.BytesIO()
np.save(buffer, value, allow_pickle=True)
return buffer.getvalue()


def deconstruct(self):
name, path, args, kwargs = super().deconstruct()
return name, path, args, kwargs


An implementation which takes a numpy array (stored as a BLOB), and which outputs json:

class NpArrayEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, numpy.ndarray):
return obj.tolist()
return json.JSONEncoder.default(self,obj)

class NpArrayField(BinaryField):
"""
Turns numpy array into json style raw binary data to store in database. When taken from the database, it outputs the json
"""
def from_db_value(self, value, expression, connection):

if value is None:
return None
return value.decode('utf-8'))


def to_python(self, value):
if value is None:
return None
return value

def get_prep_value(self, value):

if value is None:
return None

if not isinstance(value, np.ndarray):
raise ValidationError(_("NpArrayField value must be a numpy.ndarray"))

np_as_json = json.dumps({"array" : value}, cls=NpArrayEncoder)
return bytes(np_as_json, 'utf-8')


def deconstruct(self):
name, path, args, kwargs = super().deconstruct()
return name, path, args, kwargs


In case anyone was wondering, these arrays should be immutable, so it makes sense in this paradigm to store them as atomic units rather than as individual values.

Continue reading...