[Python] Using SWIG to interface numpy arrays with custom templated classes

I'm confronted with some issues getting SWIG to do what I want, if it is even possible. I have reduced the problem to the following. I have a containing a few std::deque instances of type T. T can be double, or an instance of class fxp:

template<int W, int I, bool S, ac_q_mode Q, ac_o_mode O, bool force_float>
using fxp = std::conditional_t<force_float, double, ac_fixed<W, I, S, Q, O>>;


The idea here is that I can do operations on fixed point numbers using the AC types library (for now, only ac_fixed, but to be expanded later). I added a "force_float" template parameter to override the type definition to double. Then, I have a node class containing some std::deque instances, which I would like to access from python through a SWIG generated interface.

#ifndef NODE_H
#define NODE_H

#include <deque>
#include <algorithm>
#include "ac_types.h"

template<typename T>
class node {
public:
node() {
}

~node() {
}

T pop() {
T sample = fifo.front();
fifo.pop_front();
return sample;
}

void push(const T &sample){
fifo.push_back(sample);
data.push_back(sample);
}

bool empty() {
return fifo.empty();
}

size_t size() {
return data.size();
}


template<typename U> std::deque<T> get();

std::deque<double> get(){
std::deque<double> rval(data.size());
// If we want to support get<double>() for node<double>, we can't use arg.to_double as definein ac_fixed. Use
// (double) cast instead
std::transform( data.cbegin(), data.cend(), rval.begin(), [](T arg) -> double { return (double) arg; });
return rval;
}

std::deque<T> fifo;
std::deque<T> data;
};


As I mentioned, I'd like to access fifo and data members from python. I'd like to achieve this avoiding explicit template instantiations, as this would require too much bookkeeping in tracking which fixed-point types are used in any model I build with these types. I'd think this is possible, especially as the different values of the template parameters assigned to ac_fixed classes does not require different actions converting to doubles (what I would fill the numpy array with).

Currently, my SWIG configuration looks like this:

%begin %{
#define SWIG_PYTHON_CAST_MODE
%}

%ignore log;
%ignore fx_cntxt;


%module libdsp
%{
#include "ac_types.h"
#include "node.h"
#include <deque>
#include <numpy/ndarrayobject.h>

%}

%include "numpy.i"
%include "std_deque.i"
%include "std_vector.i"
%include "std_string.i"

// Initialize NumPy API
%init %{
import_array();
%}

// Generic typemap for std::deque<T> to NumPy array conversion
%typemap(out) std::deque<T> {
$result = deque_to_numpy($1);
}

// Generic typemap for NumPy array to std::deque<T> conversion
%typemap(in) std::deque<T> (PyObject* obj) {
$1 = numpy_to_deque<T>(obj);
}

namespace std {
%template(vectori) vector<int>;
%template(vectord) vector<double>;
%template(dequei) deque<int>;
%template(dequed) deque<double>;
};



%include "ac_types.h"
%include "node.h"

%template(node1) node<fxp<16, 1, AC_RND, AC_SAT, false>>;
%template(node2) node<fxp<8, 1, AC_RND, AC_SAT, true>>;


Conversion functions are defined as

template<typename T>
PyObject* deque_to_numpy(const std::deque<T>& deque) {
npy_intp size = deque.size();

// FIXME: Can we detect integer types to return int arrays?
PyObject* numpy_array = PyArray_SimpleNew(1, &size, NPY_DOUBLE);


double* data = static_cast<double*>(PyArray_DATA((PyArrayObject*)numpy_array));
if (std::is_same<T, double>::value) {
for (npy_intp i = 0; i < size; ++i) {
data = deque;
}
} else {
for (npy_intp i = 0; i < size; ++i) {
data = deque.to_double();
}
}

return numpy_array;
}

// Function to convert NumPy array to std::deque<fxp>
template<typename T>
std::deque<T> numpy_to_deque(PyObject* numpy_array) {
std::deque<T> deque;

if (!PyArray_Check(numpy_array)) {
throw std::invalid_argument("Input is not a valid NumPy array");
}

PyArrayObject* array = (PyArrayObject*)numpy_array;
npy_intp size = PyArray_SIZE(array);

double* data = static_cast<double*>(PyArray_DATA(array));
if (std::is_same<T, double>::value) {
for (npy_intp i = 0; i < size; ++i) {
deque.push_back(data);
}
} else {
for (npy_intp i = 0; i < size; ++i) {
deque.push_back(T(data));
}
}

return deque;
}



Using the code above, I was hoping that SWIG would enable me to interface between python and the contents of node instances. However, there is no sign in the generated wrapper code of the conversion functions for example.

Continue reading...