Segfault When Import_array Not In Same Translation Unit

I'm having problems getting the NumPy C API to properly initialize. I think I've isolated the problem to calling import_array from a different translation unit, but I don't know wh

Solution 1:

After digging through the NumPy headers, I think I've found a solution:

in numpy/__multiarray_api.h, there's a section dealing with where an internal API buffer should be. For conciseness, here's the relevant snippet:

#if defined(PY_ARRAY_UNIQUE_SYMBOL)#define PyArray_API PY_ARRAY_UNIQUE_SYMBOL#endif#if defined(NO_IMPORT) || defined(NO_IMPORT_ARRAY)externvoid **PyArray_API;
#else#if defined(PY_ARRAY_UNIQUE_SYMBOL)void **PyArray_API;
#elsestaticvoid **PyArray_API=NULL;
#endif#endif

It looks like this is intended to allow multiple modules define their own internal API buffer, in which each module must call their own import_array define.

A consistent way to get several translation units to use the same internal API buffer is in every module, define PY_ARRAY_UNIQUE_SYMBOL to some library unique name, then every translation unit other than the one where the import_array wrapper is defined defines NO_IMPORT or NO_IMPORT_ARRAY. Incidentally, there are similar macros for the ufunc features: PY_UFUNC_UNIQUE_SYMBOL, and NO_IMPORT/NO_IMPORT_UFUNC.

The modified working example:

header1.hpp

#ifndef HEADER1_HPP#define HEADER1_HPP#ifndef MYLIBRARY_USE_IMPORT#define NO_IMPORT#endif#define PY_ARRAY_UNIQUE_SYMBOL MYLIBRARY_ARRAY_API#define PY_UFUNC_UNIQUE_SYMBOL MYLIBRARY_UFUNC_API#include<Python.h>#include<numpy/npy_3kcompat.h>#include<numpy/arrayobject.h>voidinitialize();

#endif

file1.cpp

#define MYLIBRARY_USE_IMPORT#include"header1.hpp"void* wrap_import_array(){
  import_array();
  return (void*) 1;
}

voidinitialize(){
  wrap_import_array();
}

file2.cpp

#include"header1.hpp"#include<iostream>intmain(){
  Py_Initialize();
  initialize();
  npy_intp dims[] = {5};
  std::cout << "creating descr" << std::endl;
  PyArray_Descr* dtype = PyArray_DescrFromType(NPY_FLOAT64);
  std::cout << "zeros" << std::endl;
  PyArray_Zeros(1, dims, dtype, 0);
  std::cout << "cleanup" << std::endl;
  return0;
}

I don't know what pitfalls there are with this hack or if there are any better alternatives, but this appears to at least compile and run without any segfaults.