如何从ctypes C函数获取打印输出到Jupyter / IPython笔记本?

介绍

假设我有这个C代码:

#include <stdio.h>

// Of course, these functions are simplified for the purposes of this question.
// The actual functions are more complex and may receive additional arguments.

void printout() {
    puts("Hello");
}
void printhere(FILE* f) {
    fputs("Hello\n", f);
}

我正在编译为共享对象(DLL):gcc -Wall -std = c99 -fPIC -shared example.c -o example.so

然后我导入它into Python 3.xJupyterIPython notebook内运行:

import ctypes
example = ctypes.cdll.LoadLibrary('./example.so')

printout = example.printout
printout.argtypes = ()
printout.restype = None

printhere = example.printhere
printhere.argtypes = (ctypes.c_void_p)  # Should have been FILE* instead
printhere.restype = None

如何执行printout()和printhere()C函数(通过ctypes)并在Jupyter / IPython笔记本中打印输出?

如果可能的话,我想避免编写更多的C代码.我更喜欢纯Python解决方案.

我也希望避免写入临时文件.但是,写入管道/插座可能是合理的.

预期的状态,当前的状态

如果我在一个Notebook单元格中键入以下代码:

print("Hi")           # Python-style print
printout()            # C-style print
printhere(something)  # C-style print
print("Bye")          # Python-style print

我想得到这个输出:

Hi
Hello
Hello
Bye

但是,相反,我只在笔记本中获得了Python风格的输出结果. C风格的输出被打印到启动笔记本进程的终端.

研究

据我所知,在Jupyter / IPython笔记本中,sys.stdout不是任何文件的包装器:

import sys

sys.stdout

# Output in command-line Python/IPython shell:
<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
# Output in IPython Notebook:
<IPython.kernel.zmq.iostream.OutStream at 0x7f39c6930438>
# Output in Jupyter:
<ipykernel.iostream.OutStream at 0x7f6dc8f2de80>

sys.stdout.fileno()

# Output in command-line Python/IPython shell:
1
# Output in command-line Jupyter and IPython notebook:
UnsupportedOperation: IOStream has no fileno.

相关问题和链接:

> Python ctypes: Python file object <-> C FILE *
> Python 3 replacement for PyFile_AsFile
> Using fopen, fwrite and fclose through ctypes
> Python ctypes DLL stdout
> Python: StringIO for Popen – StringIO中缺少fileno()的解决方法,但仅适用于subprocess.Popen.

以下两个链接使用涉及创建临时文件的类似解决方案.但是,在实现此类解决方案时必须小心,以确保以正确的顺序打印Python样式的输出和C样式的输出.

> How do I prevent a C shared library to print on stdout in python?
> Redirecting all kinds of stdout in Python

是否可以避免临时文件?

我尝试使用C open_memstream()找到解决方案并将返回的FILE *分配给stdout,但它不起作用because stdout cannot be assigned.

然后我尝试获取open_memstream()返回的流的fileno(),但我不能because it has no file descriptor.

然后我看了freopen(),但是它的API requires passing a filename.

然后我查看了Python的标准库,找到了tempfile.SpooledTemporaryFile(),它是内存中的临时文件类对象.但是,只要调用fileno(),就会将其写入磁盘.

到目前为止,我找不到任何仅限内存的解决方案.最有可能的是,无论如何我们都需要使用临时文件. (这不是什么大问题,但只是一些额外的开销和额外的清理,我宁愿避免.)

有可能使用os.pipe(),但这似乎很难没有分叉.

我终于开发出了一个解决方案.它需要将整个单元格包装在上下文管理器中(或仅包装C代码).它还使用临时文件,因为我没有使用它就找不到任何解决方案.

完整的笔记本电脑可作为GitHub Gist:https://gist.github.com/denilsonsa/9c8f5c44bf2038fd000f

第1部分:使用Python准备C库

import ctypes

# use_errno parameter is optional, because I'm not checking errno anyway.
libc = ctypes.CDLL(ctypes.util.find_library('c'), use_errno=True)

class FILE(ctypes.Structure):
    pass

FILE_p = ctypes.POINTER(FILE)

# Alternatively, we can just use:
# FILE_p = ctypes.c_void_p

# These variables, defined inside the C library, are readonly.
cstdin = FILE_p.in_dll(libc, 'stdin')
cstdout = FILE_p.in_dll(libc, 'stdout')
cstderr = FILE_p.in_dll(libc, 'stderr')

# C function to disable buffering.
csetbuf = libc.setbuf
csetbuf.argtypes = (FILE_p, ctypes.c_char_p)
csetbuf.restype = None

# C function to flush the C library buffer.
cfflush = libc.fflush
cfflush.argtypes = (FILE_p,)
cfflush.restype = ctypes.c_int

第2部分:构建我们自己的上下文管理器来捕获stdout

import io
import os
import sys
import tempfile
from contextlib import contextmanager

@contextmanager
def capture_c_stdout(encoding='utf8'):
    # Flushing, it's a good practice.
    sys.stdout.flush()
    cfflush(cstdout)

    # We need to use a actual file because we need the file descriptor number.
    with tempfile.TemporaryFile(buffering=0) as temp:
        # Saving a copy of the original stdout.
        prev_sys_stdout = sys.stdout
        prev_stdout_fd = os.dup(1)
        os.close(1)

        # Duplicating the temporary file fd into the stdout fd.
        # In other words, replacing the stdout.
        os.dup2(temp.fileno(), 1)

        # Replacing sys.stdout for Python code.
        #
        # IPython Notebook version of sys.stdout is actually an
        # in-memory OutStream, so it does not have a file descriptor.
        # We need to replace sys.stdout so that interleaved Python
        # and C output gets captured in the correct order.
        #
        # We enable line_buffering to force a flush after each line.
        # And write_through to force all data to be passed through the
        # wrapper directly into the binary temporary file.
        temp_wrapper = io.TextIOWrapper(
            temp, encoding=encoding, line_buffering=True, write_through=True)
        sys.stdout = temp_wrapper

        # Disabling buffering of C stdout.
        csetbuf(cstdout, None)

        yield

        # Must flush to clear the C library buffer.
        cfflush(cstdout)

        # Restoring stdout.
        os.dup2(prev_stdout_fd, 1)
        os.close(prev_stdout_fd)
        sys.stdout = prev_sys_stdout

        # Printing the captured output.
        temp_wrapper.seek(0)
        print(temp_wrapper.read(), end='')

乐趣:使用它!

libfoo = ctypes.CDLL('./foo.so')

printout = libfoo.printout
printout.argtypes = ()
printout.restype = None

printhere = libfoo.printhere
printhere.argtypes = (FILE_p,)
printhere.restype = None


print('Python Before capturing')
printout()  # Not captured, goes to the terminal

with capture_c_stdout():
    print('Python First')
    printout()
    print('Python Second')
    printhere(cstdout)
    print('Python Third')

print('Python After capturing')
printout()  # Not captured, goes to the terminal

输出:

Python Before capturing
Python First
C printout puts
Python Second
C printhere fputs
Python Third
Python After capturing

积分和进一步的工作

这个解决方案是阅读我在问题上链接的所有链接的结果,加上大量的反复试验.

此解决方案仅重定向stdout,重定向stdout和stderr可能会很有趣.现在,我将此作为练习留给读者. 😉

此外,此解决方案中没有异常处理(至少尚未处理).

相关文章
相关标签/搜索