60 Commits

Author SHA1 Message Date
ecd94057b7 bump version
Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>
2025-11-30 05:46:16 +05:30
68f2bc4b9b Merge pull request #8 from pythonbpf/test-workflow
Add support for using struct as value_type in hashmap
2025-11-28 15:52:48 +05:30
ccce772b51 Merge pull request #7 from pythonbpf/dependabot/github_actions/actions-c546836014
Bump the actions group across 1 directory with 5 updates
2025-11-27 14:06:17 +05:30
8ac2d67a76 Make bytes_to_python const non-static 2025-11-27 03:55:58 +05:30
d4202cdc88 Add basic userspace support for struct value type for maps 2025-11-27 03:43:24 +05:30
3335e0471b Return None instead of KeyError for failure of BpfMap::lookup for missing key 2025-11-24 23:51:08 +05:30
13554e9da9 Bump the actions group across 1 directory with 5 updates
Bumps the actions group with 5 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [actions/checkout](https://github.com/actions/checkout) | `5` | `6` |
| [actions/setup-python](https://github.com/actions/setup-python) | `5` | `6` |
| [actions/upload-artifact](https://github.com/actions/upload-artifact) | `4` | `5` |
| [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) | `3.2` | `3.3` |
| [actions/download-artifact](https://github.com/actions/download-artifact) | `5` | `6` |



Updates `actions/checkout` from 5 to 6
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v6)

Updates `actions/setup-python` from 5 to 6
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v5...v6)

Updates `actions/upload-artifact` from 4 to 5
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4...v5)

Updates `pypa/cibuildwheel` from 3.2 to 3.3
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v3.2...v3.3)

Updates `actions/download-artifact` from 5 to 6
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
- dependency-name: actions/setup-python
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
- dependency-name: actions/upload-artifact
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
- dependency-name: pypa/cibuildwheel
  dependency-version: '3.3'
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
- dependency-name: actions/download-artifact
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-11-24 01:39:00 +00:00
9a54aedb37 Update Python version link in README 2025-11-11 17:35:48 +05:30
270f4337d3 Merge pull request #6 from pythonbpf/test-workflow
Fix GH Actions, make Python3.12 the oldest supported version
2025-11-11 17:34:13 +05:30
ffc5a9f15f Add .so and .pyd wildcards to setuptools package-data in pyproject.toml 2025-11-11 17:31:09 +05:30
503e6da987 Janitorial formatting 2025-11-11 17:28:30 +05:30
8585f170cb Remove debug prints from test, fix name of Pip workflow 2025-11-11 17:21:30 +05:30
0780615fc5 Config setuptools to get library location 2025-11-11 17:12:31 +05:30
c26617d64f Overhaul pip.yml 2025-11-11 16:57:30 +05:30
303fe4e6c8 Make wheels workflow test oneline 2025-11-11 16:35:26 +05:30
9c18f5967d Fix more 2025-11-11 16:28:15 +05:30
434f145f9b Add better test to wheels workflow 2025-11-11 16:20:51 +05:30
b3e9410e0d Fix wheels workflow 2025-11-11 10:51:13 +05:30
fd739e87c1 Don't run pytest in isolated mode in pip workflow 2025-11-11 10:37:28 +05:30
87c698e940 Print pylibbpf dir in test 2025-11-11 10:29:48 +05:30
298df7ede6 Update workflows to treat Python 3.12 as minimum version 2025-11-11 10:23:58 +05:30
b48f6a8a97 Bump minimum python version to 3.12 2025-11-11 10:22:06 +05:30
6e4dc0f5aa Bump setuptools version in setup.py to use new licensing format 2025-11-11 10:13:41 +05:30
a51bed14ca Fix LICENSE and metadata in pyproject.toml 2025-11-11 10:04:13 +05:30
3a85a6446e Add Python discovery to CMakeLists.txt 2025-11-11 10:03:48 +05:30
6bc378defd Remove unnecessary metadata, fix CMake flag in setup.py 2025-11-11 10:02:13 +05:30
5c1071fac0 Merge pull request #4 from pythonbpf/perfbuf
Add userspace utils for Structs and maps, starting with PERF_EVENT_ARRAY
2025-10-20 06:14:45 +05:30
f99de9981c Fix includes for PerfEventArray 2025-10-20 06:09:07 +05:30
fa5d181e1a Fix includes for BpfProgram 2025-10-20 06:08:50 +05:30
867f142a7f Fix includes for BpfObject 2025-10-20 06:08:36 +05:30
8cc8f4267a Fix includes for BpfMap 2025-10-20 06:08:16 +05:30
ff427c2e61 Fix includes for bindings 2025-10-20 06:07:57 +05:30
fb82b609f9 Fix BpfMap includes 2025-10-20 05:58:55 +05:30
88716ce19a Fill missing fields in BpfObject's move constructor 2025-10-20 05:47:40 +05:30
003495e833 Make lost_callback type asfe in PerfEventArray 2025-10-20 05:22:31 +05:30
eebfe61ccc fix lost_callback_wrapper, clang-format 2025-10-20 05:17:40 +05:30
ec5377ba14 Remove unnecessary GIL acquisition in PerfEventArray 2025-10-20 05:14:44 +05:30
a3c3dbe141 Fix BpfMap header guard 2025-10-20 05:09:34 +05:30
3085e8155d Add __delitem__ for BpfMap in bindings 2025-10-20 05:08:15 +05:30
dd552de02c Add memory header to maps/perf_event_array.h 2025-10-20 05:04:19 +05:30
638533c19a Fix test for pip GH workflow 2025-10-20 04:48:46 +05:30
c9a152adc3 Add __version__ to __init__ 2025-10-20 04:22:14 +05:30
92e92f134a Add _make_repr to ir_to_ctypes 2025-10-20 04:18:38 +05:30
b7aa0807c5 Fix pre-commit conditions 2025-10-20 04:05:14 +05:30
ddbbce400e Use c_char type for Int8 arrays in ir_to_ctypes 2025-10-20 03:48:02 +05:30
c580aab1c4 Move Python files to pylibbpf/ 2025-10-20 03:46:30 +05:30
470afc5174 Import find_packages in setup.py 2025-10-20 03:45:59 +05:30
495318f622 Update pyproject.toml 2025-10-20 03:09:51 +05:30
bbb39898ab Fix setup.py 2025-10-20 03:06:51 +05:30
23cafa4d7b Expose classes and perform struct conversion in __init__ 2025-10-20 02:58:22 +05:30
30021e8520 Add PerfEventArray and BpfObject wrappers 2025-10-20 02:57:57 +05:30
0e454bd7d7 Add IR Types to CTypes struct convertor 2025-10-20 02:57:25 +05:30
eda08b05d7 lint fix to CMakeLists 2025-10-20 02:04:38 +05:30
1eb7ed460e Fix Bindings and PerfEventArray 2025-10-20 00:28:27 +05:30
8babf3087b Add parser and parent shared_ptr to PerfEventArray 2025-10-20 00:10:54 +05:30
cbfe6ae95e Rename BpfPerfBuffer to PerfEventArray, add struct_parser to BpfObject as a shared_ptr 2025-10-19 22:34:10 +05:30
05d5bba4f7 Add StructParser utility 2025-10-19 22:02:19 +05:30
f7874137ad Enable BpfMap to be shared, add get_parent 2025-10-19 21:33:24 +05:30
b4d0a49883 Add struct_defs_ to BpfObject 2025-10-19 21:26:20 +05:30
874d567825 Move BpfPerfBuffer under src/maps 2025-10-19 18:47:35 +05:30
26 changed files with 736 additions and 280 deletions

View File

@ -15,7 +15,7 @@ jobs:
name: Format name: Format
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v5 - uses: actions/checkout@v6
- uses: actions/setup-python@v6 - uses: actions/setup-python@v6
with: with:
python-version: "3.x" python-version: "3.x"

View File

@ -1,29 +1,38 @@
name: Pip name: Pip
on: on:
workflow_dispatch:
pull_request:
push: push:
branches: branches: [main, master]
- master pull_request:
- main branches: [main, master]
workflow_dispatch:
jobs: jobs:
build: test:
name: Test on Python ${{ matrix.python-version }}
runs-on: ubuntu-latest
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
platform: [ubuntu-latest] python-version: ['3.12', '3.13']
python-version: ["3.8", "3.12"]
runs-on: ${{ matrix.platform }}
steps: steps:
- uses: actions/checkout@v5 - name: Checkout code
uses: actions/checkout@v6
with: with:
submodules: true submodules: recursive
- uses: actions/setup-python@v6 - name: Verify submodules
run: |
echo "Checking submodule status:"
git submodule status
echo "Checking libbpf directory:"
ls -la libbpf/
echo "Checking libbpf/src:"
ls -la libbpf/src/ || echo "libbpf/src not found!"
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
@ -33,19 +42,72 @@ jobs:
sudo apt-get install -y \ sudo apt-get install -y \
libbpf-dev \ libbpf-dev \
libelf-dev \ libelf-dev \
linux-headers-generic \ zlib1g-dev \
build-essential \ build-essential \
clang \
cmake \ cmake \
ninja-build ninja-build \
pkg-config \
git \
make
- name: Add requirements - name: Install Python build dependencies
run: python -m pip install --upgrade pip wheel setuptools run: |
python -m pip install --upgrade pip
pip install --upgrade "setuptools>=77.0.0" wheel
pip install cmake ninja pybind11
- name: Build and install - name: Check build requirements
run: pip install --verbose .[test] run: |
echo "Python version:"
python --version
echo "CMake version:"
cmake --version
echo "Ninja version:"
ninja --version
echo "Setuptools version:"
python -c "import setuptools; print(setuptools.__version__)"
- name: Test import - name: Build and install pylibbpf (verbose)
run: python -c "import pylibbpf; print('Import successful')" run: |
pip install -v -e . 2>&1 | tee build.log
continue-on-error: false
- name: Test - name: Check build output
run: python -m pytest -v run: |
echo "Build directory contents:"
find build -type f -name "*.so" 2>/dev/null || echo "No .so files found in build/"
echo ""
echo "Looking for pylibbpf extension:"
find . -name "pylibbpf*.so" -o -name "pylibbpf*.pyd"
echo ""
echo "Site-packages contents:"
python -c "import site; print(site.getsitepackages())"
ls -la $(python -c "import site; print(site.getsitepackages()[0])")/pylibbpf/ || echo "pylibbpf not in site-packages"
- name: Try importing extension directly
run: |
python -c "
import sys
print('Python path:', sys.path)
try:
from pylibbpf import pylibbpf
print('Successfully imported pylibbpf.pylibbpf')
print('pylibbpf.pylibbpf members:', dir(pylibbpf))
except ImportError as e:
print(f'Failed to import pylibbpf.pylibbpf: {e}')
"
- name: Verify extension loaded
run: |
python -c "import pylibbpf; print('Members:', dir(pylibbpf)); assert hasattr(pylibbpf, 'BpfObject'), 'BpfObject not found!'; print('✓ OK')"
- name: Install test dependencies
if: success()
run: |
pip install pytest pytest-cov
- name: Run tests
if: success()
run: |
python -I -m pytest -v -s --cov=pylibbpf --cov-report=term-missing

View File

@ -15,9 +15,9 @@ jobs:
name: Build SDist name: Build SDist
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v5 - uses: actions/checkout@v6
with: with:
submodules: true submodules: recursive
- name: Build SDist - name: Build SDist
run: pipx run build --sdist run: pipx run build --sdist
@ -25,7 +25,7 @@ jobs:
- name: Check metadata - name: Check metadata
run: pipx run twine check dist/* run: pipx run twine check dist/*
- uses: actions/upload-artifact@v4 - uses: actions/upload-artifact@v5
with: with:
name: cibw-sdist name: cibw-sdist
path: dist/*.tar.gz path: dist/*.tar.gz
@ -39,31 +39,33 @@ jobs:
arch: [x86_64] arch: [x86_64]
steps: steps:
- uses: actions/checkout@v5 - uses: actions/checkout@v6
with: with:
submodules: true submodules: recursive
- name: Build wheels - name: Build wheels
uses: pypa/cibuildwheel@v3.2 uses: pypa/cibuildwheel@v3.3
env: env:
CIBW_PLATFORM: linux CIBW_PLATFORM: linux
CIBW_ARCHS_LINUX: ${{ matrix.arch }} CIBW_ARCHS_LINUX: ${{ matrix.arch }}
CIBW_BUILD: "cp38-* cp311-*" CIBW_BUILD: "cp312-* cp313-*"
CIBW_SKIP: "*-musllinux*" CIBW_SKIP: "*-musllinux*"
CIBW_TEST_COMMAND: "python -c 'import pylibbpf; print(f\"pylibbpf {pylibbpf.__version__} imported successfully\")'" CIBW_TEST_COMMAND: "python -c 'import pylibbpf; assert hasattr(pylibbpf, \"BpfObject\"), \"BpfObject not found\"; print(f\"pylibbpf {pylibbpf.__version__} OK\")'"
CIBW_TEST_SKIP: "*-linux_aarch64" CIBW_TEST_SKIP: "*-linux_aarch64"
CIBW_MANYLINUX_X86_64_IMAGE: manylinux_2_28 CIBW_MANYLINUX_X86_64_IMAGE: manylinux_2_28
CIBW_BEFORE_ALL_LINUX: | CIBW_BEFORE_ALL_LINUX: |
dnf install -y elfutils-libelf-devel zlib-devel dnf install -y elfutils-libelf-devel zlib-devel make gcc gcc-c++ git
CIBW_BEFORE_BUILD: >
pip install --upgrade "setuptools>=77.0.0" wheel cmake ninja pybind11
- name: Verify clean directory - name: Verify clean directory
run: git diff --exit-code run: git diff --exit-code
shell: bash shell: bash
- name: Upload wheels - name: Upload wheels
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v5
with: with:
name: cibw-wheels-linux-${{ matrix.arch }} name: cibw-wheels-linux-${{ matrix.arch }}
path: wheelhouse/*.whl path: wheelhouse/*.whl
@ -80,7 +82,7 @@ jobs:
steps: steps:
- name: Download all artifacts - name: Download all artifacts
uses: actions/download-artifact@v5 uses: actions/download-artifact@v6
with: with:
pattern: cibw-* pattern: cibw-*
path: dist path: dist

View File

@ -5,21 +5,32 @@ set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON) set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF) set(CMAKE_CXX_EXTENSIONS OFF)
find_package(
Python
COMPONENTS Interpreter Development.Module
REQUIRED)
# pybind11 # pybind11
include_directories(${CMAKE_SOURCE_DIR}/src) include_directories(${CMAKE_SOURCE_DIR}/src)
add_subdirectory(pybind11) add_subdirectory(pybind11)
pybind11_add_module( pybind11_add_module(
pylibbpf pylibbpf
# Core
src/core/bpf_program.h src/core/bpf_program.h
src/core/bpf_exception.h src/core/bpf_exception.h
src/core/bpf_map.h src/core/bpf_map.h
src/core/bpf_object.h src/core/bpf_object.h
src/core/bpf_perf_buffer.h
src/bindings/main.cpp
src/core/bpf_program.cpp src/core/bpf_program.cpp
src/core/bpf_map.cpp src/core/bpf_map.cpp
src/core/bpf_object.cpp src/core/bpf_object.cpp
src/core/bpf_perf_buffer.cpp) # Maps
src/maps/perf_event_array.h
src/maps/perf_event_array.cpp
# Utils
src/utils/struct_parser.h
src/utils/struct_parser.cpp
# Bindings
src/bindings/main.cpp)
# --- libbpf build rules --- # --- libbpf build rules ---
set(LIBBPF_SRC_DIR ${CMAKE_CURRENT_SOURCE_DIR}/libbpf/src) set(LIBBPF_SRC_DIR ${CMAKE_CURRENT_SOURCE_DIR}/libbpf/src)

View File

@ -11,7 +11,7 @@
</picture> </picture>
<p align="center"> <p align="center">
<!-- PyPI --> <!-- PyPI -->
<a href="https://www.python.org/downloads/release/python-3080/"><img src="https://img.shields.io/badge/python-3.8-blue.svg"></a> <a href="https://www.python.org/downloads/release/python-3120/"><img src="https://img.shields.io/badge/python-3.12-blue.svg"></a>
<a href="https://pypi.org/project/pylibbpf"><img src="https://badge.fury.io/py/pylibbpf.svg"></a> <a href="https://pypi.org/project/pylibbpf"><img src="https://badge.fury.io/py/pylibbpf.svg"></a>
<!-- <a href="https://pypi.org/project/pythonbpf/"><img src="https://img.shields.io/pypi/status/pythonbpf" alt="PyPI Status"></a> --> <!-- <a href="https://pypi.org/project/pythonbpf/"><img src="https://img.shields.io/pypi/status/pythonbpf" alt="PyPI Status"></a> -->
<a href="https://pepy.tech/project/pylibbpf"><img src="https://pepy.tech/badge/pylibbpf" alt="Downloads"></a> <a href="https://pepy.tech/project/pylibbpf"><img src="https://pepy.tech/badge/pylibbpf" alt="Downloads"></a>

View File

@ -1,10 +1,11 @@
import time import time
from ctypes import c_int32, c_int64, c_uint64, c_void_p from ctypes import c_int32, c_int64, c_uint64, c_void_p
from pylibbpf import BpfMap
from pythonbpf import BPF, bpf, bpfglobal, map, section from pythonbpf import BPF, bpf, bpfglobal, map, section
from pythonbpf.maps import HashMap from pythonbpf.maps import HashMap
from pylibbpf import BpfMap
@bpf @bpf
@map @map

46
pylibbpf/__init__.py Normal file
View File

@ -0,0 +1,46 @@
import logging
from .ir_to_ctypes import convert_structs_to_ctypes, is_pythonbpf_structs
from .pylibbpf import (
BpfException,
BpfMap,
BpfProgram,
PerfEventArray,
StructParser,
)
from .pylibbpf import (
BpfObject as _BpfObject, # C++ object (internal)
)
from .wrappers import BpfObjectWrapper
logger = logging.getLogger(__name__)
class BpfObject(BpfObjectWrapper):
"""BpfObject with automatic struct conversion"""
def __init__(self, object_path: str, structs=None):
"""Create a BPF object"""
if structs is None:
structs = {}
elif is_pythonbpf_structs(structs):
logger.info(f"Auto-converting {len(structs)} PythonBPF structs to ctypes")
structs = convert_structs_to_ctypes(structs)
# Create C++ BpfObject with converted structs
cpp_obj = _BpfObject(object_path, structs)
# Initialize wrapper
super().__init__(cpp_obj)
__all__ = [
"BpfObject",
"BpfProgram",
"BpfMap",
"PerfEventArray",
"StructParser",
"BpfException",
]
__version__ = "0.0.7"

104
pylibbpf/ir_to_ctypes.py Normal file
View File

@ -0,0 +1,104 @@
import ctypes
import logging
from llvmlite import ir
logger = logging.getLogger(__name__)
def ir_type_to_ctypes(ir_type):
"""Convert LLVM IR type to ctypes type."""
if isinstance(ir_type, ir.IntType):
width = ir_type.width
type_map = {
8: ctypes.c_uint8,
16: ctypes.c_uint16,
32: ctypes.c_uint32,
64: ctypes.c_uint64,
}
if width not in type_map:
raise ValueError(f"Unsupported integer width: {width}")
return type_map[width]
elif isinstance(ir_type, ir.ArrayType):
count = ir_type.count
element_type_ir = ir_type.element
if isinstance(element_type_ir, ir.IntType) and element_type_ir.width == 8:
# Use c_char for string fields (will have .decode())
return ctypes.c_char * count
else:
element_type = ir_type_to_ctypes(element_type_ir)
return element_type * count
elif isinstance(ir_type, ir.PointerType):
return ctypes.c_void_p
else:
raise TypeError(f"Unsupported IR type: {ir_type}")
def _make_repr(struct_name: str, fields: list):
"""Create a __repr__ function for a struct"""
def __repr__(self):
field_strs = []
for field_name, _ in fields:
value = getattr(self, field_name)
field_strs.append(f"{field_name}={value}")
return f"<{struct_name} {' '.join(field_strs)}>"
return __repr__
def convert_structs_to_ctypes(structs_sym_tab) -> dict[str, type[ctypes.Structure]]:
"""Convert PythonBPF's structs_sym_tab to ctypes.Structure classes."""
if not structs_sym_tab:
return {}
ctypes_structs = {}
for struct_name, struct_type_obj in structs_sym_tab.items():
try:
fields = []
for field_name, field_ir_type in struct_type_obj.fields.items():
field_ctypes = ir_type_to_ctypes(field_ir_type)
fields.append((field_name, field_ctypes))
repr_func = _make_repr(struct_name, fields)
struct_class = type(
struct_name,
(ctypes.Structure,),
{
"_fields_": fields,
"__module__": "pylibbpf.ir_to_ctypes",
"__doc__": f"Auto-generated ctypes structure for {struct_name}",
"__repr__": repr_func,
},
)
ctypes_structs[struct_name] = struct_class
# Pretty print field info
field_info = ", ".join(f"{name}: {typ.__name__}" for name, typ in fields)
logger.debug(f" {struct_name}({field_info})")
except Exception as e:
logger.error(f"Failed to convert struct '{struct_name}': {e}")
raise
logger.info(f"Converted struct '{struct_name}' to ctypes")
return ctypes_structs
def is_pythonbpf_structs(structs) -> bool:
"""Check if structs dict is from PythonBPF."""
if not isinstance(structs, dict) or not structs:
return False
first_value = next(iter(structs.values()))
return (
hasattr(first_value, "ir_type")
and hasattr(first_value, "fields")
and hasattr(first_value, "size")
)
__all__ = ["convert_structs_to_ctypes", "is_pythonbpf_structs"]

0
pylibbpf/py.typed Normal file
View File

77
pylibbpf/wrappers.py Normal file
View File

@ -0,0 +1,77 @@
from collections.abc import Callable
class PerfEventArrayHelper:
"""Fluent wrapper for PERF_EVENT_ARRAY maps."""
def __init__(self, bpf_map):
self._map = bpf_map
self._perf_buffer = None
def open_perf_buffer(
self,
callback: Callable,
struct_name: str = "",
page_cnt: int = 8,
lost_callback: Callable | None = None,
):
"""Open perf buffer with auto-deserialization."""
from .pylibbpf import PerfEventArray
if struct_name:
self._perf_buffer = PerfEventArray(
self._map,
page_cnt,
callback,
struct_name,
lost_callback or (lambda cpu, cnt: None),
)
else:
self._perf_buffer = PerfEventArray(
self._map, page_cnt, callback, lost_callback or (lambda cpu, cnt: None)
)
return self
def poll(self, timeout_ms: int = -1) -> int:
if not self._perf_buffer:
raise RuntimeError("Call open_perf_buffer() first")
return self._perf_buffer.poll(timeout_ms)
def consume(self) -> int:
if not self._perf_buffer:
raise RuntimeError("Call open_perf_buffer() first")
return self._perf_buffer.consume()
def __getattr__(self, name):
return getattr(self._map, name)
class BpfObjectWrapper:
"""Smart wrapper that returns map-specific helpers."""
BPF_MAP_TYPE_PERF_EVENT_ARRAY = 4
BPF_MAP_TYPE_RINGBUF = 27
def __init__(self, bpf_object):
self._obj = bpf_object
self._map_helpers = {}
def __getitem__(self, name: str):
"""Return appropriate helper based on map type."""
if name in self._map_helpers:
return self._map_helpers[name]
map_obj = self._obj[name]
map_type = map_obj.get_type()
if map_type == self.BPF_MAP_TYPE_PERF_EVENT_ARRAY:
helper = PerfEventArrayHelper(map_obj)
else:
helper = map_obj
self._map_helpers[name] = helper
return helper
def __getattr__(self, name):
return getattr(self._obj, name)

View File

@ -1,23 +1,24 @@
[build-system] [build-system]
requires = [ requires = [
"setuptools>=42", "setuptools>=77",
"wheel", "wheel",
"ninja", "ninja",
"cmake>=4.0", "cmake>=4.0",
"pybind11>=2.10",
] ]
build-backend = "setuptools.build_meta" build-backend = "setuptools.build_meta"
[project] [project]
name = "pylibbpf" name = "pylibbpf"
version = "0.0.5" version = "0.0.7"
description = "Python Bindings for Libbpf" description = "Python Bindings for Libbpf"
authors = [ authors = [
{ name = "r41k0u", email = "pragyanshchaturvedi18@gmail.com" }, { name = "r41k0u", email = "pragyanshchaturvedi18@gmail.com" },
{ name = "varun-r-mallya", email = "varunrmallya@gmail.com" } { name = "varun-r-mallya", email = "varunrmallya@gmail.com" }
] ]
readme = "README.md" readme = "README.md"
license = { file = "LICENSE" } license = "Apache-2.0"
requires-python = ">=3.8" requires-python = ">=3.12"
classifiers = [ classifiers = [
"Development Status :: 3 - Alpha", "Development Status :: 3 - Alpha",
"Intended Audience :: Developers", "Intended Audience :: Developers",
@ -28,22 +29,26 @@ classifiers = [
"Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: C++", "Programming Language :: C++",
"Topic :: Software Development :: Libraries :: Python Modules", "Topic :: Software Development :: Libraries :: Python Modules",
"Topic :: System :: Operating System Kernels :: Linux", "Topic :: System :: Operating System Kernels :: Linux",
] ]
dependencies = [
"llvmlite>=0.40.0",
]
[project.optional-dependencies] [project.optional-dependencies]
test = ["pytest>=6.0"] test = ["pytest>=6.0"]
[project.urls] [project.urls]
Homepage = "https://github.com/varun-r-mallya/pylibbpf" Homepage = "https://github.com/pythonbpf/pylibbpf"
Repository = "https://github.com/varun-r-mallya/pylibbpf" Repository = "https://github.com/pythonbpf/pylibbpf"
Issues = "https://github.com/varun-r-mallya/pylibbpf/issues" Issues = "https://github.com/pythonbpf/pylibbpf/issues"
[tool.mypy] [tool.mypy]
files = "setup.py" files = "setup.py"
python_version = "3.8" python_version = "3.12"
strict = true strict = true
show_error_codes = true show_error_codes = true
enable_error_code = ["ignore-without-code", "redundant-expr", "truthy-bool"] enable_error_code = ["ignore-without-code", "redundant-expr", "truthy-bool"]
@ -64,7 +69,7 @@ filterwarnings = [
testpaths = ["tests"] testpaths = ["tests"]
[tool.ruff] [tool.ruff]
target-version = "py38" target-version = "py312"
line-length = 88 line-length = 88
[tool.ruff.lint] [tool.ruff.lint]
@ -75,3 +80,7 @@ extend-select = [
"RUF", # Ruff-specific "RUF", # Ruff-specific
"UP", # pyupgrade "UP", # pyupgrade
] ]
[tool.setuptools]
packages = ["pylibbpf"]
package-data = {"pylibbpf" = ["*.py", "*.so", "*.pyd", "py.typed"]}

View File

@ -6,14 +6,6 @@ from pathlib import Path
from setuptools import Extension, setup from setuptools import Extension, setup
from setuptools.command.build_ext import build_ext from setuptools.command.build_ext import build_ext
# Convert distutils Windows platform specifiers to CMake -A arguments
PLAT_TO_CMAKE = {
"win32": "Win32",
"win-amd64": "x64",
"win-arm32": "ARM",
"win-arm64": "ARM64",
}
# A CMakeExtension needs a sourcedir instead of a file list. # A CMakeExtension needs a sourcedir instead of a file list.
# The name must be the _single_ output extension from the CMake build. # The name must be the _single_ output extension from the CMake build.
@ -45,8 +37,8 @@ class CMakeBuild(build_ext):
# from Python. # from Python.
cmake_args = [ cmake_args = [
f"-DCMAKE_LIBRARY_OUTPUT_DIRECTORY={extdir}{os.sep}", f"-DCMAKE_LIBRARY_OUTPUT_DIRECTORY={extdir}{os.sep}",
f"-DPYTHON_EXECUTABLE={sys.executable}",
f"-DCMAKE_BUILD_TYPE={cfg}", # not used on MSVC, but no harm f"-DCMAKE_BUILD_TYPE={cfg}", # not used on MSVC, but no harm
f"-DPython_EXECUTABLE={sys.executable}",
] ]
build_args = [] build_args = []
@ -122,31 +114,6 @@ if readme_path.exists():
long_description = readme_path.read_text(encoding="utf-8") long_description = readme_path.read_text(encoding="utf-8")
setup( setup(
name="pylibbpf", ext_modules=[CMakeExtension("pylibbpf.pylibbpf")],
version="0.0.1",
author="varun-r-mallya, r41k0u",
author_email="varunrmallyagmail.com",
description="Python Bindings for Libbpf",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/varun-r-mallya/pylibbpf",
ext_modules=[CMakeExtension("pylibbpf")],
cmdclass={"build_ext": CMakeBuild}, cmdclass={"build_ext": CMakeBuild},
zip_safe=False,
classifiers=[
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"Operating System :: POSIX :: Linux",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: C++",
"Topic :: Software Development :: Libraries :: Python Modules",
"Topic :: System :: Operating System Kernels :: Linux",
],
extras_require={"test": ["pytest>=6.0"]},
python_requires=">=3.8",
) )

View File

@ -2,20 +2,17 @@
#define STRINGIFY(x) #x #define STRINGIFY(x) #x
#define MACRO_STRINGIFY(x) STRINGIFY(x) #define MACRO_STRINGIFY(x) STRINGIFY(x)
extern "C" {
#include <libbpf.h>
}
#include "core/bpf_object.h"
#include "core/bpf_program.h"
#include "core/bpf_exception.h" #include "core/bpf_exception.h"
#include "core/bpf_map.h" #include "core/bpf_map.h"
#include "core/bpf_perf_buffer.h" #include "core/bpf_object.h"
#include "core/bpf_program.h"
#include "maps/perf_event_array.h"
#include "utils/struct_parser.h"
namespace py = pybind11; namespace py = pybind11;
PYBIND11_MODULE(pylibbpf, m) { PYBIND11_MODULE(pylibbpf, m) {
m.doc() = R"pbdoc( m.doc() = R"pbdoc(
Pylibbpf - libbpf bindings for Python Pylibbpf - libbpf bindings for Python
----------------------- -----------------------
@ -29,56 +26,77 @@ PYBIND11_MODULE(pylibbpf, m) {
BpfException BpfException
)pbdoc"; )pbdoc";
// Register the custom exception // Register the custom exception
py::register_exception<BpfException>(m, "BpfException"); py::register_exception<BpfException>(m, "BpfException");
// BpfObject // BpfObject
py::class_<BpfObject, std::shared_ptr<BpfObject>>(m, "BpfObject") py::class_<BpfObject, std::shared_ptr<BpfObject>>(m, "BpfObject")
.def(py::init<std::string>(), py::arg("object_path")) .def(py::init<std::string, py::dict>(), py::arg("object_path"),
.def("load", &BpfObject::load) py::arg("structs") = py::dict())
.def("is_loaded", &BpfObject::is_loaded) .def("load", &BpfObject::load)
.def("get_program_names", &BpfObject::get_program_names) .def("is_loaded", &BpfObject::is_loaded)
.def("get_program", &BpfObject::get_program, py::arg("name")) .def("get_program_names", &BpfObject::get_program_names)
.def("attach_all", &BpfObject::attach_all) .def("get_program", &BpfObject::get_program, py::arg("name"))
.def("get_map_names", &BpfObject::get_map_names) .def("attach_all", &BpfObject::attach_all)
.def("get_map", &BpfObject::get_map, py::arg("name")); .def("get_map_names", &BpfObject::get_map_names)
.def("get_map", &BpfObject::get_map, py::arg("name"))
.def("get_struct_defs", &BpfObject::get_struct_defs)
.def("__getitem__", &BpfObject::get_map, py::arg("name"));
// BpfProgram // BpfProgram
py::class_<BpfProgram, std::shared_ptr<BpfProgram>>(m, "BpfProgram") py::class_<BpfProgram, std::shared_ptr<BpfProgram>>(m, "BpfProgram")
.def("attach", &BpfProgram::attach) .def("attach", &BpfProgram::attach)
.def("detach", &BpfProgram::detach) .def("detach", &BpfProgram::detach)
.def("is_attached", &BpfProgram::is_attached) .def("is_attached", &BpfProgram::is_attached)
.def("get_name", &BpfProgram::get_name); .def("get_name", &BpfProgram::get_name);
// BpfMap // BpfMap
py::class_<BpfMap, std::shared_ptr<BpfMap>>(m, "BpfMap") py::class_<BpfMap, std::shared_ptr<BpfMap>>(m, "BpfMap")
.def("lookup", &BpfMap::lookup, py::arg("key")) .def("lookup", &BpfMap::lookup, py::arg("key"))
.def("update", &BpfMap::update, py::arg("key"), py::arg("value")) .def("update", &BpfMap::update, py::arg("key"), py::arg("value"))
.def("delete_elem", &BpfMap::delete_elem, py::arg("key")) .def("delete_elem", &BpfMap::delete_elem, py::arg("key"))
.def("get_next_key", &BpfMap::get_next_key, py::arg("key") = py::none()) .def("get_next_key", &BpfMap::get_next_key, py::arg("key") = py::none())
.def("items", &BpfMap::items) .def("items", &BpfMap::items)
.def("keys", &BpfMap::keys) .def("keys", &BpfMap::keys)
.def("values", &BpfMap::values) .def("values", &BpfMap::values)
.def("get_name", &BpfMap::get_name) .def("get_name", &BpfMap::get_name)
.def("get_fd", &BpfMap::get_fd) .def("get_fd", &BpfMap::get_fd)
.def("get_type", &BpfMap::get_type) .def("get_type", &BpfMap::get_type)
.def("get_key_size", &BpfMap::get_key_size) .def("get_key_size", &BpfMap::get_key_size)
.def("get_value_size", &BpfMap::get_value_size) .def("get_value_size", &BpfMap::get_value_size)
.def("get_max_entries", &BpfMap::get_max_entries); .def("get_max_entries", &BpfMap::get_max_entries)
.def("set_value_struct", &BpfMap::set_value_struct,
py::arg("struct_name"))
.def("get_value_struct_name", &BpfMap::get_value_struct_name)
.def("has_struct_value", &BpfMap::has_struct_value)
.def("__getitem__", &BpfMap::lookup, py::arg("key"))
.def("__setitem__", &BpfMap::update, py::arg("key"), py::arg("value"))
.def("__delitem__", &BpfMap::delete_elem, py::arg("key"));
py::class_<BpfPerfBuffer>(m, "BpfPerfBuffer") // StructParser
.def(py::init<int, int, py::function, py::object>(), py::class_<StructParser>(m, "StructParser")
py::arg("map_fd"), .def(py::init<py::dict>(), py::arg("structs"))
py::arg("page_cnt") = 8, .def("parse", &StructParser::parse, py::arg("struct_name"),
py::arg("callback"), py::arg("data"))
py::arg("lost_callback") = py::none()) .def("has_struct", &StructParser::has_struct, py::arg("struct_name"));
.def("poll", &BpfPerfBuffer::poll, py::arg("timeout_ms") = -1)
.def("consume", &BpfPerfBuffer::consume);
// PerfEventArray
py::class_<PerfEventArray, std::shared_ptr<PerfEventArray>>(m,
"PerfEventArray")
.def(py::init<std::shared_ptr<BpfMap>, int, py::function, py::object>(),
py::arg("map"), py::arg("page_cnt"), py::arg("callback"),
py::arg("lost_callback") = py::none())
.def(py::init<std::shared_ptr<BpfMap>, int, py::function, std::string,
py::object>(),
py::arg("map"), py::arg("page_cnt"), py::arg("callback"),
py::arg("struct_name"), py::arg("lost_callback") = py::none())
.def("poll", &PerfEventArray::poll, py::arg("timeout_ms"))
.def("consume", &PerfEventArray::consume)
.def("get_map", &PerfEventArray::get_map);
#ifdef VERSION_INFO #ifdef VERSION_INFO
m.attr("__version__") = MACRO_STRINGIFY(VERSION_INFO); m.attr("__version__") = MACRO_STRINGIFY(VERSION_INFO);
#else #else
m.attr("__version__") = "dev"; m.attr("__version__") = "dev";
#endif #endif
} }

View File

@ -1,6 +1,10 @@
#include "bpf_map.h" #include "core/bpf_map.h"
#include "bpf_exception.h" #include "core/bpf_exception.h"
#include "bpf_object.h" #include "core/bpf_object.h"
#include "utils/struct_parser.h"
#include <algorithm>
#include <cerrno>
#include <cstring>
BpfMap::BpfMap(std::shared_ptr<BpfObject> parent, struct bpf_map *raw_map, BpfMap::BpfMap(std::shared_ptr<BpfObject> parent, struct bpf_map *raw_map,
const std::string &map_name) const std::string &map_name)
@ -22,6 +26,24 @@ BpfMap::BpfMap(std::shared_ptr<BpfObject> parent, struct bpf_map *raw_map,
value_size_ = bpf_map__value_size(map_); value_size_ = bpf_map__value_size(map_);
} }
void BpfMap::set_value_struct(const std::string &struct_name) {
auto parent = parent_obj_.lock();
if (!parent) {
throw BpfException("Parent BpfObject no longer exists");
}
struct_parser_ = parent->get_struct_parser();
if (!struct_parser_) {
throw BpfException("No struct definitions available in BpfObject");
}
if (!struct_parser_->has_struct(struct_name)) {
throw BpfException("Unknown struct: " + struct_name);
}
value_struct_name_ = struct_name;
}
py::object BpfMap::lookup(const py::object &key) const { py::object BpfMap::lookup(const py::object &key) const {
if (map_fd_ < 0) if (map_fd_ < 0)
throw BpfException("Map '" + map_name_ + "' is not initialized properly"); throw BpfException("Map '" + map_name_ + "' is not initialized properly");
@ -39,7 +61,7 @@ py::object BpfMap::lookup(const py::object &key) const {
value_span.data(), value_size_, BPF_ANY); value_span.data(), value_size_, BPF_ANY);
if (ret < 0) { if (ret < 0) {
if (ret == -ENOENT) if (ret == -ENOENT)
throw py::key_error("Key not found in map '" + map_name_ + "'"); return py::none();
throw BpfException("Failed to lookup key in map '" + map_name_ + throw BpfException("Failed to lookup key in map '" + map_name_ +
"': " + std::strerror(-ret)); "': " + std::strerror(-ret));
} }
@ -212,7 +234,13 @@ void BpfMap::python_to_bytes_inplace(const py::object &obj,
} }
} }
py::object BpfMap::bytes_to_python(std::span<const uint8_t> data) { py::object BpfMap::bytes_to_python(std::span<const uint8_t> data) const {
// NOTE: Struct parsing for value type
if (struct_parser_ && !value_struct_name_.empty()) {
py::bytes py_data(reinterpret_cast<const char *>(data.data()), data.size());
return struct_parser_->parse(value_struct_name_, py_data);
}
if (data.size() == 4) { if (data.size() == 4) {
uint32_t value; uint32_t value;
std::memcpy(&value, data.data(), 4); std::memcpy(&value, data.data(), 4);

View File

@ -1,10 +1,7 @@
#ifndef PYLIBBPF_BPF_MAP_H #ifndef PYLIBBPF_BPF_MAP_H
#define PYLIBBPF_BPF_MAP_H #define PYLIBBPF_BPF_MAP_H
#include <algorithm>
#include <array> #include <array>
#include <cerrno>
#include <cstring>
#include <libbpf.h> #include <libbpf.h>
#include <pybind11/pybind11.h> #include <pybind11/pybind11.h>
#include <span> #include <span>
@ -12,10 +9,11 @@
#include <vector> #include <vector>
class BpfObject; class BpfObject;
class StructParser;
namespace py = pybind11; namespace py = pybind11;
class BpfMap { class BpfMap : public std::enable_shared_from_this<BpfMap> {
private: private:
std::weak_ptr<BpfObject> parent_obj_; std::weak_ptr<BpfObject> parent_obj_;
struct bpf_map *map_; struct bpf_map *map_;
@ -23,6 +21,11 @@ private:
std::string map_name_; std::string map_name_;
__u32 key_size_, value_size_; __u32 key_size_, value_size_;
// TODO: For now, we'll only support struct parsing for value types
// later we can extend this to keys
std::shared_ptr<StructParser> struct_parser_;
std::string value_struct_name_;
template <size_t StackSize = 64> struct BufferManager { template <size_t StackSize = 64> struct BufferManager {
std::array<uint8_t, StackSize> stack_buf; std::array<uint8_t, StackSize> stack_buf;
std::vector<uint8_t> heap_buf; std::vector<uint8_t> heap_buf;
@ -55,6 +58,7 @@ public:
py::dict items() const; py::dict items() const;
py::list keys() const; py::list keys() const;
py::list values() const; py::list values() const;
void set_value_struct(const std::string &struct_name);
[[nodiscard]] std::string get_name() const { return map_name_; } [[nodiscard]] std::string get_name() const { return map_name_; }
[[nodiscard]] int get_fd() const { return map_fd_; } [[nodiscard]] int get_fd() const { return map_fd_; }
@ -62,11 +66,20 @@ public:
[[nodiscard]] int get_key_size() const { return key_size_; }; [[nodiscard]] int get_key_size() const { return key_size_; };
[[nodiscard]] int get_value_size() const { return value_size_; }; [[nodiscard]] int get_value_size() const { return value_size_; };
[[nodiscard]] int get_max_entries() const; [[nodiscard]] int get_max_entries() const;
[[nodiscard]] std::shared_ptr<BpfObject> get_parent() const {
return parent_obj_.lock();
}
[[nodiscard]] std::string get_value_struct_name() const {
return value_struct_name_;
}
[[nodiscard]] bool has_struct_value() const {
return !value_struct_name_.empty();
}
private: private:
static void python_to_bytes_inplace(const py::object &obj, static void python_to_bytes_inplace(const py::object &obj,
std::span<uint8_t> buffer); std::span<uint8_t> buffer);
static py::object bytes_to_python(std::span<const uint8_t> data); py::object bytes_to_python(std::span<const uint8_t> data) const;
}; };
#endif // PYLIBBPF_MAPS_H #endif // PYLIBBPF_BPF_MAP_H

View File

@ -1,11 +1,15 @@
#include "bpf_object.h" #include "core/bpf_object.h"
#include "bpf_exception.h" #include "core/bpf_exception.h"
#include "bpf_map.h" #include "core/bpf_map.h"
#include "bpf_program.h" #include "core/bpf_program.h"
#include "utils/struct_parser.h"
#include <cerrno> #include <cerrno>
#include <cstring>
#include <utility>
BpfObject::BpfObject(std::string object_path) BpfObject::BpfObject(std::string object_path, py::dict structs)
: obj_(nullptr), object_path_(std::move(object_path)), loaded_(false) {} : obj_(nullptr), object_path_(std::move(object_path)), loaded_(false),
struct_defs_(structs), struct_parser_(nullptr) {}
BpfObject::~BpfObject() { BpfObject::~BpfObject() {
// Clear caches first (order matters!) // Clear caches first (order matters!)
@ -20,9 +24,13 @@ BpfObject::~BpfObject() {
} }
BpfObject::BpfObject(BpfObject &&other) noexcept BpfObject::BpfObject(BpfObject &&other) noexcept
: obj_(other.obj_), object_path_(std::move(other.object_path_)), : obj_(std::exchange(other.obj_, nullptr)),
loaded_(other.loaded_), prog_cache_(std::move(other.prog_cache_)), object_path_(std::move(other.object_path_)),
maps_cache_(std::move(other.maps_cache_)) { loaded_(std::exchange(other.loaded_, false)),
maps_cache_(std::move(other.maps_cache_)),
prog_cache_(std::move(other.prog_cache_)),
struct_defs_(std::move(other.struct_defs_)),
struct_parser_(std::move(other.struct_parser_)) {
other.obj_ = nullptr; other.obj_ = nullptr;
other.loaded_ = false; other.loaded_ = false;
@ -36,14 +44,13 @@ BpfObject &BpfObject::operator=(BpfObject &&other) noexcept {
bpf_object__close(obj_); bpf_object__close(obj_);
} }
obj_ = other.obj_; obj_ = std::exchange(other.obj_, nullptr);
object_path_ = std::move(other.object_path_); object_path_ = std::move(other.object_path_);
loaded_ = other.loaded_; loaded_ = std::exchange(other.loaded_, false);
prog_cache_ = std::move(other.prog_cache_);
maps_cache_ = std::move(other.maps_cache_); maps_cache_ = std::move(other.maps_cache_);
prog_cache_ = std::move(other.prog_cache_);
other.obj_ = nullptr; struct_defs_ = std::move(other.struct_defs_);
other.loaded_ = false; struct_parser_ = std::move(other.struct_parser_);
} }
return *this; return *this;
} }
@ -255,3 +262,11 @@ py::dict BpfObject::get_cached_maps() const {
} }
return maps; return maps;
} }
std::shared_ptr<StructParser> BpfObject::get_struct_parser() const {
if (!struct_parser_ && !struct_defs_.empty()) {
// Create parser on first access
struct_parser_ = std::make_shared<StructParser>(struct_defs_);
}
return struct_parser_;
}

View File

@ -6,12 +6,12 @@
#include <pybind11/pybind11.h> #include <pybind11/pybind11.h>
#include <string> #include <string>
#include <unordered_map> #include <unordered_map>
#include <vector>
namespace py = pybind11; namespace py = pybind11;
class BpfProgram; class BpfProgram;
class BpfMap; class BpfMap;
class StructParser;
/** /**
* BpfObject - Represents a loaded BPF object file. * BpfObject - Represents a loaded BPF object file.
@ -28,12 +28,14 @@ private:
mutable std::unordered_map<std::string, std::shared_ptr<BpfMap>> maps_cache_; mutable std::unordered_map<std::string, std::shared_ptr<BpfMap>> maps_cache_;
mutable std::unordered_map<std::string, std::shared_ptr<BpfProgram>> mutable std::unordered_map<std::string, std::shared_ptr<BpfProgram>>
prog_cache_; prog_cache_;
py::dict struct_defs_;
mutable std::shared_ptr<StructParser> struct_parser_;
std::shared_ptr<BpfProgram> _get_or_create_program(struct bpf_program *prog); std::shared_ptr<BpfProgram> _get_or_create_program(struct bpf_program *prog);
std::shared_ptr<BpfMap> _get_or_create_map(struct bpf_map *map); std::shared_ptr<BpfMap> _get_or_create_map(struct bpf_map *map);
public: public:
explicit BpfObject(std::string object_path); explicit BpfObject(std::string object_path, py::dict structs = py::dict());
~BpfObject(); ~BpfObject();
// Disable copy, allow move // Disable copy, allow move
@ -77,6 +79,10 @@ public:
[[nodiscard]] std::shared_ptr<BpfMap> get_map(const std::string &name); [[nodiscard]] std::shared_ptr<BpfMap> get_map(const std::string &name);
[[nodiscard]] struct bpf_map *find_map_by_name(const std::string &name) const; [[nodiscard]] struct bpf_map *find_map_by_name(const std::string &name) const;
[[nodiscard]] py::dict get_cached_maps() const; [[nodiscard]] py::dict get_cached_maps() const;
// Struct parsing
[[nodiscard]] py::dict get_struct_defs() const { return struct_defs_; }
[[nodiscard]] std::shared_ptr<StructParser> get_struct_parser() const;
}; };
#endif // PYLIBBPF_BPF_OBJECT_H #endif // PYLIBBPF_BPF_OBJECT_H

View File

@ -1,80 +0,0 @@
#include "bpf_perf_buffer.h"
#include "bpf_exception.h"
BpfPerfBuffer::BpfPerfBuffer(int map_fd, int page_cnt, py::function callback,
py::object lost_callback)
: pb_(nullptr), callback_(std::move(callback)),
lost_callback_(lost_callback) {
if (page_cnt <= 0 || (page_cnt & (page_cnt - 1)) != 0) {
throw BpfException("page_cnt must be a positive power of 2");
}
struct perf_buffer_opts pb_opts = {};
pb_opts.sz = sizeof(pb_opts); // Required for forward compatibility
pb_ = perf_buffer__new(
map_fd, page_cnt,
sample_callback_wrapper, // sample_cb
lost_callback.is_none() ? nullptr : lost_callback_wrapper, // lost_cb
this, // ctx
&pb_opts // opts
);
if (!pb_) {
throw BpfException("Failed to create perf buffer: " +
std::string(std::strerror(errno)));
}
}
BpfPerfBuffer::~BpfPerfBuffer() {
if (pb_) {
perf_buffer__free(pb_);
}
}
void BpfPerfBuffer::sample_callback_wrapper(void *ctx, int cpu, void *data,
unsigned int size) {
auto *self = static_cast<BpfPerfBuffer *>(ctx);
// Acquire GIL for Python calls
py::gil_scoped_acquire acquire;
try {
// Convert data to Python bytes
py::bytes py_data(static_cast<const char *>(data), size);
// Call Python callback: callback(cpu, data, size)
self->callback_(cpu, py_data, size);
} catch (const py::error_already_set &e) {
PyErr_Print();
}
}
void BpfPerfBuffer::lost_callback_wrapper(void *ctx, int cpu,
unsigned long long cnt) {
auto *self = static_cast<BpfPerfBuffer *>(ctx);
if (self->lost_callback_.is_none()) {
return;
}
py::gil_scoped_acquire acquire;
try {
self->lost_callback_(cpu, cnt);
} catch (const py::error_already_set &e) {
PyErr_Print();
}
}
int BpfPerfBuffer::poll(int timeout_ms) {
// Release GIL during blocking poll
py::gil_scoped_release release;
return perf_buffer__poll(pb_, timeout_ms);
}
int BpfPerfBuffer::consume() {
py::gil_scoped_release release;
return perf_buffer__consume(pb_);
}

View File

@ -1,31 +0,0 @@
#ifndef PYLIBBPF_BPF_PERF_BUFFER_H
#define PYLIBBPF_BPF_PERF_BUFFER_H
#include <libbpf.h>
#include <pybind11/functional.h>
#include <pybind11/pybind11.h>
namespace py = pybind11;
class BpfPerfBuffer {
private:
struct perf_buffer *pb_;
py::function callback_;
py::function lost_callback_;
// Static callback wrappers for C API
static void sample_callback_wrapper(void *ctx, int cpu, void *data,
unsigned int size);
static void lost_callback_wrapper(void *ctx, int cpu, unsigned long long cnt);
public:
BpfPerfBuffer(int map_fd, int page_cnt, py::function callback,
py::object lost_callback = py::none());
~BpfPerfBuffer();
int poll(int timeout_ms);
int consume();
[[nodiscard]] int fd() const;
};
#endif // PYLIBBPF_BPF_PERF_BUFFER_H

View File

@ -1,7 +1,8 @@
#include "bpf_program.h" #include "core/bpf_program.h"
#include "bpf_exception.h" #include "core/bpf_exception.h"
#include "bpf_object.h" #include "core/bpf_object.h"
#include <cerrno> #include <cerrno>
#include <cstring>
#include <utility> #include <utility>
BpfProgram::BpfProgram(std::shared_ptr<BpfObject> parent, BpfProgram::BpfProgram(std::shared_ptr<BpfObject> parent,

View File

@ -1,7 +1,6 @@
#ifndef PYLIBBPF_BPF_PROGRAM_H #ifndef PYLIBBPF_BPF_PROGRAM_H
#define PYLIBBPF_BPF_PROGRAM_H #define PYLIBBPF_BPF_PROGRAM_H
#include <cstring>
#include <libbpf.h> #include <libbpf.h>
#include <memory> #include <memory>
#include <string> #include <string>

View File

@ -0,0 +1,117 @@
#include "maps/perf_event_array.h"
#include "core/bpf_exception.h"
#include "core/bpf_map.h"
#include "core/bpf_object.h"
#include "utils/struct_parser.h"
#include <cerrno>
#include <cstring>
PerfEventArray::PerfEventArray(std::shared_ptr<BpfMap> map, int page_cnt,
py::function callback, py::object lost_callback)
: map_(map), pb_(nullptr), callback_(std::move(callback)),
lost_callback_(std::move(lost_callback)) {
if (map->get_type() != BPF_MAP_TYPE_PERF_EVENT_ARRAY) {
throw BpfException("Map '" + map->get_name() +
"' is not a PERF_EVENT_ARRAY");
}
if (page_cnt <= 0 || (page_cnt & (page_cnt - 1)) != 0) {
throw BpfException("page_cnt must be a positive power of 2");
}
struct perf_buffer_opts pb_opts = {};
pb_opts.sz = sizeof(pb_opts); // Required for forward compatibility
pb_ = perf_buffer__new(
map->get_fd(), page_cnt,
sample_callback_wrapper, // sample_cb
lost_callback.is_none() ? nullptr : lost_callback_wrapper, // lost_cb
this, // ctx
&pb_opts // opts
);
if (!pb_) {
throw BpfException("Failed to create perf buffer: " +
std::string(std::strerror(errno)));
}
}
PerfEventArray::PerfEventArray(std::shared_ptr<BpfMap> map, int page_cnt,
py::function callback,
const std::string &struct_name,
py::object lost_callback)
: PerfEventArray(map, page_cnt, callback, lost_callback) {
auto parent = map->get_parent();
if (!parent) {
throw BpfException("Parent BpfObject has been destroyed");
}
parser_ = parent->get_struct_parser();
struct_name_ = struct_name;
if (!parser_) {
throw BpfException("No struct definitions available");
}
}
PerfEventArray::~PerfEventArray() {
if (pb_) {
perf_buffer__free(pb_);
}
}
void PerfEventArray::sample_callback_wrapper(void *ctx, int cpu, void *data,
unsigned int size) {
auto *self = static_cast<PerfEventArray *>(ctx);
// Acquire GIL for Python calls
py::gil_scoped_acquire acquire;
try {
// Convert data to Python bytes
py::bytes py_data(static_cast<const char *>(data), size);
if (self->parser_ && !self->struct_name_.empty()) {
py::object event = self->parser_->parse(self->struct_name_, py_data);
self->callback_(cpu, event);
} else {
self->callback_(cpu, py_data);
}
} catch (const py::error_already_set &e) {
PyErr_Print();
} catch (const std::exception &e) {
py::print("C++ error in perf callback:", e.what());
}
}
void PerfEventArray::lost_callback_wrapper(void *ctx, int cpu,
unsigned long long cnt) {
auto *self = static_cast<PerfEventArray *>(ctx);
py::gil_scoped_acquire acquire;
try {
if (!self->lost_callback_.is_none()) {
py::function lost_fn = py::cast<py::function>(self->lost_callback_);
lost_fn(cpu, cnt);
} else {
py::print("Lost", cnt, "events on CPU", cpu);
}
} catch (const py::error_already_set &e) {
PyErr_Print();
}
}
int PerfEventArray::poll(int timeout_ms) {
// Release GIL during blocking poll
py::gil_scoped_release release;
return perf_buffer__poll(pb_, timeout_ms);
}
int PerfEventArray::consume() {
py::gil_scoped_release release;
return perf_buffer__consume(pb_);
}

View File

@ -0,0 +1,46 @@
#ifndef PYLIBBPF_PERF_EVENT_ARRAY_H
#define PYLIBBPF_PERF_EVENT_ARRAY_H
#include <libbpf.h>
#include <memory>
#include <pybind11/pybind11.h>
#include <string>
class StructParser;
class BpfMap;
namespace py = pybind11;
class PerfEventArray {
private:
std::shared_ptr<BpfMap> map_;
struct perf_buffer *pb_;
py::function callback_;
py::object lost_callback_;
std::shared_ptr<StructParser> parser_;
std::string struct_name_;
// Static callback wrappers for C API
static void sample_callback_wrapper(void *ctx, int cpu, void *data,
unsigned int size);
static void lost_callback_wrapper(void *ctx, int cpu, unsigned long long cnt);
public:
PerfEventArray(std::shared_ptr<BpfMap> map, int page_cnt,
py::function callback, py::object lost_callback = py::none());
PerfEventArray(std::shared_ptr<BpfMap> map, int page_cnt,
py::function callback, const std::string &struct_name,
py::object lost_callback = py::none());
~PerfEventArray();
PerfEventArray(const PerfEventArray &) = delete;
PerfEventArray &operator=(const PerfEventArray &) = delete;
int poll(int timeout_ms);
int consume();
[[nodiscard]] std::shared_ptr<BpfMap> get_map() const { return map_; }
};
#endif // PYLIBBPF_PERF_EVENT_ARRAY_H

View File

@ -0,0 +1,25 @@
#include "struct_parser.h"
#include "core/bpf_exception.h"
StructParser::StructParser(py::dict structs) {
for (auto item : structs) {
std::string name = py::str(item.first);
struct_types_[name] = py::reinterpret_borrow<py::object>(item.second);
}
}
py::object StructParser::parse(const std::string &struct_name, py::bytes data) {
auto it = struct_types_.find(struct_name);
if (it == struct_types_.end()) {
throw BpfException("Unknown struct: " + struct_name);
}
py::object struct_type = it->second;
// Use ctypes.from_buffer_copy() to create struct from bytes
return struct_type.attr("from_buffer_copy")(data);
}
bool StructParser::has_struct(const std::string &struct_name) const {
return struct_types_.find(struct_name) != struct_types_.end();
}

20
src/utils/struct_parser.h Normal file
View File

@ -0,0 +1,20 @@
#ifndef PYLIBBPF_STRUCT_PARSER_H
#define PYLIBBPF_STRUCT_PARSER_H
#include <pybind11/pybind11.h>
#include <string>
#include <unordered_map>
namespace py = pybind11;
class StructParser {
private:
std::unordered_map<std::string, py::object> struct_types_;
public:
explicit StructParser(py::dict structs);
py::object parse(const std::string &struct_name, py::bytes data);
bool has_struct(const std::string &struct_name) const;
};
#endif

View File

@ -2,6 +2,6 @@ import pylibbpf as m
def test_main(): def test_main():
assert m.__version__ == "0.0.5" assert m.__version__ == "0.0.7"
prog = m.BpfObject("tests/execve2.o") prog = m.BpfObject("tests/execve2.o", structs={})
print(prog) print(prog)