4b1860766d
* feat: Replace mplex with yamux as default multiplexer in py-libp2p * Retain Mplex alongside Yamux in new_swarm with messaging that Yamux is preferred * moved !BBHII to a constant YAMUX_HEADER_FORMAT at the top of yamux.py with a comment explaining its structure * renamed the news fragment to 534.feature.rst and updated the description * renamed the news fragment to 534.feature.rst and updated the description * added a docstring to clarify that Yamux does not support deadlines natively * Remove the __main__ block entirely from test_yamux.py * Replaced the print statements in test_yamux.py with logging.debug * Added a comment linking to the spec for clarity * Raise NotImplementedError in YamuxStream.set_deadline per review * Add muxed_conn to YamuxStream and test deadline NotImplementedError * Fix Yamux implementation to meet libp2p spec * Fix None handling in YamuxStream.read and Yamux.read_stream * Fix test_connected_peers.py to correctly handle peer connections * fix: Ensure StreamReset is raised on read after local reset in yamux * fix: Map MuxedStreamError to StreamClosed in NetStream.write for Yamux * fix: Raise MuxedStreamReset in Yamux.read_stream for closed streams * fix: Correct Yamux stream read behavior for NetStream tests Fixed est_net_stream_read_after_remote_closed by updating NetStream.read to raise StreamEOF when the stream is remotely closed and no data is available, aligning with test expectations and Fixed est_net_stream_read_until_eof by modifying YamuxStream.read to block until the stream is closed ( ecv_closed=True) for =-1 reads, ensuring data is only returned after remote closure. * fix: Correct Yamux stream read behavior for NetStream tests Fixed est_net_stream_read_after_remote_closed by updating NetStream.read to raise StreamEOF when the stream is remotely closed and no data is available, aligning with test expectations and Fixed est_net_stream_read_until_eof by modifying YamuxStream.read to block until the stream is closed ( ecv_closed=True) for =-1 reads, ensuring data is only returned after remote closure. * fix: raise StreamEOF when reading from closed stream with empty buffer * fix: prioritize returning buffered data even after stream reset * fix: prioritize returning buffered data even after stream reset * fix: Ensure test_net_stream_read_after_remote_closed_and_reset passes in full suite * fix: Add __init__.py to yamux module to fix documentation build * fix: Add __init__.py to yamux module to fix documentation build * fix: Add libp2p.stream_muxer.yamux to libp2p.stream_muxer.rst toctree * fix: Correct title underline length in libp2p.stream_muxer.yamux.rst * fix: Add a = so that is matches the libp2p.stream\_muxer.yamux length * fix(tests): Resolve race condition in network notification test * fix: fixing failing tests and examples with yamux and noise * refactor: remove debug logging and improve x25519 tests * fix: Add functionality for users to choose between Yamux and Mplex * fix: increased trio sleep to 0.1 sec for slow environment * feat: Add test for switching between Yamux and mplex * refactor: move host fixtures to interop tests * chore: Update __init__.py removing unused import removed unused ```python import os import logging ``` * lint: fix import order * fix: Resolve conftest.py conflict by removing trio test support * fix: Resolve test skipping by keeping trio test support * Fix: add a newline at end of the file --------- Co-authored-by: acul71 <luca.pisani@birdo.net> Co-authored-by: acul71 <34693171+acul71@users.noreply.github.com>
py-libp2p
The Python implementation of the libp2p networking stack.
⚠️ Warning: py-libp2p is an experimental and work-in-progress repo under development. We do not yet recommend using py-libp2p in production environments.
Read more in the documentation on ReadTheDocs. View the release notes.
Maintainers
Currently maintained by @pacrob, @seetadev and @dhuseby, looking for assistance!
Feature Breakdown
py-libp2p aims for conformity with the standard libp2p modules. Below is a breakdown of the modules we have developed, are developing, and may develop in the future.
Legend: ✅: Done 🛠️: In Progress/Usable 🌱 Prototype/Unstable ❌: Missing
Transports
| Transport | Status | Source |
|---|---|---|
libp2p-tcp |
✅ | source |
libp2p-quic |
🌱 | |
libp2p-websocket |
❌ | |
libp2p-webrtc-browser-to-server |
❌ | |
libp2p-webrtc-private-to-private |
❌ |
NAT Traversal
| NAT Traversal | Status |
|---|---|
libp2p-circuit-relay-v2 |
❌ |
libp2p-autonat |
❌ |
libp2p-hole-punching |
❌ |
Secure Communication
| Secure Communication | Status | Source |
|---|---|---|
libp2p-noise |
🌱 | source |
libp2p-tls |
❌ |
Discovery
| Discovery | Status |
|---|---|
bootstrap |
❌ |
random-walk |
❌ |
mdns-discovery |
❌ |
rendezvous |
❌ |
Peer Routing
| Peer Routing | Status |
|---|---|
libp2p-kad-dht |
❌ |
Publish/Subscribe
| Publish/Subscribe | Status | Source |
|---|---|---|
libp2p-floodsub |
✅ | source |
libp2p-gossipsub |
✅ | source |
Stream Muxers
| Stream Muxers | Status | Status |
|---|---|---|
libp2p-yamux |
🌱 | |
libp2p-mplex |
🛠️ | source |
Storage
| Storage | Status |
|---|---|
libp2p-record |
❌ |
General Purpose Utilities & Datatypes
| Utility/Datatype | Status | Source |
|---|---|---|
libp2p-ping |
✅ | source |
libp2p-peer |
✅ | source |
libp2p-identify |
✅ | source |
Explanation of Basic Two Node Communication
Core Concepts
(non-normative, useful for team notes, not a reference)
Several components of the libp2p stack take part when establishing a connection between two nodes:
- Host: a node in the libp2p network.
- Connection: the layer 3 connection between two nodes in a libp2p network.
- Transport: the component that creates a Connection, e.g. TCP, UDP, QUIC, etc.
- Streams: an abstraction on top of a Connection representing parallel conversations about different matters, each of which is identified by a protocol ID. Multiple streams are layered on top of a Connection via the Multiplexer.
- Multiplexer: a component that is responsible for wrapping messages sent on a stream with an envelope that identifies the stream they pertain to, normally via an ID. The multiplexer on the other unwraps the message and routes it internally based on the stream identification.
- Secure channel: optionally establishes a secure, encrypted, and authenticated channel over the Connection.
- Upgrader: a component that takes a raw layer 3 connection returned by the Transport, and performs the security and multiplexing negotiation to set up a secure, multiplexed channel on top of which Streams can be opened.
Communication between two hosts X and Y
(non-normative, useful for team notes, not a reference)
Initiate the connection: A host is simply a node in the libp2p network that is able to communicate with other nodes in the network. In order for X and Y to communicate with one another, one of the hosts must initiate the connection. Let's say that X is going to initiate the connection. X will first open a connection to Y. This connection is where all of the actual communication will take place.
Communication over one connection with multiple protocols: X and Y can communicate over the same connection using different protocols and the multiplexer will appropriately route messages for a given protocol to a particular handler function for that protocol, which allows for each host to handle different protocols with separate functions. Furthermore, we can use multiple streams for a given protocol that allow for the same protocol and same underlying connection to be used for communication about separate topics between nodes X and Y.
Why use multiple streams?: The purpose of using the same connection for multiple streams to communicate over is to avoid the overhead of having multiple connections between X and Y. In order for X and Y to differentiate between messages on different streams and different protocols, a multiplexer is used to encode the messages when a message will be sent and decode a message when a message is received. The multiplexer encodes the message by adding a header to the beginning of any message to be sent that contains the stream id (along with some other info). Then, the message is sent across the raw connection and the receiving host will use its multiplexer to decode the message, i.e. determine which stream id the message should be routed to.