648 Commits

Author SHA1 Message Date
c1bbb1a9b4 Merge branch 'main' into chore01 2025-09-24 22:40:38 +05:30
f13876a64f Merge pull request #953 from acul71/fix/issue-952-windows-cdci-python-version
fix: use dynamic Python version in Windows CI/CD tests
2025-09-24 19:43:06 +05:30
3c43e4682a Merge branch 'main' into fix/issue-952-windows-cdci-python-version 2025-09-23 17:43:53 -04:00
b46dae7c50 Merge pull request #917 from parth-soni07/refactor/replace-magic-numbers-with-named-constants
Refactor: Replace magic numbers with named constants and enums for clarity and maintainability
2025-09-24 03:10:48 +05:30
262e7e9834 fix: add newline to newsfragment file
Pre-commit hook fixed end-of-file formatting
2025-09-23 17:25:12 -04:00
634de8ed02 fix: use dynamic Python version in Windows CI/CD tests
Fix hardcoded py311- to use dynamic matrix.python-version variable.
Ensures Windows tests run with correct Python version and resolves
async behavior differences causing test failures.
2025-09-23 17:25:12 -04:00
17c1ced408 Merge branch 'main' into refactor/replace-magic-numbers-with-named-constants 2025-09-23 21:26:19 +05:30
d64f9e10fd Fix: lint error 2025-09-23 17:29:15 +05:30
93c2d5002f fix: GossipSub peer propagation to include FloodSub peers 2025-09-23 17:29:15 +05:30
721da9364e Fixed variable imports 2025-09-23 17:29:15 +05:30
35a4bf2d42 Update multiaddr to version 0.0.11
- Switch from git dependency to pip package
- Update from git+https://github.com/multiformats/py-multiaddr.git@b186e2ccadc22545dec4069ff313787bf29265e0
- Use multiaddr>=0.0.11 from PyPI

Fixes #934
2025-09-23 17:29:15 +05:30
02ff688b5a Added timeout passing in muxermultistream. Updated the usages. Tested the params are passed correctly 2025-09-23 17:29:15 +05:30
52625e0f68 Fix multiaddr dep to use specific commit hash to resolve install issue (#928)
* Fix multiaddr dependency to use specific commit hash to resolve installation issues

* fix: ops wrong filename
2025-09-23 17:29:15 +05:30
066c87515e Merge pull request #897 from codemaestro64/enhancement/yamuxstream-lock
enh/793: add read-write lock to yamuxstream
2025-09-23 10:48:19 +05:30
a0826b21bc Merge branch 'main' into enhancement/yamuxstream-lock 2025-09-23 10:13:46 +05:30
3e4be44fa3 Add newsfragment 2025-09-23 09:05:44 +05:30
c8c7e55d5c Add test and fix ci 2025-09-23 09:05:33 +05:30
3363f57338 Fix round robin load balancing 2025-09-23 08:58:12 +05:30
37fd2542c0 remove duplicate entry of fastecdsa (#948) 2025-09-22 15:57:06 -06:00
5e2840d5b5 Merge branch 'main' into enhancement/yamuxstream-lock 2025-09-23 00:43:22 +05:30
17838687fe Merge pull request #892 from yashksaini-coder/fix/885-Update-default-Bind-address
Fix/885 update default bind address
2025-09-23 00:23:00 +05:30
43bb36338c Merge branch 'main' into fix/885-Update-default-Bind-address 2025-09-22 22:07:12 +05:30
d519f75d69 Merge pull request #781 from GautamBytes/add-ws-transport
Add WebSocket transport support
2025-09-22 22:06:38 +05:30
7c60b81801 Merge branch 'fix/885-Update-default-Bind-address' of https://github.com/yashksaini-coder/py-libp2p into fix/885-Update-default-Bind-address 2025-09-22 21:52:51 +05:30
009fdd0d8f Increase wait time in unsubscribe_backoff test to exceed backoff duration 2025-09-22 21:51:45 +05:30
6a1b955a4e fix: implement lazy initialization for global transport registry
- Change global registry from immediate to lazy initialization
- Fix doctest failure caused by debug logging during MultiError import
- Update all functions to use get_transport_registry() instead of direct access
- Resolves CI/CD doctest failure in libp2p.rst
2025-09-21 19:29:48 -04:00
87429eb2e9 Merge branch 'main' into add-ws-transport 2025-09-22 02:36:01 +05:30
321bb86ea4 Merge branch 'main' into enhancement/yamuxstream-lock 2025-09-22 02:03:11 +05:30
c3c8d8ccb9 Merge branch 'main' into fix/885-Update-default-Bind-address 2025-09-22 02:02:56 +05:30
a01811a435 Merge pull request #893 from paschal533/keyerror-fix
fix: GossipSub peer propagation to include FloodSub peers
2025-09-22 01:57:06 +05:30
db2f3a64ea Merge branch 'main' into keyerror-fix 2025-09-22 01:56:52 +05:30
77208e95cc Refactor example scripts and core modules to enhance security by using secure loopback addresses for network binding 2025-09-20 13:32:43 +05:30
ae3e2ff943 Update examples to use wildcard addresses for network binding and improve connection timeout comments 2025-09-20 13:11:22 +05:30
a862ac83cd Invert raw format flag to determine varint format usage in main function 2025-09-18 23:37:26 +05:30
3f30ed4437 Fix typo in connection timeout comment and improve identify example output formatting 2025-09-18 21:36:25 +05:30
67a3cab2e2 Add example for new address paradigm in multiple connections 2025-09-18 02:34:01 +05:30
bf132cf3dd Fix import statements and improve error handling in examples 2025-09-18 02:22:31 +05:30
4dd2454a46 Update examples to use dynamic host IP instead of hardcoded localhost 2025-09-18 02:09:51 +05:30
b01f2bd105 Merge branch 'main' into fix/885-Update-default-Bind-address 2025-09-18 00:58:04 +05:30
1a4fe91419 doc: websocket newsframgment 2025-09-17 13:38:17 -04:00
1326592fe2 Merge branch 'main' into enhancement/yamuxstream-lock 2025-09-17 15:30:22 +05:30
5f0c5101c7 Merge branch 'main' into add-ws-transport 2025-09-17 15:27:48 +05:30
8d028a046d Merge pull request #935 from acul71/fix/issue-934-update-multiaddr
Update multiaddr to version 0.0.11
2025-09-17 15:19:13 +05:30
a0cb6e3a30 Complete WebSocket transport implementation with TLS support
- Add TLS configuration support to new_host and new_swarm functions
- Fix WebSocket transport tests (test_wss_host_pair_data_exchange, test_wss_listen_without_tls_config)
- Integrate TLS configuration with transport registry for proper WebSocket WSS support
- Move debug files to downloads directory for future reference
- All 47 WebSocket tests now passing including WSS functionality
- Maintain backward compatibility with existing code
- Resolve all type checking and linting issues
2025-09-17 03:08:24 -04:00
f4a4298c0f Restore debug tools and test client from original WebSocket implementation
- Added back debug_websocket_url.py for WebSocket URL testing
- Added back test_websocket_client.py for standalone WebSocket testing
- These tools complement the integrated WebSocket transport implementation
2025-09-17 01:00:41 -04:00
cdbd80eeba Merge remote changes with local WebSocket improvements
- Combined yashksaini-coder's flow control improvements with luca's WSS features
- Preserved comprehensive WSS support, TLS configuration, and handshake timeout
- Added production-ready buffer management and connection limits
- Maintained backward compatibility with existing WebSocket functionality
- Integrated both approaches for optimal WebSocket transport implementation
2025-09-17 01:00:15 -04:00
518c1f98b1 Update multiaddr to version 0.0.11
- Switch from git dependency to pip package
- Update from git+https://github.com/multiformats/py-multiaddr.git@b186e2ccadc22545dec4069ff313787bf29265e0
- Use multiaddr>=0.0.11 from PyPI

Fixes #934
2025-09-16 20:09:10 -04:00
284eee78d7 Merge branch 'main' into add-ws-transport 2025-09-16 01:04:16 +05:30
6e8960b383 Merge branch 'main' into enhancement/yamuxstream-lock 2025-09-16 01:03:02 +05:30
1250b2ec12 Merge branch 'main' into fix/885-Update-default-Bind-address 2025-09-16 01:02:18 +05:30
9b0f75014c Merge pull request #896 from unniznd/fix_expose_timeout_muxer_multistream
fix: Added timeout paramter into muxer multistream
2025-09-16 01:01:13 +05:30
b4a5d9037c Merge branch 'main' into enhancement/yamuxstream-lock 2025-09-16 00:47:59 +05:30
ecdb770c45 Merge branch 'main' into fix_expose_timeout_muxer_multistream 2025-09-16 00:45:26 +05:30
95e7e7b4f6 Merge branch 'main' into add-ws-transport 2025-09-15 21:20:44 +05:30
8428aff20c Merge branch 'main' into fix/885-Update-default-Bind-address 2025-09-15 06:57:27 +00:00
81cc2f06f0 Fix multiaddr dep to use specific commit hash to resolve install issue (#928)
* Fix multiaddr dependency to use specific commit hash to resolve installation issues

* fix: ops wrong filename
2025-09-14 17:45:22 -06:00
c5a2836829 stream_muxer(yamux): add ReadWriteLock to YamuxStream to prevent concurrent read/write corruption
Introduce a read/write lock abstraction and integrate it into `YamuxStream` so that simultaneous
reads and writes do not interleave, eliminating potential data corruption and race conditions.

Major changes:
- Abstract `ReadWriteLock` into its own util module
- Integrate locking into YamuxStream for `write` operations
- Ensure tests pass for lock correctness
- Fix lint & type issues discovered during review

Closes #793
2025-09-13 09:11:07 +01:00
4fdfdae9fb Refactor P2PWebSocketConnection and WebsocketTransport constructors for improved readability. Clean up whitespace and enhance logging for connection management. 2025-09-12 03:11:43 +05:30
0271a36316 Update the flow control, buffer management, and connection limits. Implement proper error handling and cleanup in P2PWebSocketConnection. Update tests for improved connection handling. 2025-09-12 03:04:38 +05:30
8793667503 Updated config & minor changes 2025-09-11 14:18:40 +05:30
771b837916 app{websocket): Refactor transport type annotations and improve event handling in QUIC connection 2025-09-10 04:15:56 +05:30
93db588b9e Merge branch 'main' into add-ws-transport 2025-09-10 02:49:52 +05:30
4a36d6efeb Replace magic number with named constants 2025-09-09 13:24:07 +05:30
7d364da950 Refactor: update examples to utilize new address paradigm with wildcard support
- Introduced `get_wildcard_address` function for explicit wildcard binding.
- Updated examples to use `get_available_interfaces` and `get_optimal_binding_address` for address selection.
- Ensured consistent usage of the new address paradigm across all example files.
- Added tests to verify the implementation of the new address paradigm and wildcard feature.
2025-09-09 12:10:28 +05:30
4e8ebf707a Merge branch 'main' into fix_expose_timeout_muxer_multistream 2025-09-09 00:43:38 +05:30
80e22f7c4a Merge branch 'libp2p:main' into fix/885-Update-default-Bind-address 2025-09-08 19:01:49 +05:30
f4d5a44521 Fix type errors and linting issues
- Fix type annotation errors in transport_registry.py and __init__.py
- Fix line length violations in test files (E501 errors)
- Fix missing return type annotations
- Fix cryptography NameAttribute type errors with type: ignore
- Fix ExceptionGroup import for cross-version compatibility
- Fix test failure in test_wss_listen_without_tls_config by handling ExceptionGroup
- Fix len() calls with None arguments in test_tcp_data_transfer.py
- Fix missing attribute access errors on interface types
- Fix boolean type expectation errors in test_js_ws_ping.py
- Fix nursery context manager type errors

All tests now pass and linting is clean.
2025-09-08 04:18:10 +02:00
afe6da5db2 Merge upstream/main into add-ws-transport
Resolved conflicts in:
- .gitignore: Combined JavaScript interop and Sphinx build ignores
- libp2p/__init__.py: Integrated QUIC transport support with WebSocket transport
- libp2p/network/swarm.py: Used upstream's improved listener handling
- pyproject.toml: Kept both WebSocket and QUIC dependencies

This merge brings in:
- QUIC transport implementation
- Enhanced swarm functionality
- Improved peer discovery
- Better error handling
- Updated dependencies and documentation

WebSocket transport implementation remains intact and functional.
2025-09-07 23:47:41 +02:00
396812e84a Experimental: Add comprehensive WebSocket and WSS implementation with tests
- Implemented full WSS support with TLS configuration
- Added handshake timeout and connection state tracking
- Created comprehensive test suite with 13+ WSS unit tests
- Added Python-to-Python WebSocket peer-to-peer tests
- Implemented multiaddr parsing for /ws, /wss, /tls/ws formats
- Added connection state tracking and concurrent close handling
- Created standalone WebSocket client for testing
- Fixed circular import issues with multiaddr utilities
- Added debug tools for WebSocket URL testing

All WebSocket transport functionality is complete and working.
Tests demonstrate WebSocket transport works correctly at the transport layer.
Higher-level libp2p protocol compatibility issues remain (same as JS interop).
2025-09-07 23:44:17 +02:00
74f4aaf136 updated random walk status in readme (#907) 2025-09-06 14:28:05 -06:00
fe662446dd Merge branch 'main' into fix/885-Update-default-Bind-address 2025-09-06 23:41:03 +05:30
a8a71b077b Merge pull request #900 from libp2p/seetadev-patch-1
Update pyproject.toml
2025-09-06 04:04:32 +05:30
b7f11ba43d Update pyproject.toml 2025-09-06 03:41:18 +05:30
a69db8a716 refactor(app): 885 Add ignore comment since SO attr not supported to Win 2025-09-06 02:00:19 +05:30
aa2a650f85 fix: update QUIC examples to use loopback address for improved security 2025-09-05 20:54:33 +05:30
030deb42b4 refactor: update examples to use available interfaces for listening addresses and improve logging configuration 2025-09-05 20:05:10 +05:30
637bd5d560 Merge branch 'libp2p:main' into fix/885-Update-default-Bind-address 2025-09-05 18:14:44 +05:30
ce3f3a8e43 Merge pull request #763 from AkMo3/main
QUIC v1
2025-09-05 11:40:47 +05:30
f3976b7d2f docs: add some documentation for QUIC transport 2025-09-05 05:41:06 +00:00
09c9709a3e Merge remote-tracking branch 'upstream/main' 2025-09-04 21:27:51 +00:00
f0b05b8307 Merge branch 'main' into fix_expose_timeout_muxer_multistream 2025-09-05 02:57:29 +05:30
9fdb36ed03 Merge branch 'main' into keyerror-fix 2025-09-05 02:57:04 +05:30
31191cbfae Merge branch 'main' into fix/885-Update-default-Bind-address 2025-09-05 02:56:39 +05:30
2fe5882013 fix: add quic utils test and improve connection performance 2025-09-04 21:25:37 +00:00
bffabd1070 Merge branch 'main' into add-ws-transport 2025-09-05 02:54:03 +05:30
9370101a84 Merge pull request #843 from unniznd/fix_pubsub_msg_id_type_inconsistency
fix: message id type inconsistency in handle ihave and message id parsing improvement in handle iwant
2025-09-04 23:39:14 +05:30
56732a1506 Merge branch 'main' into fix_pubsub_msg_id_type_inconsistency 2025-09-04 16:26:01 +05:30
68cb54ee0f Merge branch 'main' into add-ws-transport 2025-09-04 16:22:16 +05:30
f80101c4eb Merge branch 'main' into keyerror-fix 2025-09-04 16:18:41 +05:30
d268393812 Merge branch 'main' into chore01 2025-09-04 16:17:28 +05:30
4786b48364 Merge branch 'main' into fix/885-Update-default-Bind-address 2025-09-04 16:13:43 +05:30
2ee3e0b054 Merge branch 'main' into main 2025-09-04 16:13:06 +05:30
2a249b1792 Merge pull request #849 from ankur12-1610/issue-798
Enhance Bootstrap module to dial peers after address resolution.
2025-09-04 16:12:56 +05:30
c693cd9bb9 Merge branch 'main' into add-ws-transport 2025-09-04 15:19:11 +05:30
25d7706047 Added timeout passing in muxermultistream. Updated the usages. Tested the params are passed correctly 2025-09-04 14:58:22 +05:30
c5a8f26490 Merge branch 'main' into fix/885-Update-default-Bind-address 2025-09-04 14:17:35 +05:30
31c65274c3 Merge branch 'main' into keyerror-fix 2025-09-04 14:15:54 +05:30
5ec1671608 Merge branch 'main' into issue-798 2025-09-04 14:14:31 +05:30
69a0d3da9d Merge branch 'main' into main 2025-09-04 14:13:20 +05:30
431a4807fb Merge pull request #886 from yashksaini-coder/fix/cross_platform_path_tests
Fix: Cross-Platform Path Handling Standardization
2025-09-04 14:12:04 +05:30
f54a14b713 Merge branch 'main' into issue-798 2025-09-04 13:41:45 +05:30
d0c81301b5 fix: quic transport mock in quic connection 2025-09-02 18:47:07 +00:00
d2d4c4b451 fix: proper connection config setup 2025-09-02 18:27:47 +00:00
4b4214f066 fix: add mistakenly removed windows CI/CD tests 2025-09-02 17:54:40 +00:00
37a4d96f90 add rst 2025-09-02 22:23:11 +05:30
33730bdc48 fix: type assertion for config class 2025-09-02 16:39:38 +00:00
159d2cc322 Merge remote-tracking branch 'upstream/main' 2025-09-02 16:16:21 +00:00
b367ff70c3 Fix: lint error 2025-09-02 04:31:35 -07:00
9465805c3b Merge branch 'libp2p:main' into keyerror-fix 2025-09-02 14:30:13 +03:00
37652f7034 fix: GossipSub peer propagation to include FloodSub peers 2025-09-02 03:50:00 -07:00
b8217bb8a8 Merge branch 'main' into fix_pubsub_msg_id_type_inconsistency 2025-09-02 10:16:17 +05:30
a0ca284e8f Merge branch 'main' into chore01 2025-09-02 03:40:44 +05:30
809a32a712 chore: remove temp test valid script 2025-09-02 03:35:35 +05:30
ade6f5c6ad Merge branch 'libp2p:main' into fix/885-Update-default-Bind-address 2025-09-02 03:23:20 +05:30
d385cb45cf Merge branch 'libp2p:main' into fix/cross_platform_path_tests 2025-09-02 03:22:56 +05:30
05867be37e refactor: performed pre-commit checks 2025-09-02 03:06:39 +05:30
2535305123 Merge pull request #838 from unniznd/fix_multiselect_negotiate_type
fix: Added multiselect type consistency in negotiate method
2025-09-02 03:03:56 +05:30
e8d1a0fc32 chore: add newsfragment for 885 issue fix 2025-09-02 03:03:12 +05:30
5633d52a63 test: Add comprehensive tests for address validation utilities and ensure secure binding addresses (127.0.0.1) are used instead of wildcard (0.0.0.0) 2025-09-02 03:02:41 +05:30
68af8766e2 doc: Update examples documentation files for use of loopback (127.0.0.1) 2025-09-02 03:01:16 +05:30
87550113a4 chore: Update network address config to use loopback (127.0.0.1) instead of wildcard (0.0.0.0) across multiple examples and utilities. 2025-09-02 03:00:18 +05:30
9df542f97f Merge branch 'main' into fix_multiselect_negotiate_type 2025-09-02 02:38:33 +05:30
93fe070cfb Merge pull request #884 from acul71/fix/issue-883-transport-issues-todos
fix: remove unused upgrade_listener function (Issue 2 from #726)
2025-09-02 02:38:17 +05:30
7a4c955c98 Merge branch 'main' into fix/issue-883-transport-issues-todos 2025-09-02 01:50:14 +05:30
14a74fdbd1 Merge branch 'main' into fix/cross_platform_path_tests 2025-09-02 01:42:11 +05:30
934f49af83 Merge branch 'main' into fix_multiselect_negotiate_type 2025-09-02 01:40:40 +05:30
970b535b25 Merge pull request #889 from lla-dane/pubsub-record
Signed-Peer-Record support in Pubsub/Gossipsub message transfer
2025-09-02 01:39:44 +05:30
145727a9ba Refactor logging code: Remove unnecessary blank lines in logging setup and cleanup functions for improved readability. Update tests to reflect formatting changes. 2025-09-02 01:39:24 +05:30
84c1a7031a Enhance logging cleanup: Introduce global handler management for proper resource cleanup on exit and during logging setup. Update tests to ensure file handlers are closed correctly across platforms. 2025-09-02 01:23:12 +05:30
fc6b290c56 Merge branch 'main' into fix_multiselect_negotiate_type 2025-09-02 01:08:21 +05:30
20edc3830a Merge branch 'main' into fix_pubsub_msg_id_type_inconsistency 2025-09-02 01:07:16 +05:30
ef6557518c Merge branch 'main' into pubsub-record 2025-09-02 01:01:08 +05:30
6742dd38f7 Merge branch 'main' into fix/cross_platform_path_tests 2025-09-02 01:00:23 +05:30
1783a6b0b9 Merge pull request #876 from bomanaps/feat/swarm-multi-connection-support
feat(swarm): enhance swarm with retry backoff
2025-09-02 00:35:48 +05:30
1077516196 update newsfragment 2025-09-01 18:11:22 +05:30
aad87f983f Adress documentation comment 2025-09-01 11:58:42 +01:00
69680e9c1f Added negative testcases 2025-09-01 10:30:25 +05:30
7d6eb28d7c message inconsistency fixed 2025-09-01 09:48:08 +05:30
fcb35084b3 fix(docs): Update tomllib import handling and streamline pyproject path resolution 2025-09-01 03:14:09 +05:30
42c8937a8d build(app): Add fallback to os.path.join + newsfragment 886 2025-09-01 02:53:53 +05:30
64ccce17eb fix(app): 882 Comprehensive cross-platform path handling utilities 2025-09-01 02:03:51 +05:30
6a24b138dd feat: Add cross-platform path utilities module 2025-09-01 01:35:32 +05:30
eab8df84df chore: add news fragment 2025-08-31 17:09:22 +00:00
9749be6574 fix: refine selection of quic transport while init 2025-08-31 16:07:41 +00:00
186113968e chore: remove unwanted code, fix type issues and comments 2025-08-31 13:15:51 +00:00
e1141ee376 fix: fix nim interop env setup file 2025-08-31 06:47:15 +00:00
9a06ee429f Fix documentation build issues and add _build/ to .gitignore 2025-08-31 02:01:39 +01:00
526b65e1d5 style: apply ruff formatting fixes 2025-08-31 01:43:27 +01:00
59e1d9ae39 address architectural refactoring discussed 2025-08-31 01:38:29 +01:00
d620270eaf docs: add newsfragment for issue 883 - remove unused upgrade_listener function 2025-08-31 00:10:15 +02:00
31040931ea fix: remove unused upgrade_listener function (Issue 2 from #726) 2025-08-30 23:44:49 +02:00
8e74f944e1 update multiaddr dep 2025-08-30 14:18:14 +05:30
89cb8c0bd9 fix: check forced failure for nim interop 2025-08-30 14:08:53 +05:30
d97b86081b fix: add nim libp2p echo interop 2025-08-30 14:08:53 +05:30
2c03ac46ea fix: Peer ID verification during dial (#7) 2025-08-30 14:08:53 +05:30
58433f9b52 fix: changes to opening new stream, setting quic connection parameters
1. Do not dial to open a new stream, use existing swarm connection in quic transport to open new stream
2. Derive values from quic config for quic stream configuration
3. Set quic-v1 config only if enabled
2025-08-30 14:08:53 +05:30
933741b190 fix: allow accept stream to wait indefinitely 2025-08-30 14:08:53 +05:30
760f94bd81 fix: quic maddr test 2025-08-30 14:08:53 +05:30
6d1e53a4e2 fix: ignore peer id derivation for quic dial 2025-08-30 14:08:53 +05:30
5ed3707a51 fix: use ASN.1 format certificate extension 2025-08-30 14:08:53 +05:30
f550c19b2c multiple streams ping, invalid certificate handling 2025-08-30 14:08:53 +05:30
84c9ddc2dd chore: cleanup and doc gen fixes 2025-08-30 14:08:53 +05:30
a6ff93122b chore: fix linting issues 2025-08-30 14:08:53 +05:30
8e6e88140f fix: add support for rsa, ecdsa keys in quic 2025-08-30 14:08:53 +05:30
342ac746f8 fix: client certificate verification done 2025-08-30 14:08:52 +05:30
b3f0a4e8c4 DEBUG: client certificate at server 2025-08-30 14:08:32 +05:30
0f64bb49b5 chore: log cleanup 2025-08-30 14:08:22 +05:30
03bf071739 chore: cleanup and near v1 quic impl 2025-08-30 14:08:22 +05:30
c15c317514 fix: accept stream on server side 2025-08-30 14:08:22 +05:30
6c45862fe9 fix: succesfull echo example completed 2025-08-30 14:08:22 +05:30
8f0cdc9ed4 fix: succesfull echo 2025-08-30 14:08:22 +05:30
bbe632bd85 fix: initial connection succesfull 2025-08-30 14:08:21 +05:30
2689040d48 fix: handle short quic headers and compelete connection establishment 2025-08-30 14:08:06 +05:30
8263052f88 fix: peer verification successful 2025-08-30 14:08:06 +05:30
e2fee14bc5 fix: try to fix connection id updation 2025-08-30 14:08:06 +05:30
6633eb01d4 fix: add QUICTLSSecurityConfig for better security config handle 2025-08-30 14:08:06 +05:30
123c86c091 fix: duplication connection creation for same sessions 2025-08-30 14:08:06 +05:30
369f79306f chore: add logs to debug connection 2025-08-30 14:08:06 +05:30
cb6fd27626 fix: process packets received and send to quic 2025-08-30 14:08:06 +05:30
a1d1a07d4c fix: implement missing methods 2025-08-30 14:08:05 +05:30
ac01cc5038 fix: add echo example 2025-08-30 14:07:54 +05:30
94d920f365 chore: fix doc generation for quic transport 2025-08-30 14:07:31 +05:30
45c5f16379 fix: update conn and transport for security 2025-08-30 14:07:31 +05:30
ce76641ef5 temp: impl security modile 2025-08-30 14:07:31 +05:30
bc2ac47594 fix: add basic quic stream and associated tests 2025-08-30 14:07:31 +05:30
a3231af714 fix: add basic tests for listener 2025-08-30 14:07:31 +05:30
54b3055eaa fix: impl quic listener 2025-08-30 14:07:31 +05:30
446a22b0f0 temp: temporty quic impl 2025-08-30 14:07:31 +05:30
96e2149f4d added newsfragment 2025-08-29 18:06:27 +05:30
cb5bfeda39 Use the same comment in maybe_consume_peer_record function 2025-08-29 18:06:27 +05:30
b26e8333bd updated as per the suggestions in #815 2025-08-29 18:06:25 +05:30
d99b67eafa now ignoring pubsub messages upon receving invalid-signed-records 2025-08-29 18:06:09 +05:30
cdfb083c06 added tests to see if transfer works correctly 2025-08-29 18:06:09 +05:30
d4c387f923 add reissuing mechanism of records if addrs dont change as done in #815 2025-08-29 18:06:09 +05:30
56526b4870 signed-peer-record transfer integrated with pubsub rpc message trasfer 2025-08-29 18:05:46 +05:30
8f5dd3bd11 remove excessive use of trio nursery 2025-08-29 17:34:47 +05:30
997094e5b7 resolve linting errors 2025-08-29 12:55:18 +05:30
3c52b859ba improved the error message 2025-08-29 11:30:17 +05:30
426aae7efb Merge branch 'main' into fix_multiselect_negotiate_type 2025-08-29 03:25:12 +05:30
40dad64949 Merge branch 'main' into fix_pubsub_msg_id_type_inconsistency 2025-08-29 03:24:53 +05:30
d6cf83051e Merge branch 'main' into chore01 2025-08-29 03:14:51 +05:30
df39e240e7 Merge branch 'main' into feat/swarm-multi-connection-support 2025-08-29 03:11:08 +05:30
5c11ac20e7 Merge pull request #815 from lla-dane/kad-record
Signed-Peer-Record support in KAD-DHT message transfer mechanism.
2025-08-29 03:09:07 +05:30
9fa3afbb04 fix: format code to pass CI lint 2025-08-28 22:18:33 +01:00
3d1c36419c remove checkpoints, resolve logs, ttl and fix minor issues 2025-08-29 02:05:34 +05:30
c577fd2f71 feat(swarm): enhance swarm with retry backoff 2025-08-28 20:59:36 +01:00
9f80dbae12 added the testcase for StreamFailure 2025-08-27 22:05:19 +05:30
c08007feda improve error message in basic host 2025-08-27 21:54:05 +05:30
c2c4228591 added test for ADD_PROVIDER record processing 2025-08-27 13:02:32 +05:30
943bcc4d36 fix the logic error in add_provider handling 2025-08-27 10:17:40 +05:30
8100a5cd20 removed redudant check in seen seqnos and peers and added test cases of handle iwant and handle ihave 2025-08-26 21:49:12 +05:30
2006b2c92c added newsfragment 2025-08-26 12:59:18 +05:30
fe3f7adc1b fix typos 2025-08-26 12:49:51 +05:30
7b2d637382 Now using env_to_send_in_RPC for issuing records in Identify rpc messages 2025-08-26 12:49:51 +05:30
91bee9df89 Moved env_to_send_in_RPC function to libp2p/peer/peerstore.py 2025-08-26 12:49:51 +05:30
5bf9c7b537 Fix spinx error 2025-08-26 12:49:51 +05:30
8958c0fac3 Moved env_to_send_in_RPC function to libp2p/init.py 2025-08-26 12:49:51 +05:30
091ac082b9 Commented out the bool variable from env_to_send_in_RPC() at places 2025-08-26 12:49:51 +05:30
15f4a399ec Added and docstrings and removed typos 2025-08-26 12:49:51 +05:30
3917d7b596 verify peer_id in signed-record matches authenticated sender 2025-08-26 12:49:51 +05:30
3aacb3a391 remove the timeout bound from the kad-dht test 2025-08-26 12:49:51 +05:30
ba39e91a2e added test for req rejection upon invalid record transfer 2025-08-26 12:49:51 +05:30
57d1c9d807 reject dht-msgs upon receiving invalid records 2025-08-26 12:49:51 +05:30
efc899e872 fix abc.py file 2025-08-26 12:49:51 +05:30
cea1985c5c add reissuing mechanism of records if addrs dont change 2025-08-26 12:49:51 +05:30
702ad4876e remove too much repeatitive code 2025-08-26 12:49:51 +05:30
a21d9e878b recompile protobuf schema and remove typos 2025-08-26 12:49:51 +05:30
5ab68026d6 removed redundant logs 2025-08-26 12:49:51 +05:30
d1792588f9 added tests for signed-peee-record transfer in kad-dht 2025-08-26 12:49:51 +05:30
53db128f69 fix typos 2025-08-26 12:49:51 +05:30
c940dac1e6 simplify bootstrap discovery with optimized timeouts 2025-08-26 01:42:25 +05:30
6214697349 removed redundant imports 2025-08-25 23:01:35 +05:30
fb544d6db2 fixed the merge conflict gossipsub module. 2025-08-25 21:12:45 +05:30
b40d84fc26 Merge remote-tracking branch 'origin/main' into fix_pubsub_msg_id_type_inconsistency 2025-08-25 21:11:55 +05:30
cda50e0ead Merge remote-tracking branch 'origin/main' into fix_multiselect_negotiate_type 2025-08-25 21:07:49 +05:30
3b27b02a8b Merge branch 'main' into issue-798 2025-08-25 16:30:40 +05:30
fe761baa49 Merge branch 'main' into chore01 2025-08-25 16:29:23 +05:30
292bd1a942 Merge pull request #811 from yashksaini-coder/feat/804-add-thin-waist-address
 Feat: add Thin Waist address validation utilities and integrate into echo example
2025-08-25 15:52:36 +05:30
c9795e3138 Merge branch 'main' into feat/804-add-thin-waist-address 2025-08-25 15:52:14 +05:30
3baf886527 Merge branch 'main' into add-ws-transport 2025-08-25 15:31:15 +05:30
b80817b5ae Merge pull request #855 from bomanaps/tests/notifee-coverage
Add listener lifecycle tests
2025-08-25 15:29:22 +05:30
6c6adf7459 chore(app): 804 Suggested changes - Remove the comment 2025-08-25 12:43:18 +05:30
79f3a173f4 renamed newsfragments to internal 2025-08-25 06:09:40 +01:00
f0172a0ba1 Merge remote-tracking branch 'upstream/main' into add-ws-transport 2025-08-25 02:36:35 +02:00
7fb3c2da9f Add newsfragment for PR #855 (PubsubNotifee integration tests) 2025-08-24 23:31:39 +01:00
6b7f50be3d Merge branch 'libp2p:main' into tests/notifee-coverage 2025-08-24 23:03:42 +01:00
6a0a7c21e8 chore(app): Add newsfragment for 811.feature.rst 2025-08-25 01:31:30 +05:30
fde8c8f127 Merge branch 'main' into feat/804-add-thin-waist-address 2025-08-24 23:46:17 +05:30
bc1b1ed6ae fix_gossipsub_mid_type (#859)
* fix_gossipsub_mid_type

* Fix lint and add newsfragments

* Fix mid formation

* Revert "Fix mid formation"

This reverts commit 835d4ca7af58f0716db51a00a8a7aa6cc15ac0a6.
2025-08-24 12:11:36 -06:00
63a8458d45 add import to __init__ 2025-08-24 23:40:05 +05:30
ed91ee0c31 refactor(app): 804 refactored find_free_port() in address_validation.py 2025-08-24 23:28:02 +05:30
75ffb791ac fix: Ensure newline at end of file in address_validation.py and update news fragment formatting 2025-08-24 22:06:07 +05:30
cf48d2e9a4 chore(app): Add 811.internal.rst 2025-08-24 22:03:31 +05:30
88a1f0a390 cherry pick 7a1198c8c6/libp2p/utils/address_validation.py 2025-08-24 21:17:29 +05:30
b38d504fc1 Merge pull request #1 from acul71/fix/multi-address-listening-bug
Fix/multi address listening bug
2025-08-24 13:02:56 +05:30
3bd6d1f579 doc: add newsfragment 2025-08-24 02:29:23 +02:00
b6cbd78943 Fix multi-address listening bug in swarm.listen()
- Fix early return in swarm.listen() that prevented listening on all addresses
- Add comprehensive tests for multi-address listening functionality
- Ensure all available interfaces are properly bound and connectable
2025-08-24 01:49:42 +02:00
ed2716c1bf feat: Enhance echo example to dynamically find free ports and improve address handling
- Added a function to find a free port on localhost.
- Updated the run function to use the new port finding logic when a non-positive port is provided.
- Modified address printing to handle multiple listen addresses correctly.
- Improved the get_available_interfaces function to ensure the IPv4 loopback address is included.
2025-08-22 11:48:37 +05:30
5a2fca32a0 Add ip4 and tcp address resolution and fallback connection attempts 2025-08-22 02:12:42 +05:30
9efc5a1bd1 Merge branch 'libp2p:main' into tests/notifee-coverage 2025-08-21 08:07:53 +01:00
8d9b7f413d Add trio nursery address resolution and connection attempts 2025-08-21 11:20:21 +05:30
5b9bec8e28 fix: Enhance error handling in echo stream handler to manage stream closure and exceptions 2025-08-20 18:29:35 +05:30
c2c91b8c58 refactor: Improve comment formatting in test_echo_thin_waist.py for clarity 2025-08-20 18:05:20 +05:30
8a2d1f7045 Merge branch 'main' into feat/804-add-thin-waist-address 2025-08-20 18:04:45 +05:30
94d695c6bc feat: Implement Random walk in py-libp2p (#822)
* Implementing random walk in py libp2p

* Add documentation for Random Walk module implementation in py-libp2p

* Add Random Walk example for py-libp2p Kademlia DHT

* refactor: peer eviction from routing table stopped

* refactored location of random walk

* add nodesin routing table  from peerstore

* random walk working as expected

* removed extra functions

* Removed all manual triggers

* added newsfragments

* fix linting issues

* refacored logs and cleaned example file

* refactor: update RandomWalk and RTRefreshManager to use query function for peer discovery

* docs: added Random Walk example docs

* added optional argument to use random walk in kademlia DHT

* enabled random walk in example file

* Added tests for RandomWalk module

* fixed lint issues

* Update refresh interval and some more tests are added.

* Removed Random Walk module documentation file

* Extra parentheses have been removed from the random walk logs.

Co-authored-by: Paul Robinson <5199899+pacrob@users.noreply.github.com>

---------

Co-authored-by: Manu Sheel Gupta <manusheel.edu@gmail.com>
Co-authored-by: Paul Robinson <5199899+pacrob@users.noreply.github.com>
2025-08-20 05:10:06 -06:00
905f3a5708 Merge branch 'main' into feat/804-add-thin-waist-address 2025-08-20 09:59:48 +05:30
dabb3a0962 FIXME: Make TProtocol Optional[TProtocol] to keep types consistent (#770)
* FIXME: Make TProtocol Optional[TProtocol] to keep types consistent

* correct test case of test_protocol_muxer

* add newsfragment

* unit test added

---------

Co-authored-by: Manu Sheel Gupta <manusheel.edu@gmail.com>
2025-08-19 19:20:37 -06:00
69d5274891 fix: update listening address parameter in echo example to accept a list 2025-08-19 22:32:26 +05:30
3ff5728209 Merge branch 'feat/804-add-thin-waist-address' of
https://github.com/yashksaini-coder/py-libp2p into feat/804-add-thin-waist-address
2025-08-19 20:47:44 +05:30
a1b16248d3 fix: correct listening address variable in echo example and streamline address printing 2025-08-19 20:47:18 +05:30
55dd8835a7 Merge branch 'main' into feat/804-add-thin-waist-address 2025-08-19 20:02:09 +05:30
e20a9a3814 update maintainer addresses (#856) 2025-08-19 08:29:55 -06:00
7f6469d5d4 Merge remote-tracking branch 'acul71/feat/804-add-thin-waist-address' into feat/804-add-thin-waist-address 2025-08-19 19:56:20 +05:30
ee66958e7f style: fix trailing blank lines in test files 2025-08-19 11:34:40 +01:00
c306400bd9 Add initial listener lifecycle tests; pubsub integration + perf scenarios not yet implemented 2025-08-19 10:49:05 +01:00
5de09ed8a1 Improve relay selection to prioritize active reservations
Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>
2025-08-19 04:53:50 +05:30
05b372b1eb Fix linting and type checking issues for Thin Waist feature 2025-08-19 01:11:48 +02:00
a9f184be6a Merge branch 'main' into issue-798 2025-08-18 22:02:34 +05:30
a9a6ed6767 Merge branch 'main' into fix_pubsub_msg_id_type_inconsistency 2025-08-18 22:02:20 +05:30
f3cf06cd72 Merge branch 'main' into add-ws-transport 2025-08-18 22:00:20 +05:30
fe71c479dc Merge branch 'main' into feat/804-add-thin-waist-address 2025-08-18 21:55:02 +05:30
95e1f62870 Merge pull request #845 from bomanaps/feat/implement-listen_close-mynotifee
Implement listen_close notification and tests
2025-08-18 21:03:12 +05:30
9378490dcb fix: ensure loopback addresses are included in available interfaces 2025-08-18 12:40:38 +05:30
a2fcf33bc1 refactor: migrate echo example test to use Trio for process handling 2025-08-18 12:38:10 +05:30
b363d1d6d0 fix: update listening address handling to use all available interfaces 2025-08-18 12:38:04 +05:30
9a0f224a1c Merge branch 'libp2p:main' into feat/804-add-thin-waist-address 2025-08-18 11:31:28 +05:30
13379e38d8 Merge branch 'libp2p:main' into feat/implement-listen_close-mynotifee 2025-08-17 20:54:59 +01:00
09d2110d65 Remove redundant local import of Multiaddr in close() method 2025-08-17 20:29:35 +01:00
6931092eea Merge branch 'main' into issue-798 2025-08-17 17:02:20 +05:30
cff0bfc17d Merge pull request #846 from sumanjeet0012/bugfix/kbucket_split_fix
Fix: kbucket splitting in routing table.
2025-08-17 17:00:47 +05:30
163cc35cb0 Enhance Bootstrap module to dial peers after address resolution. 2025-08-17 02:12:09 +05:30
a2ad10b1e4 added newsfragments 2025-08-16 18:31:07 +05:30
7c2014087f Merge branch 'libp2p:main' into bugfix/kbucket_split_fix 2025-08-16 13:05:26 +05:30
37df8d679d fix: fixed kbucket splitting behavior in RoutingTable 2025-08-16 11:51:37 +05:30
5c78a41552 Implement closed_stream notification and tests 2025-08-15 16:02:58 +01:00
388302baa7 Added newsfragment 2025-08-15 13:57:21 +05:30
dc04270c19 fix: message id type inonsistency in handle ihave and message id parsing improvement in handle iwant 2025-08-15 13:53:24 +05:30
90f143cd88 update pyproject.toml with current maintainers (#799)
* replace ethereum author with current maintainers

* use my github handle instead of full name

* replace no-prod warning with current status message

* update maintainers blurb

---------

Co-authored-by: Manu Sheel Gupta <manusheel.edu@gmail.com>
2025-08-14 13:13:34 -06:00
1ecff5437c fixed newsfragment filename issue. 2025-08-14 07:29:06 +05:30
aa7276c863 Implement closed_stream event handling and enable related tests (#834)
* Implement closed_stream event handling and enable related tests

* Fix linting issues and ensure all tests pass

* Add logging for exception in SwarmConn and create newsfragment for closed_stream feature
2025-08-13 16:19:53 -06:00
f1872bba93 Merge branch 'main' into add-ws-transport 2025-08-13 01:20:38 +05:30
b838a0e3b6 added none type to return value of negotiate and changed caller handles to handle none. Added newsfragment. 2025-08-12 21:50:10 +05:30
b01596ad92 Revert "Compile release notes for v0.2.10"
This reverts commit 2730db4285.
2025-08-12 07:34:33 -06:00
1565d409e8 Revert "Bump version: 0.2.9 → 0.2.10"
This reverts commit 400ee9b896.
2025-08-12 07:25:21 -06:00
400ee9b896 Bump version: 0.2.9 → 0.2.10 2025-08-12 07:22:43 -06:00
2730db4285 Compile release notes for v0.2.10 2025-08-12 07:21:56 -06:00
9573ab5aea Merge branch 'main' into add-ws-transport 2025-08-12 00:24:57 +05:30
bb896dac2c Merge pull request #818 from varun-r-mallya/varun-r-mallya/protobuf-update
Update protobufs
2025-08-12 00:22:28 +05:30
a5b0db1d8f Merge remote-tracking branch 'origin/add-ws-transport' into add-ws-transport 2025-08-11 01:41:56 +02:00
fe4c17e8d1 Fix typecheck errors and improve WebSocket transport implementation
- Fix INotifee interface compliance in WebSocket demo
- Fix handler function signatures to be async (THandler compatibility)
- Fix is_closed method usage with proper type checking
- Fix pytest.raises multiple exception type issue
- Fix line length violations (E501) across multiple files
- Add debugging logging to Noise security module for troubleshooting
- Update WebSocket transport examples and tests
- Improve transport registry error handling
2025-08-11 01:25:49 +02:00
a14c42ef73 Merge branch 'main' into feat/804-add-thin-waist-address 2025-08-10 21:35:10 +05:30
af61523c87 Merge branch 'main' into varun-r-mallya/protobuf-update 2025-08-10 16:56:10 +05:30
167dfdcac1 Merge branch 'main' into add-ws-transport 2025-08-10 16:55:06 +05:30
d2fdf70692 Merge pull request #819 from lla-dane/remove-todos
Remove completed TODO task comments in Peerstore
2025-08-10 16:54:41 +05:30
09cd8b37ed Merge branch 'main' into feat/804-add-thin-waist-address 2025-08-10 13:13:44 +05:30
1ea50a3cf3 Add newsfragment to indicate updation
Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>
2025-08-10 11:47:30 +05:30
f4247faa51 added newsfragment 2025-08-10 11:39:34 +05:30
92e79bbb3f Merge branch 'main' into varun-r-mallya/protobuf-update 2025-08-10 11:29:59 +05:30
eb3121b818 remove completed TODO task comments 2025-08-10 11:28:11 +05:30
787648177f feat: enhance Makefile to keep protobufs always out of date
Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>
2025-08-10 11:25:06 +05:30
19c1f5ead0 Merge branch 'main' into add-ws-transport 2025-08-10 11:23:41 +05:30
fc9b28910a Merge pull request #817 from acul71/fix/fix-816-remove-TODO-IK-patterns-in-noise
remove deprecated IK pattern TODO
2025-08-10 11:16:44 +05:30
26d0ed2d81 remove deprecated IK pattern TODO 2025-08-09 23:25:23 +00:00
64107b4648 feat: implement WebSocket transport with transport registry system - Add transport_registry.py for centralized transport management - Integrate WebSocket transport with new registry - Add comprehensive test suite for transport registry - Include WebSocket examples and demos - Update transport initialization and swarm integration 2025-08-09 23:52:55 +02:00
618aff9368 chore: recompile protobufs
Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>
2025-08-10 01:31:40 +05:30
a6f85690bf Merge upstream/main into add-ws-transport 2025-08-09 20:18:27 +02:00
32e545d9c7 Merge pull request #813 from sumanjeet0012/feature/update-readme-status
docs: update README.md with module status and source links.
2025-08-09 19:45:47 +05:30
e712e6c0c4 docs: update README.md with module status and source links. 2025-08-09 18:41:44 +05:30
59a898c8ce Add tests for echo example and address validation utilities
- Introduced `test_echo_thin_waist.py` to validate the echo example's output for Thin Waist lines.
- Added `test_address_validation.py` to cover functions for available interfaces, optimal binding addresses, and wildcard address expansion.
- Included parameterized tests and environment checks for IPv6 support.
2025-08-09 01:24:14 +05:30
fa174230ba Refactor echo example to use optimal binding address
- Replaced hardcoded listen address with `get_optimal_binding_address` for improved flexibility.
- Imported address validation utilities in `echo.py` and updated `__init__.py` to include new functions.
2025-08-09 01:22:17 +05:30
b840eaa7e1 Implement advanced network discovery example and address validation utilities
- Added `network_discover.py` to demonstrate Thin Waist address handling.
- Introduced `address_validation.py` with functions for discovering available network interfaces, expanding wildcard addresses, and determining optimal binding addresses.
- Included fallback mechanisms for environments lacking Thin Waist support.
2025-08-09 01:22:03 +05:30
b143c96abd Merge pull request #803 from acul71/fix/fix-592-32bytes_prefix_go_fixme_comment
Remove FIXME about 32 bytes prefix in noise
2025-08-08 12:51:44 +05:30
678b920992 Remove FIXME 2025-08-08 02:01:36 +00:00
cb11f076c8 feat/606-enable-nat-traversal-via-hole-punching (#668)
* feat: base implementation of dcutr for hole-punching

* chore: removed circuit-relay imports from __init__

* feat: implemented dcutr protocol

* added test suite with mock setup

* Fix pre-commit hook issues in DCUtR implementation

* usages of CONNECT_TYPE and SYNC_TYPE have been replaced with HolePunch.Type.CONNECT and HolePunch.Type.SYNC

* added unit tests for dcutr and nat module and

* added multiaddr.get_peer_id() with proper DNS address handling and fixed method signature inconsistencies

* added assertions to verify DCUtR hole punch result in integration test

---------

Co-authored-by: Manu Sheel Gupta <manusheel.edu@gmail.com>
2025-08-07 19:00:16 -06:00
651bf0fc6e Merge branch 'main' into add-ws-transport 2025-07-27 00:05:08 +05:30
9ed44f5fa3 Merge pull request #753 from lla-dane/feat/certified-addrbook
WIP: Certified-Address-Book interface for Peer-Store
2025-07-26 23:41:28 +05:30
8786f06862 added newsfragment 2025-07-26 22:38:28 +05:30
8c96c5a941 Add the periodic peer-store cleanup in all the examples 2025-07-26 22:38:28 +05:30
16445714f7 overwrite old_addr with new_addrs in consume_peer_record 2025-07-26 22:38:28 +05:30
64bc388b33 added peer-store cleanup task in ping example 2025-07-26 22:38:28 +05:30
09e151aafc Added test for peer-store cleanup task 2025-07-26 22:38:28 +05:30
2d335d4394 Integrated Signed-peer-record transfer with identify/identify-push 2025-07-26 22:38:28 +05:30
8b8b051885 batch operations for consume_peer_record 2025-07-26 22:38:28 +05:30
07c8d4cd1f added periodic cleanup task 2025-07-26 22:38:28 +05:30
09e6feea8e merge new addresses with existing ones, in consume_peer_record 2025-07-26 22:38:28 +05:30
601a8a3ef0 enforce_peer_record_limit 2025-07-26 22:38:28 +05:30
9d597012cc fixed the linter <> protobuf issues 2025-07-26 22:38:28 +05:30
8625226be8 fix merge conflicts 2025-07-26 22:38:28 +05:30
c2b1738cd9 fix sphinx/docutils bugs 2025-07-26 22:38:28 +05:30
83acc38281 fix tox bugs 2025-07-26 22:38:28 +05:30
1899dac84c fix tox bugs 2025-07-26 22:38:28 +05:30
aab2a0b603 Completed: CertifiedAddrBook interface with related tests 2025-07-26 22:38:28 +05:30
bab08c0900 added test for Envelope.equal 2025-07-26 22:38:28 +05:30
6431fb8788 Implemented: Envelope wrapper class + linter hacks for protobuf checks 2025-07-26 22:38:28 +05:30
c8053417d5 remove the big google-protobuf file 2025-07-26 22:38:28 +05:30
6eba9d8ca0 downgrade the peer-record protobuf files to v@25.3 2025-07-26 22:38:27 +05:30
0e1b738cbb update protobuf-version 2025-07-26 22:38:27 +05:30
2ff5ae9c90 added hacks for linting errors 2025-07-26 22:38:27 +05:30
ecc443dcfe linter respacing 2025-07-26 22:38:27 +05:30
aa6039bcd3 PeerRecord class with ProtoBuff implemented 2025-07-26 22:38:27 +05:30
7a1aa548a1 Merge branch 'main' into add-ws-transport 2025-07-26 21:17:42 +05:30
8352d19113 Merge pull request #752 from Minimega12121/todo/handletimeout
todo: handle timeout
2025-07-26 21:16:12 +05:30
ceb9f7d3f7 Merge branch 'main' into todo/handletimeout 2025-07-26 20:54:53 +05:30
65faa214da Merge branch 'main' into add-ws-transport 2025-07-26 01:56:53 +05:30
9b667bd472 Merge pull request #711 from sumanjeet0012/feature/bootstrap
feat: Implemented Bootstrap module in py-libp2p
2025-07-26 01:56:16 +05:30
eca548851b added new fragment and tests 2025-07-25 16:19:29 +05:30
e91f458446 Enhance peer discovery logging and address resolution handling in BootstrapDiscovery 2025-07-24 00:11:05 +05:30
53a16d0476 Merge branch 'add-ws-transport' of https://github.com/GautamBytes/py-libp2p into add-ws-transport 2025-07-23 08:24:39 +00:00
1997777c52 Fix IPv6 host bracketing in WebSocket transport 2025-07-23 08:10:51 +00:00
8a21435fc4 Merge branch 'main' into add-ws-transport 2025-07-22 16:29:19 +00:00
0416572457 Merge branch 'main' into feature/bootstrap 2025-07-21 20:55:51 +05:30
39375fb338 Merge branch 'main' into todo/handletimeout 2025-07-21 08:17:08 -07:00
7469238bb4 Merge branch 'main' into add-ws-transport 2025-07-21 08:09:30 -07:00
8bf261ca77 Merge pull request #766 from acul71/py-multiaddr
feat: add py-multiaddr from git
2025-07-21 08:06:04 -07:00
b6c36373a9 Merge branch 'main' into add-ws-transport 2025-07-21 06:44:56 -07:00
3a927c8419 Merge branch 'main' into feature/bootstrap 2025-07-21 06:43:34 -07:00
ec92af20e7 Merge branch 'main' into py-multiaddr 2025-07-21 06:27:11 -07:00
01db5d5fa0 Merge pull request #785 from acul71/feat/issue-784-identify-push-raw-format
feat: Add identify-push raw format support and yamux logging improvent
2025-07-21 06:27:00 -07:00
21ee417793 Pin py-multiaddr dependency to specific git commit db8124e2321f316d3b7d2733c7df11d6ad9c03e6 2025-07-20 23:48:16 +02:00
4fb7132b4e Prevent crash in JS interop test
Signed-off-by: GautamBytes <manchandanigautam@gmail.com>
2025-07-20 19:10:03 +00:00
37e4fee9f8 feat: Add identify-push raw format support and yamux logging improvements
- Add comprehensive integration tests for identify-push protocol
- Support both raw protobuf and varint message formats
- Improve yamux logging integration with LIBP2P_DEBUG
- Fix RawConnError handling to reduce log noise
- Add Ctrl+C handling to identify examples
- Enhance identify-push listener/dialer demo

Fixes: #784
2025-07-20 20:19:18 +02:00
c277cce2ed Merge branch 'main' into feature/bootstrap 2025-07-20 04:39:56 -07:00
fa0b64dca8 Merge branch 'main' into add-ws-transport 2025-07-20 09:44:11 +00:00
227a5c6441 small tweak
Signed-off-by: GautamBytes <manchandanigautam@gmail.com>
2025-07-20 09:30:21 +00:00
187418378a added WebSocket transport support
Signed-off-by: GautamBytes <manchandanigautam@gmail.com>
2025-07-20 09:23:42 +00:00
048e6deb96 fix: utf-8 in reading in py-cid 2025-07-19 23:24:39 +00:00
2dc2dd4670 Merge branch 'main' into py-multiaddr 2025-07-19 02:22:39 -07:00
e6a355d395 Merge pull request #748 from Jineshbansal/add-read-write-lock
TODO: add read/write lock
2025-07-19 02:04:52 -07:00
7b181f3ce5 Merge branch 'main' into add-read-write-lock 2025-07-18 23:53:12 -07:00
0606788ab6 Merge pull request #779 from acul71/fix/issue-778-Incorrect_handling_of_raw_format_in_identify
Fix/issue 778 incorrect handling of raw format in identify
2025-07-18 22:11:58 -07:00
7d62a2f558 Merge branch 'main' into fix/issue-778-Incorrect_handling_of_raw_format_in_identify 2025-07-19 04:25:48 +02:00
26fd169ccc doc: newsfragment raw identify message 2025-07-19 04:25:06 +02:00
99db5b309f fix raw format in identify and tests 2025-07-19 04:11:27 +02:00
7cfe5b9dc7 style: add new line within newsfragment 2025-07-19 01:19:23 +02:00
092b9c0c57 chore(newsfragment): add entry to the release notes 2025-07-19 01:19:23 +02:00
fcf0546831 style: enforce consistent import block 2025-07-19 01:19:23 +02:00
85bad2d0ae replace: attributes with cache cached_property 2025-07-19 01:19:23 +02:00
11560f5cc9 TODO: throttle on async validators (#755)
* fixed todo: throttle on async validators

* added test: validate message respects concurrency limit

* added newsfragment

* added configurable validator semaphore in the PubSub constructor

* added the concurrency-checker in the original test-validate-msg test case

* separate out a _run_async_validator function

* remove redundant run_async_validator
2025-07-18 06:01:28 -06:00
3507531344 chore: clarify newline requirement in newsfragments README.md (#775)
* chore: clarify newline requirement in README

Small change in newsfragments README.md, that reduces some possible room for pull-request tox workflow errors.

* style: remove double backticks for single backticks

the linter strikes again XD.

* docs: clarify trailing newline requirement in newsfragments for lint checks

---------

Co-authored-by: Manu Sheel Gupta <manusheel.edu@gmail.com>
2025-07-17 20:43:00 -06:00
c9162beb2b add grave that were removed by mistake 2025-07-17 20:55:49 +05:30
f587e50cab Merge branch 'main' into todo/handletimeout 2025-07-17 02:36:44 -07:00
d1a0f4f767 Merge branch 'main' into py-multiaddr 2025-07-17 02:32:52 -07:00
3ca27c6e93 Merge pull request #772 from LVivona/replace/peerID/attribute
removal: private attribute for cached_property
2025-07-17 01:58:36 -07:00
b4482e1a5e Merge branch 'main' into replace/peerID/attribute 2025-07-17 01:47:06 -07:00
ae82895d86 style: add new line within newsfragment 2025-07-16 22:12:05 -04:00
9f40d97a05 chore(newsfragment): add entry to the release notes 2025-07-16 22:08:25 -04:00
6fe28dcdd3 Merge branch 'main' into py-multiaddr 2025-07-17 02:39:05 +02:00
41b1ecb67c Merge branch 'main' into feature/bootstrap 2025-07-16 15:00:35 -07:00
e3c9b4bd54 Merge branch 'main' into add-read-write-lock 2025-07-16 14:59:44 -07:00
e132b154e3 Merge branch 'main' into todo/handletimeout 2025-07-16 14:59:24 -07:00
62ea3bbf9a Merge pull request #762 from acul71/identify-fix-varint-go
feat: add length-prefixed support to identify protocol
2025-07-16 14:53:08 -07:00
430527625b Merge branch 'main' into replace/peerID/attribute 2025-07-16 17:15:10 -04:00
4115d033a8 feat: identify identify/push raw-format fix and tests 2025-07-16 20:22:45 +02:00
93fc063e70 Merge branch 'main' into todo/handletimeout 2025-07-16 10:11:23 -07:00
4bd24621f0 Merge branch 'main' into identify-fix-varint-go 2025-07-16 10:10:24 -07:00
5315816521 Merge branch 'main' into add-read-write-lock 2025-07-16 09:38:10 -07:00
d5797572ea Merge pull request #760 from Jineshbansal/improve-error-message
Improve error message
2025-07-16 09:35:20 -07:00
311b750511 add newsfragment file 2025-07-16 20:54:22 +05:30
42f07ae1ab Merge branch 'main' into add-read-write-lock 2025-07-15 14:54:29 -07:00
773962c070 Merge branch 'main' into todo/handletimeout 2025-07-15 14:53:48 -07:00
8ccf58bb83 Merge branch 'main' into improve-error-message 2025-07-15 14:50:28 -07:00
06f0c7d35c Merge branch 'main' into identify-fix-varint-go 2025-07-15 14:50:02 -07:00
ab94e77310 Merge branch 'main' into feature/bootstrap 2025-07-15 14:40:20 -07:00
23622ea1a0 style: enforce consistent import block 2025-07-15 15:28:03 -04:00
6aeb217349 replace: attributes with cache cached_property 2025-07-15 14:59:34 -04:00
003e7bf278 Merge branch 'main' into py-multiaddr 2025-07-15 10:38:01 -07:00
719246c996 Merge pull request #764 from acul71/fix/issue-757-test-peerinfo-valid-cid
fix: added valid CID and fix typecheck
2025-07-15 10:23:51 -07:00
e013e80689 doc: newsfragments 2025-07-15 15:43:48 +00:00
6f33cde9a9 feat: add py-multiaddr from git 2025-07-13 21:56:07 +00:00
9c2560d000 fix: added valid CID and fix typecheck 2025-07-13 21:28:50 +00:00
9f38d48e26 Fix valid bootstrap address in test case 2025-07-14 01:45:12 +05:30
2c1e50428a Merge branch 'feature/bootstrap' of https://github.com/sumanjeet0012/py-libp2p into feature/bootstrap 2025-07-14 01:38:53 +05:30
9e76940e75 Refactor logging configuration to reduce verbosity and improve peer discovery events 2025-07-14 01:38:15 +05:30
53614200bd doc: fix doc issues 2025-07-13 17:43:31 +02:00
41ed0769f6 Merge branch 'main' into identify-fix-varint-go 2025-07-13 17:25:51 +02:00
1c59653946 breaking: identify protocol use now prefix-length messages by default. use use_varint_format param for old raw messages 2025-07-13 17:24:56 +02:00
4bbb08ce2d feat: add length-prefixed protobuf support to identify protocol 2025-07-13 16:13:52 +02:00
912669a924 doc: newsfragment 2025-07-13 16:04:46 +02:00
8ec67289da feat: add length-prefixed protobuf support to identify protocol 2025-07-13 15:55:37 +02:00
9cd3805542 make readwrite more safe 2025-07-13 18:37:44 +05:30
b81168dae9 improve error message 2025-07-13 17:52:05 +05:30
d03bdd75d6 Merge branch 'main' into feature/bootstrap 2025-07-12 07:48:04 -07:00
8ff7bb1f20 Merge branch 'main' into todo/handletimeout 2025-07-12 07:25:08 -07:00
5fcfc677f3 fixme/correct-type (#746)
* fixme/correct-type

* added newsfragment and test
2025-07-11 15:27:17 -06:00
dd14aad47c Add tests for discovery methods in circuit_relay_v2 (#750)
* Add test for direct_connection_relay_discovery

* Add test for mux_method_relay_discovery

* Fix newsfragments
2025-07-11 14:53:27 -06:00
96434d9977 Remove .git 2025-07-10 23:59:26 +02:00
1507100632 Add interoperability test for py-libp2p and js-libp2p with enhanced logging 2025-07-10 23:59:26 +02:00
21db1c3b72 Merge branch 'main' into feature/bootstrap 2025-07-10 20:43:25 +05:30
3592ad308f Merge branch 'main' into add-read-write-lock 2025-07-10 07:52:06 -07:00
9669a92976 Fix formatting and linting issues 2025-07-10 19:25:58 +05:30
2dfee68f20 Refactor bootstrap discovery to use async methods and update bootstrap peers list 2025-07-10 19:24:09 +05:30
505d3b2a8f Bump version: 0.2.8 → 0.2.9 2025-07-09 15:19:54 -06:00
f4eb0158fe Compile release notes for v0.2.9 2025-07-09 15:18:41 -06:00
b716d64184 fix formatting and some naming in newsfragments (#754) 2025-07-09 15:13:16 -06:00
198208aef3 validate and filter bootstrap addresses during discovery initialization 2025-07-09 20:23:47 +05:30
cda163fc48 change ReadWriteLock class 2025-07-09 18:18:37 +05:30
26ed99dafd change tests path 2025-07-09 18:09:07 +05:30
a26fd95854 Merge branch 'feature/bootstrap' of https://github.com/sumanjeet0012/py-libp2p into feature/bootstrap 2025-07-09 01:46:31 +05:30
2965b4e364 DNS resolution working 2025-07-09 01:45:15 +05:30
242998ae9d add test for read-write-lock 2025-07-08 20:06:30 +05:30
5f497c7f5d add file in newsfragments folder 2025-07-08 19:17:43 +05:30
e65e38a3f1 fix: linting error related to read 2025-07-08 19:11:56 +05:30
8fb664bfdf Fix: linting errors 2025-07-08 18:34:30 +05:30
3dcd99a2d1 todo: handle timeout 2025-07-08 17:48:57 +05:30
75abc8b863 run ruff format 2025-07-08 07:35:45 +05:30
91dca97d83 TODO: add read/write lock 2025-07-07 21:55:32 +05:30
80c686ddce Merge branch 'main' into feature/bootstrap 2025-07-07 08:51:53 -07:00
0679efb299 Merge pull request #648 from lla-dane/feat/match-peerstore
WIP: Matching `py-libp2p <-> go-libp2p` PeerStore Implementation
2025-07-06 21:20:00 -07:00
b21591f8d5 remove redundants 2025-07-06 14:45:42 +05:30
d1c31483bd Implemented addr_stream in the peerstore 2025-07-06 14:45:42 +05:30
51c08de1bc test added: clear protocol data 2025-07-06 14:45:42 +05:30
faeacf686a fix typos 2025-07-06 14:45:42 +05:30
9943697054 Added docstrings 2025-07-06 14:45:42 +05:30
ff966bbfa0 Metadata: added test 2025-07-06 14:45:42 +05:30
1b025e552c Key-Book: added tests 2025-07-06 14:45:42 +05:30
4e53327079 Metrics: added tests 2025-07-06 14:45:42 +05:30
3d369bc142 Proto-Book: added tests 2025-07-06 14:45:42 +05:30
5de458482c refactor after rebase 2025-07-06 14:45:42 +05:30
f3d8cbf968 feat: Matching go-libp2p PeerStore implementation 2025-07-06 14:45:42 +05:30
e6f96d32e2 Merge pull request #640 from kaneki003/main
Identifying & resolving race-around conditions in YamuxStream
2025-07-05 14:49:35 -07:00
51313a5909 Merge branch 'main' into main 2025-07-05 14:39:31 -07:00
8d2b889605 Merge pull request #708 from lla-dane/todo/bounded-nursery
Added tests for identify push concurrency cap under high peer load
2025-07-05 14:28:31 -07:00
bfe3dee781 updated newsfragment 2025-07-04 17:32:48 +05:30
a7d122a0f9 added extra tests for identifu push for concurrency cap 2025-07-04 17:28:44 +05:30
8bfd4bde94 created concurrency limit configurable 2025-07-04 16:56:35 +05:30
383d7cb722 added tests 2025-07-04 16:56:20 +05:30
a89ba8ef81 added newsfragment 2025-07-04 16:55:56 +05:30
31b6a6f237 todo/bounded nursery in identify-push 2025-07-04 16:55:55 +05:30
5ac4fc1aba seperated tests for better understanding 2025-07-03 22:20:35 +05:30
f96fe0c1b6 Merge branch 'main' into main 2025-07-03 01:43:12 -07:00
dcb199a6b7 Merge branch 'main' into feature/bootstrap 2025-07-03 01:31:43 -07:00
3403689495 Merge pull request #626 from sukhman-sukh/limit_concurrency
Limit concurrency to push identify message to peers
2025-07-03 01:27:23 -07:00
2ee23fdec1 Fix ci 2025-07-03 11:12:05 +05:30
96c41773ea Add newsfrgment 2025-07-03 10:45:15 +05:30
16be6fab85 Merge branch 'main' into feature/bootstrap 2025-07-03 00:21:19 +05:30
83b42b7850 Merge branch 'main' into limit_concurrency 2025-07-02 10:22:21 -07:00
5a95212697 Merge branch 'main' into main 2025-07-02 10:22:01 -07:00
975ea1bd8e Merge pull request #696 from lla-dane/fix/negotiate_timeout
fix: added negotiate timeout to MuxerMultistream
2025-07-02 10:11:36 -07:00
88db4ceb21 Fix lint 2025-07-02 13:26:52 +05:30
ad0b5505ba make limit configurable in push_identify_to_peers 2025-07-02 13:12:30 +05:30
572c6915f6 added tests for negotiate/response timeout 2025-07-01 23:27:19 +05:30
01319638cd rebase with latest commit 2025-07-01 18:37:46 +05:30
a7eb9b5fbd negotiate timeout configurable in application code 2025-07-01 18:35:17 +05:30
fee4208d89 fix docstrings 2025-07-01 18:33:26 +05:30
4df454ebdc fix docstrings 2025-07-01 18:33:26 +05:30
715e528a56 DEFAULT_NEGOTIATE_TIMEOUT configurable 2025-07-01 18:33:26 +05:30
d0e73f5438 Updated newsfragment 2025-07-01 18:33:26 +05:30
8753024add Updated the timeout wrapper for read/write operations 2025-07-01 18:33:26 +05:30
621ea321ab Set default-negotiate-timeout = 5 sec 2025-07-01 18:33:26 +05:30
6d8f695778 updated multiselect.py and newsfragment 2025-07-01 18:33:26 +05:30
ba231dab79 added newsfragment 2025-07-01 18:33:26 +05:30
83d11db852 fix: added negotiate timeout to MuxerMultistream 2025-07-01 18:33:26 +05:30
cbb1e26a4f refactor fixed some lint issues 2025-06-30 23:19:03 +05:30
69a2cb00ba remove obsolete test script and add comprehensive validation tests for bootstrap addresses 2025-06-30 23:12:48 +05:30
ec20ca81dd remove unnecessary files from .gitignore 2025-06-30 22:46:19 +05:30
cbe2b4bd99 Merge branch 'main' into limit_concurrency 2025-06-30 07:47:12 -07:00
0038ef99d4 Merge branch 'main' into main 2025-06-30 07:43:32 -07:00
b5ec1bd7ee Merge branch 'main' into feature/bootstrap 2025-06-30 07:35:41 -07:00
4c739f6259 Merge pull request #707 from guha-rahul/add_degree
feat: adds degree to connect some
2025-06-30 07:33:48 -07:00
5306bdc8cb add newsfragment 2025-06-30 16:10:21 +05:30
647034221a add edge cases 2025-06-30 16:07:55 +05:30
ddbd190993 docs: added bootstrap docs in doctree 2025-06-30 11:38:06 +05:30
36be4c354b fix: ensure newline at end of file in Bootstrap peer discovery module documentation 2025-06-30 11:38:05 +05:30
befb2d31db added newsfragments 2025-06-30 11:38:05 +05:30
12ad2dcdf4 Added bootstrap module 2025-06-30 11:37:40 +05:30
ee2c979e71 Merge branch 'main' into add_degree 2025-06-29 22:19:43 -07:00
ad87e50eb7 move concurrency_limit to identify_push
Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>
2025-06-30 10:13:02 +05:30
92c9ba7e46 Merge pull request #689 from guha-rahul/write_msg_pubsub
feat: Implement WriteMsg method for efficient RPC message writing
2025-06-29 14:30:26 -07:00
ac51d87046 Merge branch 'main' into write_msg_pubsub 2025-06-29 10:12:16 -07:00
520e555b82 Merge pull request #649 from sumanjeet0012/feature/mDNS
feat: Implement Multicast DNS
2025-06-29 09:55:50 -07:00
6be05639e4 Merge branch 'main' into feature/mDNS 2025-06-29 09:43:03 -07:00
e8e0cf74d1 docs: add mDNS discovery option to new_host function docs 2025-06-29 16:38:52 +05:30
211e951678 fix: improve async validator handling in Pubsub class (#705)
Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>
2025-06-29 12:32:00 +02:00
ef16f3c993 fix: accept new streams for both DATA and WINDOW_UPDATE frames with the SYN flag (#702)
* fix: accept new streams for both  and  frames with the  flag

* doc: newsfragment

---------

Co-authored-by: Manu Sheel Gupta <manusheel.edu@gmail.com>
2025-06-29 10:50:17 +02:00
2201d9e8d2 update link 2025-06-27 13:53:06 +05:30
7ed33e9a55 smol fix , adds degree 2025-06-27 04:41:52 +05:30
4eff928a6d fix: update logging messages 2025-06-27 00:09:17 +05:30
644fc77b6c Merge branch 'main' into limit_concurrency 2025-06-26 07:46:35 -07:00
4947578139 add newsfragment 2025-06-26 19:39:41 +05:30
460f502bb9 Merge branch 'main' into write_msg_pubsub 2025-06-26 19:35:11 +05:30
6a92fa26eb Merge pull request #704 from sukhman-sukh/piggyback-gossipsub
Remove piggyback TODO from gossipsub
2025-06-26 06:53:32 -07:00
e8a484f8e4 Merge branch 'main' into piggyback-gossipsub 2025-06-26 06:44:03 -07:00
74134e9b63 Remove piggyback TODO from gossipsub
Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>
2025-06-26 15:31:16 +05:30
983a4a001c Fix pre-commit checks 2025-06-26 15:26:09 +05:30
1fb3f9c72b Fix failing ci
Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>
2025-06-26 15:26:09 +05:30
8afb99c5b1 add test for counded concurrency
Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>
2025-06-26 15:26:09 +05:30
ae16909f79 Test: Connected Peers Receive Pushes 2025-06-26 15:26:09 +05:30
b7d62c0f85 Limit concurrency to push identify message to peers
Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>
2025-06-26 15:26:09 +05:30
2277d822f1 Merge pull request #690 from mystical-prog/px-backoff
Peer Exchange and Back Off
2025-06-25 22:37:33 -07:00
b736cfa333 Merge branch 'main' into px-backoff 2025-06-25 22:28:41 -07:00
73ebd27c59 added isolated_topics_test and stress_test 2025-06-26 01:28:21 +05:30
c914818f48 fix: enhanced logging to show dependencies logs 2025-06-26 01:15:10 +05:30
5262566f6a fix: check for mDNS attribute before accessing it in BasicHost 2025-06-26 00:36:59 +05:30
f274d20715 feat: attached mdns instance with host 2025-06-25 23:44:32 +05:30
f12ca4e9c1 Merge branch 'main' into write_msg_pubsub 2025-06-24 14:42:48 -07:00
4d8afa6448 Merge branch 'main' into main 2025-06-24 14:34:35 -07:00
5a3adad093 Merge pull request #631 from Winter-Soren/feat/619-store-pubkey-peerid-peerstore
feat: store pubkey and peerid in peerstore
2025-06-24 14:20:22 -07:00
e50f9fc8e5 Merge branch 'libp2p:main' into main 2025-06-24 18:55:10 +05:30
724375e1fa updated doc-string and reverted mplex-changes 2025-06-24 18:05:15 +05:30
28d0e5759a removed redundant function and added try catch block 2025-06-24 14:25:47 +05:30
9adf9aa499 refactor: improve test structure in mDNS tests 2025-06-24 14:25:47 +05:30
dcc8bbb619 feat: add unit and integration tests for mDNS. 2025-06-24 14:25:46 +05:30
b258ff3ea2 fix: correct logger name typo and update protocol in peer info extraction 2025-06-24 14:25:46 +05:30
31b694aa29 fix: ensure newline at end of file in newsfragments/649.feature.rst 2025-06-24 14:25:45 +05:30
293087bd06 feat: added newsfragment for mDNS 2025-06-24 14:25:45 +05:30
35248f8167 fix: ensure newline at end of file in libp2p.discovery.events and libp2p.discovery.mdns documentation 2025-06-24 14:25:44 +05:30
e018af09ae feat: add documentation for libp2p.discovery.events and libp2p.discovery.mdns packages 2025-06-24 14:25:44 +05:30
7135e6cd4d fix: ensure newline at end of file in libp2p.discovery documentation 2025-06-24 14:25:43 +05:30
77a9788a69 feat: add initial documentation for libp2p.discovery package 2025-06-24 14:25:43 +05:30
555e389109 fix: correct heading formatting in mDNS example documentation 2025-06-24 14:25:42 +05:30
8f0762f95c fix: remove unnecessary blank lines in mDNS example documentation 2025-06-24 14:25:42 +05:30
67bcad1674 Refactored mDNS example and added script for example 2025-06-24 14:25:41 +05:30
3b53120092 fixed some errors during rebase 2025-06-24 14:25:40 +05:30
89ed86d903 feat: add logging for mDNS peer discovery and update dependencies 2025-06-24 14:25:40 +05:30
387f4879d1 fix lint 2025-06-24 14:25:39 +05:30
e2f95f4df3 feat: emitted event from demo file 2025-06-24 14:25:39 +05:30
f43e7e367a refactored code 2025-06-24 14:25:38 +05:30
3262749db7 added event emmiter 2025-06-24 14:25:38 +05:30
cd7eaba4a4 feat: implement mDNS discovery with PeerListener 2025-06-24 14:25:37 +05:30
6add1cb685 feat: implement broadcasting in mdns 2025-06-24 14:25:37 +05:30
742bc7bca3 feat: add stringGen function to generate random strings 2025-06-24 14:25:36 +05:30
cbd4f9b502 feat: init mDNS discovery module 2025-06-24 14:25:35 +05:30
4e2be87c73 Merge pull request #695 from LVivona/patch-1
chore(kad_dht): centralize shared values in common.py file
2025-06-23 08:55:21 -07:00
fbee0ba2ab added newsfragment 2025-06-23 01:00:46 +05:30
ea6eef6ed5 test px and backoff 2025-06-23 00:41:13 +05:30
fd818d9102 test: added tests to ensure handshake adds pubkey to existing peer ID without one; peerstore unchanged on ID mismatch 2025-06-22 16:01:02 +05:30
2c0a6c0adb Merge branch 'main' into feat/619-store-pubkey-peerid-peerstore 2025-06-22 15:26:51 +05:30
3a4338e1df chore: eliminate self.protocol_id attribute \w in PeerRouting 2025-06-22 00:25:48 -04:00
feb8db6655 style: enforce multiline import style 2025-06-22 00:15:44 -04:00
ebdde7b5aa style: enforce multiline import style for consistency 2025-06-21 15:08:11 -04:00
24e73207d2 fixed failing demo
Co-authored-by: Khwahish Patel <khwahish.p1@ahduni.edu.in>
2025-06-21 18:54:17 +05:30
303bf3060a implemented peer exchange 2025-06-21 18:54:17 +05:30
788b4cf51a added complete back_off implementation 2025-06-21 18:54:17 +05:30
b78468ca32 added params for peer exchange and back off 2025-06-21 18:54:17 +05:30
c48618825d updated protobuf for prune message 2025-06-21 18:54:16 +05:30
d7cdae8a0f intgrated n==-1 case in read() 2025-06-21 17:51:27 +05:30
df17788ec3 resolving build-fails 2025-06-21 14:10:09 +05:30
209deffc8a resolved recv_window updates,added support for read_EOF 2025-06-21 13:40:12 +05:30
0a7e13f0ed Merge branch 'libp2p:main' into main 2025-06-21 13:39:38 +05:30
811c217ee6 style: isort fix ording of imports 2025-06-20 16:01:11 -04:00
d03ca45bd8 style: fix flake8 linting errors 2025-06-20 11:57:50 -04:00
8bddbfb9bb Merge branch 'main' into write_msg_pubsub 2025-06-20 07:29:56 -07:00
79ac01308c remove: unused custom_types TProtocol import 2025-06-19 21:38:02 -04:00
dfc0bb4ec8 chore(kad_dht): centralize shared values in common.py 2025-06-19 21:24:39 -04:00
09b4c846a4 feat: add support for sparse connect (#680)
* init

* add newsfragment

* fix
2025-06-19 06:18:45 -06:00
66bd027161 Feat/587-circuit-relay (#611)
* feat: implemented setup of circuit relay and test cases

* chore: remove test files to be rewritten

* added 1 test suite for protocol

* added 1 test suite for discovery

* fixed protocol timeouts and message types to handle reservations and stream operations.

* Resolved merge conflict in libp2p/tools/utils.py by combining timeout approach with retry mechanism

* fix: linting issues

* docs: updated documentation with circuit-relay

* chore: added enums, improved typing, security and examples

* fix: created proper __init__ file to ensure importability

* fix: replace transport_opt with listen_addrs in examples, fixed typing and improved code

* fix type checking issues across relay module and test suite

* regenerated circuit_pb2 file protobuf version 3

* fixed circuit relay example and moved imports to top in test_security_multistream

* chore: moved imports to the top

* chore: fixed linting of test_circuit_v2_transport.py

---------

Co-authored-by: Manu Sheel Gupta <manusheel.edu@gmail.com>
2025-06-18 15:39:39 -06:00
8c16b316ac added newsfragement and tests that would fail without these changes but pass with them 2025-06-18 23:32:48 +05:30
d4ed859b19 Merge branch 'main' into feat/619-store-pubkey-peerid-peerstore 2025-06-18 22:45:33 +05:30
79094d70d3 Optimize pubsub publishing to support multiple topics in single RPC message (#686)
* init

* add newsfragment

* lint

---------

Co-authored-by: Manu Sheel Gupta <manusheel.edu@gmail.com>
2025-06-17 15:23:03 -06:00
2ed2587fc9 fix: removed dummy ID(b) from upgrade_security for inbound connections (#681)
* fix: removed dummy ID(b) from upgrade_security for inbound connections

* added newsfragment

* updated newsfragment
2025-06-17 06:25:50 -06:00
d61bca78ab Kademlia DHT implementation in py-libp2p (#579)
* initialise the module

* added content routing

* added routing module

* added peer routing

* added value store

* added utilities functions

* added main kademlia file

* fixed create_key_from_binary function

* example to test kademlia dht

* added protocol ID and enhanced logging for peer store size in provider and consumer nodes

* refactor: specify stream type in handle_stream method and add peer in routing table

* removed content routing

* added default value of count for finding closest peers

* added functions to find close peers

* refactor: remove content routing and enhance peer discovery

* added put value function

* added get value function

* fix: improve logging and handle key encoding in get_value method

* refactor: remove ContentRouting import from __init__.py

* refactor: improved basic kademlia example

* added protobuf files

* replaced json with protobuf

* refactor: enhance peer discovery and routing logic in KadDHT

* refactor: enhance Kademlia routing table to use PeerInfo objects and improve peer management

* refactor: enhance peer addition logic to utilize PeerInfo objects in routing table

* feat: implement content provider functionality in Kademlia DHT

* refactor: update value store to use datetime for validity management

* refactor: update RoutingTable initialization to include host reference

* refactor: enhance KBucket and RoutingTable for improved peer management and functionality

* refactor: streamline peer discovery and value storage methods in KadDHT

* refactor: update KadDHT and related classes for async peer management and enhanced value storage

* refactor: enhance ProviderStore initialization and improve peer routing integration

* test: add tests for Kademlia DHT functionality

* fix linting issues

* pydocstyle issues fixed

* CICD pipeline issues solved

* fix: update docstring format for find_peer method

* refactor: improve logging and remove unused code in DHT implementation

* refactor: clean up logging and remove unused imports in DHT and test files

* Refactor logging setup and improve DHT stream handling with varint length prefixes

* Update bootstrap peer handling in basic_dht example and refactor peer routing to accept string addresses

* Enhance peer querying in Kademlia DHT by implementing parallel queries using Trio.

* Enhance peer querying by adding deduplication checks

* Refactor DHT implementation to use varint for length prefixes and enhance logging for better traceability

* Add base58 encoding for value storage and enhance logging in basic_dht example

* Refactor Kademlia DHT to support server/client modes

* Added unit tests

* Refactor documentation to fixsome warning

* Add unit tests and remove outdated tests

* Fixed precommit errora

* Refactor error handling test to raise StringParseError for invalid bootstrap addresses

* Add libp2p.kad_dht to the list of subpackages in documentation

* Fix expiration and republish checks to use inclusive comparison

* Add __init__.py file to libp2p.kad_dht.pb package

* Refactor get value and put value to run in parallel with query timeout

* Refactor provider message handling to use parallel processing with timeout

* Add methods for provider store in KadDHT class

* Refactor KadDHT and ProviderStore methods to improve type hints and enhance parallel processing

* Add documentation for libp2p.kad_dht.pb module.

* Update documentation for libp2p.kad_dht package to include subpackages and correct formatting

* Fix formatting in documentation for libp2p.kad_dht package by correcting the subpackage reference

* Fix header formatting in libp2p.kad_dht.pb documentation

* Change log level from info to debug for various logging statements.

* fix CICD issues (post revamp)

* fixed value store unit test

* Refactored kademlia example

* Refactor Kademlia example: enhance logging, improve bootstrap node connection, and streamline server address handling

* removed bootstrap module

* Refactor Kademlia DHT example and core modules: enhance logging, remove unused code, and improve peer handling

* Added docs of kad dht example

* Update server address log file path to use the script's directory

* Refactor: Introduce DHTMode enum for clearer mode management

* moved xor_distance function to utils.py

* Enhance logging in ValueStore and KadDHT: include decoded value in debug logs and update parameter description for validity

* Add handling for closest peers in GET_VALUE response when value is not found

* Handled failure scenario for PUT_VALUE

* Remove kademlia demo from project scripts and contributing documentation

* spelling and logging

---------

Co-authored-by: pacrob <5199899+pacrob@users.noreply.github.com>
2025-06-16 14:46:40 -06:00
a3492cf82f Merge branch 'main' into feat/619-store-pubkey-peerid-peerstore 2025-06-16 22:06:47 +05:30
733ef86e62 refactor(gossipsub.py): Add helper function to fanout and gossipsub (#678)
* fanout and gossibsub helper

* add newsfragment

* remove dub fanout check
2025-06-16 07:23:31 -06:00
0caf8647c5 Merge pull request #684 from guha-rahul/use_decapsulate
fix: replace complex logic with decapsulate
2025-06-16 04:37:14 -07:00
c33ab32c33 init 2025-06-16 02:50:40 +05:30
193e8f9cb8 add newsfragment 2025-06-15 19:58:52 +05:30
10b39dad1c replace complex logic with decapsulate 2025-06-15 19:51:55 +05:30
2248108b54 fixed types and improved code 2025-06-11 23:51:57 +05:30
a762df6042 Merge branch 'main' into feat/619-store-pubkey-peerid-peerstore 2025-06-11 23:34:12 +05:30
01b9e89e83 Merge branch 'main' into main 2025-06-11 19:39:06 +05:30
d2825af045 fix(examples/echo/echo.py): Add max message length to stream.read (#671)
Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>
2025-06-10 11:44:07 -06:00
0f483dd744 Bump version: 0.2.7 → 0.2.8 2025-06-10 11:31:46 -06:00
0197b515c1 Compile release notes for v0.2.8 2025-06-10 11:31:25 -06:00
f27f4ddd85 remove references to removed setup.py (#674) 2025-06-10 11:24:34 -06:00
7e377ede36 Merge branch 'main' into feat/619-store-pubkey-peerid-peerstore 2025-06-10 21:12:28 +05:30
d733b78dba Merge branch 'libp2p:main' into main 2025-06-10 20:17:55 +05:30
e397ce25a6 Updated Yamux impl.,added tests for yamux and mplex 2025-06-10 20:12:19 +05:30
630aac703d add make pr (#672) 2025-06-10 08:34:22 -06:00
286752c517 Merge pull request #658 from AkMo3/main
fix: add connection state for net stream and gracefully handle failure
2025-06-10 01:03:18 +05:30
390ac2eb26 Merge branch 'main' into main 2025-06-10 00:53:59 +05:30
13d730ae5c fix: improve types according to new typecheck 2025-06-09 19:10:15 +00:00
4e9fa87477 Updated examples to automatically use random port (#661)
* updated examples to automatically use random port

* Refactor examples to use shared utils for port selection (#1)

---------

Co-authored-by: acul71 <34693171+acul71@users.noreply.github.com>
2025-06-09 12:59:11 -06:00
47ae20d29c fix: run pytests parallely in CI and makefile 2025-06-09 18:58:51 +00:00
f7757fa726 docs: add documentation and examples for new NetStream state management 2025-06-09 18:58:17 +00:00
5bc4d01eea fix: add connection states for net stream
Other changes:
1. Add operation validation based on states
2. Gracefully handle exceptions and cleanup
2025-06-09 18:58:17 +00:00
c83fc1582d build(deps): bump fastecdsa from 1.7.5 to 2.3.2 (#669)
Bumps [fastecdsa](https://github.com/AntonKueltz/fastecdsa) from 1.7.5 to 2.3.2.
- [Release notes](https://github.com/AntonKueltz/fastecdsa/releases)
- [Changelog](https://github.com/AntonKueltz/fastecdsa/blob/main/CHANGELOG.md)
- [Commits](https://github.com/AntonKueltz/fastecdsa/compare/v1.7.5...v2.3.2)

---
updated-dependencies:
- dependency-name: fastecdsa
  dependency-version: 2.3.2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-09 12:44:45 -06:00
22d93b39ae Add ttl for peer data expiration (#655)
* Add ttl and last_identified to peerdata

* Add test for ttl

Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>

* Fix lint and add newsfragments

Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>

* Fix failing ci

Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>

* fix ttl time from 600 to 120

Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>

* fix test ttl timeout and lint errors

Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>

* Fix docstrings

Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>

* rebase main

* remove print statement

---------

Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>
Co-authored-by: pacrob <5199899+pacrob@users.noreply.github.com>
2025-06-09 12:42:59 -06:00
bdadec7519 ft. modernise py-libp2p (#618)
* fix pyproject.toml , add ruff

* rm lock

* make progress

* add poetry lock ignore

* fix type issues

* fix tcp type errors

* fix text example - type error - wrong args

* add setuptools to dev

* test ci

* fix docs build

* fix type issues for new_swarm & new_host

* fix types in gossipsub

* fix type issues in noise

* wip: factories

* revert factories

* fix more type issues

* more type fixes

* fix: add null checks for noise protocol initialization and key handling

* corrected argument-errors in peerId and Multiaddr in peer tests

* fix: Noice - remove redundant type casts in BaseNoiseMsgReadWriter

* fix: update test_notify.py to use SwarmFactory.create_batch_and_listen, fix type hints, and comment out ClosedStream assertions

* Fix type checks for pubsub module

Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>

* Fix type checks for pubsub module-tests

Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>

* noise: add checks for uninitialized protocol and key states in PatternXX

Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>

* pubsub: add None checks for optional fields in FloodSub and Pubsub

Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>

* Fix type hints and improve testing

Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>

* remove redundant checks

Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>

* fix build issues

* add optional to trio service

* fix types

* fix type errors

* Fix type errors

Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>

* fixed more-type checks in crypto and peer_data files

* wip: factories

* replaced union with optional

* fix: type-error in interp-utils and peerinfo

* replace pyright with pyrefly

* add pyrefly.toml

* wip: fix multiselect issues

* try typecheck

* base check

* mcache test fixes , typecheck ci update

* fix ci

* will this work

* minor fix

* use poetry

* fix wokflow

* use cache,fix err

* fix pyrefly.toml

* fix pyrefly.toml

* fix cache in ci

* deploy commit

* add main baseline

* update to v5

* improve typecheck ci (#14)

* fix typo

* remove holepunching code (#16)

* fix gossipsub typeerrors (#17)

* fix: ensure initiator user includes remote peer id in handshake (#15)

* fix ci (#19)

* typefix: custom_types | core/peerinfo/test_peer_info | io/abc | pubsub/floodsub | protocol_muxer/multiselect (#18)

* fix: Typefixes in PeerInfo  (#21)

* fix minor type issue (#22)

* fix type errors in pubsub (#24)

* fix: Minor typefixes in tests (#23)

* Fix failing tests for type-fixed test/pubsub (#8)

* move pyrefly & ruff to pyproject.toml & rm .project-template (#28)

* move the async_context file to tests/core

* move crypto test to crypto folder

* fix: some typefixes (#25)

* fix type errors

* fix type issues

* fix: update gRPC API usage in autonat_pb2_grpc.py (#31)

* md: typecheck ci

* rm comments

* clean up : from review suggestions

* use | None over Optional as per new python standards

* drop supporto for py3.9

* newsfragments

---------

Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>
Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>
Co-authored-by: acul71 <luca.pisani@birdo.net>
Co-authored-by: kaneki003 <sakshamchauhan707@gmail.com>
Co-authored-by: sukhman <sukhmansinghsaluja@gmail.com>
Co-authored-by: varun-r-mallya <varunrmallya@gmail.com>
Co-authored-by: varunrmallya <100590632+varun-r-mallya@users.noreply.github.com>
Co-authored-by: lla-dane <abhinavagarwalla6@gmail.com>
Co-authored-by: Collins <ArtemisfowlX@protonmail.com>
Co-authored-by: Abhinav Agarwalla <120122716+lla-dane@users.noreply.github.com>
Co-authored-by: guha-rahul <52607971+guha-rahul@users.noreply.github.com>
Co-authored-by: Sukhman Singh <63765293+sukhman-sukh@users.noreply.github.com>
Co-authored-by: acul71 <34693171+acul71@users.noreply.github.com>
Co-authored-by: pacrob <5199899+pacrob@users.noreply.github.com>
2025-06-09 11:39:59 -06:00
d020bbc066 Add time_since_last_publish (#642)
Added `time_since_last_publish` field to gossipsub. Took reference from
https://github.com/libp2p/go-libp2p-pubsub/blob/master/gossipsub.go#L1224

Issue https://github.com/libp2p/py-libp2p/issues/636

## How was it fixed?
whenever someone publishes message to a topic or set of topics,
`time_since_last_publish` gets updated and whenever we clear fanout
peers or time exceeds ttl, we clear `time_since_last_publish` from dict.

### To-Do

Creating draft PR for now. Tests and type-binding is left for this
issue.
#### Cute Animal Picture

![put a cute animal picture link inside the
parentheses](https://i.etsystatic.com/27171676/r/il/eedb08/5303109239/il_570xN.5303109239_4o61.jpg)
2025-06-09 00:53:36 +05:30
00f10dbec3 Merge branch 'main' into add-last-publish 2025-06-08 19:19:30 +05:30
d75886b180 renamed newsfragment file causing docs ci failure 2025-06-06 17:55:40 +05:30
5ca6f26933 feat: Add blacklisting of peers (#651)
* init

* remove blacklist validation after hello packet

* add docs and newsfragment
2025-06-05 09:10:04 -06:00
a3c9ac61e6 Improve performance of read from daemon test (#646)
Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>
2025-06-05 07:25:59 -06:00
d4785b9e26 Add newsfragments to the PR
Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>
2025-06-05 14:00:23 +05:30
cef217358f fixed fanout_heartbeat bug and gossipsub join test 2025-06-05 13:39:07 +05:30
338672214c Add test for time_since_last_publish
Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>
2025-06-04 14:15:07 +05:30
c2046e6aa4 Add time_since_last_publish 2025-06-01 01:47:47 +05:30
30b5811d39 feat: store pubkey and peerid in peerstore 2025-05-29 20:07:48 +05:30
385 changed files with 48093 additions and 2826 deletions

View File

@ -16,10 +16,10 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python: ['3.9', '3.10', '3.11', '3.12', '3.13']
python: ["3.10", "3.11", "3.12", "3.13"]
toxenv: [core, interop, lint, wheel, demos]
include:
- python: '3.10'
- python: "3.10"
toxenv: docs
fail-fast: false
steps:
@ -36,17 +36,55 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- name: Install Nim for interop testing
if: matrix.toxenv == 'interop'
run: |
echo "Installing Nim for nim-libp2p interop testing..."
curl -sSf https://nim-lang.org/choosenim/init.sh | sh -s -- -y --firstInstall
echo "$HOME/.nimble/bin" >> $GITHUB_PATH
echo "$HOME/.choosenim/toolchains/nim-stable/bin" >> $GITHUB_PATH
- name: Cache nimble packages
if: matrix.toxenv == 'interop'
uses: actions/cache@v4
with:
path: |
~/.nimble
~/.choosenim/toolchains/*/lib
key: ${{ runner.os }}-nimble-${{ hashFiles('**/nim_echo_server.nim') }}
restore-keys: |
${{ runner.os }}-nimble-
- name: Build nim interop binaries
if: matrix.toxenv == 'interop'
run: |
export PATH="$HOME/.nimble/bin:$HOME/.choosenim/toolchains/nim-stable/bin:$PATH"
cd tests/interop/nim_libp2p
./scripts/setup_nim_echo.sh
- run: |
python -m pip install --upgrade pip
python -m pip install tox
- run: |
- name: Run Tests or Generate Docs
run: |
if [[ "${{ matrix.toxenv }}" == 'docs' ]]; then
export TOXENV=docs
else
export TOXENV=py${{ matrix.python }}-${{ matrix.toxenv }}
fi
# Set PATH for nim commands during tox
if [[ "${{ matrix.toxenv }}" == 'interop' ]]; then
export PATH="$HOME/.nimble/bin:$HOME/.choosenim/toolchains/nim-stable/bin:$PATH"
fi
python -m tox run -r
windows:
runs-on: windows-latest
strategy:
matrix:
python-version: ['3.11', '3.12', '3.13']
python-version: ["3.11", "3.12", "3.13"]
toxenv: [core, wheel]
fail-fast: false
steps:
@ -65,5 +103,5 @@ jobs:
if [[ "${{ matrix.toxenv }}" == "wheel" ]]; then
python -m tox run -e windows-wheel
else
python -m tox run -e py311-${{ matrix.toxenv }}
python -m tox run -e py${{ matrix.python-version }}-${{ matrix.toxenv }}
fi

14
.gitignore vendored
View File

@ -146,6 +146,9 @@ instance/
# PyBuilder
target/
# PyRight Config
pyrightconfig.json
# Jupyter Notebook
.ipynb_checkpoints
@ -171,3 +174,14 @@ env.bak/
# mkdocs documentation
/site
#lockfiles
uv.lock
poetry.lock
tests/interop/js_libp2p/js_node/node_modules/
tests/interop/js_libp2p/js_node/package-lock.json
tests/interop/js_libp2p/js_node/src/node_modules/
tests/interop/js_libp2p/js_node/src/package-lock.json
# Sphinx documentation build
_build/

View File

@ -1,59 +1,49 @@
exclude: '.project-template|docs/conf.py|.*pb2\..*'
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: check-yaml
- id: check-toml
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/asottile/pyupgrade
rev: v3.15.0
- id: check-yaml
- id: check-toml
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/asottile/pyupgrade
rev: v3.20.0
hooks:
- id: pyupgrade
args: [--py39-plus]
- repo: https://github.com/psf/black
rev: 23.9.1
- id: pyupgrade
args: [--py310-plus]
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.11.10
hooks:
- id: black
- repo: https://github.com/PyCQA/flake8
rev: 6.1.0
hooks:
- id: flake8
additional_dependencies:
- flake8-bugbear==23.9.16
exclude: setup.py
- repo: https://github.com/PyCQA/autoflake
rev: v2.2.1
hooks:
- id: autoflake
- repo: https://github.com/pycqa/isort
rev: 5.12.0
hooks:
- id: isort
- repo: https://github.com/pycqa/pydocstyle
rev: 6.3.0
hooks:
- id: pydocstyle
additional_dependencies:
- tomli # required until >= python311
- repo: https://github.com/executablebooks/mdformat
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
- id: ruff-format
- repo: https://github.com/executablebooks/mdformat
rev: 0.7.22
hooks:
- id: mdformat
- id: mdformat
additional_dependencies:
- mdformat-gfm
- repo: local
- mdformat-gfm
- repo: local
hooks:
- id: mypy-local
- id: mypy-local
name: run mypy with all dev dependencies present
entry: python -m mypy -p libp2p
entry: mypy -p libp2p
language: system
always_run: true
pass_filenames: false
- repo: local
- repo: local
hooks:
- id: check-rst-files
- id: pyrefly-local
name: run pyrefly typecheck locally
entry: pyrefly check
language: system
always_run: true
pass_filenames: false
- repo: local
hooks:
- id: check-rst-files
name: Check for .rst files in the top-level directory
entry: python -c "import glob, sys; rst_files = glob.glob('*.rst'); sys.exit(1) if rst_files else sys.exit(0)"
language: system

View File

@ -1,71 +0,0 @@
#!/usr/bin/env python3
import os
import sys
import re
from pathlib import Path
def _find_files(project_root):
path_exclude_pattern = r"\.git($|\/)|venv|_build"
file_exclude_pattern = r"fill_template_vars\.py|\.swp$"
filepaths = []
for dir_path, _dir_names, file_names in os.walk(project_root):
if not re.search(path_exclude_pattern, dir_path):
for file in file_names:
if not re.search(file_exclude_pattern, file):
filepaths.append(str(Path(dir_path, file)))
return filepaths
def _replace(pattern, replacement, project_root):
print(f"Replacing values: {pattern}")
for file in _find_files(project_root):
try:
with open(file) as f:
content = f.read()
content = re.sub(pattern, replacement, content)
with open(file, "w") as f:
f.write(content)
except UnicodeDecodeError:
pass
def main():
project_root = Path(os.path.realpath(sys.argv[0])).parent.parent
module_name = input("What is your python module name? ")
pypi_input = input(f"What is your pypi package name? (default: {module_name}) ")
pypi_name = pypi_input or module_name
repo_input = input(f"What is your github project name? (default: {pypi_name}) ")
repo_name = repo_input or pypi_name
rtd_input = input(
f"What is your readthedocs.org project name? (default: {pypi_name}) "
)
rtd_name = rtd_input or pypi_name
project_input = input(
f"What is your project name (ex: at the top of the README)? (default: {repo_name}) "
)
project_name = project_input or repo_name
short_description = input("What is a one-liner describing the project? ")
_replace("<MODULE_NAME>", module_name, project_root)
_replace("<PYPI_NAME>", pypi_name, project_root)
_replace("<REPO_NAME>", repo_name, project_root)
_replace("<RTD_NAME>", rtd_name, project_root)
_replace("<PROJECT_NAME>", project_name, project_root)
_replace("<SHORT_DESCRIPTION>", short_description, project_root)
os.makedirs(project_root / module_name, exist_ok=True)
Path(project_root / module_name / "__init__.py").touch()
Path(project_root / module_name / "py.typed").touch()
if __name__ == "__main__":
main()

View File

@ -1,39 +0,0 @@
#!/usr/bin/env python3
import os
import sys
from pathlib import Path
import subprocess
def main():
template_dir = Path(os.path.dirname(sys.argv[0]))
template_vars_file = template_dir / "template_vars.txt"
fill_template_vars_script = template_dir / "fill_template_vars.py"
with open(template_vars_file, "r") as input_file:
content_lines = input_file.readlines()
process = subprocess.Popen(
[sys.executable, str(fill_template_vars_script)],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
for line in content_lines:
process.stdin.write(line)
process.stdin.flush()
stdout, stderr = process.communicate()
if process.returncode != 0:
print(f"Error occurred: {stderr}")
sys.exit(1)
print(stdout)
if __name__ == "__main__":
main()

View File

@ -1,6 +0,0 @@
libp2p
libp2p
py-libp2p
py-libp2p
py-libp2p
The Python implementation of the libp2p networking stack

View File

@ -7,12 +7,14 @@ help:
@echo "clean-pyc - remove Python file artifacts"
@echo "clean - run clean-build and clean-pyc"
@echo "dist - build package and cat contents of the dist directory"
@echo "fix - fix formatting & linting issues with ruff"
@echo "lint - fix linting issues with pre-commit"
@echo "test - run tests quickly with the default Python"
@echo "docs - generate docs and open in browser (linux-docs for version on linux)"
@echo "package-test - build package and install it in a venv for manual testing"
@echo "notes - consume towncrier newsfragments and update release notes in docs - requires bump to be set"
@echo "release - package and upload a release (does not run notes target) - requires bump to be set"
@echo "pr - run clean, fix, lint, typecheck, and test i.e basically everything you need to do before creating a PR"
clean-build:
rm -fr build/
@ -37,8 +39,16 @@ lint:
&& pre-commit run --all-files --show-diff-on-failure \
)
fix:
python -m ruff check --fix
typecheck:
pre-commit run mypy-local --all-files && pre-commit run pyrefly-local --all-files
test:
python -m pytest tests
python -m pytest tests -n auto
pr: clean fix lint typecheck test
# protobufs management
@ -48,13 +58,19 @@ PB = libp2p/crypto/pb/crypto.proto \
libp2p/security/secio/pb/spipe.proto \
libp2p/security/noise/pb/noise.proto \
libp2p/identity/identify/pb/identify.proto \
libp2p/host/autonat/pb/autonat.proto
libp2p/host/autonat/pb/autonat.proto \
libp2p/relay/circuit_v2/pb/circuit.proto \
libp2p/relay/circuit_v2/pb/dcutr.proto \
libp2p/kad_dht/pb/kademlia.proto
PY = $(PB:.proto=_pb2.py)
PYI = $(PB:.proto=_pb2.pyi)
## Set default to `protobufs`, otherwise `format` is called when typing only `make`
all: protobufs
.PHONY: protobufs clean-proto
protobufs: $(PY)
%_pb2.py: %.proto
@ -63,6 +79,11 @@ protobufs: $(PY)
clean-proto:
rm -f $(PY) $(PYI)
# Force protobuf regeneration by making them always out of date
$(PY): FORCE
FORCE:
# docs commands
docs: check-docs
@ -80,7 +101,7 @@ validate-newsfragments:
check-docs: build-docs validate-newsfragments
build-docs:
sphinx-apidoc -o docs/ . setup.py "*conftest*" tests/
sphinx-apidoc -o docs/ . "*conftest*" tests/
$(MAKE) -C docs clean
$(MAKE) -C docs html
$(MAKE) -C docs doctest

View File

@ -12,13 +12,13 @@
[![Build Status](https://img.shields.io/github/actions/workflow/status/libp2p/py-libp2p/tox.yml?branch=main&label=build%20status)](https://github.com/libp2p/py-libp2p/actions/workflows/tox.yml)
[![Docs build](https://readthedocs.org/projects/py-libp2p/badge/?version=latest)](http://py-libp2p.readthedocs.io/en/latest/?badge=latest)
> ⚠️ **Warning:** py-libp2p is an experimental and work-in-progress repo under development. We do not yet recommend using py-libp2p in production environments.
> py-libp2p has moved beyond its experimental roots and is steadily progressing toward production readiness. The core features are stable, and were focused on refining performance, expanding protocol support, and ensuring smooth interop with other libp2p implementations. We welcome contributions and real-world usage feedback to help us reach full production maturity.
Read more in the [documentation on ReadTheDocs](https://py-libp2p.readthedocs.io/). [View the release notes](https://py-libp2p.readthedocs.io/en/latest/release_notes.html).
## Maintainers
Currently maintained by [@pacrob](https://github.com/pacrob), [@seetadev](https://github.com/seetadev) and [@dhuseby](https://github.com/dhuseby), looking for assistance!
Currently maintained by [@pacrob](https://github.com/pacrob), [@seetadev](https://github.com/seetadev) and [@dhuseby](https://github.com/dhuseby). Please reach out to us for collaboration or active feedback. If you have questions, feel free to open a new [discussion](https://github.com/libp2p/py-libp2p/discussions). We are also available on the libp2p Discord — join us at #py-libp2p [sub-channel](https://discord.gg/d92MEugb).
## Feature Breakdown
@ -34,19 +34,19 @@ ______________________________________________________________________
| -------------------------------------- | :--------: | :---------------------------------------------------------------------------------: |
| **`libp2p-tcp`** | ✅ | [source](https://github.com/libp2p/py-libp2p/blob/main/libp2p/transport/tcp/tcp.py) |
| **`libp2p-quic`** | 🌱 | |
| **`libp2p-websocket`** | | |
| **`libp2p-webrtc-browser-to-server`** | | |
| **`libp2p-webrtc-private-to-private`** | | |
| **`libp2p-websocket`** | 🌱 | |
| **`libp2p-webrtc-browser-to-server`** | 🌱 | |
| **`libp2p-webrtc-private-to-private`** | 🌱 | |
______________________________________________________________________
### NAT Traversal
| **NAT Traversal** | **Status** |
| ----------------------------- | :--------: |
| **`libp2p-circuit-relay-v2`** | |
| **`libp2p-autonat`** | |
| **`libp2p-hole-punching`** | |
| **NAT Traversal** | **Status** | **Source** |
| ----------------------------- | :--------: | :-----------------------------------------------------------------------------: |
| **`libp2p-circuit-relay-v2`** | | [source](https://github.com/libp2p/py-libp2p/tree/main/libp2p/relay/circuit_v2) |
| **`libp2p-autonat`** | ✅ | [source](https://github.com/libp2p/py-libp2p/tree/main/libp2p/host/autonat) |
| **`libp2p-hole-punching`** | | [source](https://github.com/libp2p/py-libp2p/tree/main/libp2p/relay/circuit_v2) |
______________________________________________________________________
@ -54,27 +54,27 @@ ______________________________________________________________________
| **Secure Communication** | **Status** | **Source** |
| ------------------------ | :--------: | :---------------------------------------------------------------------------: |
| **`libp2p-noise`** | 🌱 | [source](https://github.com/libp2p/py-libp2p/tree/main/libp2p/security/noise) |
| **`libp2p-tls`** | | |
| **`libp2p-noise`** | | [source](https://github.com/libp2p/py-libp2p/tree/main/libp2p/security/noise) |
| **`libp2p-tls`** | 🌱 | |
______________________________________________________________________
### Discovery
| **Discovery** | **Status** |
| -------------------- | :--------: |
| **`bootstrap`** | |
| **`random-walk`** | |
| **`mdns-discovery`** | |
| **`rendezvous`** | |
| **Discovery** | **Status** | **Source** |
| -------------------- | :--------: | :----------------------------------------------------------------------------------: |
| **`bootstrap`** | | [source](https://github.com/libp2p/py-libp2p/tree/main/libp2p/discovery/bootstrap) |
| **`random-walk`** | | [source](https://github.com/libp2p/py-libp2p/tree/main/libp2p/discovery/random_walk) |
| **`mdns-discovery`** | ✅ | [source](https://github.com/libp2p/py-libp2p/tree/main/libp2p/discovery/mdns) |
| **`rendezvous`** | 🌱 | |
______________________________________________________________________
### Peer Routing
| **Peer Routing** | **Status** |
| -------------------- | :--------: |
| **`libp2p-kad-dht`** | |
| **Peer Routing** | **Status** | **Source** |
| -------------------- | :--------: | :--------------------------------------------------------------------: |
| **`libp2p-kad-dht`** | ✅ | [source](https://github.com/libp2p/py-libp2p/tree/main/libp2p/kad_dht) |
______________________________________________________________________
@ -89,10 +89,10 @@ ______________________________________________________________________
### Stream Muxers
| **Stream Muxers** | **Status** | **Status** |
| ------------------ | :--------: | :----------------------------------------------------------------------------------------: |
| **`libp2p-yamux`** | 🌱 | |
| **`libp2p-mplex`** | 🛠️ | [source](https://github.com/libp2p/py-libp2p/blob/main/libp2p/stream_muxer/mplex/mplex.py) |
| **Stream Muxers** | **Status** | **Source** |
| ------------------ | :--------: | :-------------------------------------------------------------------------------: |
| **`libp2p-yamux`** | | [source](https://github.com/libp2p/py-libp2p/tree/main/libp2p/stream_muxer/yamux) |
| **`libp2p-mplex`** | | [source](https://github.com/libp2p/py-libp2p/tree/main/libp2p/stream_muxer/mplex) |
______________________________________________________________________
@ -100,7 +100,7 @@ ______________________________________________________________________
| **Storage** | **Status** |
| ------------------- | :--------: |
| **`libp2p-record`** | |
| **`libp2p-record`** | 🌱 |
______________________________________________________________________

View File

@ -15,14 +15,24 @@
# documentation root, use os.path.abspath to make it absolute, like shown here.
# sys.path.insert(0, os.path.abspath('.'))
import doctest
import os
import sys
from unittest.mock import MagicMock
DIR = os.path.dirname(__file__)
with open(os.path.join(DIR, "../setup.py"), "r") as f:
for line in f:
if "version=" in line:
setup_version = line.split('"')[1]
break
try:
import tomllib
except ModuleNotFoundError:
# For Python < 3.11
import tomli as tomllib # type: ignore (In case of >3.11 Pyrefly doesnt find tomli , which is right but a false flag)
# Path to pyproject.toml (assuming conf.py is in a 'docs' subdirectory)
pyproject_path = os.path.join(os.path.dirname(__file__), "..", "pyproject.toml")
with open(pyproject_path, "rb") as f:
pyproject_data = tomllib.load(f)
setup_version = pyproject_data["project"]["version"]
# -- General configuration ------------------------------------------------
@ -302,7 +312,6 @@ intersphinx_mapping = {
# -- Doctest configuration ----------------------------------------
import doctest
doctest_default_flags = (
0
@ -317,10 +326,9 @@ doctest_default_flags = (
# Mock out dependencies that are unbuildable on readthedocs, as recommended here:
# https://docs.readthedocs.io/en/rel/faq.html#i-get-import-errors-on-libraries-that-depend-on-c-modules
import sys
from unittest.mock import MagicMock
# Add new modules to mock here (it should be the same list as those excluded in setup.py)
# Add new modules to mock here (it should be the same list
# as those excluded in pyproject.toml)
MOCK_MODULES = [
"fastecdsa",
"fastecdsa.encoding",
@ -338,4 +346,4 @@ todo_include_todos = True
# Allow duplicate object descriptions
nitpicky = False
nitpick_ignore = [("py:class", "type")]
nitpick_ignore = [("py:class", "type")]

View File

@ -0,0 +1,504 @@
Circuit Relay v2 Example
========================
This example demonstrates how to use Circuit Relay v2 in py-libp2p. It includes three components:
1. A relay node that provides relay services
2. A destination node that accepts relayed connections
3. A source node that connects to the destination through the relay
Prerequisites
-------------
First, ensure you have py-libp2p installed:
.. code-block:: console
$ python -m pip install libp2p
Collecting libp2p
...
Successfully installed libp2p-x.x.x
Relay Node
----------
Create a file named ``relay_node.py`` with the following content:
.. code-block:: python
import trio
import logging
import multiaddr
import traceback
from libp2p import new_host
from libp2p.relay.circuit_v2.protocol import CircuitV2Protocol
from libp2p.relay.circuit_v2.transport import CircuitV2Transport
from libp2p.relay.circuit_v2.config import RelayConfig
from libp2p.tools.async_service import background_trio_service
from libp2p.utils import get_wildcard_address
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger("relay_node")
async def run_relay():
# Use wildcard address to listen on all interfaces
listen_addr = get_wildcard_address(9000)
host = new_host()
config = RelayConfig(
enable_hop=True, # Act as a relay
enable_stop=True, # Accept relayed connections
enable_client=False, # Don't use other relays
max_circuit_duration=3600, # 1 hour
max_circuit_bytes=1024 * 1024 * 10, # 10MB
)
# Initialize the relay protocol with allow_hop=True to act as a relay
protocol = CircuitV2Protocol(host, limits=config.limits, allow_hop=True)
print(f"Created relay protocol with hop enabled: {protocol.allow_hop}")
# Start the protocol service
async with host.run(listen_addrs=[listen_addr]):
peer_id = host.get_id()
print("\n" + "="*50)
print(f"Relay node started with ID: {peer_id}")
print(f"Relay node multiaddr: /ip4/127.0.0.1/tcp/9000/p2p/{peer_id}")
print("="*50 + "\n")
print(f"Listening on: {host.get_addrs()}")
try:
async with background_trio_service(protocol):
print("Protocol service started")
transport = CircuitV2Transport(host, protocol, config)
print("Relay service started successfully")
print(f"Relay limits: {protocol.limits}")
while True:
await trio.sleep(10)
print("Relay node still running...")
print(f"Active connections: {len(host.get_network().connections)}")
except Exception as e:
print(f"Error in relay service: {e}")
traceback.print_exc()
if __name__ == "__main__":
try:
trio.run(run_relay)
except Exception as e:
print(f"Error running relay: {e}")
traceback.print_exc()
Destination Node
----------------
Create a file named ``destination_node.py`` with the following content:
.. code-block:: python
import trio
import logging
import multiaddr
import traceback
import sys
from libp2p import new_host
from libp2p.relay.circuit_v2.protocol import CircuitV2Protocol
from libp2p.relay.circuit_v2.transport import CircuitV2Transport
from libp2p.relay.circuit_v2.config import RelayConfig
from libp2p.peer.peerinfo import info_from_p2p_addr
from libp2p.tools.async_service import background_trio_service
from libp2p.utils import get_wildcard_address
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger("destination_node")
async def handle_echo_stream(stream):
"""Handle incoming stream by echoing received data."""
try:
print(f"New echo stream from: {stream.get_protocol()}")
while True:
data = await stream.read(1024)
if not data:
print("Stream closed by remote")
break
message = data.decode('utf-8')
print(f"Received: {message}")
response = f"Echo: {message}".encode('utf-8')
await stream.write(response)
print(f"Sent response: Echo: {message}")
except Exception as e:
print(f"Error handling stream: {e}")
traceback.print_exc()
finally:
await stream.close()
print("Stream closed")
async def run_destination(relay_peer_id=None):
"""
Run a simple destination node that accepts connections.
This is a simplified version that doesn't use the relay functionality.
"""
# Create a libp2p host - use wildcard address to listen on all interfaces
listen_addr = get_wildcard_address(9001)
host = new_host()
# Configure as a relay receiver (stop)
config = RelayConfig(
enable_stop=True, # Accept relayed connections
enable_client=True, # Use relays for outbound connections
max_circuit_duration=3600, # 1 hour
max_circuit_bytes=1024 * 1024 * 10, # 10MB
)
# Initialize the relay protocol
protocol = CircuitV2Protocol(host, limits=config.limits, allow_hop=False)
async with host.run(listen_addrs=[listen_addr]):
# Print host information
dest_peer_id = host.get_id()
print("\n" + "="*50)
print(f"Destination node started with ID: {dest_peer_id}")
print(f"Use this ID in the source node: {dest_peer_id}")
print("="*50 + "\n")
print(f"Listening on: {host.get_addrs()}")
# Set stream handler for the echo protocol
host.set_stream_handler("/echo/1.0.0", handle_echo_stream)
print("Registered echo protocol handler")
# Start the protocol service in the background
async with background_trio_service(protocol):
print("Protocol service started")
# Create and register the transport
transport = CircuitV2Transport(host, protocol, config)
print("Transport created")
# Create a listener for relayed connections
listener = transport.create_listener(handle_echo_stream)
print("Created relay listener")
# Start listening for relayed connections
async with trio.open_nursery() as nursery:
await listener.listen("/p2p-circuit", nursery)
print("Destination node ready to accept relayed connections")
if not relay_peer_id:
print("No relay peer ID provided. Please enter the relay's peer ID:")
print("Waiting for relay peer ID input...")
while True:
if sys.stdin.isatty(): # Only try to read from stdin if it's a terminal
try:
relay_peer_id = input("Enter relay peer ID: ").strip()
if relay_peer_id:
break
except EOFError:
await trio.sleep(5)
else:
print("No terminal detected. Waiting for relay peer ID as command line argument.")
await trio.sleep(10)
continue
# Connect to the relay node with the provided relay peer ID
relay_addr_str = f"/ip4/127.0.0.1/tcp/9000/p2p/{relay_peer_id}"
print(f"Connecting to relay at {relay_addr_str}")
try:
# Convert string address to multiaddr, then to peer info
relay_maddr = multiaddr.Multiaddr(relay_addr_str)
relay_peer_info = info_from_p2p_addr(relay_maddr)
await host.connect(relay_peer_info)
print("Connected to relay successfully")
# Add the relay to the transport's discovery
transport.discovery._add_relay(relay_peer_info.peer_id)
print(f"Added relay {relay_peer_info.peer_id} to discovery")
# Keep the node running
while True:
await trio.sleep(10)
print("Destination node still running...")
except Exception as e:
print(f"Failed to connect to relay: {e}")
traceback.print_exc()
if __name__ == "__main__":
print("Starting destination node...")
relay_id = None
if len(sys.argv) > 1:
relay_id = sys.argv[1]
print(f"Using provided relay ID: {relay_id}")
trio.run(run_destination, relay_id)
Source Node
-----------
Create a file named ``source_node.py`` with the following content:
.. code-block:: python
import trio
import logging
import multiaddr
import traceback
import sys
from libp2p import new_host
from libp2p.peer.peerinfo import PeerInfo
from libp2p.peer.id import ID
from libp2p.relay.circuit_v2.protocol import CircuitV2Protocol
from libp2p.relay.circuit_v2.transport import CircuitV2Transport
from libp2p.relay.circuit_v2.config import RelayConfig
from libp2p.peer.peerinfo import info_from_p2p_addr
from libp2p.tools.async_service import background_trio_service
from libp2p.relay.circuit_v2.discovery import RelayInfo
from libp2p.utils import get_wildcard_address
# Configure logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger("source_node")
async def run_source(relay_peer_id=None, destination_peer_id=None):
# Create a libp2p host - use wildcard address to listen on all interfaces
listen_addr = get_wildcard_address(9002)
host = new_host()
# Configure as a relay client
config = RelayConfig(
enable_client=True, # Use relays for outbound connections
max_circuit_duration=3600, # 1 hour
max_circuit_bytes=1024 * 1024 * 10, # 10MB
)
# Initialize the relay protocol
protocol = CircuitV2Protocol(host, limits=config.limits, allow_hop=False)
# Start the protocol service
async with host.run(listen_addrs=[listen_addr]):
# Print host information
print(f"Source node started with ID: {host.get_id()}")
print(f"Listening on: {host.get_addrs()}")
# Start the protocol service in the background
async with background_trio_service(protocol):
print("Protocol service started")
# Create and register the transport
transport = CircuitV2Transport(host, protocol, config)
# Get relay peer ID if not provided
if not relay_peer_id:
print("No relay peer ID provided. Please enter the relay's peer ID:")
while True:
if sys.stdin.isatty(): # Only try to read from stdin if it's a terminal
try:
relay_peer_id = input("Enter relay peer ID: ").strip()
if relay_peer_id:
break
except EOFError:
await trio.sleep(5)
else:
print("No terminal detected. Waiting for relay peer ID as command line argument.")
await trio.sleep(10)
continue
# Connect to the relay node with the provided relay peer ID
relay_addr_str = f"/ip4/127.0.0.1/tcp/9000/p2p/{relay_peer_id}"
print(f"Connecting to relay at {relay_addr_str}")
try:
# Convert string address to multiaddr, then to peer info
relay_maddr = multiaddr.Multiaddr(relay_addr_str)
relay_peer_info = info_from_p2p_addr(relay_maddr)
await host.connect(relay_peer_info)
print("Connected to relay successfully")
# Manually add the relay to the discovery service
relay_id = relay_peer_info.peer_id
now = trio.current_time()
# Create relay info and add it to discovery
relay_info = RelayInfo(
peer_id=relay_id,
discovered_at=now,
last_seen=now
)
transport.discovery._discovered_relays[relay_id] = relay_info
print(f"Added relay {relay_id} to discovery")
# Start relay discovery in the background
async with background_trio_service(transport.discovery):
print("Relay discovery started")
# Wait for relay discovery
await trio.sleep(5)
print("Relay discovery completed")
# Get destination peer ID if not provided
if not destination_peer_id:
print("No destination peer ID provided. Please enter the destination's peer ID:")
while True:
if sys.stdin.isatty(): # Only try to read from stdin if it's a terminal
try:
destination_peer_id = input("Enter destination peer ID: ").strip()
if destination_peer_id:
break
except EOFError:
await trio.sleep(5)
else:
print("No terminal detected. Waiting for destination peer ID as command line argument.")
await trio.sleep(10)
continue
print(f"Attempting to connect to {destination_peer_id} via relay")
# Check if we have any discovered relays
discovered_relays = list(transport.discovery._discovered_relays.keys())
print(f"Discovered relays: {discovered_relays}")
try:
# Create a circuit relay multiaddr for the destination
dest_id = ID.from_base58(destination_peer_id)
# Create a circuit multiaddr that includes the relay
# Format: /ip4/127.0.0.1/tcp/9000/p2p/RELAY_ID/p2p-circuit/p2p/DEST_ID
circuit_addr = multiaddr.Multiaddr(f"{relay_addr_str}/p2p-circuit/p2p/{destination_peer_id}")
print(f"Created circuit address: {circuit_addr}")
# Dial using the circuit address
connection = await transport.dial(circuit_addr)
print("Connection established through relay!")
# Open a stream using the echo protocol
stream = await connection.new_stream("/echo/1.0.0")
# Send messages periodically
for i in range(5):
message = f"Hello from source, message {i+1}"
print(f"Sending: {message}")
await stream.write(message.encode('utf-8'))
response = await stream.read(1024)
print(f"Received: {response.decode('utf-8')}")
await trio.sleep(1)
# Close the stream
await stream.close()
print("Stream closed")
except Exception as e:
print(f"Error connecting through relay: {e}")
print("Detailed error:")
traceback.print_exc()
# Keep the node running for a while
await trio.sleep(30)
print("Source node shutting down")
except Exception as e:
print(f"Error: {e}")
traceback.print_exc()
if __name__ == "__main__":
relay_id = None
dest_id = None
# Parse command line arguments if provided
if len(sys.argv) > 1:
relay_id = sys.argv[1]
print(f"Using provided relay ID: {relay_id}")
if len(sys.argv) > 2:
dest_id = sys.argv[2]
print(f"Using provided destination ID: {dest_id}")
trio.run(run_source, relay_id, dest_id)
Running the Example
-------------------
1. First, start the relay node:
.. code-block:: console
$ python relay_node.py
Created relay protocol with hop enabled: True
==================================================
Relay node started with ID: QmaUigQJ9nJERa6GaZuyfaiX91QjYwoQJ46JS3k7ys7SLx
Relay node multiaddr: /ip4/127.0.0.1/tcp/9000/p2p/QmaUigQJ9nJERa6GaZuyfaiX91QjYwoQJ46JS3k7ys7SLx
==================================================
Listening on: [<Multiaddr /ip4/127.0.0.1/tcp/9000/p2p/QmaUigQJ9nJERa6GaZuyfaiX91QjYwoQJ46JS3k7ys7SLx>]
Protocol service started
Relay service started successfully
Relay limits: RelayLimits(duration=3600, data=10485760, max_circuit_conns=8, max_reservations=4)
Note the relay node\'s peer ID (in this example: `QmaUigQJ9nJERa6GaZuyfaiX91QjYwoQJ46JS3k7ys7SLx`). You\'ll need this for the other nodes.
2. Next, start the destination node:
.. code-block:: console
$ python destination_node.py
Starting destination node...
==================================================
Destination node started with ID: QmPBr38KeQG2ibyL4fxq6yJWpfoVNCqJMHBdNyn1Qe4h5s
Use this ID in the source node: QmPBr38KeQG2ibyL4fxq6yJWpfoVNCqJMHBdNyn1Qe4h5s
==================================================
Listening on: [<Multiaddr /ip4/127.0.0.1/tcp/9001/p2p/QmPBr38KeQG2ibyL4fxq6yJWpfoVNCqJMHBdNyn1Qe4h5s>]
Registered echo protocol handler
Protocol service started
Transport created
Created relay listener
Destination node ready to accept relayed connections
No relay peer ID provided. Please enter the relay\'s peer ID:
Waiting for relay peer ID input...
Enter relay peer ID: QmaUigQJ9nJERa6GaZuyfaiX91QjYwoQJ46JS3k7ys7SLx
Connecting to relay at /ip4/127.0.0.1/tcp/9000/p2p/QmaUigQJ9nJERa6GaZuyfaiX91QjYwoQJ46JS3k7ys7SLx
Connected to relay successfully
Added relay QmaUigQJ9nJERa6GaZuyfaiX91QjYwoQJ46JS3k7ys7SLx to discovery
Destination node still running...
Note the destination node's peer ID (in this example: `QmPBr38KeQG2ibyL4fxq6yJWpfoVNCqJMHBdNyn1Qe4h5s`). You'll need this for the source node.
3. Finally, start the source node:
.. code-block:: console
$ python source_node.py
Source node started with ID: QmPyM56cgmFoHTgvMgGfDWRdVRQznmxCDDDg2dJ8ygVXj3
Listening on: [<Multiaddr /ip4/127.0.0.1/tcp/9002/p2p/QmPyM56cgmFoHTgvMgGfDWRdVRQznmxCDDDg2dJ8ygVXj3>]
Protocol service started
No relay peer ID provided. Please enter the relay\'s peer ID:
Enter relay peer ID: QmaUigQJ9nJERa6GaZuyfaiX91QjYwoQJ46JS3k7ys7SLx
Connecting to relay at /ip4/127.0.0.1/tcp/9000/p2p/QmaUigQJ9nJERa6GaZuyfaiX91QjYwoQJ46JS3k7ys7SLx
Connected to relay successfully
Added relay QmaUigQJ9nJERa6GaZuyfaiX91QjYwoQJ46JS3k7ys7SLx to discovery
Relay discovery started
Relay discovery completed
No destination peer ID provided. Please enter the destination\'s peer ID:
Enter destination peer ID: QmPBr38KeQG2ibyL4fxq6yJWpfoVNCqJMHBdNyn1Qe4h5s
Attempting to connect to QmPBr38KeQG2ibyL4fxq6yJWpfoVNCqJMHBdNyn1Qe4h5s via relay
Discovered relays: [<libp2p.peer.id.ID (QmaUigQJ9nJERa6GaZuyfaiX91QjYwoQJ46JS3k7ys7SLx)>]
Created circuit address: /ip4/127.0.0.1/tcp/9000/p2p/QmaUigQJ9nJERa6GaZuyfaiX91QjYwoQJ46JS3k7ys7SLx/p2p-circuit/p2p/QmPBr38KeQG2ibyL4fxq6yJWpfoVNCqJMHBdNyn1Qe4h5s
At this point, the source node will establish a connection through the relay to the destination node and start sending messages.
4. Alternatively, you can provide the peer IDs as command-line arguments:
.. code-block:: console
# For the destination node (provide relay ID)
$ python destination_node.py QmaUigQJ9nJERa6GaZuyfaiX91QjYwoQJ46JS3k7ys7SLx
# For the source node (provide both relay and destination IDs)
$ python source_node.py QmaUigQJ9nJERa6GaZuyfaiX91QjYwoQJ46JS3k7ys7SLx QmPBr38KeQG2ibyL4fxq6yJWpfoVNCqJMHBdNyn1Qe4h5s
This example demonstrates how to use Circuit Relay v2 to establish connections between peers that cannot connect directly. The peer IDs are dynamically generated for each node, and the relay facilitates communication between the source and destination nodes.

View File

@ -0,0 +1,43 @@
QUIC Echo Demo
==============
This example demonstrates a simple ``echo`` protocol using **QUIC transport**.
QUIC provides built-in TLS security and stream multiplexing over UDP, making it an excellent transport choice for libp2p applications.
.. code-block:: console
$ python -m pip install libp2p
Collecting libp2p
...
Successfully installed libp2p-x.x.x
$ echo-quic-demo
Run this from the same folder in another console:
echo-quic-demo -d /ip4/127.0.0.1/udp/8000/quic-v1/p2p/16Uiu2HAmAsbxRR1HiGJRNVPQLNMeNsBCsXT3rDjoYBQzgzNpM5mJ
Waiting for incoming connection...
Copy the line that starts with ``echo-quic-demo -p 8001``, open a new terminal in the same
folder and paste it in:
.. code-block:: console
$ echo-quic-demo -d /ip4/127.0.0.1/udp/8000/quic-v1/p2p/16Uiu2HAmE3N7KauPTmHddYPsbMcBp2C6XAmprELX3YcFEN9iXiBu
I am 16Uiu2HAmE3N7KauPTmHddYPsbMcBp2C6XAmprELX3YcFEN9iXiBu
STARTING CLIENT CONNECTION PROCESS
CLIENT CONNECTED TO SERVER
Sent: hi, there!
Got: ECHO: hi, there!
**Key differences from TCP Echo:**
- Uses UDP instead of TCP: ``/udp/8000`` instead of ``/tcp/8000``
- Includes QUIC protocol identifier: ``/quic-v1`` in the multiaddr
- Built-in TLS security (no separate security transport needed)
- Native stream multiplexing over a single QUIC connection
.. literalinclude:: ../examples/echo/echo_quic.py
:language: python
:linenos:

View File

@ -12,7 +12,7 @@ This example demonstrates how to use the libp2p ``identify`` protocol.
$ identify-demo
First host listening. Run this from another console:
identify-demo -p 8889 -d /ip4/0.0.0.0/tcp/8888/p2p/QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM
identify-demo -p 8889 -d /ip4/127.0.0.1/tcp/8888/p2p/QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM
Waiting for incoming identify request...
@ -21,13 +21,13 @@ folder and paste it in:
.. code-block:: console
$ identify-demo -p 8889 -d /ip4/0.0.0.0/tcp/8888/p2p/QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM
dialer (host_b) listening on /ip4/0.0.0.0/tcp/8889
$ identify-demo -p 8889 -d /ip4/127.0.0.1/tcp/8888/p2p/QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM
dialer (host_b) listening on /ip4/127.0.0.1/tcp/8889
Second host connecting to peer: QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM
Starting identify protocol...
Identify response:
Public Key (Base64): CAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDC6c/oNPP9X13NDQ3Xrlp3zOj+ErXIWb/A4JGwWchiDBwMhMslEX3ct8CqI0BqUYKuwdFjowqqopOJ3cS2MlqtGaiP6Dg9bvGqSDoD37BpNaRVNcebRxtB0nam9SQy3PYLbHAmz0vR4ToSiL9OLRORnGOxCtHBuR8ZZ5vS0JEni8eQMpNa7IuXwyStnuty/QjugOZudBNgYSr8+9gH722KTjput5IRL7BrpIdd4HNXGVRm4b9BjNowvHu404x3a/ifeNblpy/FbYyFJEW0looygKF7hpRHhRbRKIDZt2BqOfT1sFkbqsHE85oY859+VMzP61YELgvGwai2r7KcjkW/AgMBAAE=
Listen Addresses: ['/ip4/0.0.0.0/tcp/8888/p2p/QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM']
Listen Addresses: ['/ip4/127.0.0.1/tcp/8888/p2p/QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM']
Protocols: ['/ipfs/id/1.0.0', '/ipfs/ping/1.0.0']
Observed Address: ['/ip4/127.0.0.1/tcp/38082']
Protocol Version: ipfs/0.1.0

View File

@ -34,11 +34,11 @@ There is also a more interactive version of the example which runs as separate l
==== Starting Identify-Push Listener on port 8888 ====
Listener host ready!
Listening on: /ip4/0.0.0.0/tcp/8888/p2p/QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM
Listening on: /ip4/127.0.0.1/tcp/8888/p2p/QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM
Peer ID: QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM
Run dialer with command:
identify-push-listener-dialer-demo -d /ip4/0.0.0.0/tcp/8888/p2p/QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM
identify-push-listener-dialer-demo -d /ip4/127.0.0.1/tcp/8888/p2p/QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM
Waiting for incoming connections... (Ctrl+C to exit)
@ -47,12 +47,12 @@ folder and paste it in:
.. code-block:: console
$ identify-push-listener-dialer-demo -d /ip4/0.0.0.0/tcp/8888/p2p/QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM
$ identify-push-listener-dialer-demo -d /ip4/127.0.0.1/tcp/8888/p2p/QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM
==== Starting Identify-Push Dialer on port 8889 ====
Dialer host ready!
Listening on: /ip4/0.0.0.0/tcp/8889/p2p/QmZyXwVuTaBcDeRsSkJpOpWrSt
Listening on: /ip4/127.0.0.1/tcp/8889/p2p/QmZyXwVuTaBcDeRsSkJpOpWrSt
Connecting to peer: QmUiN4R3fNrCoQugGgmmb3v35neMEjKFNrsbNGVDsRHWpM
Successfully connected to listener!

124
docs/examples.kademlia.rst Normal file
View File

@ -0,0 +1,124 @@
Kademlia DHT Demo
=================
This example demonstrates a Kademlia Distributed Hash Table (DHT) implementation with both value storage/retrieval and content provider advertisement/discovery functionality.
.. code-block:: console
$ python -m pip install libp2p
Collecting libp2p
...
Successfully installed libp2p-x.x.x
$ cd examples/kademlia
$ python kademlia.py --mode server
2025-06-13 19:51:25,424 - kademlia-example - INFO - Running in server mode on port 0
2025-06-13 19:51:25,426 - kademlia-example - INFO - Connected to bootstrap nodes: []
2025-06-13 19:51:25,426 - kademlia-example - INFO - To connect to this node, use: --bootstrap /ip4/127.0.0.1/tcp/28910/p2p/16Uiu2HAm7EsNv5vvjPAehGAVfChjYjD63ZHyWogQRdzntSbAg9ef
2025-06-13 19:51:25,426 - kademlia-example - INFO - Saved server address to log: /ip4/127.0.0.1/tcp/28910/p2p/16Uiu2HAm7EsNv5vvjPAehGAVfChjYjD63ZHyWogQRdzntSbAg9ef
2025-06-13 19:51:25,427 - kademlia-example - INFO - DHT service started in SERVER mode
2025-06-13 19:51:25,427 - kademlia-example - INFO - Stored value 'Hello message from Sumanjeet' with key: FVDjasarSFDoLPMdgnp1dHSbW2ZAfN8NU2zNbCQeczgP
2025-06-13 19:51:25,427 - kademlia-example - INFO - Successfully advertised as server for content: 361f2ed1183bca491b8aec11f0b9e5c06724759b0f7480ae7fb4894901993bc8
Copy the line that starts with ``--bootstrap``, open a new terminal in the same folder and run the client:
.. code-block:: console
$ python kademlia.py --mode client --bootstrap /ip4/127.0.0.1/tcp/28910/p2p/16Uiu2HAm7EsNv5vvjPAehGAVfChjYjD63ZHyWogQRdzntSbAg9ef
2025-06-13 19:51:37,022 - kademlia-example - INFO - Running in client mode on port 0
2025-06-13 19:51:37,026 - kademlia-example - INFO - Connected to bootstrap nodes: [<libp2p.peer.id.ID (16Uiu2HAm7EsNv5vvjPAehGAVfChjYjD63ZHyWogQRdzntSbAg9ef)>]
2025-06-13 19:51:37,027 - kademlia-example - INFO - DHT service started in CLIENT mode
2025-06-13 19:51:37,027 - kademlia-example - INFO - Looking up key: FVDjasarSFDoLPMdgnp1dHSbW2ZAfN8NU2zNbCQeczgP
2025-06-13 19:51:37,031 - kademlia-example - INFO - Retrieved value: Hello message from Sumanjeet
2025-06-13 19:51:37,031 - kademlia-example - INFO - Looking for servers of content: 361f2ed1183bca491b8aec11f0b9e5c06724759b0f7480ae7fb4894901993bc8
2025-06-13 19:51:37,035 - kademlia-example - INFO - Found 1 servers for content: ['16Uiu2HAm7EsNv5vvjPAehGAVfChjYjD63ZHyWogQRdzntSbAg9ef']
Alternatively, if you run the server first, the client can automatically extract the bootstrap address from the server log file:
.. code-block:: console
$ python kademlia.py --mode client
2025-06-13 19:51:37,022 - kademlia-example - INFO - Running in client mode on port 0
2025-06-13 19:51:37,026 - kademlia-example - INFO - Connected to bootstrap nodes: [<libp2p.peer.id.ID (16Uiu2HAm7EsNv5vvjPAehGAVfChjYjD63ZHyWogQRdzntSbAg9ef)>]
2025-06-13 19:51:37,027 - kademlia-example - INFO - DHT service started in CLIENT mode
2025-06-13 19:51:37,027 - kademlia-example - INFO - Looking up key: FVDjasarSFDoLPMdgnp1dHSbW2ZAfN8NU2zNbCQeczgP
2025-06-13 19:51:37,031 - kademlia-example - INFO - Retrieved value: Hello message from Sumanjeet
2025-06-13 19:51:37,031 - kademlia-example - INFO - Looking for servers of content: 361f2ed1183bca491b8aec11f0b9e5c06724759b0f7480ae7fb4894901993bc8
2025-06-13 19:51:37,035 - kademlia-example - INFO - Found 1 servers for content: ['16Uiu2HAm7EsNv5vvjPAehGAVfChjYjD63ZHyWogQRdzntSbAg9ef']
The demo showcases key DHT operations:
- **Value Storage & Retrieval**: The server stores a value, and the client retrieves it
- **Content Provider Discovery**: The server advertises content, and the client finds providers
- **Peer Discovery**: Automatic bootstrap and peer routing using the Kademlia algorithm
- **Network Resilience**: Distributed storage across multiple nodes (when available)
Command Line Options
--------------------
The Kademlia demo supports several command line options for customization:
.. code-block:: console
$ python kademlia.py --help
usage: kademlia.py [-h] [--mode MODE] [--port PORT] [--bootstrap [BOOTSTRAP ...]] [--verbose]
Kademlia DHT example with content server functionality
options:
-h, --help show this help message and exit
--mode MODE Run as a server or client node (default: server)
--port PORT Port to listen on (0 for random) (default: 0)
--bootstrap [BOOTSTRAP ...]
Multiaddrs of bootstrap nodes. Provide a space-separated list of addresses.
This is required for client mode.
--verbose Enable verbose logging
**Examples:**
Start server on a specific port:
.. code-block:: console
$ python kademlia.py --mode server --port 8000
Start client with verbose logging:
.. code-block:: console
$ python kademlia.py --mode client --verbose
Connect to multiple bootstrap nodes:
.. code-block:: console
$ python kademlia.py --mode client --bootstrap /ip4/127.0.0.1/tcp/8000/p2p/... /ip4/127.0.0.1/tcp/8001/p2p/...
How It Works
------------
The Kademlia DHT implementation demonstrates several key concepts:
**Server Mode:**
- Stores key-value pairs in the distributed hash table
- Advertises itself as a content provider for specific content
- Handles incoming DHT requests from other nodes
- Maintains routing table with known peers
**Client Mode:**
- Connects to bootstrap nodes to join the network
- Retrieves values by their keys from the DHT
- Discovers content providers for specific content
- Performs network lookups using the Kademlia algorithm
**Key Components:**
- **Routing Table**: Organizes peers in k-buckets based on XOR distance
- **Value Store**: Manages key-value storage with TTL (time-to-live)
- **Provider Store**: Tracks which peers provide specific content
- **Peer Routing**: Implements iterative lookups to find closest peers
The full source code for this example is below:
.. literalinclude:: ../examples/kademlia/kademlia.py
:language: python
:linenos:

64
docs/examples.mDNS.rst Normal file
View File

@ -0,0 +1,64 @@
mDNS Peer Discovery Example
===========================
This example demonstrates how to use mDNS (Multicast DNS) for peer discovery in py-libp2p.
Prerequisites
-------------
First, ensure you have py-libp2p installed and your environment is activated:
.. code-block:: console
$ python -m pip install libp2p
Running the Example
-------------------
The mDNS demo script allows you to discover peers on your local network using mDNS. To start a peer, run:
.. code-block:: console
$ mdns-demo
You should see output similar to:
.. code-block:: console
Run this from another console to start another peer on a different port:
python mdns-demo -p <ANOTHER_PORT>
Waiting for mDNS peer discovery events...
2025-06-20 23:28:12,052 - libp2p.example.discovery.mdns - INFO - Starting peer Discovery
To discover peers, open another terminal and run the same command with a different port:
.. code-block:: console
$ python mdns-demo -p 9001
You should see output indicating that a new peer has been discovered:
.. code-block:: console
Run this from the same folder in another console to start another peer on a different port:
python mdns-demo -p <ANOTHER_PORT>
Waiting for mDNS peer discovery events...
2025-06-20 23:43:43,786 - libp2p.example.discovery.mdns - INFO - Starting peer Discovery
2025-06-20 23:43:43,790 - libp2p.example.discovery.mdns - INFO - Discovered: 16Uiu2HAmGxy5NdQEjZWtrYUMrzdp3Syvg7MB2E5Lx8weA9DanYxj
When a new peer is discovered, its peer ID will be printed in the console output.
How it Works
------------
- Each node advertises itself on the local network using mDNS.
- When a new peer is discovered, the handler prints its peer ID.
- This is useful for local peer discovery without requiring a DHT or bootstrap nodes.
You can modify the script to perform additional actions when peers are discovered, such as opening streams or exchanging messages.

View File

@ -0,0 +1,194 @@
Multiple Connections Per Peer
=============================
This example demonstrates how to use the multiple connections per peer feature in py-libp2p.
Overview
--------
The multiple connections per peer feature allows a libp2p node to maintain multiple network connections to the same peer. This provides several benefits:
- **Improved reliability**: If one connection fails, others remain available
- **Better performance**: Load can be distributed across multiple connections
- **Enhanced throughput**: Multiple streams can be created in parallel
- **Fault tolerance**: Redundant connections provide backup paths
Configuration
-------------
The feature is configured through the `ConnectionConfig` class:
.. code-block:: python
from libp2p.network.swarm import ConnectionConfig
# Default configuration
config = ConnectionConfig()
print(f"Max connections per peer: {config.max_connections_per_peer}")
print(f"Load balancing strategy: {config.load_balancing_strategy}")
# Custom configuration
custom_config = ConnectionConfig(
max_connections_per_peer=5,
connection_timeout=60.0,
load_balancing_strategy="least_loaded"
)
Load Balancing Strategies
-------------------------
Two load balancing strategies are available:
**Round Robin** (default)
Cycles through connections in order, distributing load evenly.
**Least Loaded**
Selects the connection with the fewest active streams.
API Usage
---------
The new API provides direct access to multiple connections:
.. code-block:: python
from libp2p import new_swarm
# Create swarm with multiple connections support
swarm = new_swarm()
# Dial a peer - returns list of connections
connections = await swarm.dial_peer(peer_id)
print(f"Established {len(connections)} connections")
# Get all connections to a peer
peer_connections = swarm.get_connections(peer_id)
# Get all connections (across all peers)
all_connections = swarm.get_connections()
# Get the complete connections map
connections_map = swarm.get_connections_map()
# Backward compatibility - get single connection
single_conn = swarm.get_connection(peer_id)
Backward Compatibility
----------------------
Existing code continues to work through backward compatibility features:
.. code-block:: python
# Legacy 1:1 mapping (returns first connection for each peer)
legacy_connections = swarm.connections_legacy
# Single connection access (returns first available connection)
conn = swarm.get_connection(peer_id)
Example
-------
A complete working example is available in the `examples/doc-examples/multiple_connections_example.py` file.
Production Configuration
-------------------------
For production use, consider these settings:
**RetryConfig Parameters**
The `RetryConfig` class controls connection retry behavior with exponential backoff:
- **max_retries**: Maximum number of retry attempts before giving up (default: 3)
- **initial_delay**: Initial delay in seconds before the first retry (default: 0.1s)
- **max_delay**: Maximum delay cap to prevent excessive wait times (default: 30.0s)
- **backoff_multiplier**: Exponential backoff multiplier - each retry multiplies delay by this factor (default: 2.0)
- **jitter_factor**: Random jitter (0.0-1.0) to prevent synchronized retries (default: 0.1)
**ConnectionConfig Parameters**
The `ConnectionConfig` class manages multi-connection behavior:
- **max_connections_per_peer**: Maximum connections allowed to a single peer (default: 3)
- **connection_timeout**: Timeout for establishing new connections in seconds (default: 30.0s)
- **load_balancing_strategy**: Strategy for distributing streams ("round_robin" or "least_loaded")
**Load Balancing Strategies Explained**
- **round_robin**: Cycles through connections in order, distributing load evenly. Simple and predictable.
- **least_loaded**: Selects the connection with the fewest active streams. Better for performance but more complex.
.. code-block:: python
from libp2p.network.swarm import ConnectionConfig, RetryConfig
# Production-ready configuration
retry_config = RetryConfig(
max_retries=3, # Maximum retry attempts before giving up
initial_delay=0.1, # Start with 100ms delay
max_delay=30.0, # Cap exponential backoff at 30 seconds
backoff_multiplier=2.0, # Double delay each retry (100ms -> 200ms -> 400ms)
jitter_factor=0.1 # Add 10% random jitter to prevent thundering herd
)
connection_config = ConnectionConfig(
max_connections_per_peer=3, # Allow up to 3 connections per peer
connection_timeout=30.0, # 30 second timeout for new connections
load_balancing_strategy="round_robin" # Simple, predictable load distribution
)
swarm = new_swarm(
retry_config=retry_config,
connection_config=connection_config
)
**How RetryConfig Works in Practice**
With the configuration above, connection retries follow this pattern:
1. **Attempt 1**: Immediate connection attempt
2. **Attempt 2**: Wait 100ms ± 10ms jitter, then retry
3. **Attempt 3**: Wait 200ms ± 20ms jitter, then retry
4. **Attempt 4**: Wait 400ms ± 40ms jitter, then retry
5. **Attempt 5**: Wait 800ms ± 80ms jitter, then retry
6. **Attempt 6**: Wait 1.6s ± 160ms jitter, then retry
7. **Attempt 7**: Wait 3.2s ± 320ms jitter, then retry
8. **Attempt 8**: Wait 6.4s ± 640ms jitter, then retry
9. **Attempt 9**: Wait 12.8s ± 1.28s jitter, then retry
10. **Attempt 10**: Wait 25.6s ± 2.56s jitter, then retry
11. **Attempt 11**: Wait 30.0s (capped) ± 3.0s jitter, then retry
12. **Attempt 12**: Wait 30.0s (capped) ± 3.0s jitter, then retry
13. **Give up**: After 12 retries (3 initial + 9 retries), connection fails
The jitter prevents multiple clients from retrying simultaneously, reducing server load.
**Parameter Tuning Guidelines**
**For Development/Testing:**
- Use lower `max_retries` (1-2) and shorter delays for faster feedback
- Example: `RetryConfig(max_retries=2, initial_delay=0.01, max_delay=0.1)`
**For Production:**
- Use moderate `max_retries` (3-5) with reasonable delays for reliability
- Example: `RetryConfig(max_retries=5, initial_delay=0.1, max_delay=60.0)`
**For High-Latency Networks:**
- Use higher `max_retries` (5-10) with longer delays
- Example: `RetryConfig(max_retries=8, initial_delay=0.5, max_delay=120.0)`
**For Load Balancing:**
- Use `round_robin` for simple, predictable behavior
- Use `least_loaded` when you need optimal performance and can handle complexity
Architecture
------------
The implementation follows the same architectural patterns as the Go and JavaScript reference implementations:
- **Core data structure**: `dict[ID, list[INetConn]]` for 1:many mapping
- **API consistency**: Methods like `get_connections()` match reference implementations
- **Load balancing**: Integrated at the API level for optimal performance
- **Backward compatibility**: Maintains existing interfaces for gradual migration
This design ensures consistency across libp2p implementations while providing the benefits of multiple connections per peer.

View File

@ -15,7 +15,7 @@ This example demonstrates how to create a chat application using libp2p's PubSub
2025-04-06 23:59:17,471 - pubsub-demo - INFO - Your selected topic is: pubsub-chat
2025-04-06 23:59:17,472 - pubsub-demo - INFO - Using random available port: 33269
2025-04-06 23:59:17,490 - pubsub-demo - INFO - Node started with peer ID: QmcJnocH1d1tz3Zp4MotVDjNfNFawXHw2dpB9tMYGTXJp7
2025-04-06 23:59:17,490 - pubsub-demo - INFO - Listening on: /ip4/0.0.0.0/tcp/33269
2025-04-06 23:59:17,490 - pubsub-demo - INFO - Listening on: /ip4/127.0.0.1/tcp/33269
2025-04-06 23:59:17,490 - pubsub-demo - INFO - Initializing PubSub and GossipSub...
2025-04-06 23:59:17,491 - pubsub-demo - INFO - Pubsub and GossipSub services started.
2025-04-06 23:59:17,491 - pubsub-demo - INFO - Pubsub ready.
@ -35,7 +35,7 @@ Copy the line that starts with ``pubsub-demo -d``, open a new terminal and paste
2025-04-07 00:00:59,846 - pubsub-demo - INFO - Your selected topic is: pubsub-chat
2025-04-07 00:00:59,846 - pubsub-demo - INFO - Using random available port: 51977
2025-04-07 00:00:59,864 - pubsub-demo - INFO - Node started with peer ID: QmYQKCm95Ut1aXsjHmWVYqdaVbno1eKTYC8KbEVjqUaKaQ
2025-04-07 00:00:59,864 - pubsub-demo - INFO - Listening on: /ip4/0.0.0.0/tcp/51977
2025-04-07 00:00:59,864 - pubsub-demo - INFO - Listening on: /ip4/127.0.0.1/tcp/51977
2025-04-07 00:00:59,864 - pubsub-demo - INFO - Initializing PubSub and GossipSub...
2025-04-07 00:00:59,864 - pubsub-demo - INFO - Pubsub and GossipSub services started.
2025-04-07 00:00:59,865 - pubsub-demo - INFO - Pubsub ready.

View File

@ -0,0 +1,131 @@
Random Walk Example
===================
This example demonstrates the Random Walk module's peer discovery capabilities using real libp2p hosts and Kademlia DHT.
It shows how the Random Walk module automatically discovers new peers and maintains routing table health.
The Random Walk implementation performs the following key operations:
* **Automatic Peer Discovery**: Generates random peer IDs and queries the DHT network to discover new peers
* **Routing Table Maintenance**: Periodically refreshes the routing table to maintain network connectivity
* **Connection Management**: Maintains optimal connections to healthy peers in the network
* **Real-time Statistics**: Displays routing table size, connected peers, and peerstore statistics
.. code-block:: console
$ python -m pip install libp2p
Collecting libp2p
...
Successfully installed libp2p-x.x.x
$ cd examples/random_walk
$ python random_walk.py --mode server
2025-08-12 19:51:25,424 - random-walk-example - INFO - === Random Walk Example for py-libp2p ===
2025-08-12 19:51:25,424 - random-walk-example - INFO - Mode: server, Port: 0 Demo interval: 30s
2025-08-12 19:51:25,426 - random-walk-example - INFO - Starting server node on port 45123
2025-08-12 19:51:25,426 - random-walk-example - INFO - Node peer ID: 16Uiu2HAm7EsNv5vvjPAehGAVfChjYjD63ZHyWogQRdzntSbAg9ef
2025-08-12 19:51:25,426 - random-walk-example - INFO - Node address: /ip4/127.0.0.1/tcp/45123/p2p/16Uiu2HAm7EsNv5vvjPAehGAVfChjYjD63ZHyWogQRdzntSbAg9ef
2025-08-12 19:51:25,427 - random-walk-example - INFO - Initial routing table size: 0
2025-08-12 19:51:25,427 - random-walk-example - INFO - DHT service started in SERVER mode
2025-08-12 19:51:25,430 - libp2p.discovery.random_walk.rt_refresh_manager - INFO - RT Refresh Manager started
2025-08-12 19:51:55,432 - random-walk-example - INFO - --- Iteration 1 ---
2025-08-12 19:51:55,432 - random-walk-example - INFO - Routing table size: 15
2025-08-12 19:51:55,432 - random-walk-example - INFO - Connected peers: 8
2025-08-12 19:51:55,432 - random-walk-example - INFO - Peerstore size: 42
You can also run the example in client mode:
.. code-block:: console
$ python random_walk.py --mode client
2025-08-12 19:52:15,424 - random-walk-example - INFO - === Random Walk Example for py-libp2p ===
2025-08-12 19:52:15,424 - random-walk-example - INFO - Mode: client, Port: 0 Demo interval: 30s
2025-08-12 19:52:15,426 - random-walk-example - INFO - Starting client node on port 51234
2025-08-12 19:52:15,426 - random-walk-example - INFO - Node peer ID: 16Uiu2HAmAbc123xyz...
2025-08-12 19:52:15,427 - random-walk-example - INFO - DHT service started in CLIENT mode
2025-08-12 19:52:45,432 - random-walk-example - INFO - --- Iteration 1 ---
2025-08-12 19:52:45,432 - random-walk-example - INFO - Routing table size: 8
2025-08-12 19:52:45,432 - random-walk-example - INFO - Connected peers: 5
2025-08-12 19:52:45,432 - random-walk-example - INFO - Peerstore size: 25
Command Line Options
--------------------
The example supports several command-line options:
.. code-block:: console
$ python random_walk.py --help
usage: random_walk.py [-h] [--mode {server,client}] [--port PORT]
[--demo-interval DEMO_INTERVAL] [--verbose]
Random Walk Example for py-libp2p Kademlia DHT
optional arguments:
-h, --help show this help message and exit
--mode {server,client}
Node mode: server (DHT server), or client (DHT client)
--port PORT Port to listen on (0 for random)
--demo-interval DEMO_INTERVAL
Interval between random walk demonstrations in seconds
--verbose Enable verbose logging
Key Features Demonstrated
-------------------------
**Automatic Random Walk Discovery**
The example shows how the Random Walk module automatically:
* Generates random 256-bit peer IDs for discovery queries
* Performs concurrent random walks to maximize peer discovery
* Validates discovered peers and adds them to the routing table
* Maintains routing table health through periodic refreshes
**Real-time Network Statistics**
The example displays live statistics every 30 seconds (configurable):
* **Routing Table Size**: Number of peers in the Kademlia routing table
* **Connected Peers**: Number of actively connected peers
* **Peerstore Size**: Total number of known peers with addresses
**Connection Management**
The example includes sophisticated connection management:
* Automatically maintains connections to healthy peers
* Filters for compatible peers (TCP + IPv4 addresses)
* Reconnects to maintain optimal network connectivity
* Handles connection failures gracefully
**DHT Integration**
Shows seamless integration between Random Walk and Kademlia DHT:
* RT Refresh Manager coordinates with the DHT routing table
* Peer discovery feeds directly into DHT operations
* Both SERVER and CLIENT modes supported
* Bootstrap connectivity to public IPFS nodes
Understanding the Output
------------------------
When you run the example, you'll see periodic statistics that show how the Random Walk module is working:
* **Initial Phase**: Routing table starts empty and quickly discovers peers
* **Growth Phase**: Routing table size increases as more peers are discovered
* **Maintenance Phase**: Routing table size stabilizes as the system maintains optimal peer connections
The Random Walk module runs automatically in the background, performing peer discovery queries every few minutes to ensure the routing table remains populated with fresh, reachable peers.
Configuration
-------------
The Random Walk module can be configured through the following parameters in ``libp2p.discovery.random_walk.config``:
* ``RANDOM_WALK_ENABLED``: Enable/disable automatic random walks (default: True)
* ``REFRESH_INTERVAL``: Time between automatic refreshes in seconds (default: 300)
* ``RANDOM_WALK_CONCURRENCY``: Number of concurrent random walks (default: 3)
* ``MIN_RT_REFRESH_THRESHOLD``: Minimum routing table size before triggering refresh (default: 4)
See Also
--------
* :doc:`examples.kademlia` - Kademlia DHT value storage and content routing
* :doc:`libp2p.discovery.random_walk` - Random Walk module API documentation

View File

@ -9,5 +9,11 @@ Examples
examples.identify_push
examples.chat
examples.echo
examples.echo_quic
examples.ping
examples.pubsub
examples.circuit_relay
examples.kademlia
examples.mDNS
examples.random_walk
examples.multiple_connections

View File

@ -28,6 +28,11 @@ For Python, the most common transport is TCP. Here's how to set up a basic TCP t
.. literalinclude:: ../examples/doc-examples/example_transport.py
:language: python
Also, QUIC is a modern transport protocol that provides built-in TLS security and stream multiplexing over UDP:
.. literalinclude:: ../examples/doc-examples/example_quic_transport.py
:language: python
Connection Encryption
^^^^^^^^^^^^^^^^^^^^^

View File

@ -12,10 +12,6 @@ The Python implementation of the libp2p networking stack
getting_started
release_notes
.. toctree::
:maxdepth: 1
:caption: Community
.. toctree::
:maxdepth: 1
:caption: py-libp2p

View File

@ -0,0 +1,13 @@
libp2p.discovery.bootstrap package
==================================
Submodules
----------
Module contents
---------------
.. automodule:: libp2p.discovery.bootstrap
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,21 @@
libp2p.discovery.events package
===============================
Submodules
----------
libp2p.discovery.events.peerDiscovery module
--------------------------------------------
.. automodule:: libp2p.discovery.events.peerDiscovery
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.discovery.events
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,45 @@
libp2p.discovery.mdns package
=============================
Submodules
----------
libp2p.discovery.mdns.broadcaster module
----------------------------------------
.. automodule:: libp2p.discovery.mdns.broadcaster
:members:
:undoc-members:
:show-inheritance:
libp2p.discovery.mdns.listener module
-------------------------------------
.. automodule:: libp2p.discovery.mdns.listener
:members:
:undoc-members:
:show-inheritance:
libp2p.discovery.mdns.mdns module
---------------------------------
.. automodule:: libp2p.discovery.mdns.mdns
:members:
:undoc-members:
:show-inheritance:
libp2p.discovery.mdns.utils module
----------------------------------
.. automodule:: libp2p.discovery.mdns.utils
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.discovery.mdns
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,48 @@
libp2p.discovery.random_walk package
====================================
The Random Walk module implements a peer discovery mechanism.
It performs random walks through the DHT network to discover new peers and maintain routing table health through periodic refreshes.
Submodules
----------
libp2p.discovery.random_walk.config module
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: libp2p.discovery.random_walk.config
:members:
:undoc-members:
:show-inheritance:
libp2p.discovery.random_walk.exceptions module
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: libp2p.discovery.random_walk.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.discovery.random_walk.random_walk module
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: libp2p.discovery.random_walk.random_walk
:members:
:undoc-members:
:show-inheritance:
libp2p.discovery.random_walk.rt_refresh_manager module
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: libp2p.discovery.random_walk.rt_refresh_manager
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.discovery.random_walk
:members:
:undoc-members:
:show-inheritance:

24
docs/libp2p.discovery.rst Normal file
View File

@ -0,0 +1,24 @@
libp2p.discovery package
========================
Subpackages
-----------
.. toctree::
:maxdepth: 4
libp2p.discovery.bootstrap
libp2p.discovery.events
libp2p.discovery.mdns
libp2p.discovery.random_walk
Submodules
----------
Module contents
---------------
.. automodule:: libp2p.discovery
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,22 @@
libp2p.kad\_dht.pb package
==========================
Submodules
----------
libp2p.kad_dht.pb.kademlia_pb2 module
-------------------------------------
.. automodule:: libp2p.kad_dht.pb.kademlia_pb2
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.kad_dht.pb
:no-index:
:members:
:undoc-members:
:show-inheritance:

77
docs/libp2p.kad_dht.rst Normal file
View File

@ -0,0 +1,77 @@
libp2p.kad\_dht package
=======================
Subpackages
-----------
.. toctree::
:maxdepth: 4
libp2p.kad_dht.pb
Submodules
----------
libp2p.kad\_dht.kad\_dht module
-------------------------------
.. automodule:: libp2p.kad_dht.kad_dht
:members:
:undoc-members:
:show-inheritance:
libp2p.kad\_dht.peer\_routing module
------------------------------------
.. automodule:: libp2p.kad_dht.peer_routing
:members:
:undoc-members:
:show-inheritance:
libp2p.kad\_dht.provider\_store module
--------------------------------------
.. automodule:: libp2p.kad_dht.provider_store
:members:
:undoc-members:
:show-inheritance:
libp2p.kad\_dht.routing\_table module
-------------------------------------
.. automodule:: libp2p.kad_dht.routing_table
:members:
:undoc-members:
:show-inheritance:
libp2p.kad\_dht.utils module
----------------------------
.. automodule:: libp2p.kad_dht.utils
:members:
:undoc-members:
:show-inheritance:
libp2p.kad\_dht.value\_store module
-----------------------------------
.. automodule:: libp2p.kad_dht.value_store
:members:
:undoc-members:
:show-inheritance:
libp2p.kad\_dht.pb
------------------
.. automodule:: libp2p.kad_dht.pb
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.kad_dht
:members:
:undoc-members:
:show-inheritance:

View File

@ -0,0 +1,22 @@
libp2p.relay.circuit_v2.pb package
==================================
Submodules
----------
libp2p.relay.circuit_v2.pb.circuit_pb2 module
---------------------------------------------
.. automodule:: libp2p.relay.circuit_v2.pb.circuit_pb2
:members:
:show-inheritance:
:undoc-members:
Module contents
---------------
.. automodule:: libp2p.relay.circuit_v2.pb
:members:
:show-inheritance:
:undoc-members:
:no-index:

View File

@ -0,0 +1,70 @@
libp2p.relay.circuit_v2 package
===============================
Subpackages
-----------
.. toctree::
:maxdepth: 4
libp2p.relay.circuit_v2.pb
Submodules
----------
libp2p.relay.circuit_v2.protocol module
---------------------------------------
.. automodule:: libp2p.relay.circuit_v2.protocol
:members:
:show-inheritance:
:undoc-members:
libp2p.relay.circuit_v2.transport module
----------------------------------------
.. automodule:: libp2p.relay.circuit_v2.transport
:members:
:show-inheritance:
:undoc-members:
libp2p.relay.circuit_v2.discovery module
----------------------------------------
.. automodule:: libp2p.relay.circuit_v2.discovery
:members:
:show-inheritance:
:undoc-members:
libp2p.relay.circuit_v2.resources module
----------------------------------------
.. automodule:: libp2p.relay.circuit_v2.resources
:members:
:show-inheritance:
:undoc-members:
libp2p.relay.circuit_v2.config module
-------------------------------------
.. automodule:: libp2p.relay.circuit_v2.config
:members:
:show-inheritance:
:undoc-members:
libp2p.relay.circuit_v2.protocol_buffer module
----------------------------------------------
.. automodule:: libp2p.relay.circuit_v2.protocol_buffer
:members:
:show-inheritance:
:undoc-members:
Module contents
---------------
.. automodule:: libp2p.relay.circuit_v2
:members:
:show-inheritance:
:undoc-members:
:no-index:

19
docs/libp2p.relay.rst Normal file
View File

@ -0,0 +1,19 @@
libp2p.relay package
====================
Subpackages
-----------
.. toctree::
:maxdepth: 4
libp2p.relay.circuit_v2
Module contents
---------------
.. automodule:: libp2p.relay
:members:
:show-inheritance:
:undoc-members:
:no-index:

View File

@ -8,13 +8,16 @@ Subpackages
:maxdepth: 4
libp2p.crypto
libp2p.discovery
libp2p.host
libp2p.identity
libp2p.io
libp2p.kad_dht
libp2p.network
libp2p.peer
libp2p.protocol_muxer
libp2p.pubsub
libp2p.relay
libp2p.security
libp2p.stream_muxer
libp2p.tools

View File

@ -0,0 +1,77 @@
libp2p.transport.quic package
=============================
Submodules
----------
libp2p.transport.quic.config module
-----------------------------------
.. automodule:: libp2p.transport.quic.config
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.quic.connection module
---------------------------------------
.. automodule:: libp2p.transport.quic.connection
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.quic.exceptions module
---------------------------------------
.. automodule:: libp2p.transport.quic.exceptions
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.quic.listener module
-------------------------------------
.. automodule:: libp2p.transport.quic.listener
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.quic.security module
-------------------------------------
.. automodule:: libp2p.transport.quic.security
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.quic.stream module
-----------------------------------
.. automodule:: libp2p.transport.quic.stream
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.quic.transport module
--------------------------------------
.. automodule:: libp2p.transport.quic.transport
:members:
:undoc-members:
:show-inheritance:
libp2p.transport.quic.utils module
----------------------------------
.. automodule:: libp2p.transport.quic.utils
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: libp2p.transport.quic
:members:
:undoc-members:
:show-inheritance:

View File

@ -9,6 +9,11 @@ Subpackages
libp2p.transport.tcp
.. toctree::
:maxdepth: 4
libp2p.transport.quic
Submodules
----------

View File

@ -3,6 +3,110 @@ Release Notes
.. towncrier release notes start
py-libp2p v0.2.9 (2025-07-09)
-----------------------------
Breaking Changes
~~~~~~~~~~~~~~~~
- Reordered the arguments to ``upgrade_security`` to place ``is_initiator`` before ``peer_id``, and made ``peer_id`` optional.
This allows the method to reflect the fact that peer identity is not required for inbound connections. (`#681 <https://github.com/libp2p/py-libp2p/issues/681>`__)
Bugfixes
~~~~~~~~
- Add timeout wrappers in:
1. ``multiselect.py``: ``negotiate`` function
2. ``multiselect_client.py``: ``select_one_of`` , ``query_multistream_command`` functions
to prevent indefinite hangs when a remote peer does not respond. (`#696 <https://github.com/libp2p/py-libp2p/issues/696>`__)
- Align stream creation logic with yamux specification (`#701 <https://github.com/libp2p/py-libp2p/issues/701>`__)
- Fixed an issue in ``Pubsub`` where async validators were not handled reliably under concurrency. Now uses a safe aggregator list for consistent behavior. (`#702 <https://github.com/libp2p/py-libp2p/issues/702>`__)
Features
~~~~~~~~
- Added support for ``Kademlia DHT`` in py-libp2p. (`#579 <https://github.com/libp2p/py-libp2p/issues/579>`__)
- Limit concurrency in ``push_identify_to_peers`` to prevent resource congestion under high peer counts. (`#621 <https://github.com/libp2p/py-libp2p/issues/621>`__)
- Store public key and peer ID in peerstore during handshake
Modified the InsecureTransport class to accept an optional peerstore parameter and updated the handshake process to store the received public key and peer ID in the peerstore when available.
Added test cases to verify:
1. The peerstore remains unchanged when handshake fails due to peer ID mismatch
2. The handshake correctly adds a public key to a peer ID that already exists in the peerstore but doesn't have a public key yet (`#631 <https://github.com/libp2p/py-libp2p/issues/631>`__)
- Fixed several flow-control and concurrency issues in the ``YamuxStream`` class. Previously, stress-testing revealed that transferring data over ``DEFAULT_WINDOW_SIZE`` would break the stream due to inconsistent window update handling and lock management. The fixes include:
- Removed sending of window updates during writes to maintain correct flow-control.
- Added proper timeout handling when releasing and acquiring locks to prevent concurrency errors.
- Corrected the ``read`` function to properly handle window updates for both ``read_until_EOF`` and ``read_n_bytes``.
- Added event logging at ``send_window_updates`` and ``waiting_for_window_updates`` for better observability. (`#639 <https://github.com/libp2p/py-libp2p/issues/639>`__)
- Added support for ``Multicast DNS`` in py-libp2p (`#649 <https://github.com/libp2p/py-libp2p/issues/649>`__)
- Optimized pubsub publishing to send multiple topics in a single message instead of separate messages per topic. (`#685 <https://github.com/libp2p/py-libp2p/issues/685>`__)
- Optimized pubsub message writing by implementing a write_msg() method that uses pre-allocated buffers and single write operations, improving performance by eliminating separate varint prefix encoding and write operations in FloodSub and GossipSub. (`#687 <https://github.com/libp2p/py-libp2p/issues/687>`__)
- Added peer exchange and backoff logic as part of Gossipsub v1.1 upgrade (`#690 <https://github.com/libp2p/py-libp2p/issues/690>`__)
Internal Changes - for py-libp2p Contributors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Added sparse connect utility function to pubsub test utilities for creating test networks with configurable connectivity. (`#679 <https://github.com/libp2p/py-libp2p/issues/679>`__)
- Added comprehensive tests for pubsub connection utility functions to verify degree limits are enforced, excess peers are handled correctly, and edge cases (degree=0, negative values, empty lists) are managed gracefully. (`#707 <https://github.com/libp2p/py-libp2p/issues/707>`__)
- Added extra tests for identify push concurrency cap under high peer load (`#708 <https://github.com/libp2p/py-libp2p/issues/708>`__)
Miscellaneous Changes
~~~~~~~~~~~~~~~~~~~~~
- `#678 <https://github.com/libp2p/py-libp2p/issues/678>`__, `#684 <https://github.com/libp2p/py-libp2p/issues/684>`__
py-libp2p v0.2.8 (2025-06-10)
-----------------------------
Breaking Changes
~~~~~~~~~~~~~~~~
- The `NetStream.state` property is now async and requires `await`. Update any direct state access to use `await stream.state`. (`#300 <https://github.com/libp2p/py-libp2p/issues/300>`__)
Bugfixes
~~~~~~~~
- Added proper state management and resource cleanup to `NetStream`, fixing memory leaks and improved error handling. (`#300 <https://github.com/libp2p/py-libp2p/issues/300>`__)
Improved Documentation
~~~~~~~~~~~~~~~~~~~~~~
- Updated examples to automatically use random port, when `-p` flag is not given (`#661 <https://github.com/libp2p/py-libp2p/issues/661>`__)
Features
~~~~~~~~
- Allow passing `listen_addrs` to `new_swarm` to customize swarm listening behavior. (`#616 <https://github.com/libp2p/py-libp2p/issues/616>`__)
- Feature: Support for sending `ls` command over `multistream-select` to list supported protocols from remote peer.
This allows inspecting which protocol handlers a peer supports at runtime. (`#622 <https://github.com/libp2p/py-libp2p/issues/622>`__)
- implement AsyncContextManager for IMuxedStream to support async with (`#629 <https://github.com/libp2p/py-libp2p/issues/629>`__)
- feat: add method to compute time since last message published by a peer and remove fanout peers based on ttl. (`#636 <https://github.com/libp2p/py-libp2p/issues/636>`__)
- implement blacklist management for `pubsub.Pubsub` with methods to get, add, remove, check, and clear blacklisted peer IDs. (`#641 <https://github.com/libp2p/py-libp2p/issues/641>`__)
- fix: remove expired peers from peerstore based on TTL (`#650 <https://github.com/libp2p/py-libp2p/issues/650>`__)
Internal Changes - for py-libp2p Contributors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Modernizes several aspects of the project, notably using ``pyproject.toml`` for project info instead of ``setup.py``, using ``ruff`` to replace several separate linting tools, and ``pyrefly`` in addition to ``mypy`` for typing. Also includes changes across the codebase to conform to new linting and typing rules. (`#618 <https://github.com/libp2p/py-libp2p/issues/618>`__)
Removals
~~~~~~~~
- Removes support for python 3.9 and updates some code conventions, notably using ``|`` operator in typing instead of ``Optional`` or ``Union`` (`#618 <https://github.com/libp2p/py-libp2p/issues/618>`__)
py-libp2p v0.2.7 (2025-05-22)
-----------------------------

View File

@ -0,0 +1,90 @@
"""
Advanced demonstration of Thin Waist address handling.
Run:
python -m examples.advanced.network_discovery
"""
from __future__ import annotations
from multiaddr import Multiaddr
try:
from libp2p.utils.address_validation import (
expand_wildcard_address,
get_available_interfaces,
get_optimal_binding_address,
get_wildcard_address,
)
except ImportError:
# Fallbacks if utilities are missing - use minimal network discovery
import socket
def get_available_interfaces(port: int, protocol: str = "tcp"):
# Try to get local network interfaces, fallback to loopback
addrs = []
try:
# Get hostname IP (better than hardcoded localhost)
hostname = socket.gethostname()
local_ip = socket.gethostbyname(hostname)
if local_ip != "127.0.0.1":
addrs.append(Multiaddr(f"/ip4/{local_ip}/{protocol}/{port}"))
except Exception:
pass
# Always include loopback as fallback
addrs.append(Multiaddr(f"/ip4/127.0.0.1/{protocol}/{port}"))
return addrs
def expand_wildcard_address(addr: Multiaddr, port: int | None = None):
if port is None:
return [addr]
addr_str = str(addr).rsplit("/", 1)[0]
return [Multiaddr(addr_str + f"/{port}")]
def get_optimal_binding_address(port: int, protocol: str = "tcp"):
# Try to get a non-loopback address first
interfaces = get_available_interfaces(port, protocol)
for addr in interfaces:
if "127.0.0.1" not in str(addr):
return addr
# Fallback to loopback if no other interfaces found
return Multiaddr(f"/ip4/127.0.0.1/{protocol}/{port}")
def get_wildcard_address(port: int, protocol: str = "tcp"):
return Multiaddr(f"/ip4/0.0.0.0/{protocol}/{port}")
def main() -> None:
port = 8080
interfaces = get_available_interfaces(port)
print(f"Discovered interfaces for port {port}:")
for a in interfaces:
print(f" - {a}")
# Demonstrate wildcard address as a feature
wildcard_v4 = get_wildcard_address(port)
print(f"\nWildcard address (feature): {wildcard_v4}")
expanded_v4 = expand_wildcard_address(wildcard_v4)
print("\nExpanded IPv4 wildcard:")
for a in expanded_v4:
print(f" - {a}")
wildcard_v6 = Multiaddr(f"/ip6/::/tcp/{port}")
expanded_v6 = expand_wildcard_address(wildcard_v6)
print("\nExpanded IPv6 wildcard:")
for a in expanded_v6:
print(f" - {a}")
print("\nOptimal binding address heuristic result:")
print(f" -> {get_optimal_binding_address(port)}")
override_port = 9000
overridden = expand_wildcard_address(wildcard_v4, port=override_port)
print(f"\nPort override expansion to {override_port}:")
for a in overridden:
print(f" - {a}")
if __name__ == "__main__":
main()

View File

@ -0,0 +1,158 @@
import argparse
import logging
import secrets
import trio
from libp2p import new_host
from libp2p.abc import PeerInfo
from libp2p.crypto.secp256k1 import create_new_key_pair
from libp2p.discovery.events.peerDiscovery import peerDiscovery
# Configure logging
logger = logging.getLogger("libp2p.discovery.bootstrap")
logger.setLevel(logging.INFO)
handler = logging.StreamHandler()
handler.setFormatter(
logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
)
logger.addHandler(handler)
# Configure root logger to only show warnings and above to reduce noise
# This prevents verbose DEBUG messages from multiaddr, DNS, etc.
logging.getLogger().setLevel(logging.WARNING)
# Specifically silence noisy libraries
logging.getLogger("multiaddr").setLevel(logging.WARNING)
logging.getLogger("root").setLevel(logging.WARNING)
def on_peer_discovery(peer_info: PeerInfo) -> None:
"""Handler for peer discovery events."""
logger.info(f"🔍 Discovered peer: {peer_info.peer_id}")
logger.debug(f" Addresses: {[str(addr) for addr in peer_info.addrs]}")
# Example bootstrap peers
BOOTSTRAP_PEERS = [
"/dnsaddr/github.com/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
"/dnsaddr/cloudflare.com/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
"/dnsaddr/google.com/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
"/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
"/ip6/2604:a880:1:20::203:d001/tcp/4001/p2p/QmSoLPppuBtQSGwKDZT2M73ULpjvfd3aZ6ha4oFGL1KrGM",
"/ip4/128.199.219.111/tcp/4001/p2p/QmSoLV4Bbm51jM9C4gDYZQ9Cy3U6aXMJDAbzgu2fzaDs64",
"/ip4/104.236.76.40/tcp/4001/p2p/QmSoLV4Bbm51jM9C4gDYZQ9Cy3U6aXMJDAbzgu2fzaDs64",
"/ip4/178.62.158.247/tcp/4001/p2p/QmSoLer265NRgSp2LA3dPaeykiS1J6DifTC88f5uVQKNAd",
"/ip6/2604:a880:1:20::203:d001/tcp/4001/p2p/QmSoLPppuBtQSGwKDZT2M73ULpjvfd3aZ6ha4oFGL1KrGM",
"/ip6/2400:6180:0:d0::151:6001/tcp/4001/p2p/QmSoLSafTMBsPKadTEgaXctDQVcqN88CNLHXMkTNwMKPnu",
"/ip6/2a03:b0c0:0:1010::23:1001/tcp/4001/p2p/QmSoLueR4xBeUbY9WZ9xGUUxunbKWcrNFTDAadQJmocnWm",
]
async def run(port: int, bootstrap_addrs: list[str]) -> None:
"""Run the bootstrap discovery example."""
from libp2p.utils.address_validation import (
find_free_port,
get_available_interfaces,
get_optimal_binding_address,
)
if port <= 0:
port = find_free_port()
# Generate key pair
secret = secrets.token_bytes(32)
key_pair = create_new_key_pair(secret)
# Create listen addresses for all available interfaces
listen_addrs = get_available_interfaces(port)
# Register peer discovery handler
peerDiscovery.register_peer_discovered_handler(on_peer_discovery)
logger.info("🚀 Starting Bootstrap Discovery Example")
logger.info(f"🌐 Bootstrap peers: {len(bootstrap_addrs)}")
print("\n" + "=" * 60)
print("Bootstrap Discovery Example")
print("=" * 60)
print("This example demonstrates connecting to bootstrap peers.")
print("Watch the logs for peer discovery events!")
print("Press Ctrl+C to exit.")
print("=" * 60)
# Create and run host with bootstrap discovery
host = new_host(key_pair=key_pair, bootstrap=bootstrap_addrs)
try:
async with host.run(listen_addrs=listen_addrs):
# Get all available addresses with peer ID
all_addrs = host.get_addrs()
logger.info("Listener ready, listening on:")
print("Listener ready, listening on:")
for addr in all_addrs:
logger.info(f"{addr}")
print(f"{addr}")
# Display optimal address for reference
optimal_addr = get_optimal_binding_address(port)
optimal_addr_with_peer = f"{optimal_addr}/p2p/{host.get_id().to_string()}"
logger.info(f"Optimal address: {optimal_addr_with_peer}")
print(f"Optimal address: {optimal_addr_with_peer}")
# Keep running and log peer discovery events
await trio.sleep_forever()
except KeyboardInterrupt:
logger.info("👋 Shutting down...")
def main() -> None:
"""Main entry point."""
description = """
Bootstrap Discovery Example for py-libp2p
This example demonstrates how to use bootstrap peers for peer discovery.
Bootstrap peers are predefined peers that help new nodes join the network.
Usage:
python bootstrap.py -p 8000
python bootstrap.py -p 8001 --custom-bootstrap \\
"/ip4/[HOST_IP]/tcp/8000/p2p/QmYourPeerID"
"""
parser = argparse.ArgumentParser(
description=description, formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument(
"-p", "--port", default=0, type=int, help="Port to listen on (default: random)"
)
parser.add_argument(
"--custom-bootstrap",
nargs="*",
help="Custom bootstrap addresses (space-separated)",
)
parser.add_argument(
"-v", "--verbose", action="store_true", help="Enable verbose output"
)
args = parser.parse_args()
if args.verbose:
logger.setLevel(logging.DEBUG)
# Use custom bootstrap addresses if provided, otherwise use defaults
bootstrap_addrs = (
args.custom_bootstrap if args.custom_bootstrap else BOOTSTRAP_PEERS
)
try:
trio.run(run, args.port, bootstrap_addrs)
except KeyboardInterrupt:
logger.info("Exiting...")
if __name__ == "__main__":
main()

View File

@ -1,4 +1,5 @@
import argparse
import logging
import sys
import multiaddr
@ -17,6 +18,11 @@ from libp2p.peer.peerinfo import (
info_from_p2p_addr,
)
# Configure minimal logging
logging.basicConfig(level=logging.WARNING)
logging.getLogger("multiaddr").setLevel(logging.WARNING)
logging.getLogger("libp2p").setLevel(logging.WARNING)
PROTOCOL_ID = TProtocol("/chat/1.0.0")
MAX_READ_LEN = 2**32 - 1
@ -40,10 +46,21 @@ async def write_data(stream: INetStream) -> None:
async def run(port: int, destination: str) -> None:
localhost_ip = "127.0.0.1"
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
from libp2p.utils.address_validation import (
find_free_port,
get_available_interfaces,
get_optimal_binding_address,
)
if port <= 0:
port = find_free_port()
listen_addrs = get_available_interfaces(port)
host = new_host()
async with host.run(listen_addrs=[listen_addr]), trio.open_nursery() as nursery:
async with host.run(listen_addrs=listen_addrs), trio.open_nursery() as nursery:
# Start the peer-store cleanup task
nursery.start_soon(host.get_peerstore().start_cleanup_task, 60)
if not destination: # its the server
async def stream_handler(stream: INetStream) -> None:
@ -52,10 +69,19 @@ async def run(port: int, destination: str) -> None:
host.set_stream_handler(PROTOCOL_ID, stream_handler)
# Get all available addresses with peer ID
all_addrs = host.get_addrs()
print("Listener ready, listening on:\n")
for addr in all_addrs:
print(f"{addr}")
# Use optimal address for the client command
optimal_addr = get_optimal_binding_address(port)
optimal_addr_with_peer = f"{optimal_addr}/p2p/{host.get_id().to_string()}"
print(
"Run this from the same folder in another console:\n\n"
f"chat-demo -p {int(port) + 1} "
f"-d /ip4/{localhost_ip}/tcp/{port}/p2p/{host.get_id().pretty()}\n"
f"\nRun this from the same folder in another console:\n\n"
f"chat-demo -d {optimal_addr_with_peer}\n"
)
print("Waiting for incoming connection...")
@ -84,12 +110,10 @@ def main() -> None:
where <DESTINATION> is the multiaddress of the previous listener host.
"""
example_maddr = (
"/ip4/127.0.0.1/tcp/8000/p2p/QmQn4SwGkDZKkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
"/ip4/[HOST_IP]/tcp/8000/p2p/QmQn4SwGkDZKkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
)
parser = argparse.ArgumentParser(description=description)
parser.add_argument(
"-p", "--port", default=8000, type=int, help="source port number"
)
parser.add_argument("-p", "--port", default=0, type=int, help="source port number")
parser.add_argument(
"-d",
"--destination",
@ -98,9 +122,6 @@ def main() -> None:
)
args = parser.parse_args()
if not args.port:
raise RuntimeError("was not able to determine a local port")
try:
trio.run(run, *(args.port, args.destination))
except KeyboardInterrupt:

View File

@ -1,6 +1,5 @@
import secrets
import multiaddr
import trio
from libp2p import (
@ -9,9 +8,10 @@ from libp2p import (
from libp2p.crypto.secp256k1 import (
create_new_key_pair,
)
from libp2p.security.insecure.transport import (
PLAINTEXT_PROTOCOL_ID,
InsecureTransport,
from libp2p.security.insecure.transport import PLAINTEXT_PROTOCOL_ID, InsecureTransport
from libp2p.utils.address_validation import (
get_available_interfaces,
get_optimal_binding_address,
)
@ -27,6 +27,9 @@ async def main():
# secure_bytes_provider: Optional function to generate secure random bytes
# (defaults to secrets.token_bytes)
secure_bytes_provider=None, # Use default implementation
# peerstore: Optional peerstore to store peer IDs and public keys
# (defaults to None)
peerstore=None,
)
# Create a security options dictionary mapping protocol ID to transport
@ -35,17 +38,19 @@ async def main():
# Create a host with the key pair and insecure transport
host = new_host(key_pair=key_pair, sec_opt=security_options)
# Configure the listening address
# Configure the listening address using the new paradigm
port = 8000
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
listen_addrs = get_available_interfaces(port)
optimal_addr = get_optimal_binding_address(port)
# Start the host
async with host.run(listen_addrs=[listen_addr]):
async with host.run(listen_addrs=listen_addrs):
print(
"libp2p has started with insecure transport "
"(not recommended for production)"
)
print("libp2p is listening on:", host.get_addrs())
print(f"Optimal address: {optimal_addr}")
# Keep the host running
await trio.sleep_forever()

View File

@ -1,6 +1,5 @@
import secrets
import multiaddr
import trio
from libp2p import (
@ -9,8 +8,14 @@ from libp2p import (
from libp2p.crypto.secp256k1 import (
create_new_key_pair,
)
from libp2p.security.noise.transport import PROTOCOL_ID as NOISE_PROTOCOL_ID
from libp2p.security.noise.transport import Transport as NoiseTransport
from libp2p.security.noise.transport import (
PROTOCOL_ID as NOISE_PROTOCOL_ID,
Transport as NoiseTransport,
)
from libp2p.utils.address_validation import (
get_available_interfaces,
get_optimal_binding_address,
)
async def main():
@ -37,14 +42,16 @@ async def main():
# Create a host with the key pair and Noise security
host = new_host(key_pair=key_pair, sec_opt=security_options)
# Configure the listening address
# Configure the listening address using the new paradigm
port = 8000
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
listen_addrs = get_available_interfaces(port)
optimal_addr = get_optimal_binding_address(port)
# Start the host
async with host.run(listen_addrs=[listen_addr]):
async with host.run(listen_addrs=listen_addrs):
print("libp2p has started with Noise encryption")
print("libp2p is listening on:", host.get_addrs())
print(f"Optimal address: {optimal_addr}")
# Keep the host running
await trio.sleep_forever()

View File

@ -1,6 +1,5 @@
import secrets
import multiaddr
import trio
from libp2p import (
@ -9,8 +8,14 @@ from libp2p import (
from libp2p.crypto.secp256k1 import (
create_new_key_pair,
)
from libp2p.security.secio.transport import ID as SECIO_PROTOCOL_ID
from libp2p.security.secio.transport import Transport as SecioTransport
from libp2p.security.secio.transport import (
ID as SECIO_PROTOCOL_ID,
Transport as SecioTransport,
)
from libp2p.utils.address_validation import (
get_available_interfaces,
get_optimal_binding_address,
)
async def main():
@ -22,9 +27,6 @@ async def main():
secio_transport = SecioTransport(
# local_key_pair: The key pair used for libp2p identity and authentication
local_key_pair=key_pair,
# secure_bytes_provider: Optional function to generate secure random bytes
# (defaults to secrets.token_bytes)
secure_bytes_provider=None, # Use default implementation
)
# Create a security options dictionary mapping protocol ID to transport
@ -33,14 +35,16 @@ async def main():
# Create a host with the key pair and SECIO security
host = new_host(key_pair=key_pair, sec_opt=security_options)
# Configure the listening address
# Configure the listening address using the new paradigm
port = 8000
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
listen_addrs = get_available_interfaces(port)
optimal_addr = get_optimal_binding_address(port)
# Start the host
async with host.run(listen_addrs=[listen_addr]):
async with host.run(listen_addrs=listen_addrs):
print("libp2p has started with SECIO encryption")
print("libp2p is listening on:", host.get_addrs())
print(f"Optimal address: {optimal_addr}")
# Keep the host running
await trio.sleep_forever()

View File

@ -1,6 +1,5 @@
import secrets
import multiaddr
import trio
from libp2p import (
@ -9,10 +8,13 @@ from libp2p import (
from libp2p.crypto.secp256k1 import (
create_new_key_pair,
)
from libp2p.security.noise.transport import PROTOCOL_ID as NOISE_PROTOCOL_ID
from libp2p.security.noise.transport import Transport as NoiseTransport
from libp2p.stream_muxer.mplex.mplex import (
MPLEX_PROTOCOL_ID,
from libp2p.security.noise.transport import (
PROTOCOL_ID as NOISE_PROTOCOL_ID,
Transport as NoiseTransport,
)
from libp2p.utils.address_validation import (
get_available_interfaces,
get_optimal_binding_address,
)
@ -37,23 +39,19 @@ async def main():
# Create a security options dictionary mapping protocol ID to transport
security_options = {NOISE_PROTOCOL_ID: noise_transport}
# Create a muxer options dictionary mapping protocol ID to muxer class
# We don't need to instantiate the muxer here, the host will do that for us
muxer_options = {MPLEX_PROTOCOL_ID: None}
# Create a host with the key pair, Noise security, and mplex multiplexer
host = new_host(
key_pair=key_pair, sec_opt=security_options, muxer_opt=muxer_options
)
host = new_host(key_pair=key_pair, sec_opt=security_options)
# Configure the listening address
# Configure the listening address using the new paradigm
port = 8000
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
listen_addrs = get_available_interfaces(port)
optimal_addr = get_optimal_binding_address(port)
# Start the host
async with host.run(listen_addrs=[listen_addr]):
async with host.run(listen_addrs=listen_addrs):
print("libp2p has started with Noise encryption and mplex multiplexing")
print("libp2p is listening on:", host.get_addrs())
print(f"Optimal address: {optimal_addr}")
# Keep the host running
await trio.sleep_forever()

View File

@ -0,0 +1,271 @@
"""
Enhanced NetStream Example for py-libp2p with State Management
This example demonstrates the new NetStream features including:
- State tracking and transitions
- Proper error handling and validation
- Resource cleanup and event notifications
- Thread-safe operations with Trio locks
Based on the standard echo demo but enhanced to show NetStream state management.
"""
import argparse
import random
import secrets
import multiaddr
import trio
from libp2p import (
new_host,
)
from libp2p.crypto.secp256k1 import (
create_new_key_pair,
)
from libp2p.custom_types import (
TProtocol,
)
from libp2p.network.stream.exceptions import (
StreamClosed,
StreamEOF,
StreamReset,
)
from libp2p.network.stream.net_stream import (
NetStream,
StreamState,
)
from libp2p.peer.peerinfo import (
info_from_p2p_addr,
)
from libp2p.utils.address_validation import (
get_available_interfaces,
get_optimal_binding_address,
)
PROTOCOL_ID = TProtocol("/echo/1.0.0")
async def enhanced_echo_handler(stream: NetStream) -> None:
"""
Enhanced echo handler that demonstrates NetStream state management.
"""
print(f"New connection established: {stream}")
print(f"Initial stream state: {await stream.state}")
try:
# Verify stream is in expected initial state
assert await stream.state == StreamState.OPEN
assert await stream.is_readable()
assert await stream.is_writable()
print("✓ Stream initialized in OPEN state")
# Read incoming data with proper state checking
print("Waiting for client data...")
while await stream.is_readable():
try:
# Read data from client
data = await stream.read(1024)
if not data:
print("Received empty data, client may have closed")
break
print(f"Received: {data.decode('utf-8').strip()}")
# Check if we can still write before echoing
if await stream.is_writable():
await stream.write(data)
print(f"Echoed: {data.decode('utf-8').strip()}")
else:
print("Cannot echo - stream not writable")
break
except StreamEOF:
print("Client closed their write side (EOF)")
break
except StreamReset:
print("Stream was reset by client")
return
except StreamClosed as e:
print(f"Stream operation failed: {e}")
break
# Demonstrate graceful closure
current_state = await stream.state
print(f"Current state before close: {current_state}")
if current_state not in [StreamState.CLOSE_BOTH, StreamState.RESET]:
await stream.close()
print("Server closed write side")
final_state = await stream.state
print(f"Final stream state: {final_state}")
except Exception as e:
print(f"Handler error: {e}")
# Reset stream on unexpected errors
if await stream.state not in [StreamState.RESET, StreamState.CLOSE_BOTH]:
await stream.reset()
print("Stream reset due to error")
async def enhanced_client_demo(stream: NetStream) -> None:
"""
Enhanced client that demonstrates various NetStream state scenarios.
"""
print(f"Client stream established: {stream}")
print(f"Initial state: {await stream.state}")
try:
# Verify initial state
assert await stream.state == StreamState.OPEN
print("✓ Client stream in OPEN state")
# Scenario 1: Normal communication
message = b"Hello from enhanced NetStream client!\n"
if await stream.is_writable():
await stream.write(message)
print(f"Sent: {message.decode('utf-8').strip()}")
else:
print("Cannot write - stream not writable")
return
# Close write side to signal EOF to server
await stream.close()
print("Client closed write side")
# Verify state transition
state_after_close = await stream.state
print(f"State after close: {state_after_close}")
assert state_after_close == StreamState.CLOSE_WRITE
assert await stream.is_readable() # Should still be readable
assert not await stream.is_writable() # Should not be writable
# Try to write (should fail)
try:
await stream.write(b"This should fail")
print("ERROR: Write succeeded when it should have failed!")
except StreamClosed as e:
print(f"✓ Expected error when writing to closed stream: {e}")
# Read the echo response
if await stream.is_readable():
try:
response = await stream.read()
print(f"Received echo: {response.decode('utf-8').strip()}")
except StreamEOF:
print("Server closed their write side")
except StreamReset:
print("Stream was reset")
# Check final state
final_state = await stream.state
print(f"Final client state: {final_state}")
except Exception as e:
print(f"Client error: {e}")
# Reset on error
await stream.reset()
print("Client reset stream due to error")
async def run_enhanced_demo(
port: int, destination: str, seed: int | None = None
) -> None:
"""
Run enhanced echo demo with NetStream state management.
"""
# Use the new address paradigm
listen_addrs = get_available_interfaces(port)
optimal_addr = get_optimal_binding_address(port)
# Generate or use provided key
if seed:
random.seed(seed)
secret_number = random.getrandbits(32 * 8)
secret = secret_number.to_bytes(length=32, byteorder="big")
else:
secret = secrets.token_bytes(32)
host = new_host(key_pair=create_new_key_pair(secret))
async with host.run(listen_addrs=listen_addrs):
print(f"Host ID: {host.get_id().to_string()}")
print("=" * 60)
if not destination: # Server mode
print("🖥️ ENHANCED ECHO SERVER MODE")
print("=" * 60)
# type: ignore: Stream is type of NetStream
host.set_stream_handler(PROTOCOL_ID, enhanced_echo_handler)
# Use optimal address for client command
optimal_addr_with_peer = f"{optimal_addr}/p2p/{host.get_id().to_string()}"
print(
"Run client from another console:\n"
f"python3 example_net_stream.py "
f"-d {optimal_addr_with_peer}\n"
)
print("Waiting for connections...")
print("Press Ctrl+C to stop server")
await trio.sleep_forever()
else: # Client mode
print("📱 ENHANCED ECHO CLIENT MODE")
print("=" * 60)
# Connect to server
maddr = multiaddr.Multiaddr(destination)
info = info_from_p2p_addr(maddr)
await host.connect(info)
print(f"Connected to server: {info.peer_id.pretty()}")
# Create stream and run enhanced demo
stream = await host.new_stream(info.peer_id, [PROTOCOL_ID])
if isinstance(stream, NetStream):
await enhanced_client_demo(stream)
print("\n" + "=" * 60)
print("CLIENT DEMO COMPLETE")
def main() -> None:
example_maddr = (
"/ip4/[HOST_IP]/tcp/8000/p2p/QmQn4SwGkDZKkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
)
parser = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument("-p", "--port", default=0, type=int, help="source port number")
parser.add_argument(
"-d",
"--destination",
type=str,
help=f"destination multiaddr string, e.g. {example_maddr}",
)
parser.add_argument(
"-s",
"--seed",
type=int,
help="seed for deterministic peer ID generation",
)
parser.add_argument(
"--demo-states", action="store_true", help="run state transition demo only"
)
args = parser.parse_args()
try:
trio.run(run_enhanced_demo, args.port, args.destination, args.seed)
except KeyboardInterrupt:
print("\n👋 Demo interrupted by user")
except Exception as e:
print(f"❌ Demo failed: {e}")
if __name__ == "__main__":
main()

View File

@ -1,6 +1,6 @@
import secrets
import multiaddr
from multiaddr import Multiaddr
import trio
from libp2p import (
@ -12,10 +12,13 @@ from libp2p.crypto.secp256k1 import (
from libp2p.peer.peerinfo import (
info_from_p2p_addr,
)
from libp2p.security.noise.transport import PROTOCOL_ID as NOISE_PROTOCOL_ID
from libp2p.security.noise.transport import Transport as NoiseTransport
from libp2p.stream_muxer.mplex.mplex import (
MPLEX_PROTOCOL_ID,
from libp2p.security.noise.transport import (
PROTOCOL_ID as NOISE_PROTOCOL_ID,
Transport as NoiseTransport,
)
from libp2p.utils.address_validation import (
get_available_interfaces,
get_optimal_binding_address,
)
@ -40,23 +43,19 @@ async def main():
# Create a security options dictionary mapping protocol ID to transport
security_options = {NOISE_PROTOCOL_ID: noise_transport}
# Create a muxer options dictionary mapping protocol ID to muxer class
# We don't need to instantiate the muxer here, the host will do that for us
muxer_options = {MPLEX_PROTOCOL_ID: None}
# Create a host with the key pair, Noise security, and mplex multiplexer
host = new_host(
key_pair=key_pair, sec_opt=security_options, muxer_opt=muxer_options
)
host = new_host(key_pair=key_pair, sec_opt=security_options)
# Configure the listening address
# Configure the listening address using the new paradigm
port = 8000
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
listen_addrs = get_available_interfaces(port)
optimal_addr = get_optimal_binding_address(port)
# Start the host
async with host.run(listen_addrs=[listen_addr]):
async with host.run(listen_addrs=listen_addrs):
print("libp2p has started")
print("libp2p is listening on:", host.get_addrs())
print(f"Optimal address: {optimal_addr}")
# Connect to bootstrap peers manually
bootstrap_list = [
@ -68,7 +67,7 @@ async def main():
for addr in bootstrap_list:
try:
peer_info = info_from_p2p_addr(multiaddr.Multiaddr(addr))
peer_info = info_from_p2p_addr(Multiaddr(addr))
await host.connect(peer_info)
print(f"Connected to {peer_info.peer_id.to_string()}")
except Exception as e:

View File

@ -0,0 +1,49 @@
import secrets
import trio
from libp2p import (
new_host,
)
from libp2p.crypto.secp256k1 import (
create_new_key_pair,
)
from libp2p.utils.address_validation import (
get_available_interfaces,
get_optimal_binding_address,
)
async def main():
# Create a key pair for the host
secret = secrets.token_bytes(32)
key_pair = create_new_key_pair(secret)
# Create a host with the key pair
host = new_host(key_pair=key_pair, enable_quic=True)
# Configure the listening address using the new paradigm
port = 8000
listen_addrs = get_available_interfaces(port, protocol="udp")
# Convert TCP addresses to QUIC-v1 addresses
quic_addrs = []
for addr in listen_addrs:
addr_str = str(addr).replace("/tcp/", "/udp/") + "/quic-v1"
from multiaddr import Multiaddr
quic_addrs.append(Multiaddr(addr_str))
optimal_addr = get_optimal_binding_address(port, protocol="udp")
optimal_quic_str = str(optimal_addr).replace("/tcp/", "/udp/") + "/quic-v1"
# Start the host
async with host.run(listen_addrs=quic_addrs):
print("libp2p has started with QUIC transport")
print("libp2p is listening on:", host.get_addrs())
print(f"Optimal address: {optimal_quic_str}")
# Keep the host running
await trio.sleep_forever()
# Run the async function
trio.run(main)

View File

@ -1,6 +1,5 @@
import secrets
import multiaddr
import trio
from libp2p import (
@ -9,10 +8,13 @@ from libp2p import (
from libp2p.crypto.secp256k1 import (
create_new_key_pair,
)
from libp2p.security.noise.transport import PROTOCOL_ID as NOISE_PROTOCOL_ID
from libp2p.security.noise.transport import Transport as NoiseTransport
from libp2p.stream_muxer.mplex.mplex import (
MPLEX_PROTOCOL_ID,
from libp2p.security.noise.transport import (
PROTOCOL_ID as NOISE_PROTOCOL_ID,
Transport as NoiseTransport,
)
from libp2p.utils.address_validation import (
get_available_interfaces,
get_optimal_binding_address,
)
@ -37,23 +39,19 @@ async def main():
# Create a security options dictionary mapping protocol ID to transport
security_options = {NOISE_PROTOCOL_ID: noise_transport}
# Create a muxer options dictionary mapping protocol ID to muxer class
# We don't need to instantiate the muxer here, the host will do that for us
muxer_options = {MPLEX_PROTOCOL_ID: None}
# Create a host with the key pair, Noise security, and mplex multiplexer
host = new_host(
key_pair=key_pair, sec_opt=security_options, muxer_opt=muxer_options
)
host = new_host(key_pair=key_pair, sec_opt=security_options)
# Configure the listening address
# Configure the listening address using the new paradigm
port = 8000
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
listen_addrs = get_available_interfaces(port)
optimal_addr = get_optimal_binding_address(port)
# Start the host
async with host.run(listen_addrs=[listen_addr]):
async with host.run(listen_addrs=listen_addrs):
print("libp2p has started")
print("libp2p is listening on:", host.get_addrs())
print(f"Optimal address: {optimal_addr}")
# Keep the host running
await trio.sleep_forever()

View File

@ -1,6 +1,5 @@
import secrets
import multiaddr
import trio
from libp2p import (
@ -9,6 +8,10 @@ from libp2p import (
from libp2p.crypto.secp256k1 import (
create_new_key_pair,
)
from libp2p.utils.address_validation import (
get_available_interfaces,
get_optimal_binding_address,
)
async def main():
@ -19,14 +22,16 @@ async def main():
# Create a host with the key pair
host = new_host(key_pair=key_pair)
# Configure the listening address
# Configure the listening address using the new paradigm
port = 8000
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
listen_addrs = get_available_interfaces(port)
optimal_addr = get_optimal_binding_address(port)
# Start the host
async with host.run(listen_addrs=[listen_addr]):
async with host.run(listen_addrs=listen_addrs):
print("libp2p has started with TCP transport")
print("libp2p is listening on:", host.get_addrs())
print(f"Optimal address: {optimal_addr}")
# Keep the host running
await trio.sleep_forever()

View File

@ -0,0 +1,210 @@
#!/usr/bin/env python3
"""
Example demonstrating multiple connections per peer support in libp2p.
This example shows how to:
1. Configure multiple connections per peer
2. Use different load balancing strategies
3. Access multiple connections through the new API
4. Maintain backward compatibility
5. Use the new address paradigm for network configuration
"""
import logging
import trio
from libp2p import new_swarm
from libp2p.network.swarm import ConnectionConfig, RetryConfig
from libp2p.utils import get_available_interfaces, get_optimal_binding_address
# Set up logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
async def example_basic_multiple_connections() -> None:
"""Example of basic multiple connections per peer usage."""
logger.info("Creating swarm with multiple connections support...")
# Create swarm with default configuration
swarm = new_swarm()
default_connection = ConnectionConfig()
logger.info(f"Swarm created with peer ID: {swarm.get_peer_id()}")
logger.info(
f"Connection config: max_connections_per_peer="
f"{default_connection.max_connections_per_peer}"
)
await swarm.close()
logger.info("Basic multiple connections example completed")
async def example_custom_connection_config() -> None:
"""Example of custom connection configuration."""
logger.info("Creating swarm with custom connection configuration...")
# Custom connection configuration for high-performance scenarios
connection_config = ConnectionConfig(
max_connections_per_peer=5, # More connections per peer
connection_timeout=60.0, # Longer timeout
load_balancing_strategy="least_loaded", # Use least loaded strategy
)
# Create swarm with custom connection config
swarm = new_swarm(connection_config=connection_config)
logger.info("Custom connection config applied:")
logger.info(
f" Max connections per peer: {connection_config.max_connections_per_peer}"
)
logger.info(f" Connection timeout: {connection_config.connection_timeout}s")
logger.info(
f" Load balancing strategy: {connection_config.load_balancing_strategy}"
)
await swarm.close()
logger.info("Custom connection config example completed")
async def example_multiple_connections_api() -> None:
"""Example of using the new multiple connections API."""
logger.info("Demonstrating multiple connections API...")
connection_config = ConnectionConfig(
max_connections_per_peer=3, load_balancing_strategy="round_robin"
)
swarm = new_swarm(connection_config=connection_config)
logger.info("Multiple connections API features:")
logger.info(" - dial_peer() returns list[INetConn]")
logger.info(" - get_connections(peer_id) returns list[INetConn]")
logger.info(" - get_connections_map() returns dict[ID, list[INetConn]]")
logger.info(
" - get_connection(peer_id) returns INetConn | None (backward compatibility)"
)
await swarm.close()
logger.info("Multiple connections API example completed")
async def example_backward_compatibility() -> None:
"""Example of backward compatibility features."""
logger.info("Demonstrating backward compatibility...")
swarm = new_swarm()
logger.info("Backward compatibility features:")
logger.info(" - connections_legacy property provides 1:1 mapping")
logger.info(" - get_connection() method for single connection access")
logger.info(" - Existing code continues to work")
await swarm.close()
logger.info("Backward compatibility example completed")
async def example_network_address_paradigm() -> None:
"""Example of using the new address paradigm with multiple connections."""
logger.info("Demonstrating network address paradigm...")
# Get available interfaces using the new paradigm
port = 8000 # Example port
available_interfaces = get_available_interfaces(port)
logger.info(f"Available interfaces: {available_interfaces}")
# Get optimal binding address
optimal_address = get_optimal_binding_address(port)
logger.info(f"Optimal binding address: {optimal_address}")
# Create connection config for multiple connections with network awareness
connection_config = ConnectionConfig(
max_connections_per_peer=3, load_balancing_strategy="round_robin"
)
# Create swarm with address paradigm
swarm = new_swarm(connection_config=connection_config)
logger.info("Network address paradigm features:")
logger.info(" - get_available_interfaces() for interface discovery")
logger.info(" - get_optimal_binding_address() for smart address selection")
logger.info(" - Multiple connections with proper network binding")
await swarm.close()
logger.info("Network address paradigm example completed")
async def example_production_ready_config() -> None:
"""Example of production-ready configuration."""
logger.info("Creating swarm with production-ready configuration...")
# Get optimal network configuration using the new paradigm
port = 8001 # Example port
optimal_address = get_optimal_binding_address(port)
logger.info(f"Using optimal binding address: {optimal_address}")
# Production-ready retry configuration
retry_config = RetryConfig(
max_retries=3, # Reasonable retry limit
initial_delay=0.1, # Quick initial retry
max_delay=30.0, # Cap exponential backoff
backoff_multiplier=2.0, # Standard exponential backoff
jitter_factor=0.1, # Small jitter to prevent thundering herd
)
# Production-ready connection configuration
connection_config = ConnectionConfig(
max_connections_per_peer=3, # Balance between performance and resource usage
connection_timeout=30.0, # Reasonable timeout
load_balancing_strategy="round_robin", # Simple, predictable strategy
)
# Create swarm with production config
swarm = new_swarm(retry_config=retry_config, connection_config=connection_config)
logger.info("Production-ready configuration applied:")
logger.info(
f" Retry: {retry_config.max_retries} retries, "
f"{retry_config.max_delay}s max delay"
)
logger.info(f" Connections: {connection_config.max_connections_per_peer} per peer")
logger.info(f" Load balancing: {connection_config.load_balancing_strategy}")
await swarm.close()
logger.info("Production-ready configuration example completed")
async def main() -> None:
"""Run all examples."""
logger.info("Multiple Connections Per Peer Examples")
logger.info("=" * 50)
try:
await example_basic_multiple_connections()
logger.info("-" * 30)
await example_custom_connection_config()
logger.info("-" * 30)
await example_multiple_connections_api()
logger.info("-" * 30)
await example_backward_compatibility()
logger.info("-" * 30)
await example_network_address_paradigm()
logger.info("-" * 30)
await example_production_ready_config()
logger.info("-" * 30)
logger.info("All examples completed successfully!")
except Exception as e:
logger.error(f"Example failed: {e}")
raise
if __name__ == "__main__":
trio.run(main)

View File

@ -1,4 +1,7 @@
import argparse
import logging
import random
import secrets
import multiaddr
import trio
@ -12,49 +15,81 @@ from libp2p.crypto.secp256k1 import (
from libp2p.custom_types import (
TProtocol,
)
from libp2p.network.stream.exceptions import (
StreamEOF,
)
from libp2p.network.stream.net_stream import (
INetStream,
)
from libp2p.peer.peerinfo import (
info_from_p2p_addr,
)
from libp2p.utils.address_validation import (
find_free_port,
get_available_interfaces,
get_optimal_binding_address,
)
# Configure minimal logging
logging.basicConfig(level=logging.WARNING)
logging.getLogger("multiaddr").setLevel(logging.WARNING)
logging.getLogger("libp2p").setLevel(logging.WARNING)
PROTOCOL_ID = TProtocol("/echo/1.0.0")
MAX_READ_LEN = 2**32 - 1
async def _echo_stream_handler(stream: INetStream) -> None:
# Wait until EOF
msg = await stream.read()
await stream.write(msg)
await stream.close()
try:
peer_id = stream.muxed_conn.peer_id
print(f"Received connection from {peer_id}")
# Wait until EOF
msg = await stream.read(MAX_READ_LEN)
print(f"Echoing message: {msg.decode('utf-8')}")
await stream.write(msg)
except StreamEOF:
print("Stream closed by remote peer.")
except Exception as e:
print(f"Error in echo handler: {e}")
finally:
await stream.close()
async def run(port: int, destination: str, seed: int = None) -> None:
localhost_ip = "127.0.0.1"
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
async def run(port: int, destination: str, seed: int | None = None) -> None:
if port <= 0:
port = find_free_port()
listen_addr = get_available_interfaces(port)
if seed:
import random
random.seed(seed)
secret_number = random.getrandbits(32 * 8)
secret = secret_number.to_bytes(length=32, byteorder="big")
else:
import secrets
secret = secrets.token_bytes(32)
host = new_host(key_pair=create_new_key_pair(secret))
async with host.run(listen_addrs=[listen_addr]):
async with host.run(listen_addrs=listen_addr), trio.open_nursery() as nursery:
# Start the peer-store cleanup task
nursery.start_soon(host.get_peerstore().start_cleanup_task, 60)
print(f"I am {host.get_id().to_string()}")
if not destination: # its the server
host.set_stream_handler(PROTOCOL_ID, _echo_stream_handler)
# Print all listen addresses with peer ID (JS parity)
print("Listener ready, listening on:\n")
peer_id = host.get_id().to_string()
for addr in listen_addr:
print(f"{addr}/p2p/{peer_id}")
# Get optimal address for display
optimal_addr = get_optimal_binding_address(port)
optimal_addr_with_peer = f"{optimal_addr}/p2p/{peer_id}"
print(
"Run this from the same folder in another console:\n\n"
f"echo-demo -p {int(port) + 1} "
f"-d /ip4/{localhost_ip}/tcp/{port}/p2p/{host.get_id().pretty()}\n"
"\nRun this from the same folder in another console:\n\n"
f"echo-demo -d {optimal_addr_with_peer}\n"
)
print("Waiting for incoming connections...")
await trio.sleep_forever()
@ -73,9 +108,8 @@ async def run(port: int, destination: str, seed: int = None) -> None:
msg = b"hi, there!\n"
await stream.write(msg)
# Notify the other side about EOF
await stream.close()
response = await stream.read()
await stream.close()
print(f"Sent: {msg.decode('utf-8')}")
print(f"Got: {response.decode('utf-8')}")
@ -91,12 +125,10 @@ def main() -> None:
where <DESTINATION> is the multiaddress of the previous listener host.
"""
example_maddr = (
"/ip4/127.0.0.1/tcp/8000/p2p/QmQn4SwGkDZKkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
"/ip4/[HOST_IP]/tcp/8000/p2p/QmQn4SwGkDZKkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
)
parser = argparse.ArgumentParser(description=description)
parser.add_argument(
"-p", "--port", default=8000, type=int, help="source port number"
)
parser.add_argument("-p", "--port", default=0, type=int, help="source port number")
parser.add_argument(
"-d",
"--destination",
@ -110,10 +142,6 @@ def main() -> None:
help="provide a seed to the random number generator (e.g. to fix peer IDs across runs)", # noqa: E501
)
args = parser.parse_args()
if not args.port:
raise RuntimeError("was not able to determine a local port")
try:
trio.run(run, args.port, args.destination, args.seed)
except KeyboardInterrupt:

207
examples/echo/echo_quic.py Normal file
View File

@ -0,0 +1,207 @@
#!/usr/bin/env python3
"""
QUIC Echo Example - Fixed version with proper client/server separation
This program demonstrates a simple echo protocol using QUIC transport where a peer
listens for connections and copies back any input received on a stream.
Fixed to properly separate client and server modes - clients don't start listeners.
"""
import argparse
import logging
from multiaddr import Multiaddr
import trio
from libp2p import new_host
from libp2p.crypto.secp256k1 import create_new_key_pair
from libp2p.custom_types import TProtocol
from libp2p.network.stream.net_stream import INetStream
from libp2p.peer.peerinfo import info_from_p2p_addr
# Configure minimal logging
logging.basicConfig(level=logging.WARNING)
logging.getLogger("multiaddr").setLevel(logging.WARNING)
logging.getLogger("libp2p").setLevel(logging.WARNING)
PROTOCOL_ID = TProtocol("/echo/1.0.0")
async def _echo_stream_handler(stream: INetStream) -> None:
try:
msg = await stream.read()
await stream.write(msg)
await stream.close()
except Exception as e:
print(f"Echo handler error: {e}")
try:
await stream.close()
except: # noqa: E722
pass
async def run_server(port: int, seed: int | None = None) -> None:
"""Run echo server with QUIC transport."""
from libp2p.utils.address_validation import (
find_free_port,
get_available_interfaces,
get_optimal_binding_address,
)
if port <= 0:
port = find_free_port()
# For QUIC, we need UDP addresses - use the new address paradigm
tcp_addrs = get_available_interfaces(port)
# Convert TCP addresses to QUIC addresses
quic_addrs = []
for addr in tcp_addrs:
addr_str = str(addr).replace("/tcp/", "/udp/") + "/quic"
quic_addrs.append(Multiaddr(addr_str))
if seed:
import random
random.seed(seed)
secret_number = random.getrandbits(32 * 8)
secret = secret_number.to_bytes(length=32, byteorder="big")
else:
import secrets
secret = secrets.token_bytes(32)
# Create host with QUIC transport
host = new_host(
enable_quic=True,
key_pair=create_new_key_pair(secret),
)
# Server mode: start listener
async with host.run(listen_addrs=quic_addrs):
try:
print(f"I am {host.get_id().to_string()}")
host.set_stream_handler(PROTOCOL_ID, _echo_stream_handler)
# Get all available addresses with peer ID
all_addrs = host.get_addrs()
print("Listener ready, listening on:")
for addr in all_addrs:
print(f"{addr}")
# Use optimal address for the client command
optimal_tcp = get_optimal_binding_address(port)
optimal_quic_str = str(optimal_tcp).replace("/tcp/", "/udp/") + "/quic"
peer_id = host.get_id().to_string()
optimal_quic_with_peer = f"{optimal_quic_str}/p2p/{peer_id}"
print(
f"\nRun this from the same folder in another console:\n\n"
f"python3 ./examples/echo/echo_quic.py -d {optimal_quic_with_peer}\n"
)
print("Waiting for incoming QUIC connections...")
await trio.sleep_forever()
except KeyboardInterrupt:
print("Closing server gracefully...")
await host.close()
return
async def run_client(destination: str, seed: int | None = None) -> None:
"""Run echo client with QUIC transport."""
if seed:
import random
random.seed(seed)
secret_number = random.getrandbits(32 * 8)
secret = secret_number.to_bytes(length=32, byteorder="big")
else:
import secrets
secret = secrets.token_bytes(32)
# Create host with QUIC transport
host = new_host(
enable_quic=True,
key_pair=create_new_key_pair(secret),
)
# Client mode: NO listener, just connect
async with host.run(listen_addrs=[]): # Empty listen_addrs for client
print(f"I am {host.get_id().to_string()}")
maddr = Multiaddr(destination)
info = info_from_p2p_addr(maddr)
# Connect to server
print("STARTING CLIENT CONNECTION PROCESS")
await host.connect(info)
print("CLIENT CONNECTED TO SERVER")
# Start a stream with the destination
stream = await host.new_stream(info.peer_id, [PROTOCOL_ID])
msg = b"hi, there!\n"
await stream.write(msg)
response = await stream.read()
print(f"Sent: {msg.decode('utf-8')}")
print(f"Got: {response.decode('utf-8')}")
await stream.close()
await host.disconnect(info.peer_id)
async def run(port: int, destination: str, seed: int | None = None) -> None:
"""
Run echo server or client with QUIC transport.
Fixed version that properly separates client and server modes.
"""
if not destination: # Server mode
await run_server(port, seed)
else: # Client mode
await run_client(destination, seed)
def main() -> None:
"""Main function - help text updated for QUIC."""
description = """
This program demonstrates a simple echo protocol using QUIC
transport where a peer listens for connections and copies back
any input received on a stream.
QUIC provides built-in TLS security and stream multiplexing over UDP.
To use it, first run 'echo-quic-demo -p <PORT>', where <PORT> is
the UDP port number. Then, run another host with ,
'echo-quic-demo -d <DESTINATION>'
where <DESTINATION> is the QUIC multiaddress of the previous listener host.
"""
example_maddr = "/ip4/[HOST_IP]/udp/8000/quic/p2p/QmQn4SwGkDZKkUEpBRBv"
parser = argparse.ArgumentParser(description=description)
parser.add_argument("-p", "--port", default=0, type=int, help="UDP port number")
parser.add_argument(
"-d",
"--destination",
type=str,
help=f"destination multiaddr string, e.g. {example_maddr}",
)
parser.add_argument(
"-s",
"--seed",
type=int,
help="provide a seed to the random number generator",
)
args = parser.parse_args()
try:
trio.run(run, args.port, args.destination, args.seed)
except KeyboardInterrupt:
pass
if __name__ == "__main__":
main()

View File

@ -1,6 +1,7 @@
import argparse
import base64
import logging
import sys
import multiaddr
import trio
@ -8,14 +9,22 @@ import trio
from libp2p import (
new_host,
)
from libp2p.identity.identify.identify import ID as IDENTIFY_PROTOCOL_ID
from libp2p.identity.identify.pb.identify_pb2 import (
Identify,
from libp2p.identity.identify.identify import (
ID as IDENTIFY_PROTOCOL_ID,
identify_handler_for,
parse_identify_response,
)
from libp2p.identity.identify.pb.identify_pb2 import Identify
from libp2p.peer.envelope import debug_dump_envelope, unmarshal_envelope
from libp2p.peer.peerinfo import (
info_from_p2p_addr,
)
# Configure minimal logging
logging.basicConfig(level=logging.WARNING)
logging.getLogger("multiaddr").setLevel(logging.WARNING)
logging.getLogger("libp2p").setLevel(logging.WARNING)
logger = logging.getLogger("libp2p.identity.identify-example")
@ -30,10 +39,11 @@ def decode_multiaddrs(raw_addrs):
return decoded_addrs
def print_identify_response(identify_response):
def print_identify_response(identify_response: Identify):
"""Pretty-print Identify response."""
public_key_b64 = base64.b64encode(identify_response.public_key).decode("utf-8")
listen_addrs = decode_multiaddrs(identify_response.listen_addrs)
signed_peer_record = unmarshal_envelope(identify_response.signedPeerRecord)
try:
observed_addr_decoded = decode_multiaddrs([identify_response.observed_addr])
except Exception:
@ -49,48 +59,211 @@ def print_identify_response(identify_response):
f" Agent Version: {identify_response.agent_version}"
)
debug_dump_envelope(signed_peer_record)
async def run(port: int, destination: str) -> None:
localhost_ip = "0.0.0.0"
async def run(port: int, destination: str, use_varint_format: bool = True) -> None:
from libp2p.utils.address_validation import (
get_available_interfaces,
get_optimal_binding_address,
)
if not destination:
# Create first host (listener)
listen_addr = multiaddr.Multiaddr(f"/ip4/{localhost_ip}/tcp/{port}")
if port <= 0:
from libp2p.utils.address_validation import find_free_port
port = find_free_port()
listen_addrs = get_available_interfaces(port)
host_a = new_host()
async with host_a.run(listen_addrs=[listen_addr]):
print(
"First host listening. Run this from another console:\n\n"
f"identify-demo -p {int(port) + 1} "
f"-d /ip4/{localhost_ip}/tcp/{port}/p2p/{host_a.get_id().pretty()}\n"
)
print("Waiting for incoming identify request...")
await trio.sleep_forever()
# Set up identify handler with specified format
# Set use_varint_format = False, if want to checkout the Signed-PeerRecord
identify_handler = identify_handler_for(
host_a, use_varint_format=use_varint_format
)
host_a.set_stream_handler(IDENTIFY_PROTOCOL_ID, identify_handler)
async with (
host_a.run(listen_addrs=listen_addrs),
trio.open_nursery() as nursery,
):
# Start the peer-store cleanup task
nursery.start_soon(host_a.get_peerstore().start_cleanup_task, 60)
# Get all available addresses with peer ID
all_addrs = host_a.get_addrs()
if use_varint_format:
format_name = "length-prefixed"
print(f"First host listening (using {format_name} format).")
print("Listener ready, listening on:\n")
for addr in all_addrs:
print(f"{addr}")
# Use optimal address for the client command
optimal_addr = get_optimal_binding_address(port)
optimal_addr_with_peer = (
f"{optimal_addr}/p2p/{host_a.get_id().to_string()}"
)
print(
f"\nRun this from the same folder in another console:\n\n"
f"identify-demo -d {optimal_addr_with_peer}\n"
)
print("Waiting for incoming identify request...")
else:
format_name = "raw protobuf"
print(f"First host listening (using {format_name} format).")
print("Listener ready, listening on:\n")
for addr in all_addrs:
print(f"{addr}")
# Use optimal address for the client command
optimal_addr = get_optimal_binding_address(port)
optimal_addr_with_peer = (
f"{optimal_addr}/p2p/{host_a.get_id().to_string()}"
)
print(
f"\nRun this from the same folder in another console:\n\n"
f"identify-demo -d {optimal_addr_with_peer}\n"
)
print("Waiting for incoming identify request...")
# Add a custom handler to show connection events
async def custom_identify_handler(stream):
peer_id = stream.muxed_conn.peer_id
print(f"\n🔗 Received identify request from peer: {peer_id}")
# Show remote address in multiaddr format
try:
from libp2p.identity.identify.identify import (
_remote_address_to_multiaddr,
)
remote_address = stream.get_remote_address()
if remote_address:
observed_multiaddr = _remote_address_to_multiaddr(
remote_address
)
# Add the peer ID to create a complete multiaddr
complete_multiaddr = f"{observed_multiaddr}/p2p/{peer_id}"
print(f" Remote address: {complete_multiaddr}")
else:
print(f" Remote address: {remote_address}")
except Exception:
print(f" Remote address: {stream.get_remote_address()}")
# Call the original handler
await identify_handler(stream)
print(f"✅ Successfully processed identify request from {peer_id}")
# Replace the handler with our custom one
host_a.set_stream_handler(IDENTIFY_PROTOCOL_ID, custom_identify_handler)
try:
await trio.sleep_forever()
except KeyboardInterrupt:
print("\n🛑 Shutting down listener...")
logger.info("Listener interrupted by user")
return
else:
# Create second host (dialer)
print(f"dialer (host_b) listening on /ip4/{localhost_ip}/tcp/{port}")
listen_addr = multiaddr.Multiaddr(f"/ip4/{localhost_ip}/tcp/{port}")
from libp2p.utils.address_validation import (
find_free_port,
get_available_interfaces,
get_optimal_binding_address,
)
if port <= 0:
port = find_free_port()
listen_addrs = get_available_interfaces(port)
host_b = new_host()
async with host_b.run(listen_addrs=[listen_addr]):
async with (
host_b.run(listen_addrs=listen_addrs),
trio.open_nursery() as nursery,
):
# Start the peer-store cleanup task
nursery.start_soon(host_b.get_peerstore().start_cleanup_task, 60)
# Connect to the first host
print(f"dialer (host_b) listening on {host_b.get_addrs()[0]}")
maddr = multiaddr.Multiaddr(destination)
info = info_from_p2p_addr(maddr)
print(f"Second host connecting to peer: {info.peer_id}")
await host_b.connect(info)
try:
await host_b.connect(info)
except Exception as e:
error_msg = str(e)
if "unable to connect" in error_msg or "SwarmException" in error_msg:
print(f"\n❌ Cannot connect to peer: {info.peer_id}")
print(f" Address: {destination}")
print(f" Error: {error_msg}")
print(
"\n💡 Make sure the peer is running and the address is correct."
)
return
else:
# Re-raise other exceptions
raise
stream = await host_b.new_stream(info.peer_id, (IDENTIFY_PROTOCOL_ID,))
try:
print("Starting identify protocol...")
response = await stream.read()
# Read the response using the utility function
from libp2p.utils.varint import read_length_prefixed_protobuf
response = await read_length_prefixed_protobuf(
stream, use_varint_format
)
full_response = response
await stream.close()
identify_msg = Identify()
identify_msg.ParseFromString(response)
# Parse the response using the robust protocol-level function
# This handles both old and new formats automatically
identify_msg = parse_identify_response(full_response)
print_identify_response(identify_msg)
except Exception as e:
print(f"Identify protocol error: {e}")
error_msg = str(e)
print(f"Identify protocol error: {error_msg}")
# Check for specific format mismatch errors
if "Error parsing message" in error_msg or "DecodeError" in error_msg:
print("\n" + "=" * 60)
print("FORMAT MISMATCH DETECTED!")
print("=" * 60)
if use_varint_format:
print(
"You are using length-prefixed format (default) but the "
"listener"
)
print("is using raw protobuf format.")
print(
"\nTo fix this, run the dialer with the --raw-format flag:"
)
print(f"identify-demo --raw-format -d {destination}")
else:
print("You are using raw protobuf format but the listener")
print("is using length-prefixed format (default).")
print(
"\nTo fix this, run the dialer without the --raw-format "
"flag:"
)
print(f"identify-demo -d {destination}")
print("=" * 60)
else:
import traceback
traceback.print_exc()
return
@ -98,34 +271,55 @@ async def run(port: int, destination: str) -> None:
def main() -> None:
description = """
This program demonstrates the libp2p identify protocol.
First run identify-demo -p <PORT>' to start a listener.
First run 'identify-demo -p <PORT> [--raw-format]' to start a listener.
Then run 'identify-demo <ANOTHER_PORT> -d <DESTINATION>'
where <DESTINATION> is the multiaddress shown by the listener.
Use --raw-format to send raw protobuf messages (old format) instead of
length-prefixed protobuf messages (new format, default).
"""
example_maddr = (
"/ip4/127.0.0.1/tcp/8888/p2p/QmQn4SwGkDZkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
"/ip4/[HOST_IP]/tcp/8888/p2p/QmQn4SwGkDZKkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
)
parser = argparse.ArgumentParser(description=description)
parser.add_argument(
"-p", "--port", default=8888, type=int, help="source port number"
)
parser.add_argument("-p", "--port", default=0, type=int, help="source port number")
parser.add_argument(
"-d",
"--destination",
type=str,
help=f"destination multiaddr string, e.g. {example_maddr}",
)
parser.add_argument(
"--raw-format",
action="store_true",
help=(
"use raw protobuf format (old format) instead of "
"length-prefixed (new format)"
),
)
args = parser.parse_args()
if not args.port:
raise RuntimeError("failed to determine local port")
# Determine format: use varint (length-prefixed) if --raw-format is specified,
# otherwise use raw protobuf format (old format)
use_varint_format = not args.raw_format
try:
trio.run(run, *(args.port, args.destination))
if args.destination:
# Run in dialer mode
trio.run(run, *(args.port, args.destination, use_varint_format))
else:
# Run in listener mode
trio.run(run, *(args.port, args.destination, use_varint_format))
except KeyboardInterrupt:
pass
print("\n👋 Goodbye!")
logger.info("Application interrupted by user")
except Exception as e:
print(f"\n❌ Error: {str(e)}")
logger.error("Error: %s", str(e))
sys.exit(1)
if __name__ == "__main__":

View File

@ -11,35 +11,178 @@ This example shows how to:
import logging
import multiaddr
import trio
from libp2p import (
new_host,
)
from libp2p.abc import (
INetStream,
)
from libp2p.crypto.secp256k1 import (
create_new_key_pair,
)
from libp2p.custom_types import (
TProtocol,
)
from libp2p.identity.identify import (
identify_handler_for,
from libp2p.identity.identify.pb.identify_pb2 import (
Identify,
)
from libp2p.identity.identify_push import (
ID_PUSH,
identify_push_handler_for,
push_identify_to_peer,
)
from libp2p.peer.peerinfo import (
info_from_p2p_addr,
)
from libp2p.utils.address_validation import (
get_available_interfaces,
)
# Configure logging
logger = logging.getLogger(__name__)
def create_custom_identify_handler(host, host_name: str):
"""Create a custom identify handler that displays received information."""
async def handle_identify(stream: INetStream) -> None:
peer_id = stream.muxed_conn.peer_id
print(f"\n🔍 {host_name} received identify request from peer: {peer_id}")
# Get the standard identify response using the existing function
from libp2p.identity.identify.identify import (
_mk_identify_protobuf,
_remote_address_to_multiaddr,
)
# Get observed address
observed_multiaddr = None
try:
remote_address = stream.get_remote_address()
if remote_address:
observed_multiaddr = _remote_address_to_multiaddr(remote_address)
except Exception:
pass
# Build the identify protobuf
identify_msg = _mk_identify_protobuf(host, observed_multiaddr)
response_data = identify_msg.SerializeToString()
print(f" 📋 {host_name} identify information:")
if identify_msg.HasField("protocol_version"):
print(f" Protocol Version: {identify_msg.protocol_version}")
if identify_msg.HasField("agent_version"):
print(f" Agent Version: {identify_msg.agent_version}")
if identify_msg.HasField("public_key"):
print(f" Public Key: {identify_msg.public_key.hex()[:16]}...")
if identify_msg.listen_addrs:
print(" Listen Addresses:")
for addr_bytes in identify_msg.listen_addrs:
addr = multiaddr.Multiaddr(addr_bytes)
print(f" - {addr}")
if identify_msg.protocols:
print(" Supported Protocols:")
for protocol in identify_msg.protocols:
print(f" - {protocol}")
# Send the response
await stream.write(response_data)
await stream.close()
return handle_identify
def create_custom_identify_push_handler(host, host_name: str):
"""Create a custom identify/push handler that displays received information."""
async def handle_identify_push(stream: INetStream) -> None:
peer_id = stream.muxed_conn.peer_id
print(f"\n📤 {host_name} received identify/push from peer: {peer_id}")
try:
# Read the identify message using the utility function
from libp2p.utils.varint import read_length_prefixed_protobuf
data = await read_length_prefixed_protobuf(stream, use_varint_format=True)
# Parse the identify message
identify_msg = Identify()
identify_msg.ParseFromString(data)
print(" 📋 Received identify information:")
if identify_msg.HasField("protocol_version"):
print(f" Protocol Version: {identify_msg.protocol_version}")
if identify_msg.HasField("agent_version"):
print(f" Agent Version: {identify_msg.agent_version}")
if identify_msg.HasField("public_key"):
print(f" Public Key: {identify_msg.public_key.hex()[:16]}...")
if identify_msg.HasField("observed_addr") and identify_msg.observed_addr:
observed_addr = multiaddr.Multiaddr(identify_msg.observed_addr)
print(f" Observed Address: {observed_addr}")
if identify_msg.listen_addrs:
print(" Listen Addresses:")
for addr_bytes in identify_msg.listen_addrs:
addr = multiaddr.Multiaddr(addr_bytes)
print(f" - {addr}")
if identify_msg.protocols:
print(" Supported Protocols:")
for protocol in identify_msg.protocols:
print(f" - {protocol}")
# Update the peerstore with the new information
from libp2p.identity.identify_push.identify_push import (
_update_peerstore_from_identify,
)
await _update_peerstore_from_identify(
host.get_peerstore(), peer_id, identify_msg
)
print(f"{host_name} updated peerstore with new information")
except Exception as e:
print(f" ❌ Error processing identify/push: {e}")
finally:
await stream.close()
return handle_identify_push
async def display_peerstore_info(host, host_name: str, peer_id, description: str):
"""Display peerstore information for a specific peer."""
peerstore = host.get_peerstore()
try:
addrs = peerstore.addrs(peer_id)
except Exception:
addrs = []
try:
protocols = peerstore.get_protocols(peer_id)
except Exception:
protocols = []
print(f"\n📚 {host_name} peerstore for {description}:")
print(f" Peer ID: {peer_id}")
if addrs:
print(" Addresses:")
for addr in addrs:
print(f" - {addr}")
else:
print(" Addresses: None")
if protocols:
print(" Protocols:")
for protocol in protocols:
print(f" - {protocol}")
else:
print(" Protocols: None")
async def main() -> None:
print("\n==== Starting Identify-Push Example ====\n")
print("\n==== Starting Enhanced Identify-Push Example ====\n")
# Create key pairs for the two hosts
key_pair_1 = create_new_key_pair()
@ -48,45 +191,57 @@ async def main() -> None:
# Create the first host
host_1 = new_host(key_pair=key_pair_1)
# Set up the identify and identify/push handlers
host_1.set_stream_handler(TProtocol("/ipfs/id/1.0.0"), identify_handler_for(host_1))
host_1.set_stream_handler(ID_PUSH, identify_push_handler_for(host_1))
# Set up custom identify and identify/push handlers
host_1.set_stream_handler(
TProtocol("/ipfs/id/1.0.0"), create_custom_identify_handler(host_1, "Host 1")
)
host_1.set_stream_handler(
ID_PUSH, create_custom_identify_push_handler(host_1, "Host 1")
)
# Create the second host
host_2 = new_host(key_pair=key_pair_2)
# Set up the identify and identify/push handlers
host_2.set_stream_handler(TProtocol("/ipfs/id/1.0.0"), identify_handler_for(host_2))
host_2.set_stream_handler(ID_PUSH, identify_push_handler_for(host_2))
# Set up custom identify and identify/push handlers
host_2.set_stream_handler(
TProtocol("/ipfs/id/1.0.0"), create_custom_identify_handler(host_2, "Host 2")
)
host_2.set_stream_handler(
ID_PUSH, create_custom_identify_push_handler(host_2, "Host 2")
)
# Start listening on random ports using the run context manager
import multiaddr
# Start listening on available interfaces using random ports
listen_addrs_1 = get_available_interfaces(0) # 0 for random port
listen_addrs_2 = get_available_interfaces(0) # 0 for random port
listen_addr_1 = multiaddr.Multiaddr("/ip4/127.0.0.1/tcp/0")
listen_addr_2 = multiaddr.Multiaddr("/ip4/127.0.0.1/tcp/0")
async with (
host_1.run(listen_addrs_1),
host_2.run(listen_addrs_2),
trio.open_nursery() as nursery,
):
# Start the peer-store cleanup task
nursery.start_soon(host_1.get_peerstore().start_cleanup_task, 60)
nursery.start_soon(host_2.get_peerstore().start_cleanup_task, 60)
async with host_1.run([listen_addr_1]), host_2.run([listen_addr_2]):
# Get the addresses of both hosts
addr_1 = host_1.get_addrs()[0]
logger.info(f"Host 1 listening on {addr_1}")
print(f"Host 1 listening on {addr_1}")
print(f"Peer ID: {host_1.get_id().pretty()}")
addr_2 = host_2.get_addrs()[0]
logger.info(f"Host 2 listening on {addr_2}")
print(f"Host 2 listening on {addr_2}")
print(f"Peer ID: {host_2.get_id().pretty()}")
print("\nConnecting Host 2 to Host 1...")
print("🏠 Host Configuration:")
print(f" Host 1: {addr_1}")
print(f" Host 1 Peer ID: {host_1.get_id().pretty()}")
print(f" Host 2: {addr_2}")
print(f" Host 2 Peer ID: {host_2.get_id().pretty()}")
print("\n🔗 Connecting Host 2 to Host 1...")
# Connect host_2 to host_1
peer_info = info_from_p2p_addr(addr_1)
await host_2.connect(peer_info)
logger.info("Host 2 connected to Host 1")
print("Host 2 successfully connected to Host 1")
print("Host 2 successfully connected to Host 1")
# Run the identify protocol from host_2 to host_1
# (so Host 1 learns Host 2's address)
print("\n🔄 Running identify protocol (Host 2 → Host 1)...")
from libp2p.identity.identify.identify import ID as IDENTIFY_PROTOCOL_ID
stream = await host_2.new_stream(host_1.get_id(), (IDENTIFY_PROTOCOL_ID,))
@ -94,64 +249,58 @@ async def main() -> None:
await stream.close()
# Run the identify protocol from host_1 to host_2
# (so Host 2 learns Host 1's address)
print("\n🔄 Running identify protocol (Host 1 → Host 2)...")
stream = await host_1.new_stream(host_2.get_id(), (IDENTIFY_PROTOCOL_ID,))
response = await stream.read()
await stream.close()
# --- NEW CODE: Update Host 1's peerstore with Host 2's addresses ---
from libp2p.identity.identify.pb.identify_pb2 import (
Identify,
)
# Update Host 1's peerstore with Host 2's addresses
identify_msg = Identify()
identify_msg.ParseFromString(response)
peerstore_1 = host_1.get_peerstore()
peer_id_2 = host_2.get_id()
for addr_bytes in identify_msg.listen_addrs:
maddr = multiaddr.Multiaddr(addr_bytes)
# TTL can be any positive int
peerstore_1.add_addr(
peer_id_2,
maddr,
ttl=3600,
)
# --- END NEW CODE ---
peerstore_1.add_addr(peer_id_2, maddr, ttl=3600)
# Now Host 1's peerstore should have Host 2's address
peerstore_1 = host_1.get_peerstore()
peer_id_2 = host_2.get_id()
addrs_1_for_2 = peerstore_1.addrs(peer_id_2)
logger.info(
f"[DEBUG] Host 1 peerstore addresses for Host 2 before push: "
f"{addrs_1_for_2}"
)
print(
f"[DEBUG] Host 1 peerstore addresses for Host 2 before push: "
f"{addrs_1_for_2}"
# Display peerstore information before push
await display_peerstore_info(
host_1, "Host 1", peer_id_2, "Host 2 (before push)"
)
# Push identify information from host_1 to host_2
logger.info("Host 1 pushing identify information to Host 2")
print("\nHost 1 pushing identify information to Host 2...")
print("\n📤 Host 1 pushing identify information to Host 2...")
try:
# Call push_identify_to_peer which now returns a boolean
success = await push_identify_to_peer(host_1, host_2.get_id())
if success:
logger.info("Identify push completed successfully")
print("Identify push completed successfully!")
print("Identify push completed successfully!")
else:
logger.warning("Identify push didn't complete successfully")
print("\nWarning: Identify push didn't complete successfully")
print("⚠️ Identify push didn't complete successfully")
except Exception as e:
logger.error(f"Error during identify push: {str(e)}")
print(f"\nError during identify push: {str(e)}")
print(f"Error during identify push: {str(e)}")
# Add this at the end of your async with block:
await trio.sleep(0.5) # Give background tasks time to finish
# Give a moment for the identify/push processing to complete
await trio.sleep(0.5)
# Display peerstore information after push
await display_peerstore_info(host_1, "Host 1", peer_id_2, "Host 2 (after push)")
await display_peerstore_info(
host_2, "Host 2", host_1.get_id(), "Host 1 (after push)"
)
# Give more time for background tasks to finish and connections to stabilize
print("\n⏳ Waiting for background tasks to complete...")
await trio.sleep(1.0)
# Gracefully close connections to prevent connection errors
print("🔌 Closing connections...")
await host_2.disconnect(host_1.get_id())
await trio.sleep(0.2)
print("\n🎉 Example completed successfully!")
if __name__ == "__main__":

View File

@ -14,7 +14,7 @@ Usage:
python identify_push_listener_dialer.py
# Then in another console, run as a dialer (default port 8889):
python identify_push_listener_dialer.py -d /ip4/127.0.0.1/tcp/8888/p2p/PEER_ID
python identify_push_listener_dialer.py -d /ip4/[HOST_IP]/tcp/8888/p2p/PEER_ID
(where PEER_ID is the peer ID displayed by the listener)
"""
@ -38,40 +38,73 @@ from libp2p.crypto.secp256k1 import (
create_new_key_pair,
)
from libp2p.identity.identify import (
ID as ID_IDENTIFY,
identify_handler_for,
)
from libp2p.identity.identify import ID as ID_IDENTIFY
from libp2p.identity.identify.identify import (
_remote_address_to_multiaddr,
)
from libp2p.identity.identify.pb.identify_pb2 import (
Identify,
)
from libp2p.identity.identify_push import (
ID_PUSH as ID_IDENTIFY_PUSH,
identify_push_handler_for,
push_identify_to_peer,
)
from libp2p.identity.identify_push import ID_PUSH as ID_IDENTIFY_PUSH
from libp2p.peer.peerinfo import (
info_from_p2p_addr,
)
# Configure minimal logging
logging.basicConfig(level=logging.WARNING)
logging.getLogger("multiaddr").setLevel(logging.WARNING)
logging.getLogger("libp2p").setLevel(logging.WARNING)
# Configure logging
logger = logging.getLogger("libp2p.identity.identify-push-example")
# Default port configuration
DEFAULT_PORT = 8888
def custom_identify_push_handler_for(host):
def custom_identify_push_handler_for(host, use_varint_format: bool = True):
"""
Create a custom handler for the identify/push protocol that logs and prints
the identity information received from the dialer.
Args:
host: The libp2p host
use_varint_format: If True, expect length-prefixed format; if False, expect
raw protobuf
"""
async def handle_identify_push(stream: INetStream) -> None:
peer_id = stream.muxed_conn.peer_id
# Get remote address information
try:
# Read the identify message from the stream
data = await stream.read()
remote_address = stream.get_remote_address()
if remote_address:
observed_multiaddr = _remote_address_to_multiaddr(remote_address)
logger.info(
"Connection from remote peer %s, address: %s, multiaddr: %s",
peer_id,
remote_address,
observed_multiaddr,
)
print(f"\n🔗 Received identify/push request from peer: {peer_id}")
# Add the peer ID to create a complete multiaddr
complete_multiaddr = f"{observed_multiaddr}/p2p/{peer_id}"
print(f" Remote address: {complete_multiaddr}")
except Exception as e:
logger.error("Error getting remote address: %s", e)
print(f"\n🔗 Received identify/push request from peer: {peer_id}")
try:
# Use the utility function to read the protobuf message
from libp2p.utils.varint import read_length_prefixed_protobuf
data = await read_length_prefixed_protobuf(stream, use_varint_format)
identify_msg = Identify()
identify_msg.ParseFromString(data)
@ -120,11 +153,41 @@ def custom_identify_push_handler_for(host):
await _update_peerstore_from_identify(peerstore, peer_id, identify_msg)
logger.info("Successfully processed identify/push from peer %s", peer_id)
print(f"\nSuccessfully processed identify/push from peer {peer_id}")
print(f"Successfully processed identify/push from peer {peer_id}")
except Exception as e:
logger.error("Error processing identify/push from %s: %s", peer_id, e)
print(f"\nError processing identify/push from {peer_id}: {e}")
error_msg = str(e)
logger.error(
"Error processing identify/push from %s: %s", peer_id, error_msg
)
print(f"\nError processing identify/push from {peer_id}: {error_msg}")
# Check for specific format mismatch errors
if (
"Error parsing message" in error_msg
or "DecodeError" in error_msg
or "ParseFromString" in error_msg
):
print("\n" + "=" * 60)
print("FORMAT MISMATCH DETECTED!")
print("=" * 60)
if use_varint_format:
print(
"You are using length-prefixed format (default) but the "
"dialer is using raw protobuf format."
)
print("\nTo fix this, run the dialer with the --raw-format flag:")
print(
"identify-push-listener-dialer-demo --raw-format -d <ADDRESS>"
)
else:
print("You are using raw protobuf format but the dialer")
print("is using length-prefixed format (default).")
print(
"\nTo fix this, run the dialer without the --raw-format flag:"
)
print("identify-push-listener-dialer-demo -d <ADDRESS>")
print("=" * 60)
finally:
# Close the stream after processing
await stream.close()
@ -132,9 +195,20 @@ def custom_identify_push_handler_for(host):
return handle_identify_push
async def run_listener(port: int) -> None:
async def run_listener(
port: int, use_varint_format: bool = True, raw_format_flag: bool = False
) -> None:
"""Run a host in listener mode."""
print(f"\n==== Starting Identify-Push Listener on port {port} ====\n")
from libp2p.utils.address_validation import find_free_port, get_available_interfaces
if port <= 0:
port = find_free_port()
format_name = "length-prefixed" if use_varint_format else "raw protobuf"
print(
f"\n==== Starting Identify-Push Listener on port {port} "
f"(using {format_name} format) ====\n"
)
# Create key pair for the listener
key_pair = create_new_key_pair()
@ -142,35 +216,65 @@ async def run_listener(port: int) -> None:
# Create the listener host
host = new_host(key_pair=key_pair)
# Set up the identify and identify/push handlers
host.set_stream_handler(ID_IDENTIFY, identify_handler_for(host))
host.set_stream_handler(ID_IDENTIFY_PUSH, custom_identify_push_handler_for(host))
# Set up the identify and identify/push handlers with specified format
host.set_stream_handler(
ID_IDENTIFY, identify_handler_for(host, use_varint_format=use_varint_format)
)
host.set_stream_handler(
ID_IDENTIFY_PUSH,
custom_identify_push_handler_for(host, use_varint_format=use_varint_format),
)
# Start listening
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
# Start listening on all available interfaces
listen_addrs = get_available_interfaces(port)
async with host.run([listen_addr]):
addr = host.get_addrs()[0]
logger.info("Listener host ready!")
print("Listener host ready!")
try:
async with host.run(listen_addrs):
all_addrs = host.get_addrs()
logger.info("Listener host ready!")
print("Listener host ready!")
logger.info(f"Listening on: {addr}")
print(f"Listening on: {addr}")
logger.info("Listener ready, listening on:")
print("Listener ready, listening on:")
for addr in all_addrs:
logger.info(f"{addr}")
print(f"{addr}")
logger.info(f"Peer ID: {host.get_id().pretty()}")
print(f"Peer ID: {host.get_id().pretty()}")
logger.info(f"Peer ID: {host.get_id().pretty()}")
print(f"Peer ID: {host.get_id().pretty()}")
print("\nRun dialer with command:")
print(f"identify-push-listener-dialer-demo -d {addr}")
print("\nWaiting for incoming connections... (Ctrl+C to exit)")
# Use the first address as the default for the dialer command
default_addr = all_addrs[0]
print("\nRun this from the same folder in another console:")
if raw_format_flag:
print(
f"identify-push-listener-dialer-demo -d {default_addr} --raw-format"
)
else:
print(f"identify-push-listener-dialer-demo -d {default_addr}")
print("\nWaiting for incoming identify/push requests... (Ctrl+C to exit)")
# Keep running until interrupted
await trio.sleep_forever()
# Keep running until interrupted
try:
await trio.sleep_forever()
except KeyboardInterrupt:
print("\n🛑 Shutting down listener...")
logger.info("Listener interrupted by user")
return
except Exception as e:
logger.error(f"Listener error: {e}")
raise
async def run_dialer(port: int, destination: str) -> None:
async def run_dialer(
port: int, destination: str, use_varint_format: bool = True
) -> None:
"""Run a host in dialer mode that connects to a listener."""
print(f"\n==== Starting Identify-Push Dialer on port {port} ====\n")
format_name = "length-prefixed" if use_varint_format else "raw protobuf"
print(
f"\n==== Starting Identify-Push Dialer on port {port} "
f"(using {format_name} format) ====\n"
)
# Create key pair for the dialer
key_pair = create_new_key_pair()
@ -178,14 +282,21 @@ async def run_dialer(port: int, destination: str) -> None:
# Create the dialer host
host = new_host(key_pair=key_pair)
# Set up the identify and identify/push handlers
host.set_stream_handler(ID_IDENTIFY, identify_handler_for(host))
host.set_stream_handler(ID_IDENTIFY_PUSH, identify_push_handler_for(host))
# Set up the identify and identify/push handlers with specified format
host.set_stream_handler(
ID_IDENTIFY, identify_handler_for(host, use_varint_format=use_varint_format)
)
host.set_stream_handler(
ID_IDENTIFY_PUSH,
identify_push_handler_for(host, use_varint_format=use_varint_format),
)
# Start listening on a different port
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
# Start listening on available interfaces
from libp2p.utils.address_validation import get_available_interfaces
async with host.run([listen_addr]):
listen_addrs = get_available_interfaces(port)
async with host.run(listen_addrs):
logger.info("Dialer host ready!")
print("Dialer host ready!")
@ -201,7 +312,9 @@ async def run_dialer(port: int, destination: str) -> None:
try:
await host.connect(peer_info)
logger.info("Successfully connected to listener!")
print("Successfully connected to listener!")
print("Successfully connected to listener!")
print(f" Connected to: {peer_info.peer_id}")
print(f" Full address: {destination}")
# Push identify information to the listener
logger.info("Pushing identify information to listener...")
@ -209,11 +322,13 @@ async def run_dialer(port: int, destination: str) -> None:
try:
# Call push_identify_to_peer which returns a boolean
success = await push_identify_to_peer(host, peer_info.peer_id)
success = await push_identify_to_peer(
host, peer_info.peer_id, use_varint_format=use_varint_format
)
if success:
logger.info("Identify push completed successfully!")
print("Identify push completed successfully!")
print("Identify push completed successfully!")
logger.info("Example completed successfully!")
print("\nExample completed successfully!")
@ -224,64 +339,114 @@ async def run_dialer(port: int, destination: str) -> None:
logger.warning("Example completed with warnings.")
print("Example completed with warnings.")
except Exception as e:
logger.error(f"Error during identify push: {str(e)}")
print(f"\nError during identify push: {str(e)}")
error_msg = str(e)
logger.error(f"Error during identify push: {error_msg}")
print(f"\nError during identify push: {error_msg}")
# Check for specific format mismatch errors
if (
"Error parsing message" in error_msg
or "DecodeError" in error_msg
or "ParseFromString" in error_msg
):
print("\n" + "=" * 60)
print("FORMAT MISMATCH DETECTED!")
print("=" * 60)
if use_varint_format:
print(
"You are using length-prefixed format (default) but the "
"listener is using raw protobuf format."
)
print(
"\nTo fix this, run the dialer with the --raw-format flag:"
)
print(
f"identify-push-listener-dialer-demo --raw-format -d "
f"{destination}"
)
else:
print("You are using raw protobuf format but the listener")
print("is using length-prefixed format (default).")
print(
"\nTo fix this, run the dialer without the --raw-format "
"flag:"
)
print(f"identify-push-listener-dialer-demo -d {destination}")
print("=" * 60)
logger.error("Example completed with errors.")
print("Example completed with errors.")
# Continue execution despite the push error
except Exception as e:
logger.error(f"Error during dialer operation: {str(e)}")
print(f"\nError during dialer operation: {str(e)}")
raise
error_msg = str(e)
if "unable to connect" in error_msg or "SwarmException" in error_msg:
print(f"\n❌ Cannot connect to peer: {peer_info.peer_id}")
print(f" Address: {destination}")
print(f" Error: {error_msg}")
print("\n💡 Make sure the peer is running and the address is correct.")
return
else:
logger.error(f"Error during dialer operation: {error_msg}")
print(f"\nError during dialer operation: {error_msg}")
raise
def main() -> None:
"""Parse arguments and start the appropriate mode."""
description = """
This program demonstrates the libp2p identify/push protocol.
Without arguments, it runs as a listener on port 8888.
With -d parameter, it runs as a dialer on port 8889.
"""
Without arguments, it runs as a listener on random port.
With -d parameter, it runs as a dialer on random port.
example = (
f"/ip4/127.0.0.1/tcp/{DEFAULT_PORT}/p2p/"
"QmQn4SwGkDZkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
)
Port 0 (default) means the OS will automatically assign an available port.
This prevents port conflicts when running multiple instances.
Use --raw-format to send raw protobuf messages (old format) instead of
length-prefixed protobuf messages (new format, default).
"""
parser = argparse.ArgumentParser(description=description)
parser.add_argument(
"-p",
"--port",
default=0,
type=int,
help=(
f"port to listen on (default: {DEFAULT_PORT} for listener, "
f"{DEFAULT_PORT + 1} for dialer)"
),
help="source port number (0 = random available port)",
)
parser.add_argument(
"-d",
"--destination",
type=str,
help=f"destination multiaddr string, e.g. {example}",
help="destination multiaddr string",
)
parser.add_argument(
"--raw-format",
action="store_true",
help=(
"use raw protobuf format (old format) instead of "
"length-prefixed (new format)"
),
)
args = parser.parse_args()
# Determine format: raw format if --raw-format is specified, otherwise
# length-prefixed
use_varint_format = not args.raw_format
try:
if args.destination:
# Run in dialer mode with default port DEFAULT_PORT + 1 if not specified
port = args.port if args.port is not None else DEFAULT_PORT + 1
trio.run(run_dialer, port, args.destination)
# Run in dialer mode with random available port if not specified
trio.run(run_dialer, args.port, args.destination, use_varint_format)
else:
# Run in listener mode with default port DEFAULT_PORT if not specified
port = args.port if args.port is not None else DEFAULT_PORT
trio.run(run_listener, port)
# Run in listener mode with random available port if not specified
trio.run(run_listener, args.port, use_varint_format, args.raw_format)
except KeyboardInterrupt:
print("\nInterrupted by user")
logger.info("Interrupted by user")
print("\n👋 Goodbye!")
logger.info("Application interrupted by user")
except Exception as e:
print(f"\nError: {str(e)}")
print(f"\nError: {str(e)}")
logger.error("Error: %s", str(e))
sys.exit(1)

View File

@ -0,0 +1,321 @@
#!/usr/bin/env python
"""
A basic example of using the Kademlia DHT implementation, with all setup logic inlined.
This example demonstrates both value storage/retrieval and content server
advertisement/discovery.
"""
import argparse
import logging
import os
import random
import secrets
import sys
import base58
from multiaddr import (
Multiaddr,
)
import trio
from libp2p import (
new_host,
)
from libp2p.abc import (
IHost,
)
from libp2p.crypto.secp256k1 import (
create_new_key_pair,
)
from libp2p.kad_dht.kad_dht import (
DHTMode,
KadDHT,
)
from libp2p.kad_dht.utils import (
create_key_from_binary,
)
from libp2p.tools.async_service import (
background_trio_service,
)
from libp2p.tools.utils import (
info_from_p2p_addr,
)
from libp2p.utils.paths import get_script_dir, join_paths
# Configure logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
handlers=[logging.StreamHandler()],
)
logger = logging.getLogger("kademlia-example")
# Configure DHT module loggers to inherit from the parent logger
# This ensures all kademlia-example.* loggers use the same configuration
# Get the directory where this script is located
SCRIPT_DIR = get_script_dir(__file__)
SERVER_ADDR_LOG = join_paths(SCRIPT_DIR, "server_node_addr.txt")
# Set the level for all child loggers
for module in [
"kad_dht",
"value_store",
"peer_routing",
"routing_table",
"provider_store",
]:
child_logger = logging.getLogger(f"kademlia-example.{module}")
child_logger.setLevel(logging.INFO)
child_logger.propagate = True # Allow propagation to parent
# File to store node information
bootstrap_nodes = []
# function to take bootstrap_nodes as input and connects to them
async def connect_to_bootstrap_nodes(host: IHost, bootstrap_addrs: list[str]) -> None:
"""
Connect to the bootstrap nodes provided in the list.
params: host: The host instance to connect to
bootstrap_addrs: List of bootstrap node addresses
Returns
-------
None
"""
for addr in bootstrap_addrs:
try:
peerInfo = info_from_p2p_addr(Multiaddr(addr))
host.get_peerstore().add_addrs(peerInfo.peer_id, peerInfo.addrs, 3600)
await host.connect(peerInfo)
except Exception as e:
logger.error(f"Failed to connect to bootstrap node {addr}: {e}")
def save_server_addr(addr: str) -> None:
"""Append the server's multiaddress to the log file."""
try:
with open(SERVER_ADDR_LOG, "w") as f:
f.write(addr + "\n")
logger.info(f"Saved server address to log: {addr}")
except Exception as e:
logger.error(f"Failed to save server address: {e}")
def load_server_addrs() -> list[str]:
"""Load all server multiaddresses from the log file."""
if not os.path.exists(SERVER_ADDR_LOG):
return []
try:
with open(SERVER_ADDR_LOG) as f:
return [line.strip() for line in f if line.strip()]
except Exception as e:
logger.error(f"Failed to load server addresses: {e}")
return []
async def run_node(
port: int, mode: str, bootstrap_addrs: list[str] | None = None
) -> None:
"""Run a node that serves content in the DHT with setup inlined."""
try:
if port <= 0:
port = random.randint(10000, 60000)
logger.debug(f"Using port: {port}")
# Convert string mode to DHTMode enum
if mode is None or mode.upper() == "CLIENT":
dht_mode = DHTMode.CLIENT
elif mode.upper() == "SERVER":
dht_mode = DHTMode.SERVER
else:
logger.error(f"Invalid mode: {mode}. Must be 'client' or 'server'")
sys.exit(1)
# Load server addresses for client mode
if dht_mode == DHTMode.CLIENT:
server_addrs = load_server_addrs()
if server_addrs:
logger.info(f"Loaded {len(server_addrs)} server addresses from log")
bootstrap_nodes.append(server_addrs[0]) # Use the first server address
else:
logger.warning("No server addresses found in log file")
if bootstrap_addrs:
for addr in bootstrap_addrs:
bootstrap_nodes.append(addr)
key_pair = create_new_key_pair(secrets.token_bytes(32))
host = new_host(key_pair=key_pair)
from libp2p.utils.address_validation import (
get_available_interfaces,
get_optimal_binding_address,
)
listen_addrs = get_available_interfaces(port)
async with host.run(listen_addrs=listen_addrs), trio.open_nursery() as nursery:
# Start the peer-store cleanup task
nursery.start_soon(host.get_peerstore().start_cleanup_task, 60)
peer_id = host.get_id().pretty()
# Get all available addresses with peer ID
all_addrs = host.get_addrs()
logger.info("Listener ready, listening on:")
for addr in all_addrs:
logger.info(f"{addr}")
# Use optimal address for the bootstrap command
optimal_addr = get_optimal_binding_address(port)
optimal_addr_with_peer = f"{optimal_addr}/p2p/{host.get_id().to_string()}"
bootstrap_cmd = f"--bootstrap {optimal_addr_with_peer}"
logger.info("To connect to this node, use: %s", bootstrap_cmd)
await connect_to_bootstrap_nodes(host, bootstrap_nodes)
dht = KadDHT(host, dht_mode)
# take all peer ids from the host and add them to the dht
for peer_id in host.get_peerstore().peer_ids():
await dht.routing_table.add_peer(peer_id)
logger.info(f"Connected to bootstrap nodes: {host.get_connected_peers()}")
# Save server address in server mode
if dht_mode == DHTMode.SERVER:
save_server_addr(str(optimal_addr_with_peer))
# Start the DHT service
async with background_trio_service(dht):
logger.info(f"DHT service started in {dht_mode.value} mode")
val_key = create_key_from_binary(b"py-libp2p kademlia example value")
content = b"Hello from python node "
content_key = create_key_from_binary(content)
if dht_mode == DHTMode.SERVER:
# Store a value in the DHT
msg = "Hello message from Sumanjeet"
val_data = msg.encode()
await dht.put_value(val_key, val_data)
logger.info(
f"Stored value '{val_data.decode()}'"
f"with key: {base58.b58encode(val_key).decode()}"
)
# Advertise as content server
success = await dht.provider_store.provide(content_key)
if success:
logger.info(
"Successfully advertised as server"
f"for content: {content_key.hex()}"
)
else:
logger.warning("Failed to advertise as content server")
else:
# retrieve the value
logger.info(
"Looking up key: %s", base58.b58encode(val_key).decode()
)
val_data = await dht.get_value(val_key)
if val_data:
try:
logger.info(f"Retrieved value: {val_data.decode()}")
except UnicodeDecodeError:
logger.info(f"Retrieved value (bytes): {val_data!r}")
else:
logger.warning("Failed to retrieve value")
# Also check if we can find servers for our own content
logger.info("Looking for servers of content: %s", content_key.hex())
providers = await dht.provider_store.find_providers(content_key)
if providers:
logger.info(
"Found %d servers for content: %s",
len(providers),
[p.peer_id.pretty() for p in providers],
)
else:
logger.warning(
"No servers found for content %s", content_key.hex()
)
# Keep the node running
while True:
logger.info(
"Status - Connected peers: %d,"
"Peers in store: %d, Values in store: %d",
len(dht.host.get_connected_peers()),
len(dht.host.get_peerstore().peer_ids()),
len(dht.value_store.store),
)
await trio.sleep(10)
except Exception as e:
logger.error(f"Server node error: {e}", exc_info=True)
sys.exit(1)
def parse_args():
"""Parse command line arguments."""
parser = argparse.ArgumentParser(
description="Kademlia DHT example with content server functionality"
)
parser.add_argument(
"--mode",
default="server",
help="Run as a server or client node",
)
parser.add_argument(
"--port",
type=int,
default=0,
help="Port to listen on (0 for random)",
)
parser.add_argument(
"--bootstrap",
type=str,
nargs="*",
help=(
"Multiaddrs of bootstrap nodes. "
"Provide a space-separated list of addresses. "
"This is required for client mode."
),
)
# add option to use verbose logging
parser.add_argument(
"--verbose",
action="store_true",
help="Enable verbose logging",
)
args = parser.parse_args()
# Set logging level based on verbosity
if args.verbose:
logging.getLogger().setLevel(logging.DEBUG)
else:
logging.getLogger().setLevel(logging.INFO)
return args
def main():
"""Main entry point for the kademlia demo."""
try:
args = parse_args()
logger.info(
"Running in %s mode on port %d",
args.mode,
args.port,
)
trio.run(run_node, args.port, args.mode, args.bootstrap)
except Exception as e:
logger.critical(f"Script failed: {e}", exc_info=True)
sys.exit(1)
if __name__ == "__main__":
main()

90
examples/mDNS/mDNS.py Normal file
View File

@ -0,0 +1,90 @@
import argparse
import logging
import secrets
import trio
from libp2p import (
new_host,
)
from libp2p.abc import PeerInfo
from libp2p.crypto.secp256k1 import (
create_new_key_pair,
)
from libp2p.discovery.events.peerDiscovery import peerDiscovery
# Configure minimal logging
logging.basicConfig(level=logging.WARNING)
logging.getLogger("multiaddr").setLevel(logging.WARNING)
logging.getLogger("libp2p").setLevel(logging.WARNING)
logger = logging.getLogger("libp2p.discovery.mdns")
logger.setLevel(logging.INFO)
handler = logging.StreamHandler()
handler.setFormatter(
logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
)
logger.addHandler(handler)
def onPeerDiscovery(peerinfo: PeerInfo):
logger.info(f"Discovered: {peerinfo.peer_id}")
async def run(port: int) -> None:
from libp2p.utils.address_validation import find_free_port, get_available_interfaces
if port <= 0:
port = find_free_port()
secret = secrets.token_bytes(32)
key_pair = create_new_key_pair(secret)
listen_addrs = get_available_interfaces(port)
peerDiscovery.register_peer_discovered_handler(onPeerDiscovery)
logger.info("Starting peer Discovery")
host = new_host(key_pair=key_pair, enable_mDNS=True)
async with host.run(listen_addrs=listen_addrs), trio.open_nursery() as nursery:
# Start the peer-store cleanup task
nursery.start_soon(host.get_peerstore().start_cleanup_task, 60)
# Get all available addresses with peer ID
all_addrs = host.get_addrs()
print("Listener ready, listening on:")
for addr in all_addrs:
print(f"{addr}")
print(
"\nRun this from the same folder in another console to "
"start another peer on a different port:\n\n"
"mdns-demo -p <ANOTHER_PORT>\n"
)
print("Waiting for mDNS peer discovery events...\n")
await trio.sleep_forever()
def main() -> None:
description = """
This program demonstrates mDNS peer discovery using libp2p.
To use it, run 'mdns-demo -p <PORT>', where <PORT> is the port number.
Start multiple peers on different ports to see discovery in action.
"""
parser = argparse.ArgumentParser(description=description)
parser.add_argument("-p", "--port", default=0, type=int, help="source port number")
parser.add_argument(
"-v", "--verbose", action="store_true", help="Enable verbose output"
)
args = parser.parse_args()
if args.verbose:
logger.setLevel(logging.DEBUG)
try:
trio.run(run, args.port)
except KeyboardInterrupt:
logger.info("Exiting...")
if __name__ == "__main__":
main()

View File

@ -1,4 +1,5 @@
import argparse
import logging
import multiaddr
import trio
@ -16,6 +17,11 @@ from libp2p.peer.peerinfo import (
info_from_p2p_addr,
)
# Configure minimal logging
logging.basicConfig(level=logging.WARNING)
logging.getLogger("multiaddr").setLevel(logging.WARNING)
logging.getLogger("libp2p").setLevel(logging.WARNING)
PING_PROTOCOL_ID = TProtocol("/ipfs/ping/1.0.0")
PING_LENGTH = 32
RESP_TIMEOUT = 60
@ -55,18 +61,38 @@ async def send_ping(stream: INetStream) -> None:
async def run(port: int, destination: str) -> None:
localhost_ip = "127.0.0.1"
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
host = new_host(listen_addrs=[listen_addr])
from libp2p.utils.address_validation import (
find_free_port,
get_available_interfaces,
get_optimal_binding_address,
)
if port <= 0:
port = find_free_port()
listen_addrs = get_available_interfaces(port)
host = new_host(listen_addrs=listen_addrs)
async with host.run(listen_addrs=listen_addrs), trio.open_nursery() as nursery:
# Start the peer-store cleanup task
nursery.start_soon(host.get_peerstore().start_cleanup_task, 60)
async with host.run(listen_addrs=[listen_addr]), trio.open_nursery() as nursery:
if not destination:
host.set_stream_handler(PING_PROTOCOL_ID, handle_ping)
# Get all available addresses with peer ID
all_addrs = host.get_addrs()
print("Listener ready, listening on:\n")
for addr in all_addrs:
print(f"{addr}")
# Use optimal address for the client command
optimal_addr = get_optimal_binding_address(port)
optimal_addr_with_peer = f"{optimal_addr}/p2p/{host.get_id().to_string()}"
print(
"Run this from the same folder in another console:\n\n"
f"ping-demo -p {int(port) + 1} "
f"-d /ip4/{localhost_ip}/tcp/{port}/p2p/{host.get_id().pretty()}\n"
f"\nRun this from the same folder in another console:\n\n"
f"ping-demo -d {optimal_addr_with_peer}\n"
)
print("Waiting for incoming connection...")
@ -92,14 +118,12 @@ def main() -> None:
"""
example_maddr = (
"/ip4/127.0.0.1/tcp/8000/p2p/QmQn4SwGkDZKkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
"/ip4/[HOST_IP]/tcp/8000/p2p/QmQn4SwGkDZKkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
)
parser = argparse.ArgumentParser(description=description)
parser.add_argument("-p", "--port", default=0, type=int, help="source port number")
parser.add_argument(
"-p", "--port", default=8000, type=int, help="source port number"
)
parser.add_argument(
"-d",
"--destination",
@ -108,9 +132,6 @@ def main() -> None:
)
args = parser.parse_args()
if not args.port:
raise RuntimeError("failed to determine local port")
try:
trio.run(run, *(args.port, args.destination))
except KeyboardInterrupt:

View File

@ -1,9 +1,5 @@
import argparse
import logging
import socket
from typing import (
Optional,
)
import base58
import multiaddr
@ -34,6 +30,9 @@ from libp2p.stream_muxer.mplex.mplex import (
from libp2p.tools.async_service.trio_service import (
background_trio_service,
)
from libp2p.utils.address_validation import (
find_free_port,
)
# Configure logging
logging.basicConfig(
@ -80,13 +79,6 @@ async def publish_loop(pubsub, topic, termination_event):
await trio.sleep(1) # Avoid tight loop on error
def find_free_port():
"""Find a free port on localhost."""
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(("", 0)) # Bind to a free port provided by the OS
return s.getsockname()[1]
async def monitor_peer_topics(pubsub, nursery, termination_event):
"""
Monitor for new topics that peers are subscribed to and
@ -109,15 +101,17 @@ async def monitor_peer_topics(pubsub, nursery, termination_event):
await trio.sleep(2)
async def run(topic: str, destination: Optional[str], port: Optional[int]) -> None:
# Initialize network settings
localhost_ip = "127.0.0.1"
async def run(topic: str, destination: str | None, port: int | None) -> None:
from libp2p.utils.address_validation import (
get_available_interfaces,
get_optimal_binding_address,
)
if port is None or port == 0:
port = find_free_port()
logger.info(f"Using random available port: {port}")
listen_addr = multiaddr.Multiaddr(f"/ip4/0.0.0.0/tcp/{port}")
listen_addrs = get_available_interfaces(port)
# Create a new libp2p host
host = new_host(
@ -146,9 +140,11 @@ async def run(topic: str, destination: Optional[str], port: Optional[int]) -> No
pubsub = Pubsub(host, gossipsub)
termination_event = trio.Event() # Event to signal termination
async with host.run(listen_addrs=[listen_addr]), trio.open_nursery() as nursery:
async with host.run(listen_addrs=listen_addrs), trio.open_nursery() as nursery:
# Start the peer-store cleanup task
nursery.start_soon(host.get_peerstore().start_cleanup_task, 60)
logger.info(f"Node started with peer ID: {host.get_id()}")
logger.info(f"Listening on: {listen_addr}")
logger.info("Initializing PubSub and GossipSub...")
async with background_trio_service(pubsub):
async with background_trio_service(gossipsub):
@ -162,10 +158,21 @@ async def run(topic: str, destination: Optional[str], port: Optional[int]) -> No
if not destination:
# Server mode
# Get all available addresses with peer ID
all_addrs = host.get_addrs()
logger.info("Listener ready, listening on:")
for addr in all_addrs:
logger.info(f"{addr}")
# Use optimal address for the client command
optimal_addr = get_optimal_binding_address(port)
optimal_addr_with_peer = (
f"{optimal_addr}/p2p/{host.get_id().to_string()}"
)
logger.info(
"Run this script in another console with:\n"
f"pubsub-demo "
f"-d /ip4/{localhost_ip}/tcp/{port}/p2p/{host.get_id()}\n"
f"\nRun this from the same folder in another console:\n\n"
f"pubsub-demo -d {optimal_addr_with_peer}\n"
)
logger.info("Waiting for peers...")
@ -187,11 +194,6 @@ async def run(topic: str, destination: Optional[str], port: Optional[int]) -> No
f"Connecting to peer: {info.peer_id} "
f"using protocols: {protocols_in_maddr}"
)
logger.info(
"Run this script in another console with:\n"
f"pubsub-demo "
f"-d /ip4/{localhost_ip}/tcp/{port}/p2p/{host.get_id()}\n"
)
try:
await host.connect(info)
logger.info(f"Connected to peer: {info.peer_id}")

View File

@ -0,0 +1,228 @@
"""
Random Walk Example for py-libp2p Kademlia DHT
This example demonstrates the Random Walk module's peer discovery capabilities
using real libp2p hosts and Kademlia DHT. It shows how the Random Walk module
automatically discovers new peers and maintains routing table health.
Usage:
# Start server nodes (they will discover peers via random walk)
python3 random_walk.py --mode server
"""
import argparse
import logging
import random
import secrets
import sys
import trio
from libp2p import new_host
from libp2p.abc import IHost
from libp2p.crypto.secp256k1 import create_new_key_pair
from libp2p.kad_dht.kad_dht import DHTMode, KadDHT
from libp2p.tools.async_service import background_trio_service
# Simple logging configuration
def setup_logging(verbose: bool = False):
"""Setup unified logging configuration."""
level = logging.DEBUG if verbose else logging.INFO
logging.basicConfig(
level=level,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
handlers=[logging.StreamHandler()],
)
# Configure key module loggers
for module in ["libp2p.discovery.random_walk", "libp2p.kad_dht"]:
logging.getLogger(module).setLevel(level)
# Suppress noisy logs
logging.getLogger("multiaddr").setLevel(logging.WARNING)
logger = logging.getLogger("random-walk-example")
# Default bootstrap nodes
DEFAULT_BOOTSTRAP_NODES = [
"/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
]
def filter_compatible_peer_info(peer_info) -> bool:
"""Filter peer info to check if it has compatible addresses (TCP + IPv4)."""
if not hasattr(peer_info, "addrs") or not peer_info.addrs:
return False
for addr in peer_info.addrs:
addr_str = str(addr)
if "/tcp/" in addr_str and "/ip4/" in addr_str and "/quic" not in addr_str:
return True
return False
async def maintain_connections(host: IHost) -> None:
"""Maintain connections to ensure the host remains connected to healthy peers."""
while True:
try:
connected_peers = host.get_connected_peers()
list_peers = host.get_peerstore().peers_with_addrs()
if len(connected_peers) < 20:
logger.debug("Reconnecting to maintain peer connections...")
# Find compatible peers
compatible_peers = []
for peer_id in list_peers:
try:
peer_info = host.get_peerstore().peer_info(peer_id)
if filter_compatible_peer_info(peer_info):
compatible_peers.append(peer_id)
except Exception:
continue
# Connect to random subset of compatible peers
if compatible_peers:
random_peers = random.sample(
compatible_peers, min(50, len(compatible_peers))
)
for peer_id in random_peers:
if peer_id not in connected_peers:
try:
with trio.move_on_after(5):
peer_info = host.get_peerstore().peer_info(peer_id)
await host.connect(peer_info)
logger.debug(f"Connected to peer: {peer_id}")
except Exception as e:
logger.debug(f"Failed to connect to {peer_id}: {e}")
await trio.sleep(15)
except Exception as e:
logger.error(f"Error maintaining connections: {e}")
async def demonstrate_random_walk_discovery(dht: KadDHT, interval: int = 30) -> None:
"""Demonstrate Random Walk peer discovery with periodic statistics."""
iteration = 0
while True:
iteration += 1
logger.info(f"--- Iteration {iteration} ---")
logger.info(f"Routing table size: {dht.get_routing_table_size()}")
logger.info(f"Connected peers: {len(dht.host.get_connected_peers())}")
logger.info(f"Peerstore size: {len(dht.host.get_peerstore().peer_ids())}")
await trio.sleep(interval)
async def run_node(port: int, mode: str, demo_interval: int = 30) -> None:
"""Run a node that demonstrates Random Walk peer discovery."""
try:
if port <= 0:
port = random.randint(10000, 60000)
logger.info(f"Starting {mode} node on port {port}")
# Determine DHT mode
dht_mode = DHTMode.SERVER if mode == "server" else DHTMode.CLIENT
# Create host and DHT
key_pair = create_new_key_pair(secrets.token_bytes(32))
host = new_host(key_pair=key_pair, bootstrap=DEFAULT_BOOTSTRAP_NODES)
from libp2p.utils.address_validation import get_available_interfaces
listen_addrs = get_available_interfaces(port)
async with host.run(listen_addrs=listen_addrs), trio.open_nursery() as nursery:
# Start maintenance tasks
nursery.start_soon(host.get_peerstore().start_cleanup_task, 60)
nursery.start_soon(maintain_connections, host)
peer_id = host.get_id().pretty()
logger.info(f"Node peer ID: {peer_id}")
# Get all available addresses with peer ID
all_addrs = host.get_addrs()
logger.info("Listener ready, listening on:")
for addr in all_addrs:
logger.info(f"{addr}")
# Create and start DHT with Random Walk enabled
dht = KadDHT(host, dht_mode, enable_random_walk=True)
logger.info(f"Initial routing table size: {dht.get_routing_table_size()}")
async with background_trio_service(dht):
logger.info(f"DHT service started in {dht_mode.value} mode")
logger.info(f"Random Walk enabled: {dht.is_random_walk_enabled()}")
async with trio.open_nursery() as task_nursery:
# Start demonstration and status reporting
task_nursery.start_soon(
demonstrate_random_walk_discovery, dht, demo_interval
)
# Periodic status updates
async def status_reporter():
while True:
await trio.sleep(30)
logger.debug(
f"Connected: {len(dht.host.get_connected_peers())}, "
f"Routing table: {dht.get_routing_table_size()}, "
f"Peerstore: {len(dht.host.get_peerstore().peer_ids())}"
)
task_nursery.start_soon(status_reporter)
await trio.sleep_forever()
except Exception as e:
logger.error(f"Node error: {e}", exc_info=True)
sys.exit(1)
def parse_args():
"""Parse command line arguments."""
parser = argparse.ArgumentParser(
description="Random Walk Example for py-libp2p Kademlia DHT",
)
parser.add_argument(
"--mode",
choices=["server", "client"],
default="server",
help="Node mode: server (DHT server), or client (DHT client)",
)
parser.add_argument(
"--port", type=int, default=0, help="Port to listen on (0 for random)"
)
parser.add_argument(
"--demo-interval",
type=int,
default=30,
help="Interval between random walk demonstrations in seconds",
)
parser.add_argument("--verbose", action="store_true", help="Enable verbose logging")
return parser.parse_args()
def main():
"""Main entry point for the random walk example."""
try:
args = parse_args()
setup_logging(args.verbose)
logger.info("=== Random Walk Example for py-libp2p ===")
logger.info(
f"Mode: {args.mode}, Port: {args.port} Demo interval: {args.demo_interval}s"
)
trio.run(run_node, args.port, args.mode, args.demo_interval)
except KeyboardInterrupt:
logger.info("Received interrupt signal, shutting down...")
except Exception as e:
logger.critical(f"Example failed: {e}", exc_info=True)
sys.exit(1)
if __name__ == "__main__":
main()

View File

@ -0,0 +1,446 @@
#!/usr/bin/env python3
"""
TCP P2P Data Transfer Test
This test proves that TCP peer-to-peer data transfer works correctly in libp2p.
This serves as a baseline to compare with WebSocket tests.
"""
import pytest
from multiaddr import Multiaddr
import trio
from libp2p import create_yamux_muxer_option, new_host
from libp2p.crypto.secp256k1 import create_new_key_pair
from libp2p.custom_types import TProtocol
from libp2p.peer.peerinfo import info_from_p2p_addr
from libp2p.security.insecure.transport import PLAINTEXT_PROTOCOL_ID, InsecureTransport
# Test protocol for data exchange
TCP_DATA_PROTOCOL = TProtocol("/test/tcp-data-exchange/1.0.0")
async def create_tcp_host_pair():
"""Create a pair of hosts configured for TCP communication."""
# Create key pairs
key_pair_a = create_new_key_pair()
key_pair_b = create_new_key_pair()
# Create security options (using plaintext for simplicity)
def security_options(kp):
return {
PLAINTEXT_PROTOCOL_ID: InsecureTransport(
local_key_pair=kp, secure_bytes_provider=None, peerstore=None
)
}
# Host A (listener) - TCP transport (default)
host_a = new_host(
key_pair=key_pair_a,
sec_opt=security_options(key_pair_a),
muxer_opt=create_yamux_muxer_option(),
listen_addrs=[Multiaddr("/ip4/127.0.0.1/tcp/0")],
)
# Host B (dialer) - TCP transport (default)
host_b = new_host(
key_pair=key_pair_b,
sec_opt=security_options(key_pair_b),
muxer_opt=create_yamux_muxer_option(),
listen_addrs=[Multiaddr("/ip4/127.0.0.1/tcp/0")],
)
return host_a, host_b
@pytest.mark.trio
async def test_tcp_basic_connection():
"""Test basic TCP connection establishment."""
host_a, host_b = await create_tcp_host_pair()
connection_established = False
async def connection_handler(stream):
nonlocal connection_established
connection_established = True
await stream.close()
host_a.set_stream_handler(TCP_DATA_PROTOCOL, connection_handler)
async with (
host_a.run(listen_addrs=[Multiaddr("/ip4/127.0.0.1/tcp/0")]),
host_b.run(listen_addrs=[]),
):
# Get host A's listen address
listen_addrs = host_a.get_addrs()
assert listen_addrs, "Host A should have listen addresses"
# Extract TCP address
tcp_addr = None
for addr in listen_addrs:
if "/tcp/" in str(addr) and "/ws" not in str(addr):
tcp_addr = addr
break
assert tcp_addr, f"No TCP address found in {listen_addrs}"
print(f"🔗 Host A listening on: {tcp_addr}")
# Create peer info for host A
peer_info = info_from_p2p_addr(tcp_addr)
# Host B connects to host A
await host_b.connect(peer_info)
print("✅ TCP connection established")
# Open a stream to test the connection
stream = await host_b.new_stream(peer_info.peer_id, [TCP_DATA_PROTOCOL])
await stream.close()
# Wait a bit for the handler to be called
await trio.sleep(0.1)
assert connection_established, "TCP connection handler should have been called"
print("✅ TCP basic connection test successful!")
@pytest.mark.trio
async def test_tcp_data_transfer():
"""Test TCP peer-to-peer data transfer."""
host_a, host_b = await create_tcp_host_pair()
# Test data
test_data = b"Hello TCP P2P Data Transfer! This is a test message."
received_data = None
transfer_complete = trio.Event()
async def data_handler(stream):
nonlocal received_data
try:
# Read the incoming data
received_data = await stream.read(len(test_data))
# Echo it back to confirm successful transfer
await stream.write(received_data)
await stream.close()
transfer_complete.set()
except Exception as e:
print(f"Handler error: {e}")
transfer_complete.set()
host_a.set_stream_handler(TCP_DATA_PROTOCOL, data_handler)
async with (
host_a.run(listen_addrs=[Multiaddr("/ip4/127.0.0.1/tcp/0")]),
host_b.run(listen_addrs=[]),
):
# Get host A's listen address
listen_addrs = host_a.get_addrs()
assert listen_addrs, "Host A should have listen addresses"
# Extract TCP address
tcp_addr = None
for addr in listen_addrs:
if "/tcp/" in str(addr) and "/ws" not in str(addr):
tcp_addr = addr
break
assert tcp_addr, f"No TCP address found in {listen_addrs}"
print(f"🔗 Host A listening on: {tcp_addr}")
# Create peer info for host A
peer_info = info_from_p2p_addr(tcp_addr)
# Host B connects to host A
await host_b.connect(peer_info)
print("✅ TCP connection established")
# Open a stream for data transfer
stream = await host_b.new_stream(peer_info.peer_id, [TCP_DATA_PROTOCOL])
print("✅ TCP stream opened")
# Send test data
await stream.write(test_data)
print(f"📤 Sent data: {test_data}")
# Read echoed data back
echoed_data = await stream.read(len(test_data))
print(f"📥 Received echo: {echoed_data}")
await stream.close()
# Wait for transfer to complete
with trio.fail_after(5.0): # 5 second timeout
await transfer_complete.wait()
# Verify data transfer
assert received_data == test_data, (
f"Data mismatch: {received_data} != {test_data}"
)
assert echoed_data == test_data, f"Echo mismatch: {echoed_data} != {test_data}"
print("✅ TCP P2P data transfer successful!")
print(f" Original: {test_data}")
print(f" Received: {received_data}")
print(f" Echoed: {echoed_data}")
@pytest.mark.trio
async def test_tcp_large_data_transfer():
"""Test TCP with larger data payloads."""
host_a, host_b = await create_tcp_host_pair()
# Large test data (10KB)
test_data = b"TCP Large Data Test! " * 500 # ~10KB
received_data = None
transfer_complete = trio.Event()
async def large_data_handler(stream):
nonlocal received_data
try:
# Read data in chunks
chunks = []
total_received = 0
expected_size = len(test_data)
while total_received < expected_size:
chunk = await stream.read(min(1024, expected_size - total_received))
if not chunk:
break
chunks.append(chunk)
total_received += len(chunk)
received_data = b"".join(chunks)
# Send back confirmation
await stream.write(b"RECEIVED_OK")
await stream.close()
transfer_complete.set()
except Exception as e:
print(f"Large data handler error: {e}")
transfer_complete.set()
host_a.set_stream_handler(TCP_DATA_PROTOCOL, large_data_handler)
async with (
host_a.run(listen_addrs=[Multiaddr("/ip4/127.0.0.1/tcp/0")]),
host_b.run(listen_addrs=[]),
):
# Get host A's listen address
listen_addrs = host_a.get_addrs()
assert listen_addrs, "Host A should have listen addresses"
# Extract TCP address
tcp_addr = None
for addr in listen_addrs:
if "/tcp/" in str(addr) and "/ws" not in str(addr):
tcp_addr = addr
break
assert tcp_addr, f"No TCP address found in {listen_addrs}"
print(f"🔗 Host A listening on: {tcp_addr}")
print(f"📊 Test data size: {len(test_data)} bytes")
# Create peer info for host A
peer_info = info_from_p2p_addr(tcp_addr)
# Host B connects to host A
await host_b.connect(peer_info)
print("✅ TCP connection established")
# Open a stream for data transfer
stream = await host_b.new_stream(peer_info.peer_id, [TCP_DATA_PROTOCOL])
print("✅ TCP stream opened")
# Send large test data in chunks
chunk_size = 1024
sent_bytes = 0
for i in range(0, len(test_data), chunk_size):
chunk = test_data[i : i + chunk_size]
await stream.write(chunk)
sent_bytes += len(chunk)
if sent_bytes % (chunk_size * 4) == 0: # Progress every 4KB
print(f"📤 Sent {sent_bytes}/{len(test_data)} bytes")
print(f"📤 Sent all {len(test_data)} bytes")
# Read confirmation
confirmation = await stream.read(1024)
print(f"📥 Received confirmation: {confirmation}")
await stream.close()
# Wait for transfer to complete
with trio.fail_after(10.0): # 10 second timeout for large data
await transfer_complete.wait()
# Verify data transfer
assert received_data is not None, "No data was received"
assert received_data == test_data, (
"Large data transfer failed:"
+ f" sizes {len(received_data)} != {len(test_data)}"
)
assert confirmation == b"RECEIVED_OK", f"Confirmation failed: {confirmation}"
print("✅ TCP large data transfer successful!")
print(f" Data size: {len(test_data)} bytes")
print(f" Received: {len(received_data)} bytes")
print(f" Match: {received_data == test_data}")
@pytest.mark.trio
async def test_tcp_bidirectional_transfer():
"""Test bidirectional data transfer over TCP."""
host_a, host_b = await create_tcp_host_pair()
# Test data
data_a_to_b = b"Message from Host A to Host B via TCP"
data_b_to_a = b"Response from Host B to Host A via TCP"
received_on_a = None
received_on_b = None
transfer_complete_a = trio.Event()
transfer_complete_b = trio.Event()
async def handler_a(stream):
nonlocal received_on_a
try:
# Read data from B
received_on_a = await stream.read(len(data_b_to_a))
print(f"🅰️ Host A received: {received_on_a}")
await stream.close()
transfer_complete_a.set()
except Exception as e:
print(f"Handler A error: {e}")
transfer_complete_a.set()
async def handler_b(stream):
nonlocal received_on_b
try:
# Read data from A
received_on_b = await stream.read(len(data_a_to_b))
print(f"🅱️ Host B received: {received_on_b}")
await stream.close()
transfer_complete_b.set()
except Exception as e:
print(f"Handler B error: {e}")
transfer_complete_b.set()
# Set up handlers on both hosts
protocol_a_to_b = TProtocol("/test/tcp-a-to-b/1.0.0")
protocol_b_to_a = TProtocol("/test/tcp-b-to-a/1.0.0")
host_a.set_stream_handler(protocol_b_to_a, handler_a)
host_b.set_stream_handler(protocol_a_to_b, handler_b)
async with (
host_a.run(listen_addrs=[Multiaddr("/ip4/127.0.0.1/tcp/0")]),
host_b.run(listen_addrs=[Multiaddr("/ip4/127.0.0.1/tcp/0")]),
):
# Get addresses
addrs_a = host_a.get_addrs()
addrs_b = host_b.get_addrs()
assert addrs_a and addrs_b, "Both hosts should have addresses"
# Extract TCP addresses
tcp_addr_a = next(
(
addr
for addr in addrs_a
if "/tcp/" in str(addr) and "/ws" not in str(addr)
),
None,
)
tcp_addr_b = next(
(
addr
for addr in addrs_b
if "/tcp/" in str(addr) and "/ws" not in str(addr)
),
None,
)
assert tcp_addr_a and tcp_addr_b, (
f"TCP addresses not found: A={addrs_a}, B={addrs_b}"
)
print(f"🔗 Host A listening on: {tcp_addr_a}")
print(f"🔗 Host B listening on: {tcp_addr_b}")
# Create peer infos
peer_info_a = info_from_p2p_addr(tcp_addr_a)
peer_info_b = info_from_p2p_addr(tcp_addr_b)
# Establish connections
await host_b.connect(peer_info_a)
await host_a.connect(peer_info_b)
print("✅ Bidirectional TCP connections established")
# Send data A -> B
stream_a_to_b = await host_a.new_stream(peer_info_b.peer_id, [protocol_a_to_b])
await stream_a_to_b.write(data_a_to_b)
print(f"📤 A->B: {data_a_to_b}")
await stream_a_to_b.close()
# Send data B -> A
stream_b_to_a = await host_b.new_stream(peer_info_a.peer_id, [protocol_b_to_a])
await stream_b_to_a.write(data_b_to_a)
print(f"📤 B->A: {data_b_to_a}")
await stream_b_to_a.close()
# Wait for both transfers to complete
with trio.fail_after(5.0):
await transfer_complete_a.wait()
await transfer_complete_b.wait()
# Verify bidirectional transfer
assert received_on_a == data_b_to_a, f"A received wrong data: {received_on_a}"
assert received_on_b == data_a_to_b, f"B received wrong data: {received_on_b}"
print("✅ TCP bidirectional data transfer successful!")
print(f" A->B: {data_a_to_b}")
print(f" B->A: {data_b_to_a}")
print(f" ✓ A got: {received_on_a}")
print(f" ✓ B got: {received_on_b}")
if __name__ == "__main__":
# Run tests directly
import logging
logging.basicConfig(level=logging.INFO)
print("🧪 Running TCP P2P Data Transfer Tests")
print("=" * 50)
async def run_all_tcp_tests():
try:
print("\n1. Testing basic TCP connection...")
await test_tcp_basic_connection()
except Exception as e:
print(f"❌ Basic TCP connection test failed: {e}")
return
try:
print("\n2. Testing TCP data transfer...")
await test_tcp_data_transfer()
except Exception as e:
print(f"❌ TCP data transfer test failed: {e}")
return
try:
print("\n3. Testing TCP large data transfer...")
await test_tcp_large_data_transfer()
except Exception as e:
print(f"❌ TCP large data transfer test failed: {e}")
return
try:
print("\n4. Testing TCP bidirectional transfer...")
await test_tcp_bidirectional_transfer()
except Exception as e:
print(f"❌ TCP bidirectional transfer test failed: {e}")
return
print("\n" + "=" * 50)
print("🏁 TCP P2P Tests Complete - All Tests PASSED!")
trio.run(run_all_tcp_tests)

View File

@ -0,0 +1,210 @@
#!/usr/bin/env python3
"""
Demo script showing the new transport integration capabilities in py-libp2p.
This script demonstrates:
1. How to use the transport registry
2. How to create transports dynamically based on multiaddrs
3. How to register custom transports
4. How the new system automatically selects the right transport
"""
import asyncio
import logging
from pathlib import Path
import sys
# Add the libp2p directory to the path so we can import it
sys.path.insert(0, str(Path(__file__).parent.parent))
import multiaddr
from libp2p.transport import (
create_transport,
create_transport_for_multiaddr,
get_supported_transport_protocols,
get_transport_registry,
register_transport,
)
from libp2p.transport.tcp.tcp import TCP
from libp2p.transport.upgrader import TransportUpgrader
# Set up logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
def demo_transport_registry():
"""Demonstrate the transport registry functionality."""
print("🔧 Transport Registry Demo")
print("=" * 50)
# Get the global registry
registry = get_transport_registry()
# Show supported protocols
supported = get_supported_transport_protocols()
print(f"Supported transport protocols: {supported}")
# Show registered transports
print("\nRegistered transports:")
for protocol in supported:
transport_class = registry.get_transport(protocol)
class_name = transport_class.__name__ if transport_class else "None"
print(f" {protocol}: {class_name}")
print()
def demo_transport_factory():
"""Demonstrate the transport factory functions."""
print("🏭 Transport Factory Demo")
print("=" * 50)
# Create a dummy upgrader for WebSocket transport
upgrader = TransportUpgrader({}, {})
# Create transports using the factory function
try:
tcp_transport = create_transport("tcp")
print(f"✅ Created TCP transport: {type(tcp_transport).__name__}")
ws_transport = create_transport("ws", upgrader)
print(f"✅ Created WebSocket transport: {type(ws_transport).__name__}")
except Exception as e:
print(f"❌ Error creating transport: {e}")
print()
def demo_multiaddr_transport_selection():
"""Demonstrate automatic transport selection based on multiaddrs."""
print("🎯 Multiaddr Transport Selection Demo")
print("=" * 50)
# Create a dummy upgrader
upgrader = TransportUpgrader({}, {})
# Test different multiaddr types
test_addrs = [
"/ip4/127.0.0.1/tcp/8080",
"/ip4/127.0.0.1/tcp/8080/ws",
"/ip6/::1/tcp/8080/ws",
"/dns4/example.com/tcp/443/ws",
]
for addr_str in test_addrs:
try:
maddr = multiaddr.Multiaddr(addr_str)
transport = create_transport_for_multiaddr(maddr, upgrader)
if transport:
print(f"{addr_str} -> {type(transport).__name__}")
else:
print(f"{addr_str} -> No transport found")
except Exception as e:
print(f"{addr_str} -> Error: {e}")
print()
def demo_custom_transport_registration():
"""Demonstrate how to register custom transports."""
print("🔧 Custom Transport Registration Demo")
print("=" * 50)
# Show current supported protocols
print(f"Before registration: {get_supported_transport_protocols()}")
# Register a custom transport (using TCP as an example)
class CustomTCPTransport(TCP):
"""Custom TCP transport for demonstration."""
def __init__(self):
super().__init__()
self.custom_flag = True
# Register the custom transport
register_transport("custom_tcp", CustomTCPTransport)
# Show updated supported protocols
print(f"After registration: {get_supported_transport_protocols()}")
# Test creating the custom transport
try:
custom_transport = create_transport("custom_tcp")
print(f"✅ Created custom transport: {type(custom_transport).__name__}")
# Check if it has the custom flag (type-safe way)
if hasattr(custom_transport, "custom_flag"):
flag_value = getattr(custom_transport, "custom_flag", "Not found")
print(f" Custom flag: {flag_value}")
else:
print(" Custom flag: Not found")
except Exception as e:
print(f"❌ Error creating custom transport: {e}")
print()
def demo_integration_with_libp2p():
"""Demonstrate how the new system integrates with libp2p."""
print("🚀 Libp2p Integration Demo")
print("=" * 50)
print("The new transport system integrates seamlessly with libp2p:")
print()
print("1. ✅ Automatic transport selection based on multiaddr")
print("2. ✅ Support for WebSocket (/ws) protocol")
print("3. ✅ Fallback to TCP for backward compatibility")
print("4. ✅ Easy registration of new transport protocols")
print("5. ✅ No changes needed to existing libp2p code")
print()
print("Example usage in libp2p:")
print(" # This will automatically use WebSocket transport")
print(" host = new_host(listen_addrs=['/ip4/127.0.0.1/tcp/8080/ws'])")
print()
print(" # This will automatically use TCP transport")
print(" host = new_host(listen_addrs=['/ip4/127.0.0.1/tcp/8080'])")
print()
print()
async def main():
"""Run all demos."""
print("🎉 Py-libp2p Transport Integration Demo")
print("=" * 60)
print()
# Run all demos
demo_transport_registry()
demo_transport_factory()
demo_multiaddr_transport_selection()
demo_custom_transport_registration()
demo_integration_with_libp2p()
print("🎯 Summary of New Features:")
print("=" * 40)
print("✅ Transport Registry: Central registry for all transport implementations")
print("✅ Dynamic Transport Selection: Automatic selection based on multiaddr")
print("✅ WebSocket Support: Full /ws protocol support")
print("✅ Extensible Architecture: Easy to add new transport protocols")
print("✅ Backward Compatibility: Existing TCP code continues to work")
print("✅ Factory Functions: Simple API for creating transports")
print()
print("🚀 The transport system is now ready for production use!")
if __name__ == "__main__":
try:
asyncio.run(main())
except KeyboardInterrupt:
print("\n👋 Demo interrupted by user")
except Exception as e:
print(f"\n❌ Demo failed with error: {e}")
import traceback
traceback.print_exc()

View File

@ -0,0 +1,220 @@
#!/usr/bin/env python3
"""
Simple TCP echo demo to verify basic libp2p functionality.
"""
import argparse
import logging
import traceback
import multiaddr
import trio
from libp2p.crypto.secp256k1 import create_new_key_pair
from libp2p.custom_types import TProtocol
from libp2p.host.basic_host import BasicHost
from libp2p.network.swarm import Swarm
from libp2p.peer.id import ID
from libp2p.peer.peerinfo import info_from_p2p_addr
from libp2p.peer.peerstore import PeerStore
from libp2p.security.insecure.transport import PLAINTEXT_PROTOCOL_ID, InsecureTransport
from libp2p.stream_muxer.yamux.yamux import Yamux
from libp2p.transport.tcp.tcp import TCP
from libp2p.transport.upgrader import TransportUpgrader
# Enable debug logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger("libp2p.tcp-example")
# Simple echo protocol
ECHO_PROTOCOL_ID = TProtocol("/echo/1.0.0")
async def echo_handler(stream):
"""Simple echo handler that echoes back any data received."""
try:
data = await stream.read(1024)
if data:
message = data.decode("utf-8", errors="replace")
print(f"📥 Received: {message}")
print(f"📤 Echoing back: {message}")
await stream.write(data)
await stream.close()
except Exception as e:
logger.error(f"Echo handler error: {e}")
await stream.close()
def create_tcp_host():
"""Create a host with TCP transport."""
# Create key pair and peer store
key_pair = create_new_key_pair()
peer_id = ID.from_pubkey(key_pair.public_key)
peer_store = PeerStore()
peer_store.add_key_pair(peer_id, key_pair)
# Create transport upgrader with plaintext security
upgrader = TransportUpgrader(
secure_transports_by_protocol={
TProtocol(PLAINTEXT_PROTOCOL_ID): InsecureTransport(key_pair)
},
muxer_transports_by_protocol={TProtocol("/yamux/1.0.0"): Yamux},
)
# Create TCP transport
transport = TCP()
# Create swarm and host
swarm = Swarm(peer_id, peer_store, upgrader, transport)
host = BasicHost(swarm)
return host
async def run(port: int, destination: str) -> None:
localhost_ip = "0.0.0.0"
if not destination:
# Create first host (listener) with TCP transport
listen_addr = multiaddr.Multiaddr(f"/ip4/{localhost_ip}/tcp/{port}")
try:
host = create_tcp_host()
logger.debug("Created TCP host")
# Set up echo handler
host.set_stream_handler(ECHO_PROTOCOL_ID, echo_handler)
async with (
host.run(listen_addrs=[listen_addr]),
trio.open_nursery() as (nursery),
):
# Start the peer-store cleanup task
nursery.start_soon(host.get_peerstore().start_cleanup_task, 60)
# Get the actual address and replace 0.0.0.0 with 127.0.0.1 for client
# connections
addrs = host.get_addrs()
logger.debug(f"Host addresses: {addrs}")
if not addrs:
print("❌ Error: No addresses found for the host")
return
server_addr = str(addrs[0])
client_addr = server_addr.replace("/ip4/0.0.0.0/", "/ip4/127.0.0.1/")
print("🌐 TCP Server Started Successfully!")
print("=" * 50)
print(f"📍 Server Address: {client_addr}")
print("🔧 Protocol: /echo/1.0.0")
print("🚀 Transport: TCP")
print()
print("📋 To test the connection, run this in another terminal:")
print(f" python test_tcp_echo.py -d {client_addr}")
print()
print("⏳ Waiting for incoming TCP connections...")
print("" * 50)
await trio.sleep_forever()
except Exception as e:
print(f"❌ Error creating TCP server: {e}")
traceback.print_exc()
return
else:
# Create second host (dialer) with TCP transport
listen_addr = multiaddr.Multiaddr(f"/ip4/{localhost_ip}/tcp/{port}")
try:
# Create a single host for client operations
host = create_tcp_host()
# Start the host for client operations
async with (
host.run(listen_addrs=[listen_addr]),
trio.open_nursery() as (nursery),
):
# Start the peer-store cleanup task
nursery.start_soon(host.get_peerstore().start_cleanup_task, 60)
maddr = multiaddr.Multiaddr(destination)
info = info_from_p2p_addr(maddr)
print("🔌 TCP Client Starting...")
print("=" * 40)
print(f"🎯 Target Peer: {info.peer_id}")
print(f"📍 Target Address: {destination}")
print()
try:
print("🔗 Connecting to TCP server...")
await host.connect(info)
print("✅ Successfully connected to TCP server!")
except Exception as e:
error_msg = str(e)
print("\n❌ Connection Failed!")
print(f" Peer ID: {info.peer_id}")
print(f" Address: {destination}")
print(f" Error: {error_msg}")
return
# Create a stream and send test data
try:
stream = await host.new_stream(info.peer_id, [ECHO_PROTOCOL_ID])
except Exception as e:
print(f"❌ Failed to create stream: {e}")
return
try:
print("🚀 Starting Echo Protocol Test...")
print("" * 40)
# Send test data
test_message = b"Hello TCP Transport!"
print(f"📤 Sending message: {test_message.decode('utf-8')}")
await stream.write(test_message)
# Read response
print("⏳ Waiting for server response...")
response = await stream.read(1024)
print(f"📥 Received response: {response.decode('utf-8')}")
await stream.close()
print("" * 40)
if response == test_message:
print("🎉 Echo test successful!")
print("✅ TCP transport is working perfectly!")
else:
print("❌ Echo test failed!")
except Exception as e:
print(f"Echo protocol error: {e}")
traceback.print_exc()
print("✅ TCP demo completed successfully!")
except Exception as e:
print(f"❌ Error creating TCP client: {e}")
traceback.print_exc()
return
def main() -> None:
description = "Simple TCP echo demo for libp2p"
parser = argparse.ArgumentParser(description=description)
parser.add_argument("-p", "--port", default=0, type=int, help="source port number")
parser.add_argument(
"-d", "--destination", type=str, help="destination multiaddr string"
)
args = parser.parse_args()
try:
trio.run(run, args.port, args.destination)
except KeyboardInterrupt:
pass
if __name__ == "__main__":
main()

View File

@ -0,0 +1,145 @@
#!/usr/bin/env python3
"""
Simple test script to verify WebSocket transport functionality.
"""
import asyncio
import logging
from pathlib import Path
import sys
# Add the libp2p directory to the path so we can import it
sys.path.insert(0, str(Path(__file__).parent))
import multiaddr
from libp2p.transport import create_transport, create_transport_for_multiaddr
from libp2p.transport.upgrader import TransportUpgrader
# Set up logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
async def test_websocket_transport():
"""Test basic WebSocket transport functionality."""
print("🧪 Testing WebSocket Transport Functionality")
print("=" * 50)
# Create a dummy upgrader
upgrader = TransportUpgrader({}, {})
# Test creating WebSocket transport
try:
ws_transport = create_transport("ws", upgrader)
print(f"✅ WebSocket transport created: {type(ws_transport).__name__}")
# Test creating transport from multiaddr
ws_maddr = multiaddr.Multiaddr("/ip4/127.0.0.1/tcp/8080/ws")
ws_transport_from_maddr = create_transport_for_multiaddr(ws_maddr, upgrader)
print(
f"✅ WebSocket transport from multiaddr: "
f"{type(ws_transport_from_maddr).__name__}"
)
# Test creating listener
handler_called = False
async def test_handler(conn):
nonlocal handler_called
handler_called = True
print(f"✅ Connection handler called with: {type(conn).__name__}")
await conn.close()
listener = ws_transport.create_listener(test_handler)
print(f"✅ WebSocket listener created: {type(listener).__name__}")
# Test that the transport can be used
print(
f"✅ WebSocket transport supports dialing: {hasattr(ws_transport, 'dial')}"
)
print(
f"✅ WebSocket transport supports listening: "
f"{hasattr(ws_transport, 'create_listener')}"
)
print("\n🎯 WebSocket Transport Test Results:")
print("✅ Transport creation: PASS")
print("✅ Multiaddr parsing: PASS")
print("✅ Listener creation: PASS")
print("✅ Interface compliance: PASS")
except Exception as e:
print(f"❌ WebSocket transport test failed: {e}")
import traceback
traceback.print_exc()
return False
return True
async def test_transport_registry():
"""Test the transport registry functionality."""
print("\n🔧 Testing Transport Registry")
print("=" * 30)
from libp2p.transport import (
get_supported_transport_protocols,
get_transport_registry,
)
registry = get_transport_registry()
supported = get_supported_transport_protocols()
print(f"Supported protocols: {supported}")
# Test getting transports
for protocol in supported:
transport_class = registry.get_transport(protocol)
class_name = transport_class.__name__ if transport_class else "None"
print(f" {protocol}: {class_name}")
# Test creating transports through registry
upgrader = TransportUpgrader({}, {})
for protocol in supported:
try:
transport = registry.create_transport(protocol, upgrader)
if transport:
print(f"{protocol}: Created successfully")
else:
print(f"{protocol}: Failed to create")
except Exception as e:
print(f"{protocol}: Error - {e}")
async def main():
"""Run all tests."""
print("🚀 WebSocket Transport Integration Test Suite")
print("=" * 60)
print()
# Run tests
success = await test_websocket_transport()
await test_transport_registry()
print("\n" + "=" * 60)
if success:
print("🎉 All tests passed! WebSocket transport is working correctly.")
else:
print("❌ Some tests failed. Check the output above for details.")
print("\n🚀 WebSocket transport is ready for use in py-libp2p!")
if __name__ == "__main__":
try:
asyncio.run(main())
except KeyboardInterrupt:
print("\n👋 Test interrupted by user")
except Exception as e:
print(f"\n❌ Test failed with error: {e}")
import traceback
traceback.print_exc()

View File

@ -0,0 +1,448 @@
import argparse
import logging
import signal
import sys
import traceback
import multiaddr
import trio
from libp2p.abc import INotifee
from libp2p.crypto.ed25519 import create_new_key_pair as create_ed25519_key_pair
from libp2p.crypto.secp256k1 import create_new_key_pair
from libp2p.custom_types import TProtocol
from libp2p.host.basic_host import BasicHost
from libp2p.network.swarm import Swarm
from libp2p.peer.id import ID
from libp2p.peer.peerinfo import info_from_p2p_addr
from libp2p.peer.peerstore import PeerStore
from libp2p.security.insecure.transport import PLAINTEXT_PROTOCOL_ID, InsecureTransport
from libp2p.security.noise.transport import (
PROTOCOL_ID as NOISE_PROTOCOL_ID,
Transport as NoiseTransport,
)
from libp2p.stream_muxer.yamux.yamux import Yamux
from libp2p.transport.upgrader import TransportUpgrader
from libp2p.transport.websocket.transport import WebsocketTransport
# Enable debug logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger("libp2p.websocket-example")
# Suppress KeyboardInterrupt by handling SIGINT directly
def signal_handler(signum, frame):
print("✅ Clean exit completed.")
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
# Simple echo protocol
ECHO_PROTOCOL_ID = TProtocol("/echo/1.0.0")
async def echo_handler(stream):
"""Simple echo handler that echoes back any data received."""
try:
data = await stream.read(1024)
if data:
message = data.decode("utf-8", errors="replace")
print(f"📥 Received: {message}")
print(f"📤 Echoing back: {message}")
await stream.write(data)
await stream.close()
except Exception as e:
logger.error(f"Echo handler error: {e}")
await stream.close()
def create_websocket_host(listen_addrs=None, use_plaintext=False):
"""Create a host with WebSocket transport."""
# Create key pair and peer store
key_pair = create_new_key_pair()
peer_id = ID.from_pubkey(key_pair.public_key)
peer_store = PeerStore()
peer_store.add_key_pair(peer_id, key_pair)
if use_plaintext:
# Create transport upgrader with plaintext security
upgrader = TransportUpgrader(
secure_transports_by_protocol={
TProtocol(PLAINTEXT_PROTOCOL_ID): InsecureTransport(key_pair)
},
muxer_transports_by_protocol={TProtocol("/yamux/1.0.0"): Yamux},
)
else:
# Create separate Ed25519 key for Noise protocol
noise_key_pair = create_ed25519_key_pair()
# Create Noise transport
noise_transport = NoiseTransport(
libp2p_keypair=key_pair,
noise_privkey=noise_key_pair.private_key,
early_data=None,
with_noise_pipes=False,
)
# Create transport upgrader with Noise security
upgrader = TransportUpgrader(
secure_transports_by_protocol={
TProtocol(NOISE_PROTOCOL_ID): noise_transport
},
muxer_transports_by_protocol={TProtocol("/yamux/1.0.0"): Yamux},
)
# Create WebSocket transport
transport = WebsocketTransport(upgrader)
# Create swarm and host
swarm = Swarm(peer_id, peer_store, upgrader, transport)
host = BasicHost(swarm)
return host
async def run(port: int, destination: str, use_plaintext: bool = False) -> None:
localhost_ip = "0.0.0.0"
if not destination:
# Create first host (listener) with WebSocket transport
listen_addr = multiaddr.Multiaddr(f"/ip4/{localhost_ip}/tcp/{port}/ws")
try:
host = create_websocket_host(use_plaintext=use_plaintext)
logger.debug(f"Created host with use_plaintext={use_plaintext}")
# Set up echo handler
host.set_stream_handler(ECHO_PROTOCOL_ID, echo_handler)
# Add connection event handlers for debugging
class DebugNotifee(INotifee):
async def opened_stream(self, network, stream):
pass
async def closed_stream(self, network, stream):
pass
async def connected(self, network, conn):
print(
f"🔗 New libp2p connection established: "
f"{conn.muxed_conn.peer_id}"
)
if hasattr(conn.muxed_conn, "get_security_protocol"):
security = conn.muxed_conn.get_security_protocol()
else:
security = "Unknown"
print(f" Security: {security}")
async def disconnected(self, network, conn):
print(f"🔌 libp2p connection closed: {conn.muxed_conn.peer_id}")
async def listen(self, network, multiaddr):
pass
async def listen_close(self, network, multiaddr):
pass
host.get_network().register_notifee(DebugNotifee())
# Create a cancellation token for clean shutdown
cancel_scope = trio.CancelScope()
async def signal_handler():
with trio.open_signal_receiver(signal.SIGINT, signal.SIGTERM) as (
signal_receiver
):
async for sig in signal_receiver:
print(f"\n🛑 Received signal {sig}")
print("✅ Shutting down WebSocket server...")
cancel_scope.cancel()
return
async with (
host.run(listen_addrs=[listen_addr]),
trio.open_nursery() as (nursery),
):
# Start the peer-store cleanup task
nursery.start_soon(host.get_peerstore().start_cleanup_task, 60)
# Start the signal handler
nursery.start_soon(signal_handler)
# Get the actual address and replace 0.0.0.0 with 127.0.0.1 for client
# connections
addrs = host.get_addrs()
logger.debug(f"Host addresses: {addrs}")
if not addrs:
print("❌ Error: No addresses found for the host")
print("Debug: host.get_addrs() returned empty list")
return
server_addr = str(addrs[0])
client_addr = server_addr.replace("/ip4/0.0.0.0/", "/ip4/127.0.0.1/")
print("🌐 WebSocket Server Started Successfully!")
print("=" * 50)
print(f"📍 Server Address: {client_addr}")
print("🔧 Protocol: /echo/1.0.0")
print("🚀 Transport: WebSocket (/ws)")
print()
print("📋 To test the connection, run this in another terminal:")
plaintext_flag = " --plaintext" if use_plaintext else ""
print(f" python websocket_demo.py -d {client_addr}{plaintext_flag}")
print()
print("⏳ Waiting for incoming WebSocket connections...")
print("" * 50)
# Add a custom handler to show connection events
async def custom_echo_handler(stream):
peer_id = stream.muxed_conn.peer_id
print("\n🔗 New WebSocket Connection!")
print(f" Peer ID: {peer_id}")
print(" Protocol: /echo/1.0.0")
# Show remote address in multiaddr format
try:
remote_address = stream.get_remote_address()
if remote_address:
print(f" Remote: {remote_address}")
except Exception:
print(" Remote: Unknown")
print("" * 40)
# Call the original handler
await echo_handler(stream)
print("" * 40)
print(f"✅ Echo request completed for peer: {peer_id}")
print()
# Replace the handler with our custom one
host.set_stream_handler(ECHO_PROTOCOL_ID, custom_echo_handler)
# Wait indefinitely or until cancelled
with cancel_scope:
await trio.sleep_forever()
except Exception as e:
print(f"❌ Error creating WebSocket server: {e}")
traceback.print_exc()
return
else:
# Create second host (dialer) with WebSocket transport
listen_addr = multiaddr.Multiaddr(f"/ip4/{localhost_ip}/tcp/{port}/ws")
try:
# Create a single host for client operations
host = create_websocket_host(use_plaintext=use_plaintext)
# Start the host for client operations
async with (
host.run(listen_addrs=[listen_addr]),
trio.open_nursery() as (nursery),
):
# Start the peer-store cleanup task
nursery.start_soon(host.get_peerstore().start_cleanup_task, 60)
# Add connection event handlers for debugging
class ClientDebugNotifee(INotifee):
async def opened_stream(self, network, stream):
pass
async def closed_stream(self, network, stream):
pass
async def connected(self, network, conn):
print(
f"🔗 Client: libp2p connection established: "
f"{conn.muxed_conn.peer_id}"
)
async def disconnected(self, network, conn):
print(
f"🔌 Client: libp2p connection closed: "
f"{conn.muxed_conn.peer_id}"
)
async def listen(self, network, multiaddr):
pass
async def listen_close(self, network, multiaddr):
pass
host.get_network().register_notifee(ClientDebugNotifee())
maddr = multiaddr.Multiaddr(destination)
info = info_from_p2p_addr(maddr)
print("🔌 WebSocket Client Starting...")
print("=" * 40)
print(f"🎯 Target Peer: {info.peer_id}")
print(f"📍 Target Address: {destination}")
print()
try:
print("🔗 Connecting to WebSocket server...")
print(f" Security: {'Plaintext' if use_plaintext else 'Noise'}")
await host.connect(info)
print("✅ Successfully connected to WebSocket server!")
except Exception as e:
error_msg = str(e)
print("\n❌ Connection Failed!")
print(f" Peer ID: {info.peer_id}")
print(f" Address: {destination}")
print(f" Security: {'Plaintext' if use_plaintext else 'Noise'}")
print(f" Error: {error_msg}")
print(f" Error type: {type(e).__name__}")
# Add more detailed error information for debugging
if hasattr(e, "__cause__") and e.__cause__:
print(f" Root cause: {e.__cause__}")
print(f" Root cause type: {type(e.__cause__).__name__}")
print()
print("💡 Troubleshooting:")
print(" • Make sure the WebSocket server is running")
print(" • Check that the server address is correct")
print(" • Verify the server is listening on the right port")
print(
" • Ensure both client and server use the same sec protocol"
)
if not use_plaintext:
print(" • Noise over WebSocket may have compatibility issues")
return
# Create a stream and send test data
try:
stream = await host.new_stream(info.peer_id, [ECHO_PROTOCOL_ID])
except Exception as e:
print(f"❌ Failed to create stream: {e}")
return
try:
print("🚀 Starting Echo Protocol Test...")
print("" * 40)
# Send test data
test_message = b"Hello WebSocket Transport!"
print(f"📤 Sending message: {test_message.decode('utf-8')}")
await stream.write(test_message)
# Read response
print("⏳ Waiting for server response...")
response = await stream.read(1024)
print(f"📥 Received response: {response.decode('utf-8')}")
await stream.close()
print("" * 40)
if response == test_message:
print("🎉 Echo test successful!")
print("✅ WebSocket transport is working perfectly!")
print("✅ Client completed successfully, exiting.")
else:
print("❌ Echo test failed!")
print(" Response doesn't match sent data.")
print(f" Sent: {test_message}")
print(f" Received: {response}")
except Exception as e:
error_msg = str(e)
print(f"Echo protocol error: {error_msg}")
traceback.print_exc()
finally:
# Ensure stream is closed
try:
if stream:
# Check if stream has is_closed method and use it
has_is_closed = hasattr(stream, "is_closed") and callable(
getattr(stream, "is_closed")
)
if has_is_closed:
# type: ignore[attr-defined]
if not await stream.is_closed():
await stream.close()
else:
# Fallback: just try to close the stream
await stream.close()
except Exception:
pass
# host.run() context manager handles cleanup automatically
print()
print("🎉 WebSocket Demo Completed Successfully!")
print("=" * 50)
print("✅ WebSocket transport is working perfectly!")
print("✅ Echo protocol communication successful!")
print("✅ libp2p integration verified!")
print()
print("🚀 Your WebSocket transport is ready for production use!")
# Add a small delay to ensure all cleanup is complete
await trio.sleep(0.1)
except Exception as e:
print(f"❌ Error creating WebSocket client: {e}")
traceback.print_exc()
return
def main() -> None:
description = """
This program demonstrates the libp2p WebSocket transport.
First run
'python websocket_demo.py -p <PORT> [--plaintext]' to start a WebSocket server.
Then run
'python websocket_demo.py <ANOTHER_PORT> -d <DESTINATION> [--plaintext]'
where <DESTINATION> is the multiaddress shown by the server.
By default, this example uses Noise encryption for secure communication.
Use --plaintext for testing with unencrypted communication
(not recommended for production).
"""
example_maddr = (
"/ip4/127.0.0.1/tcp/8888/ws/p2p/QmQn4SwGkDZKkUEpBRBvTmheQycxAHJUNmVEnjA2v1qe8Q"
)
parser = argparse.ArgumentParser(description=description)
parser.add_argument("-p", "--port", default=0, type=int, help="source port number")
parser.add_argument(
"-d",
"--destination",
type=str,
help=f"destination multiaddr string, e.g. {example_maddr}",
)
parser.add_argument(
"--plaintext",
action="store_true",
help=(
"use plaintext security instead of Noise encryption "
"(not recommended for production)"
),
)
args = parser.parse_args()
# Determine security mode: use Noise by default,
# plaintext if --plaintext is specified
use_plaintext = args.plaintext
try:
trio.run(run, args.port, args.destination, use_plaintext)
except KeyboardInterrupt:
# This is expected when Ctrl+C is pressed
# The signal handler already printed the shutdown message
print("✅ Clean exit completed.")
return
except Exception as e:
print(f"❌ Unexpected error: {e}")
return
if __name__ == "__main__":
main()

View File

@ -1,3 +1,12 @@
"""Libp2p Python implementation."""
import logging
import ssl
from libp2p.transport.quic.utils import is_quic_multiaddr
from typing import Any
from libp2p.transport.quic.transport import QUICTransport
from libp2p.transport.quic.config import QUICTransportConfig
from collections.abc import (
Mapping,
Sequence,
@ -6,19 +15,17 @@ from importlib.metadata import version as __version
from typing import (
Literal,
Optional,
Type,
cast,
)
import multiaddr
from libp2p.abc import (
IHost,
IMuxedConn,
INetworkService,
IPeerRouting,
IPeerStore,
ISecureTransport,
ITransport,
)
from libp2p.crypto.keys import (
KeyPair,
@ -41,33 +48,44 @@ from libp2p.host.routed_host import (
from libp2p.network.swarm import (
Swarm,
)
from libp2p.network.config import (
ConnectionConfig,
RetryConfig
)
from libp2p.peer.id import (
ID,
)
from libp2p.peer.peerstore import (
PeerStore,
create_signed_peer_record,
)
from libp2p.security.insecure.transport import (
PLAINTEXT_PROTOCOL_ID,
InsecureTransport,
)
from libp2p.security.noise.transport import PROTOCOL_ID as NOISE_PROTOCOL_ID
from libp2p.security.noise.transport import Transport as NoiseTransport
from libp2p.security.noise.transport import (
PROTOCOL_ID as NOISE_PROTOCOL_ID,
Transport as NoiseTransport,
)
import libp2p.security.secio.transport as secio
from libp2p.stream_muxer.mplex.mplex import (
MPLEX_PROTOCOL_ID,
Mplex,
)
from libp2p.stream_muxer.yamux.yamux import (
PROTOCOL_ID as YAMUX_PROTOCOL_ID,
Yamux,
)
from libp2p.stream_muxer.yamux.yamux import PROTOCOL_ID as YAMUX_PROTOCOL_ID
from libp2p.transport.tcp.tcp import (
TCP,
)
from libp2p.transport.upgrader import (
TransportUpgrader,
)
from libp2p.transport.transport_registry import (
create_transport_for_multiaddr,
get_supported_transport_protocols,
)
from libp2p.utils.logging import (
setup_logging,
)
@ -81,7 +99,9 @@ DEFAULT_MUXER = "YAMUX"
# Multiplexer options
MUXER_YAMUX = "YAMUX"
MUXER_MPLEX = "MPLEX"
DEFAULT_NEGOTIATE_TIMEOUT = 5
logger = logging.getLogger(__name__)
def set_default_muxer(muxer_name: Literal["YAMUX", "MPLEX"]) -> None:
"""
@ -150,15 +170,20 @@ def get_default_muxer_options() -> TMuxerOptions:
else: # YAMUX is default
return create_yamux_muxer_option()
def new_swarm(
key_pair: Optional[KeyPair] = None,
muxer_opt: Optional[TMuxerOptions] = None,
sec_opt: Optional[TSecurityOptions] = None,
peerstore_opt: Optional[IPeerStore] = None,
muxer_preference: Optional[Literal["YAMUX", "MPLEX"]] = None,
listen_addrs: Optional[Sequence[multiaddr.Multiaddr]] = None,
key_pair: KeyPair | None = None,
muxer_opt: TMuxerOptions | None = None,
sec_opt: TSecurityOptions | None = None,
peerstore_opt: IPeerStore | None = None,
muxer_preference: Literal["YAMUX", "MPLEX"] | None = None,
listen_addrs: Sequence[multiaddr.Multiaddr] | None = None,
enable_quic: bool = False,
retry_config: Optional["RetryConfig"] = None,
connection_config: ConnectionConfig | QUICTransportConfig | None = None,
tls_client_config: ssl.SSLContext | None = None,
tls_server_config: ssl.SSLContext | None = None,
) -> INetworkService:
logger.debug(f"new_swarm: enable_quic={enable_quic}, listen_addrs={listen_addrs}")
"""
Create a swarm instance based on the parameters.
@ -168,6 +193,8 @@ def new_swarm(
:param peerstore_opt: optional peerstore
:param muxer_preference: optional explicit muxer preference
:param listen_addrs: optional list of multiaddrs to listen on
:param enable_quic: enable quic for transport
:param quic_transport_opt: options for transport
:return: return a default swarm instance
Note: Yamux (/yamux/1.0.0) is the preferred stream multiplexer
@ -180,16 +207,48 @@ def new_swarm(
id_opt = generate_peer_id_from(key_pair)
transport: TCP | QUICTransport | ITransport
quic_transport_opt = connection_config if isinstance(connection_config, QUICTransportConfig) else None
if listen_addrs is None:
transport = TCP()
else:
addr = listen_addrs[0]
if addr.__contains__("tcp"):
transport = TCP()
elif addr.__contains__("quic"):
raise ValueError("QUIC not yet supported")
if enable_quic:
transport = QUICTransport(key_pair.private_key, config=quic_transport_opt)
else:
raise ValueError(f"Unknown transport in listen_addrs: {listen_addrs}")
transport = TCP()
else:
# Use transport registry to select the appropriate transport
from libp2p.transport.transport_registry import create_transport_for_multiaddr
# Create a temporary upgrader for transport selection
# We'll create the real upgrader later with the proper configuration
temp_upgrader = TransportUpgrader(
secure_transports_by_protocol={},
muxer_transports_by_protocol={}
)
addr = listen_addrs[0]
logger.debug(f"new_swarm: Creating transport for address: {addr}")
transport_maybe = create_transport_for_multiaddr(
addr,
temp_upgrader,
private_key=key_pair.private_key,
config=quic_transport_opt,
tls_client_config=tls_client_config,
tls_server_config=tls_server_config
)
if transport_maybe is None:
raise ValueError(f"Unsupported transport for listen_addrs: {listen_addrs}")
transport = transport_maybe
logger.debug(f"new_swarm: Created transport: {type(transport)}")
# If enable_quic is True but we didn't get a QUIC transport, force QUIC
if enable_quic and not isinstance(transport, QUICTransport):
logger.debug(f"new_swarm: Forcing QUIC transport (enable_quic=True but got {type(transport)})")
transport = QUICTransport(key_pair.private_key, config=quic_transport_opt)
logger.debug(f"new_swarm: Final transport type: {type(transport)}")
# Generate X25519 keypair for Noise
noise_key_pair = create_new_x25519_key_pair()
@ -200,7 +259,9 @@ def new_swarm(
key_pair, noise_privkey=noise_key_pair.private_key
),
TProtocol(secio.ID): secio.Transport(key_pair),
TProtocol(PLAINTEXT_PROTOCOL_ID): InsecureTransport(key_pair),
TProtocol(PLAINTEXT_PROTOCOL_ID): InsecureTransport(
key_pair, peerstore=peerstore_opt
),
}
# Use given muxer preference if provided, otherwise use global default
@ -228,21 +289,36 @@ def new_swarm(
muxer_transports_by_protocol=muxer_transports_by_protocol,
)
peerstore = peerstore_opt or PeerStore()
# Store our key pair in peerstore
peerstore.add_key_pair(id_opt, key_pair)
return Swarm(id_opt, peerstore, upgrader, transport)
return Swarm(
id_opt,
peerstore,
upgrader,
transport,
retry_config=retry_config,
connection_config=connection_config
)
def new_host(
key_pair: Optional[KeyPair] = None,
muxer_opt: Optional[TMuxerOptions] = None,
sec_opt: Optional[TSecurityOptions] = None,
peerstore_opt: Optional[IPeerStore] = None,
disc_opt: Optional[IPeerRouting] = None,
muxer_preference: Optional[Literal["YAMUX", "MPLEX"]] = None,
listen_addrs: Sequence[multiaddr.Multiaddr] = None,
key_pair: KeyPair | None = None,
muxer_opt: TMuxerOptions | None = None,
sec_opt: TSecurityOptions | None = None,
peerstore_opt: IPeerStore | None = None,
disc_opt: IPeerRouting | None = None,
muxer_preference: Literal["YAMUX", "MPLEX"] | None = None,
listen_addrs: Sequence[multiaddr.Multiaddr] | None = None,
enable_mDNS: bool = False,
bootstrap: list[str] | None = None,
negotiate_timeout: int = DEFAULT_NEGOTIATE_TIMEOUT,
enable_quic: bool = False,
quic_transport_opt: QUICTransportConfig | None = None,
tls_client_config: ssl.SSLContext | None = None,
tls_server_config: ssl.SSLContext | None = None,
) -> IHost:
"""
Create a new libp2p host based on the given parameters.
@ -254,20 +330,39 @@ def new_host(
:param disc_opt: optional discovery
:param muxer_preference: optional explicit muxer preference
:param listen_addrs: optional list of multiaddrs to listen on
:param enable_mDNS: whether to enable mDNS discovery
:param bootstrap: optional list of bootstrap peer addresses as strings
:param enable_quic: optinal choice to use QUIC for transport
:param quic_transport_opt: optional configuration for quic transport
:param tls_client_config: optional TLS client configuration for WebSocket transport
:param tls_server_config: optional TLS server configuration for WebSocket transport
:return: return a host instance
"""
if not enable_quic and quic_transport_opt is not None:
logger.warning(f"QUIC config provided but QUIC not enabled, ignoring QUIC config")
swarm = new_swarm(
enable_quic=enable_quic,
key_pair=key_pair,
muxer_opt=muxer_opt,
sec_opt=sec_opt,
peerstore_opt=peerstore_opt,
muxer_preference=muxer_preference,
listen_addrs=listen_addrs,
connection_config=quic_transport_opt if enable_quic else None,
tls_client_config=tls_client_config,
tls_server_config=tls_server_config
)
if disc_opt is not None:
return RoutedHost(swarm, disc_opt)
return BasicHost(swarm)
return RoutedHost(swarm, disc_opt, enable_mDNS, bootstrap)
return BasicHost(
network=swarm,
enable_mDNS=enable_mDNS,
bootstrap=bootstrap,
negotitate_timeout=negotiate_timeout
)
__version__ = __version("libp2p")

File diff suppressed because it is too large Load Diff

View File

@ -116,15 +116,15 @@ def initialize_pair(
EncryptionParameters(
cipher_type,
hash_type,
first_half[0:iv_size],
first_half[iv_size + cipher_key_size :],
first_half[iv_size : iv_size + cipher_key_size],
bytes(first_half[0:iv_size]),
bytes(first_half[iv_size + cipher_key_size :]),
bytes(first_half[iv_size : iv_size + cipher_key_size]),
),
EncryptionParameters(
cipher_type,
hash_type,
second_half[0:iv_size],
second_half[iv_size + cipher_key_size :],
second_half[iv_size : iv_size + cipher_key_size],
bytes(second_half[0:iv_size]),
bytes(second_half[iv_size + cipher_key_size :]),
bytes(second_half[iv_size : iv_size + cipher_key_size]),
),
)

View File

@ -9,29 +9,40 @@ from libp2p.crypto.keys import (
if sys.platform != "win32":
from fastecdsa import (
curve as curve_types,
keys,
point,
)
from fastecdsa import curve as curve_types
from fastecdsa.encoding.sec1 import (
SEC1Encoder,
)
else:
from coincurve import PrivateKey as CPrivateKey
from coincurve import PublicKey as CPublicKey
from coincurve import (
PrivateKey as CPrivateKey,
PublicKey as CPublicKey,
)
def infer_local_type(curve: str) -> object:
"""
Convert a str representation of some elliptic curve to a
representation understood by the backend of this module.
"""
if curve != "P-256":
raise NotImplementedError("Only P-256 curve is supported")
if sys.platform != "win32":
if sys.platform != "win32":
def infer_local_type(curve: str) -> curve_types.Curve:
"""
Convert a str representation of some elliptic curve to a
representation understood by the backend of this module.
"""
if curve != "P-256":
raise NotImplementedError("Only P-256 curve is supported")
return curve_types.P256
return "P-256" # coincurve only supports P-256
else:
def infer_local_type(curve: str) -> str:
"""
Convert a str representation of some elliptic curve to a
representation understood by the backend of this module.
"""
if curve != "P-256":
raise NotImplementedError("Only P-256 curve is supported")
return "P-256" # coincurve only supports P-256
if sys.platform != "win32":
@ -68,7 +79,10 @@ if sys.platform != "win32":
return cls(private_key_impl, curve_type)
def to_bytes(self) -> bytes:
return keys.export_key(self.impl, self.curve)
key_str = keys.export_key(self.impl, self.curve)
if key_str is None:
raise Exception("Key not found")
return key_str.encode()
def get_type(self) -> KeyType:
return KeyType.ECC_P256

View File

@ -4,8 +4,10 @@ from Crypto.Hash import (
from nacl.exceptions import (
BadSignatureError,
)
from nacl.public import PrivateKey as PrivateKeyImpl
from nacl.public import PublicKey as PublicKeyImpl
from nacl.public import (
PrivateKey as PrivateKeyImpl,
PublicKey as PublicKeyImpl,
)
from nacl.signing import (
SigningKey,
VerifyKey,
@ -48,7 +50,7 @@ class Ed25519PrivateKey(PrivateKey):
self.impl = impl
@classmethod
def new(cls, seed: bytes = None) -> "Ed25519PrivateKey":
def new(cls, seed: bytes | None = None) -> "Ed25519PrivateKey":
if not seed:
seed = utils.random()
@ -75,7 +77,7 @@ class Ed25519PrivateKey(PrivateKey):
return Ed25519PublicKey(self.impl.public_key)
def create_new_key_pair(seed: bytes = None) -> KeyPair:
def create_new_key_pair(seed: bytes | None = None) -> KeyPair:
private_key = Ed25519PrivateKey.new(seed)
public_key = private_key.get_public_key()
return KeyPair(private_key, public_key)

View File

@ -1,6 +1,6 @@
from collections.abc import Callable
import sys
from typing import (
Callable,
cast,
)

View File

@ -81,12 +81,10 @@ class PrivateKey(Key):
"""A ``PrivateKey`` represents a cryptographic private key."""
@abstractmethod
def sign(self, data: bytes) -> bytes:
...
def sign(self, data: bytes) -> bytes: ...
@abstractmethod
def get_public_key(self) -> PublicKey:
...
def get_public_key(self) -> PublicKey: ...
def _serialize_to_protobuf(self) -> crypto_pb2.PrivateKey:
"""Return the protobuf representation of this ``Key``."""

View File

@ -13,7 +13,7 @@ _sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x1dlibp2p/crypto/pb/crypto.proto\x12\tcrypto.pb\"?\n\tPublicKey\x12$\n\x08key_type\x18\x01 \x02(\x0e\x32\x12.crypto.pb.KeyType\x12\x0c\n\x04\x64\x61ta\x18\x02 \x02(\x0c\"@\n\nPrivateKey\x12$\n\x08key_type\x18\x01 \x02(\x0e\x32\x12.crypto.pb.KeyType\x12\x0c\n\x04\x64\x61ta\x18\x02 \x02(\x0c*G\n\x07KeyType\x12\x07\n\x03RSA\x10\x00\x12\x0b\n\x07\x45\x64\x32\x35\x35\x31\x39\x10\x01\x12\r\n\tSecp256k1\x10\x02\x12\t\n\x05\x45\x43\x44SA\x10\x03\x12\x0c\n\x08\x45\x43\x43_P256\x10\x04')
DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x1dlibp2p/crypto/pb/crypto.proto\x12\tcrypto.pb\"?\n\tPublicKey\x12$\n\x08key_type\x18\x01 \x02(\x0e\x32\x12.crypto.pb.KeyType\x12\x0c\n\x04\x64\x61ta\x18\x02 \x02(\x0c\"@\n\nPrivateKey\x12$\n\x08key_type\x18\x01 \x02(\x0e\x32\x12.crypto.pb.KeyType\x12\x0c\n\x04\x64\x61ta\x18\x02 \x02(\x0c*S\n\x07KeyType\x12\x07\n\x03RSA\x10\x00\x12\x0b\n\x07\x45\x64\x32\x35\x35\x31\x39\x10\x01\x12\r\n\tSecp256k1\x10\x02\x12\t\n\x05\x45\x43\x44SA\x10\x03\x12\x0c\n\x08\x45\x43\x43_P256\x10\x04\x12\n\n\x06X25519\x10\x05')
_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, globals())
_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'libp2p.crypto.pb.crypto_pb2', globals())
@ -21,7 +21,7 @@ if _descriptor._USE_C_DESCRIPTORS == False:
DESCRIPTOR._options = None
_KEYTYPE._serialized_start=175
_KEYTYPE._serialized_end=246
_KEYTYPE._serialized_end=258
_PUBLICKEY._serialized_start=44
_PUBLICKEY._serialized_end=107
_PRIVATEKEY._serialized_start=109

View File

@ -28,6 +28,7 @@ class _KeyTypeEnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTy
Secp256k1: _KeyType.ValueType # 2
ECDSA: _KeyType.ValueType # 3
ECC_P256: _KeyType.ValueType # 4
X25519: _KeyType.ValueType # 5
class KeyType(_KeyType, metaclass=_KeyTypeEnumTypeWrapper): ...
@ -36,6 +37,7 @@ Ed25519: KeyType.ValueType # 1
Secp256k1: KeyType.ValueType # 2
ECDSA: KeyType.ValueType # 3
ECC_P256: KeyType.ValueType # 4
X25519: KeyType.ValueType # 5
global___KeyType = KeyType
@typing.final

View File

@ -37,7 +37,7 @@ class Secp256k1PrivateKey(PrivateKey):
self.impl = impl
@classmethod
def new(cls, secret: bytes = None) -> "Secp256k1PrivateKey":
def new(cls, secret: bytes | None = None) -> "Secp256k1PrivateKey":
private_key_impl = coincurve.PrivateKey(secret)
return cls(private_key_impl)
@ -65,7 +65,7 @@ class Secp256k1PrivateKey(PrivateKey):
return Secp256k1PublicKey(public_key_impl)
def create_new_key_pair(secret: bytes = None) -> KeyPair:
def create_new_key_pair(secret: bytes | None = None) -> KeyPair:
"""
Returns a new Secp256k1 keypair derived from the provided ``secret``, a
sequence of bytes corresponding to some integer between 0 and the group

View File

@ -1,31 +1,21 @@
from collections.abc import (
Awaitable,
Callable,
Mapping,
)
from typing import (
TYPE_CHECKING,
Callable,
NewType,
Union,
)
from typing import TYPE_CHECKING, NewType, Union, cast
from libp2p.transport.quic.stream import QUICStream
if TYPE_CHECKING:
from libp2p.abc import (
IMuxedConn,
INetStream,
ISecureTransport,
)
from libp2p.abc import IMuxedConn, IMuxedStream, INetStream, ISecureTransport
from libp2p.transport.quic.connection import QUICConnection
else:
class INetStream:
pass
class IMuxedConn:
pass
class ISecureTransport:
pass
IMuxedConn = cast(type, object)
INetStream = cast(type, object)
ISecureTransport = cast(type, object)
IMuxedStream = cast(type, object)
QUICConnection = cast(type, object)
from libp2p.io.abc import (
ReadWriteCloser,
@ -38,12 +28,15 @@ from libp2p.pubsub.pb import (
)
TProtocol = NewType("TProtocol", str)
StreamHandlerFn = Callable[["INetStream"], Awaitable[None]]
StreamHandlerFn = Callable[[INetStream], Awaitable[None]]
THandler = Callable[[ReadWriteCloser], Awaitable[None]]
TSecurityOptions = Mapping[TProtocol, "ISecureTransport"]
TMuxerClass = type["IMuxedConn"]
TSecurityOptions = Mapping[TProtocol, ISecureTransport]
TMuxerClass = type[IMuxedConn]
TMuxerOptions = Mapping[TProtocol, TMuxerClass]
SyncValidatorFn = Callable[[ID, rpc_pb2.Message], bool]
AsyncValidatorFn = Callable[[ID, rpc_pb2.Message], Awaitable[bool]]
ValidatorFn = Union[SyncValidatorFn, AsyncValidatorFn]
UnsubscribeFn = Callable[[], Awaitable[None]]
TQUICStreamHandlerFn = Callable[[QUICStream], Awaitable[None]]
TQUICConnHandlerFn = Callable[[QUICConnection], Awaitable[None]]
MessageID = NewType("MessageID", str)

View File

View File

@ -0,0 +1,5 @@
"""Bootstrap peer discovery module for py-libp2p."""
from .bootstrap import BootstrapDiscovery
__all__ = ["BootstrapDiscovery"]

View File

@ -0,0 +1,312 @@
import logging
from multiaddr import Multiaddr
from multiaddr.resolvers import DNSResolver
import trio
from libp2p.abc import ID, INetworkService, PeerInfo
from libp2p.discovery.bootstrap.utils import validate_bootstrap_addresses
from libp2p.discovery.events.peerDiscovery import peerDiscovery
from libp2p.network.exceptions import SwarmException
from libp2p.peer.peerinfo import info_from_p2p_addr
from libp2p.peer.peerstore import PERMANENT_ADDR_TTL
logger = logging.getLogger("libp2p.discovery.bootstrap")
resolver = DNSResolver()
DEFAULT_CONNECTION_TIMEOUT = 10
class BootstrapDiscovery:
"""
Bootstrap-based peer discovery for py-libp2p.
Connects to predefined bootstrap peers and adds them to peerstore.
"""
def __init__(self, swarm: INetworkService, bootstrap_addrs: list[str]):
"""
Initialize BootstrapDiscovery.
Args:
swarm: The network service (swarm) instance
bootstrap_addrs: List of bootstrap peer multiaddresses
"""
self.swarm = swarm
self.peerstore = swarm.peerstore
self.bootstrap_addrs = bootstrap_addrs or []
self.discovered_peers: set[str] = set()
self.connection_timeout: int = DEFAULT_CONNECTION_TIMEOUT
async def start(self) -> None:
"""Process bootstrap addresses and emit peer discovery events in parallel."""
logger.info(
f"Starting bootstrap discovery with "
f"{len(self.bootstrap_addrs)} bootstrap addresses"
)
# Show all bootstrap addresses being processed
for i, addr in enumerate(self.bootstrap_addrs):
logger.debug(f"{i + 1}. {addr}")
# Validate and filter bootstrap addresses
self.bootstrap_addrs = validate_bootstrap_addresses(self.bootstrap_addrs)
logger.info(f"Valid addresses after validation: {len(self.bootstrap_addrs)}")
# Use Trio nursery for PARALLEL address processing
try:
async with trio.open_nursery() as nursery:
logger.debug(
f"Starting {len(self.bootstrap_addrs)} parallel address "
f"processing tasks"
)
# Start all bootstrap address processing tasks in parallel
for addr_str in self.bootstrap_addrs:
logger.debug(f"Starting parallel task for: {addr_str}")
nursery.start_soon(self._process_bootstrap_addr, addr_str)
# The nursery will wait for all address processing tasks to complete
logger.debug(
"Nursery active - waiting for address processing tasks to complete"
)
except trio.Cancelled:
logger.debug("Bootstrap address processing cancelled - cleaning up tasks")
raise
except Exception as e:
logger.error(f"Bootstrap address processing failed: {e}")
raise
logger.info("Bootstrap discovery startup complete - all tasks finished")
def stop(self) -> None:
"""Clean up bootstrap discovery resources."""
logger.info("Stopping bootstrap discovery and cleaning up tasks")
# Clear discovered peers
self.discovered_peers.clear()
logger.debug("Bootstrap discovery cleanup completed")
async def _process_bootstrap_addr(self, addr_str: str) -> None:
"""Convert string address to PeerInfo and add to peerstore."""
try:
try:
multiaddr = Multiaddr(addr_str)
except Exception as e:
logger.debug(f"Invalid multiaddr format '{addr_str}': {e}")
return
if self.is_dns_addr(multiaddr):
resolved_addrs = await resolver.resolve(multiaddr)
if resolved_addrs is None:
logger.warning(f"DNS resolution returned None for: {addr_str}")
return
peer_id_str = multiaddr.get_peer_id()
if peer_id_str is None:
logger.warning(f"Missing peer ID in DNS address: {addr_str}")
return
peer_id = ID.from_base58(peer_id_str)
addrs = [addr for addr in resolved_addrs]
if not addrs:
logger.warning(f"No addresses resolved for DNS address: {addr_str}")
return
peer_info = PeerInfo(peer_id, addrs)
await self.add_addr(peer_info)
else:
peer_info = info_from_p2p_addr(multiaddr)
await self.add_addr(peer_info)
except Exception as e:
logger.warning(f"Failed to process bootstrap address {addr_str}: {e}")
def is_dns_addr(self, addr: Multiaddr) -> bool:
"""Check if the address is a DNS address."""
return any(protocol.name == "dnsaddr" for protocol in addr.protocols())
async def add_addr(self, peer_info: PeerInfo) -> None:
"""
Add a peer to the peerstore, emit discovery event,
and attempt connection in parallel.
"""
logger.debug(
f"Adding peer {peer_info.peer_id} with {len(peer_info.addrs)} addresses"
)
# Skip if it's our own peer
if peer_info.peer_id == self.swarm.get_peer_id():
logger.debug(f"Skipping own peer ID: {peer_info.peer_id}")
return
# Filter addresses to only include IPv4+TCP (only supported protocol)
ipv4_tcp_addrs = []
filtered_out_addrs = []
for addr in peer_info.addrs:
if self._is_ipv4_tcp_addr(addr):
ipv4_tcp_addrs.append(addr)
else:
filtered_out_addrs.append(addr)
# Log filtering results
logger.debug(
f"Address filtering for {peer_info.peer_id}: "
f"{len(ipv4_tcp_addrs)} IPv4+TCP, {len(filtered_out_addrs)} filtered"
)
# Skip peer if no IPv4+TCP addresses available
if not ipv4_tcp_addrs:
logger.warning(
f"❌ No IPv4+TCP addresses for {peer_info.peer_id} - "
f"skipping connection attempts"
)
return
# Add only IPv4+TCP addresses to peerstore
self.peerstore.add_addrs(peer_info.peer_id, ipv4_tcp_addrs, PERMANENT_ADDR_TTL)
# Only emit discovery event if this is the first time we see this peer
peer_id_str = str(peer_info.peer_id)
if peer_id_str not in self.discovered_peers:
# Track discovered peer
self.discovered_peers.add(peer_id_str)
# Emit peer discovery event
peerDiscovery.emit_peer_discovered(peer_info)
logger.info(f"Peer discovered: {peer_info.peer_id}")
# Connect to peer (parallel across different bootstrap addresses)
logger.debug("Connecting to discovered peer...")
await self._connect_to_peer(peer_info.peer_id)
else:
logger.debug(
f"Additional addresses added for existing peer: {peer_info.peer_id}"
)
# Even for existing peers, try to connect if not already connected
if peer_info.peer_id not in self.swarm.connections:
logger.debug("Connecting to existing peer...")
await self._connect_to_peer(peer_info.peer_id)
async def _connect_to_peer(self, peer_id: ID) -> None:
"""
Attempt to establish a connection to a peer with timeout.
Uses swarm.dial_peer to connect using addresses stored in peerstore.
Times out after self.connection_timeout seconds to prevent hanging.
"""
logger.debug(f"Connection attempt for peer: {peer_id}")
# Pre-connection validation: Check if already connected
if peer_id in self.swarm.connections:
logger.debug(
f"Already connected to {peer_id} - skipping connection attempt"
)
return
# Check available addresses before attempting connection
available_addrs = self.peerstore.addrs(peer_id)
logger.debug(f"Connecting to {peer_id} ({len(available_addrs)} addresses)")
if not available_addrs:
logger.error(f"❌ No addresses available for {peer_id} - cannot connect")
return
# Record start time for connection attempt monitoring
connection_start_time = trio.current_time()
try:
with trio.move_on_after(self.connection_timeout):
# Log connection attempt
logger.debug(
f"Attempting connection to {peer_id} using "
f"{len(available_addrs)} addresses"
)
# Use swarm.dial_peer to connect using stored addresses
await self.swarm.dial_peer(peer_id)
# Calculate connection time
connection_time = trio.current_time() - connection_start_time
# Post-connection validation: Verify connection was actually established
if peer_id in self.swarm.connections:
logger.info(
f"✅ Connected to {peer_id} (took {connection_time:.2f}s)"
)
else:
logger.warning(
f"Dial succeeded but connection not found for {peer_id}"
)
except trio.TooSlowError:
logger.warning(
f"❌ Connection to {peer_id} timed out after {self.connection_timeout}s"
)
except SwarmException as e:
# Calculate failed connection time
failed_connection_time = trio.current_time() - connection_start_time
# Enhanced error logging
error_msg = str(e)
if "no addresses established a successful connection" in error_msg:
logger.warning(
f"❌ Failed to connect to {peer_id} after trying all "
f"{len(available_addrs)} addresses "
f"(took {failed_connection_time:.2f}s)"
)
# Log individual address failures if this is a MultiError
if (
e.__cause__ is not None
and hasattr(e.__cause__, "exceptions")
and getattr(e.__cause__, "exceptions", None) is not None
):
exceptions_list = getattr(e.__cause__, "exceptions")
logger.debug("📋 Individual address failure details:")
for i, addr_exception in enumerate(exceptions_list, 1):
logger.debug(f"Address {i}: {addr_exception}")
# Also log the actual address that failed
if i <= len(available_addrs):
logger.debug(f"Failed address: {available_addrs[i - 1]}")
else:
logger.warning("No detailed exception information available")
else:
logger.warning(
f"❌ Failed to connect to {peer_id}: {e} "
f"(took {failed_connection_time:.2f}s)"
)
except Exception as e:
# Handle unexpected errors that aren't swarm-specific
failed_connection_time = trio.current_time() - connection_start_time
logger.error(
f"❌ Unexpected error connecting to {peer_id}: "
f"{e} (took {failed_connection_time:.2f}s)"
)
# Don't re-raise to prevent killing the nursery and other parallel tasks
def _is_ipv4_tcp_addr(self, addr: Multiaddr) -> bool:
"""
Check if address is IPv4 with TCP protocol only.
Filters out IPv6, UDP, QUIC, WebSocket, and other unsupported protocols.
Only IPv4+TCP addresses are supported by the current transport.
"""
try:
protocols = addr.protocols()
# Must have IPv4 protocol
has_ipv4 = any(p.name == "ip4" for p in protocols)
if not has_ipv4:
return False
# Must have TCP protocol
has_tcp = any(p.name == "tcp" for p in protocols)
if not has_tcp:
return False
return True
except Exception:
# If we can't parse the address, don't use it
return False

View File

@ -0,0 +1,51 @@
"""Utility functions for bootstrap discovery."""
import logging
from multiaddr import Multiaddr
from libp2p.peer.peerinfo import InvalidAddrError, PeerInfo, info_from_p2p_addr
logger = logging.getLogger("libp2p.discovery.bootstrap.utils")
def validate_bootstrap_addresses(addrs: list[str]) -> list[str]:
"""
Validate and filter bootstrap addresses.
:param addrs: List of bootstrap address strings
:return: List of valid bootstrap addresses
"""
valid_addrs = []
for addr_str in addrs:
try:
# Try to parse as multiaddr
multiaddr = Multiaddr(addr_str)
# Try to extract peer info (this validates the p2p component)
info_from_p2p_addr(multiaddr)
valid_addrs.append(addr_str)
logger.debug(f"Valid bootstrap address: {addr_str}")
except (InvalidAddrError, ValueError, Exception) as e:
logger.warning(f"Invalid bootstrap address '{addr_str}': {e}")
continue
return valid_addrs
def parse_bootstrap_peer_info(addr_str: str) -> PeerInfo | None:
"""
Parse bootstrap address string into PeerInfo.
:param addr_str: Bootstrap address string
:return: PeerInfo object or None if parsing fails
"""
try:
multiaddr = Multiaddr(addr_str)
return info_from_p2p_addr(multiaddr)
except Exception as e:
logger.error(f"Failed to parse bootstrap address '{addr_str}': {e}")
return None

View File

View File

@ -0,0 +1,26 @@
from collections.abc import (
Callable,
)
from libp2p.abc import (
PeerInfo,
)
TTL: int = 60 * 60 # Time-to-live for discovered peers in seconds
class PeerDiscovery:
def __init__(self) -> None:
self._peer_discovered_handlers: list[Callable[[PeerInfo], None]] = []
def register_peer_discovered_handler(
self, handler: Callable[[PeerInfo], None]
) -> None:
self._peer_discovered_handlers.append(handler)
def emit_peer_discovered(self, peer_info: PeerInfo) -> None:
for handler in self._peer_discovered_handlers:
handler(peer_info)
peerDiscovery = PeerDiscovery()

View File

View File

@ -0,0 +1,91 @@
import logging
import socket
from zeroconf import (
EventLoopBlocked,
ServiceInfo,
Zeroconf,
)
logger = logging.getLogger("libp2p.discovery.mdns.broadcaster")
class PeerBroadcaster:
"""
Broadcasts this peer's presence on the local network using mDNS/zeroconf.
Registers a service with the peer's ID in the TXT record as per libp2p spec.
"""
def __init__(
self,
zeroconf: Zeroconf,
service_type: str,
service_name: str,
peer_id: str,
port: int,
):
self.zeroconf = zeroconf
self.service_type = service_type
self.peer_id = peer_id
self.port = port
self.service_name = service_name
# Get the local IP address
local_ip = self._get_local_ip()
hostname = socket.gethostname()
self.service_info = ServiceInfo(
type_=self.service_type,
name=self.service_name,
port=self.port,
properties={b"id": self.peer_id.encode()},
server=f"{hostname}.local.",
addresses=[socket.inet_aton(local_ip)],
)
def _get_local_ip(self) -> str:
"""Get the local IP address of this machine"""
try:
# Connect to a remote address to determine the local IP
# This doesn't actually send data
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:
s.connect(("8.8.8.8", 80))
local_ip = s.getsockname()[0]
return local_ip
except Exception:
# Fallback to localhost if we can't determine the IP
return "127.0.0.1"
def register(self) -> None:
"""Register the peer's mDNS service on the network."""
try:
self.zeroconf.register_service(self.service_info)
logger.debug(f"mDNS service registered: {self.service_name}")
except EventLoopBlocked as e:
logger.warning(
"EventLoopBlocked while registering mDNS '%s': %s", self.service_name, e
)
except Exception as e:
logger.error(
"Unexpected error during mDNS registration for '%s': %r",
self.service_name,
e,
)
def unregister(self) -> None:
"""Unregister the peer's mDNS service from the network."""
try:
self.zeroconf.unregister_service(self.service_info)
logger.debug(f"mDNS service unregistered: {self.service_name}")
except EventLoopBlocked as e:
logger.warning(
"EventLoopBlocked while unregistering mDNS '%s': %s",
self.service_name,
e,
)
except Exception as e:
logger.error(
"Unexpected error during mDNS unregistration for '%s': %r",
self.service_name,
e,
)

View File

@ -0,0 +1,83 @@
import logging
import socket
from zeroconf import (
ServiceBrowser,
ServiceInfo,
ServiceListener,
Zeroconf,
)
from libp2p.abc import IPeerStore, Multiaddr
from libp2p.discovery.events.peerDiscovery import peerDiscovery
from libp2p.peer.id import ID
from libp2p.peer.peerinfo import PeerInfo
logger = logging.getLogger("libp2p.discovery.mdns.listener")
class PeerListener(ServiceListener):
"""mDNS listener — now a true ServiceListener subclass."""
def __init__(
self,
peerstore: IPeerStore,
zeroconf: Zeroconf,
service_type: str,
service_name: str,
) -> None:
self.peerstore = peerstore
self.zeroconf = zeroconf
self.service_type = service_type
self.service_name = service_name
self.discovered_services: dict[str, ID] = {}
self.browser = ServiceBrowser(self.zeroconf, self.service_type, listener=self)
def add_service(self, zc: Zeroconf, type_: str, name: str) -> None:
if name == self.service_name:
return
logger.debug(f"Adding service: {name}")
info = zc.get_service_info(type_, name, timeout=5000)
if not info:
return
peer_info = self._extract_peer_info(info)
if peer_info:
self.discovered_services[name] = peer_info.peer_id
self.peerstore.add_addrs(peer_info.peer_id, peer_info.addrs, 10)
peerDiscovery.emit_peer_discovered(peer_info)
logger.debug(f"Discovered Peer: {peer_info.peer_id}")
def remove_service(self, zc: Zeroconf, type_: str, name: str) -> None:
if name == self.service_name:
return
logger.debug(f"Removing service: {name}")
peer_id = self.discovered_services.pop(name)
self.peerstore.clear_addrs(peer_id)
logger.debug(f"Removed Peer: {peer_id}")
def update_service(self, zc: Zeroconf, type_: str, name: str) -> None:
info = zc.get_service_info(type_, name, timeout=5000)
if not info:
return
peer_info = self._extract_peer_info(info)
if peer_info:
self.peerstore.clear_addrs(peer_info.peer_id)
self.peerstore.add_addrs(peer_info.peer_id, peer_info.addrs, 10)
logger.debug(f"Updated Peer {peer_info.peer_id}")
def _extract_peer_info(self, info: ServiceInfo) -> PeerInfo | None:
try:
addrs = [
Multiaddr(f"/ip4/{socket.inet_ntoa(addr)}/tcp/{info.port}")
for addr in info.addresses
]
pid_bytes = info.properties.get(b"id")
if not pid_bytes:
return None
pid = ID.from_base58(pid_bytes.decode())
return PeerInfo(peer_id=pid, addrs=addrs)
except Exception:
return None
def stop(self) -> None:
self.browser.cancel()

View File

@ -0,0 +1,73 @@
"""
mDNS-based peer discovery for py-libp2p.
Conforms to https://github.com/libp2p/specs/blob/master/discovery/mdns.md
Uses zeroconf for mDNS broadcast/listen. Async operations use trio.
"""
import logging
from zeroconf import (
Zeroconf,
)
from libp2p.abc import (
INetworkService,
)
from .broadcaster import (
PeerBroadcaster,
)
from .listener import (
PeerListener,
)
from .utils import (
stringGen,
)
logger = logging.getLogger("libp2p.discovery.mdns")
SERVICE_TYPE = "_p2p._udp.local."
MCAST_PORT = 5353
MCAST_ADDR = "224.0.0.251"
class MDNSDiscovery:
"""
mDNS-based peer discovery for py-libp2p, using zeroconf.
Conforms to the libp2p mDNS discovery spec.
"""
def __init__(self, swarm: INetworkService, port: int = 8000):
self.peer_id = str(swarm.get_peer_id())
self.port = port
self.zeroconf = Zeroconf()
self.serviceName = f"{stringGen()}.{SERVICE_TYPE}"
self.peerstore = swarm.peerstore
self.swarm = swarm
self.broadcaster = PeerBroadcaster(
zeroconf=self.zeroconf,
service_type=SERVICE_TYPE,
service_name=self.serviceName,
peer_id=self.peer_id,
port=self.port,
)
self.listener = PeerListener(
zeroconf=self.zeroconf,
peerstore=self.peerstore,
service_type=SERVICE_TYPE,
service_name=self.serviceName,
)
def start(self) -> None:
"""Register this peer and start listening for others."""
logger.debug(
f"Starting mDNS discovery for peer {self.peer_id} on port {self.port}"
)
self.broadcaster.register()
# Listener is started in constructor
def stop(self) -> None:
"""Unregister this peer and clean up zeroconf resources."""
logger.debug("Stopping mDNS discovery")
self.broadcaster.unregister()
self.zeroconf.close()

View File

@ -0,0 +1,11 @@
import random
import string
def stringGen(len: int = 63) -> str:
"""Generate a random string of lowercase letters and digits."""
charset = string.ascii_lowercase + string.digits
result = []
for _ in range(len):
result.append(random.choice(charset))
return "".join(result)

View File

@ -0,0 +1,17 @@
"""Random walk discovery modules for py-libp2p."""
from .rt_refresh_manager import RTRefreshManager
from .random_walk import RandomWalk
from .exceptions import (
RoutingTableRefreshError,
RandomWalkError,
PeerValidationError,
)
__all__ = [
"RTRefreshManager",
"RandomWalk",
"RoutingTableRefreshError",
"RandomWalkError",
"PeerValidationError",
]

View File

@ -0,0 +1,16 @@
from typing import Final
# Timing constants (matching go-libp2p)
PEER_PING_TIMEOUT: Final[float] = 10.0 # seconds
REFRESH_QUERY_TIMEOUT: Final[float] = 60.0 # seconds
REFRESH_INTERVAL: Final[float] = 300.0 # 5 minutes
SUCCESSFUL_OUTBOUND_QUERY_GRACE_PERIOD: Final[float] = 60.0 # 1 minute
# Routing table thresholds
MIN_RT_REFRESH_THRESHOLD: Final[int] = 4 # Minimum peers before triggering refresh
MAX_N_BOOTSTRAPPERS: Final[int] = 2 # Maximum bootstrap peers to try
# Random walk specific
RANDOM_WALK_CONCURRENCY: Final[int] = 3 # Number of concurrent random walks
RANDOM_WALK_ENABLED: Final[bool] = True # Enable automatic random walks
RANDOM_WALK_RT_THRESHOLD: Final[int] = 20 # RT size threshold for peerstore fallback

View File

@ -0,0 +1,19 @@
from libp2p.exceptions import BaseLibp2pError
class RoutingTableRefreshError(BaseLibp2pError):
"""Base exception for routing table refresh operations."""
pass
class RandomWalkError(RoutingTableRefreshError):
"""Exception raised during random walk operations."""
pass
class PeerValidationError(RoutingTableRefreshError):
"""Exception raised when peer validation fails."""
pass

View File

@ -0,0 +1,218 @@
from collections.abc import Awaitable, Callable
import logging
import secrets
import trio
from libp2p.abc import IHost
from libp2p.discovery.random_walk.config import (
RANDOM_WALK_CONCURRENCY,
RANDOM_WALK_RT_THRESHOLD,
REFRESH_QUERY_TIMEOUT,
)
from libp2p.discovery.random_walk.exceptions import RandomWalkError
from libp2p.peer.id import ID
from libp2p.peer.peerinfo import PeerInfo
logger = logging.getLogger("libp2p.discovery.random_walk")
class RandomWalk:
"""
Random Walk implementation for peer discovery in Kademlia DHT.
Generates random peer IDs and performs FIND_NODE queries to discover
new peers and populate the routing table.
"""
def __init__(
self,
host: IHost,
local_peer_id: ID,
query_function: Callable[[bytes], Awaitable[list[ID]]],
):
"""
Initialize Random Walk module.
Args:
host: The libp2p host instance
local_peer_id: Local peer ID
query_function: Function to query for closest peers given target key bytes
"""
self.host = host
self.local_peer_id = local_peer_id
self.query_function = query_function
def generate_random_peer_id(self) -> str:
"""
Generate a completely random peer ID
for random walk queries.
Returns:
Random peer ID as string
"""
# Generate 32 random bytes (256 bits) - same as go-libp2p
random_bytes = secrets.token_bytes(32)
# Convert to hex string for query
return random_bytes.hex()
async def perform_random_walk(self) -> list[PeerInfo]:
"""
Perform a single random walk operation.
Returns:
List of validated peers discovered during the walk
"""
try:
# Generate random peer ID
random_peer_id = self.generate_random_peer_id()
logger.info(f"Starting random walk for peer ID: {random_peer_id}")
# Perform FIND_NODE query
discovered_peer_ids: list[ID] = []
with trio.move_on_after(REFRESH_QUERY_TIMEOUT):
# Call the query function with target key bytes
target_key = bytes.fromhex(random_peer_id)
discovered_peer_ids = await self.query_function(target_key) or []
if not discovered_peer_ids:
logger.debug(f"No peers discovered in random walk for {random_peer_id}")
return []
logger.info(
f"Discovered {len(discovered_peer_ids)} peers in random walk "
f"for {random_peer_id[:8]}..." # Show only first 8 chars for brevity
)
# Convert peer IDs to PeerInfo objects and validate
validated_peers: list[PeerInfo] = []
for peer_id in discovered_peer_ids:
try:
# Get addresses from peerstore
addrs = self.host.get_peerstore().addrs(peer_id)
if addrs:
peer_info = PeerInfo(peer_id, addrs)
validated_peers.append(peer_info)
except Exception as e:
logger.debug(f"Failed to create PeerInfo for {peer_id}: {e}")
continue
return validated_peers
except Exception as e:
logger.error(f"Random walk failed: {e}")
raise RandomWalkError(f"Random walk operation failed: {e}") from e
async def run_concurrent_random_walks(
self, count: int = RANDOM_WALK_CONCURRENCY, current_routing_table_size: int = 0
) -> list[PeerInfo]:
"""
Run multiple random walks concurrently.
Args:
count: Number of concurrent random walks to perform
current_routing_table_size: Current size of routing table (for optimization)
Returns:
Combined list of all validated peers discovered
"""
all_validated_peers: list[PeerInfo] = []
logger.info(f"Starting {count} concurrent random walks")
# First, try to add peers from peerstore if routing table is small
if current_routing_table_size < RANDOM_WALK_RT_THRESHOLD:
try:
peerstore_peers = self._get_peerstore_peers()
if peerstore_peers:
logger.debug(
f"RT size ({current_routing_table_size}) below threshold, "
f"adding {len(peerstore_peers)} peerstore peers"
)
all_validated_peers.extend(peerstore_peers)
except Exception as e:
logger.warning(f"Error processing peerstore peers: {e}")
async def single_walk() -> None:
try:
peers = await self.perform_random_walk()
all_validated_peers.extend(peers)
except Exception as e:
logger.warning(f"Concurrent random walk failed: {e}")
return
# Run concurrent random walks
async with trio.open_nursery() as nursery:
for _ in range(count):
nursery.start_soon(single_walk)
# Remove duplicates based on peer ID
unique_peers = {}
for peer in all_validated_peers:
unique_peers[peer.peer_id] = peer
result = list(unique_peers.values())
logger.info(
f"Concurrent random walks completed: {len(result)} unique peers discovered"
)
return result
def _get_peerstore_peers(self) -> list[PeerInfo]:
"""
Get peer info objects from the host's peerstore.
Returns:
List of PeerInfo objects from peerstore
"""
try:
peerstore = self.host.get_peerstore()
peer_ids = peerstore.peers_with_addrs()
peer_infos = []
for peer_id in peer_ids:
try:
# Skip local peer
if peer_id == self.local_peer_id:
continue
peer_info = peerstore.peer_info(peer_id)
if peer_info and peer_info.addrs:
# Filter for compatible addresses (TCP + IPv4)
if self._has_compatible_addresses(peer_info):
peer_infos.append(peer_info)
except Exception as e:
logger.debug(f"Error getting peer info for {peer_id}: {e}")
return peer_infos
except Exception as e:
logger.warning(f"Error accessing peerstore: {e}")
return []
def _has_compatible_addresses(self, peer_info: PeerInfo) -> bool:
"""
Check if a peer has TCP+IPv4 compatible addresses.
Args:
peer_info: PeerInfo to check
Returns:
True if peer has compatible addresses
"""
if not peer_info.addrs:
return False
for addr in peer_info.addrs:
addr_str = str(addr)
# Check for TCP and IPv4 compatibility, avoid QUIC
if "/tcp/" in addr_str and "/ip4/" in addr_str and "/quic" not in addr_str:
return True
return False

View File

@ -0,0 +1,208 @@
from collections.abc import Awaitable, Callable
import logging
import time
from typing import Protocol
import trio
from libp2p.abc import IHost
from libp2p.discovery.random_walk.config import (
MIN_RT_REFRESH_THRESHOLD,
RANDOM_WALK_CONCURRENCY,
RANDOM_WALK_ENABLED,
REFRESH_INTERVAL,
)
from libp2p.discovery.random_walk.exceptions import RoutingTableRefreshError
from libp2p.discovery.random_walk.random_walk import RandomWalk
from libp2p.peer.id import ID
from libp2p.peer.peerinfo import PeerInfo
class RoutingTableProtocol(Protocol):
"""Protocol for routing table operations needed by RT refresh manager."""
def size(self) -> int:
"""Return the current size of the routing table."""
...
async def add_peer(self, peer_obj: PeerInfo) -> bool:
"""Add a peer to the routing table."""
...
logger = logging.getLogger("libp2p.discovery.random_walk.rt_refresh_manager")
class RTRefreshManager:
"""
Routing Table Refresh Manager for py-libp2p.
Manages periodic routing table refreshes and random walk operations
to maintain routing table health and discover new peers.
"""
def __init__(
self,
host: IHost,
routing_table: RoutingTableProtocol,
local_peer_id: ID,
query_function: Callable[[bytes], Awaitable[list[ID]]],
enable_auto_refresh: bool = RANDOM_WALK_ENABLED,
refresh_interval: float = REFRESH_INTERVAL,
min_refresh_threshold: int = MIN_RT_REFRESH_THRESHOLD,
):
"""
Initialize RT Refresh Manager.
Args:
host: The libp2p host instance
routing_table: Routing table of host
local_peer_id: Local peer ID
query_function: Function to query for closest peers given target key bytes
enable_auto_refresh: Whether to enable automatic refresh
refresh_interval: Interval between refreshes in seconds
min_refresh_threshold: Minimum RT size before triggering refresh
"""
self.host = host
self.routing_table = routing_table
self.local_peer_id = local_peer_id
self.query_function = query_function
self.enable_auto_refresh = enable_auto_refresh
self.refresh_interval = refresh_interval
self.min_refresh_threshold = min_refresh_threshold
# Initialize random walk module
self.random_walk = RandomWalk(
host=host,
local_peer_id=self.local_peer_id,
query_function=query_function,
)
# Control variables
self._running = False
self._nursery: trio.Nursery | None = None
# Tracking
self._last_refresh_time = 0.0
self._refresh_done_callbacks: list[Callable[[], None]] = []
async def start(self) -> None:
"""Start the RT Refresh Manager."""
if self._running:
logger.warning("RT Refresh Manager is already running")
return
self._running = True
logger.info("Starting RT Refresh Manager")
# Start the main loop
async with trio.open_nursery() as nursery:
self._nursery = nursery
nursery.start_soon(self._main_loop)
async def stop(self) -> None:
"""Stop the RT Refresh Manager."""
if not self._running:
return
logger.info("Stopping RT Refresh Manager")
self._running = False
async def _main_loop(self) -> None:
"""Main loop for the RT Refresh Manager."""
logger.info("RT Refresh Manager main loop started")
# Initial refresh if auto-refresh is enabled
if self.enable_auto_refresh:
await self._do_refresh(force=True)
try:
while self._running:
async with trio.open_nursery() as nursery:
# Schedule periodic refresh if enabled
if self.enable_auto_refresh:
nursery.start_soon(self._periodic_refresh_task)
except Exception as e:
logger.error(f"RT Refresh Manager main loop error: {e}")
finally:
logger.info("RT Refresh Manager main loop stopped")
async def _periodic_refresh_task(self) -> None:
"""Task for periodic refreshes."""
while self._running:
await trio.sleep(self.refresh_interval)
if self._running:
await self._do_refresh()
async def _do_refresh(self, force: bool = False) -> None:
"""
Perform routing table refresh operation.
Args:
force: Whether to force refresh regardless of timing
"""
try:
current_time = time.time()
# Check if refresh is needed
if not force:
if current_time - self._last_refresh_time < self.refresh_interval:
logger.debug("Skipping refresh: interval not elapsed")
return
if self.routing_table.size() >= self.min_refresh_threshold:
logger.debug("Skipping refresh: routing table size above threshold")
return
logger.info(f"Starting routing table refresh (force={force})")
start_time = current_time
# Perform random walks to discover new peers
logger.info("Running concurrent random walks to discover new peers")
current_rt_size = self.routing_table.size()
discovered_peers = await self.random_walk.run_concurrent_random_walks(
count=RANDOM_WALK_CONCURRENCY,
current_routing_table_size=current_rt_size,
)
# Add discovered peers to routing table
added_count = 0
for peer_info in discovered_peers:
result = await self.routing_table.add_peer(peer_info)
if result:
added_count += 1
self._last_refresh_time = current_time
duration = time.time() - start_time
logger.info(
f"Routing table refresh completed: "
f"{added_count}/{len(discovered_peers)} peers added, "
f"RT size: {self.routing_table.size()}, "
f"duration: {duration:.2f}s"
)
# Notify refresh completion
for callback in self._refresh_done_callbacks:
try:
callback()
except Exception as e:
logger.warning(f"Refresh callback error: {e}")
except Exception as e:
logger.error(f"Routing table refresh failed: {e}")
raise RoutingTableRefreshError(f"Refresh operation failed: {e}") from e
def add_refresh_done_callback(self, callback: Callable[[], None]) -> None:
"""Add a callback to be called when refresh completes."""
self._refresh_done_callbacks.append(callback)
def remove_refresh_done_callback(self, callback: Callable[[], None]) -> None:
"""Remove a refresh completion callback."""
if callback in self._refresh_done_callbacks:
self._refresh_done_callbacks.remove(callback)

View File

@ -1,7 +1,4 @@
import logging
from typing import (
Union,
)
from libp2p.custom_types import (
TProtocol,
@ -94,7 +91,7 @@ class AutoNATService:
finally:
await stream.close()
async def _handle_request(self, request: Union[bytes, Message]) -> Message:
async def _handle_request(self, request: bytes | Message) -> Message:
"""
Process an AutoNAT protocol request.

View File

@ -84,26 +84,23 @@ class AutoNAT:
request: Any,
target: str,
options: tuple[Any, ...] = (),
channel_credentials: Optional[Any] = None,
call_credentials: Optional[Any] = None,
channel_credentials: Any | None = None,
call_credentials: Any | None = None,
insecure: bool = False,
compression: Optional[Any] = None,
wait_for_ready: Optional[bool] = None,
timeout: Optional[float] = None,
metadata: Optional[list[tuple[str, str]]] = None,
compression: Any | None = None,
wait_for_ready: bool | None = None,
timeout: float | None = None,
metadata: list[tuple[str, str]] | None = None,
) -> Any:
return grpc.experimental.unary_unary(
request,
target,
channel = grpc.secure_channel(target, channel_credentials) if channel_credentials else grpc.insecure_channel(target)
return channel.unary_unary(
"/autonat.pb.AutoNAT/Dial",
autonat__pb2.Message.SerializeToString,
autonat__pb2.Message.FromString,
options,
channel_credentials,
insecure,
call_credentials,
compression,
wait_for_ready,
timeout,
metadata,
request_serializer=autonat__pb2.Message.SerializeToString,
response_deserializer=autonat__pb2.Message.FromString,
_registered_method=True,
)(
request,
timeout=timeout,
metadata=metadata,
wait_for_ready=wait_for_ready,
)

View File

@ -3,6 +3,7 @@ from collections.abc import (
Sequence,
)
from contextlib import (
AbstractAsyncContextManager,
asynccontextmanager,
)
import logging
@ -28,6 +29,8 @@ from libp2p.custom_types import (
StreamHandlerFn,
TProtocol,
)
from libp2p.discovery.bootstrap.bootstrap import BootstrapDiscovery
from libp2p.discovery.mdns.mdns import MDNSDiscovery
from libp2p.host.defaults import (
get_default_protocols,
)
@ -40,6 +43,7 @@ from libp2p.peer.id import (
from libp2p.peer.peerinfo import (
PeerInfo,
)
from libp2p.peer.peerstore import create_signed_peer_record
from libp2p.protocol_muxer.exceptions import (
MultiselectClientError,
MultiselectError,
@ -69,6 +73,7 @@ if TYPE_CHECKING:
logger = logging.getLogger("libp2p.network.basic_host")
DEFAULT_NEGOTIATE_TIMEOUT = 5
class BasicHost(IHost):
@ -88,15 +93,31 @@ class BasicHost(IHost):
def __init__(
self,
network: INetworkService,
default_protocols: "OrderedDict[TProtocol, StreamHandlerFn]" = None,
enable_mDNS: bool = False,
bootstrap: list[str] | None = None,
default_protocols: Optional["OrderedDict[TProtocol, StreamHandlerFn]"] = None,
negotitate_timeout: int = DEFAULT_NEGOTIATE_TIMEOUT,
) -> None:
self._network = network
self._network.set_stream_handler(self._swarm_stream_handler)
self.peerstore = self._network.peerstore
self.negotiate_timeout = negotitate_timeout
# Protocol muxing
default_protocols = default_protocols or get_default_protocols(self)
self.multiselect = Multiselect(default_protocols)
self.multiselect = Multiselect(dict(default_protocols.items()))
self.multiselect_client = MultiselectClient()
if enable_mDNS:
self.mDNS = MDNSDiscovery(network)
if bootstrap:
self.bootstrap = BootstrapDiscovery(network, bootstrap)
# Cache a signed-record if the local-node in the PeerStore
envelope = create_signed_peer_record(
self.get_id(),
self.get_addrs(),
self.get_private_key(),
)
self.get_peerstore().set_local_record(envelope)
def get_id(self) -> ID:
"""
@ -147,19 +168,35 @@ class BasicHost(IHost):
"""
return list(self._network.connections.keys())
@asynccontextmanager
async def run(
def run(
self, listen_addrs: Sequence[multiaddr.Multiaddr]
) -> AsyncIterator[None]:
) -> AbstractAsyncContextManager[None]:
"""
Run the host instance and listen to ``listen_addrs``.
:param listen_addrs: a sequence of multiaddrs that we want to listen to
"""
network = self.get_network()
async with background_trio_service(network):
await network.listen(*listen_addrs)
yield
@asynccontextmanager
async def _run() -> AsyncIterator[None]:
network = self.get_network()
async with background_trio_service(network):
await network.listen(*listen_addrs)
if hasattr(self, "mDNS") and self.mDNS is not None:
logger.debug("Starting mDNS Discovery")
self.mDNS.start()
if hasattr(self, "bootstrap") and self.bootstrap is not None:
logger.debug("Starting Bootstrap Discovery")
await self.bootstrap.start()
try:
yield
finally:
if hasattr(self, "mDNS") and self.mDNS is not None:
self.mDNS.stop()
if hasattr(self, "bootstrap") and self.bootstrap is not None:
self.bootstrap.stop()
return _run()
def set_stream_handler(
self, protocol_id: TProtocol, stream_handler: StreamHandlerFn
@ -173,7 +210,9 @@ class BasicHost(IHost):
self.multiselect.add_handler(protocol_id, stream_handler)
async def new_stream(
self, peer_id: ID, protocol_ids: Sequence[TProtocol]
self,
peer_id: ID,
protocol_ids: Sequence[TProtocol],
) -> INetStream:
"""
:param peer_id: peer_id that host is connecting
@ -185,7 +224,9 @@ class BasicHost(IHost):
# Perform protocol muxing to determine protocol to use
try:
selected_protocol = await self.multiselect_client.select_one_of(
list(protocol_ids), MultiselectCommunicator(net_stream)
list(protocol_ids),
MultiselectCommunicator(net_stream),
self.negotiate_timeout,
)
except MultiselectClientError as error:
logger.debug("fail to open a stream to peer %s, error=%s", peer_id, error)
@ -195,7 +236,12 @@ class BasicHost(IHost):
net_stream.set_protocol(selected_protocol)
return net_stream
async def send_command(self, peer_id: ID, command: str) -> list[str]:
async def send_command(
self,
peer_id: ID,
command: str,
response_timeout: int = DEFAULT_NEGOTIATE_TIMEOUT,
) -> list[str]:
"""
Send a multistream-select command to the specified peer and return
the response.
@ -209,7 +255,7 @@ class BasicHost(IHost):
try:
response = await self.multiselect_client.query_multistream_command(
MultiselectCommunicator(new_stream), command
MultiselectCommunicator(new_stream), command, response_timeout
)
except MultiselectClientError as error:
logger.debug("fail to open a stream to peer %s, error=%s", peer_id, error)
@ -229,7 +275,7 @@ class BasicHost(IHost):
:param peer_info: peer_info of the peer we want to connect to
:type peer_info: peer.peerinfo.PeerInfo
"""
self.peerstore.add_addrs(peer_info.peer_id, peer_info.addrs, 10)
self.peerstore.add_addrs(peer_info.peer_id, peer_info.addrs, 120)
# there is already a connection to this peer
if peer_info.peer_id in self._network.connections:
@ -248,8 +294,13 @@ class BasicHost(IHost):
# Perform protocol muxing to determine protocol to use
try:
protocol, handler = await self.multiselect.negotiate(
MultiselectCommunicator(net_stream)
MultiselectCommunicator(net_stream), self.negotiate_timeout
)
if protocol is None:
await net_stream.reset()
raise StreamFailure(
"Failed to negotiate protocol: no protocol selected"
)
except MultiselectError as error:
peer_id = net_stream.muxed_conn.peer_id
logger.debug(
@ -257,7 +308,23 @@ class BasicHost(IHost):
)
await net_stream.reset()
return
if protocol is None:
logger.debug(
"no protocol negotiated, closing stream from peer %s",
net_stream.muxed_conn.peer_id,
)
await net_stream.reset()
return
net_stream.set_protocol(protocol)
if handler is None:
logger.debug(
"no handler for protocol %s, closing stream from peer %s",
protocol,
net_stream.muxed_conn.peer_id,
)
await net_stream.reset()
return
await handler(net_stream)
def get_live_peers(self) -> list[ID]:
@ -275,13 +342,13 @@ class BasicHost(IHost):
:param peer_id: ID of the peer to check
:return: True if peer has an active connection, False otherwise
"""
return peer_id in self._network.connections
return len(self._network.get_connections(peer_id)) > 0
def get_peer_connection_info(self, peer_id: ID) -> Optional[INetConn]:
def get_peer_connection_info(self, peer_id: ID) -> INetConn | None:
"""
Get connection information for a specific peer if connected.
:param peer_id: ID of the peer to get info for
:return: Connection object if peer is connected, None otherwise
"""
return self._network.connections.get(peer_id)
return self._network.get_connection(peer_id)

View File

@ -9,13 +9,13 @@ from libp2p.abc import (
IHost,
)
from libp2p.host.ping import (
ID as PingID,
handle_ping,
)
from libp2p.host.ping import ID as PingID
from libp2p.identity.identify.identify import (
ID as IdentifyID,
identify_handler_for,
)
from libp2p.identity.identify.identify import ID as IdentifyID
if TYPE_CHECKING:
from libp2p.custom_types import (
@ -26,5 +26,8 @@ if TYPE_CHECKING:
def get_default_protocols(host: IHost) -> "OrderedDict[TProtocol, StreamHandlerFn]":
return OrderedDict(
((IdentifyID, identify_handler_for(host)), (PingID, handle_ping))
(
(IdentifyID, identify_handler_for(host, use_varint_format=True)),
(PingID, handle_ping),
)
)

View File

@ -18,8 +18,14 @@ from libp2p.peer.peerinfo import (
class RoutedHost(BasicHost):
_router: IPeerRouting
def __init__(self, network: INetworkService, router: IPeerRouting):
super().__init__(network)
def __init__(
self,
network: INetworkService,
router: IPeerRouting,
enable_mDNS: bool = False,
bootstrap: list[str] | None = None,
):
super().__init__(network, enable_mDNS, bootstrap)
self._router = router
async def connect(self, peer_info: PeerInfo) -> None:
@ -40,8 +46,8 @@ class RoutedHost(BasicHost):
found_peer_info = await self._router.find_peer(peer_info.peer_id)
if not found_peer_info:
raise ConnectionFailure("Unable to find Peer address")
self.peerstore.add_addrs(peer_info.peer_id, found_peer_info.addrs, 10)
self.peerstore.add_addrs(peer_info.peer_id, peer_info.addrs, 10)
self.peerstore.add_addrs(peer_info.peer_id, found_peer_info.addrs, 120)
self.peerstore.add_addrs(peer_info.peer_id, peer_info.addrs, 120)
# there is already a connection to this peer
if peer_info.peer_id in self._network.connections:

View File

@ -1,7 +1,4 @@
import logging
from typing import (
Optional,
)
from multiaddr import (
Multiaddr,
@ -18,8 +15,11 @@ from libp2p.custom_types import (
from libp2p.network.stream.exceptions import (
StreamClosed,
)
from libp2p.peer.peerstore import env_to_send_in_RPC
from libp2p.utils import (
decode_varint_with_size,
get_agent_version,
varint,
)
from .pb.identify_pb2 import (
@ -40,8 +40,8 @@ def _multiaddr_to_bytes(maddr: Multiaddr) -> bytes:
def _remote_address_to_multiaddr(
remote_address: Optional[tuple[str, int]]
) -> Optional[Multiaddr]:
remote_address: tuple[str, int] | None,
) -> Multiaddr | None:
"""Convert a (host, port) tuple to a Multiaddr."""
if remote_address is None:
return None
@ -58,11 +58,14 @@ def _remote_address_to_multiaddr(
def _mk_identify_protobuf(
host: IHost, observed_multiaddr: Optional[Multiaddr]
host: IHost, observed_multiaddr: Multiaddr | None
) -> Identify:
public_key = host.get_public_key()
laddrs = host.get_addrs()
protocols = host.get_mux().get_protocols()
protocols = tuple(str(p) for p in host.get_mux().get_protocols() if p is not None)
# Create a signed peer-record for the remote peer
envelope_bytes, _ = env_to_send_in_RPC(host)
observed_addr = observed_multiaddr.to_bytes() if observed_multiaddr else b""
return Identify(
@ -72,24 +75,64 @@ def _mk_identify_protobuf(
listen_addrs=map(_multiaddr_to_bytes, laddrs),
observed_addr=observed_addr,
protocols=protocols,
signedPeerRecord=envelope_bytes,
)
def identify_handler_for(host: IHost) -> StreamHandlerFn:
def parse_identify_response(response: bytes) -> Identify:
"""
Parse identify response that could be either:
- Old format: raw protobuf
- New format: length-prefixed protobuf
This function provides backward and forward compatibility.
"""
# Try new format first: length-prefixed protobuf
if len(response) >= 1:
length, varint_size = decode_varint_with_size(response)
if varint_size > 0 and length > 0 and varint_size + length <= len(response):
protobuf_data = response[varint_size : varint_size + length]
try:
identify_response = Identify()
identify_response.ParseFromString(protobuf_data)
# Sanity check: must have agent_version (protocol_version is optional)
if identify_response.agent_version:
logger.debug(
"Parsed length-prefixed identify response (new format)"
)
return identify_response
except Exception:
pass # Fall through to old format
# Fall back to old format: raw protobuf
try:
identify_response = Identify()
identify_response.ParseFromString(response)
logger.debug("Parsed raw protobuf identify response (old format)")
return identify_response
except Exception as e:
logger.error(f"Failed to parse identify response: {e}")
logger.error(f"Response length: {len(response)}")
logger.error(f"Response hex: {response.hex()}")
raise
def identify_handler_for(
host: IHost, use_varint_format: bool = True
) -> StreamHandlerFn:
async def handle_identify(stream: INetStream) -> None:
# get observed address from ``stream``
peer_id = (
stream.muxed_conn.peer_id
) # remote peer_id is in class Mplex (mplex.py )
observed_multiaddr: Multiaddr | None = None
# Get the remote address
try:
remote_address = stream.get_remote_address()
# Convert to multiaddr
if remote_address:
observed_multiaddr = _remote_address_to_multiaddr(remote_address)
else:
observed_multiaddr = None
logger.debug(
"Connection from remote peer %s, address: %s, multiaddr: %s",
peer_id,
@ -104,7 +147,21 @@ def identify_handler_for(host: IHost) -> StreamHandlerFn:
response = protobuf.SerializeToString()
try:
await stream.write(response)
if use_varint_format:
# Send length-prefixed protobuf message (new format)
await stream.write(varint.encode_uvarint(len(response)))
await stream.write(response)
logger.debug(
"Sent new format (length-prefixed) identify response to %s",
peer_id,
)
else:
# Send raw protobuf message (old format for backward compatibility)
await stream.write(response)
logger.debug(
"Sent old format (raw protobuf) identify response to %s",
peer_id,
)
except StreamClosed:
logger.debug("Fail to respond to %s request: stream closed", ID)
else:

View File

@ -9,4 +9,5 @@ message Identify {
repeated bytes listen_addrs = 2;
optional bytes observed_addr = 4;
repeated string protocols = 3;
optional bytes signedPeerRecord = 8;
}

View File

@ -13,7 +13,7 @@ _sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n*libp2p/identity/identify/pb/identify.proto\x12\x0bidentify.pb\"\x8f\x01\n\x08Identify\x12\x18\n\x10protocol_version\x18\x05 \x01(\t\x12\x15\n\ragent_version\x18\x06 \x01(\t\x12\x12\n\npublic_key\x18\x01 \x01(\x0c\x12\x14\n\x0clisten_addrs\x18\x02 \x03(\x0c\x12\x15\n\robserved_addr\x18\x04 \x01(\x0c\x12\x11\n\tprotocols\x18\x03 \x03(\t')
DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n*libp2p/identity/identify/pb/identify.proto\x12\x0bidentify.pb\"\xa9\x01\n\x08Identify\x12\x18\n\x10protocol_version\x18\x05 \x01(\t\x12\x15\n\ragent_version\x18\x06 \x01(\t\x12\x12\n\npublic_key\x18\x01 \x01(\x0c\x12\x14\n\x0clisten_addrs\x18\x02 \x03(\x0c\x12\x15\n\robserved_addr\x18\x04 \x01(\x0c\x12\x11\n\tprotocols\x18\x03 \x03(\t\x12\x18\n\x10signedPeerRecord\x18\x08 \x01(\x0c')
_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, globals())
_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'libp2p.identity.identify.pb.identify_pb2', globals())
@ -21,5 +21,5 @@ if _descriptor._USE_C_DESCRIPTORS == False:
DESCRIPTOR._options = None
_IDENTIFY._serialized_start=60
_IDENTIFY._serialized_end=203
_IDENTIFY._serialized_end=229
# @@protoc_insertion_point(module_scope)

View File

@ -22,10 +22,12 @@ class Identify(google.protobuf.message.Message):
LISTEN_ADDRS_FIELD_NUMBER: builtins.int
OBSERVED_ADDR_FIELD_NUMBER: builtins.int
PROTOCOLS_FIELD_NUMBER: builtins.int
SIGNEDPEERRECORD_FIELD_NUMBER: builtins.int
protocol_version: builtins.str
agent_version: builtins.str
public_key: builtins.bytes
observed_addr: builtins.bytes
signedPeerRecord: builtins.bytes
@property
def listen_addrs(self) -> google.protobuf.internal.containers.RepeatedScalarFieldContainer[builtins.bytes]: ...
@property
@ -39,8 +41,9 @@ class Identify(google.protobuf.message.Message):
listen_addrs: collections.abc.Iterable[builtins.bytes] | None = ...,
observed_addr: builtins.bytes | None = ...,
protocols: collections.abc.Iterable[builtins.str] | None = ...,
signedPeerRecord: builtins.bytes | None = ...,
) -> None: ...
def HasField(self, field_name: typing.Literal["agent_version", b"agent_version", "observed_addr", b"observed_addr", "protocol_version", b"protocol_version", "public_key", b"public_key"]) -> builtins.bool: ...
def ClearField(self, field_name: typing.Literal["agent_version", b"agent_version", "listen_addrs", b"listen_addrs", "observed_addr", b"observed_addr", "protocol_version", b"protocol_version", "protocols", b"protocols", "public_key", b"public_key"]) -> None: ...
def HasField(self, field_name: typing.Literal["agent_version", b"agent_version", "observed_addr", b"observed_addr", "protocol_version", b"protocol_version", "public_key", b"public_key", "signedPeerRecord", b"signedPeerRecord"]) -> builtins.bool: ...
def ClearField(self, field_name: typing.Literal["agent_version", b"agent_version", "listen_addrs", b"listen_addrs", "observed_addr", b"observed_addr", "protocol_version", b"protocol_version", "protocols", b"protocols", "public_key", b"public_key", "signedPeerRecord", b"signedPeerRecord"]) -> None: ...
global___Identify = Identify

View File

@ -1,7 +1,4 @@
import logging
from typing import (
Optional,
)
from multiaddr import (
Multiaddr,
@ -23,11 +20,16 @@ from libp2p.custom_types import (
from libp2p.network.stream.exceptions import (
StreamClosed,
)
from libp2p.peer.envelope import consume_envelope
from libp2p.peer.id import (
ID,
)
from libp2p.utils import (
get_agent_version,
varint,
)
from libp2p.utils.varint import (
read_length_prefixed_protobuf,
)
from ..identify.identify import (
@ -43,22 +45,31 @@ logger = logging.getLogger(__name__)
ID_PUSH = TProtocol("/ipfs/id/push/1.0.0")
PROTOCOL_VERSION = "ipfs/0.1.0"
AGENT_VERSION = get_agent_version()
CONCURRENCY_LIMIT = 10
def identify_push_handler_for(host: IHost) -> StreamHandlerFn:
def identify_push_handler_for(
host: IHost, use_varint_format: bool = True
) -> StreamHandlerFn:
"""
Create a handler for the identify/push protocol.
This handler receives pushed identify messages from remote peers and updates
the local peerstore with the new information.
Args:
host: The libp2p host.
use_varint_format: True=length-prefixed, False=raw protobuf.
"""
async def handle_identify_push(stream: INetStream) -> None:
peer_id = stream.muxed_conn.peer_id
try:
# Read the identify message from the stream
data = await stream.read()
# Use the utility function to read the protobuf message
data = await read_length_prefixed_protobuf(stream, use_varint_format)
identify_msg = Identify()
identify_msg.ParseFromString(data)
@ -68,6 +79,11 @@ def identify_push_handler_for(host: IHost) -> StreamHandlerFn:
)
logger.debug("Successfully processed identify/push from peer %s", peer_id)
# Send acknowledgment to indicate successful processing
# This ensures the sender knows the message was received before closing
await stream.write(b"OK")
except StreamClosed:
logger.debug(
"Stream closed while processing identify/push from %s", peer_id
@ -76,7 +92,10 @@ def identify_push_handler_for(host: IHost) -> StreamHandlerFn:
logger.error("Error processing identify/push from %s: %s", peer_id, e)
finally:
# Close the stream after processing
await stream.close()
try:
await stream.close()
except Exception:
pass # Ignore errors when closing
return handle_identify_push
@ -122,6 +141,19 @@ async def _update_peerstore_from_identify(
except Exception as e:
logger.error("Error updating protocols for peer %s: %s", peer_id, e)
if identify_msg.HasField("signedPeerRecord"):
try:
# Convert the signed-peer-record(Envelope) from prtobuf bytes
envelope, _ = consume_envelope(
identify_msg.signedPeerRecord, "libp2p-peer-record"
)
# Use a default TTL of 2 hours (7200 seconds)
if not peerstore.consume_peer_record(envelope, 7200):
logger.error("Updating Certified-Addr-Book was unsuccessful")
except Exception as e:
logger.error(
"Error updating the certified addr book for peer %s: %s", peer_id, e
)
# Update observed address if present
if identify_msg.HasField("observed_addr") and identify_msg.observed_addr:
try:
@ -135,7 +167,11 @@ async def _update_peerstore_from_identify(
async def push_identify_to_peer(
host: IHost, peer_id: ID, observed_multiaddr: Optional[Multiaddr] = None
host: IHost,
peer_id: ID,
observed_multiaddr: Multiaddr | None = None,
limit: trio.Semaphore = trio.Semaphore(CONCURRENCY_LIMIT),
use_varint_format: bool = True,
) -> bool:
"""
Push an identify message to a specific peer.
@ -143,52 +179,91 @@ async def push_identify_to_peer(
This function opens a stream to the peer using the identify/push protocol,
sends the identify message, and closes the stream.
Returns
-------
bool
True if the push was successful, False otherwise.
Args:
host: The libp2p host.
peer_id: The peer ID to push to.
observed_multiaddr: The observed multiaddress (optional).
limit: Semaphore for concurrency control.
use_varint_format: True=length-prefixed, False=raw protobuf.
Returns:
bool: True if the push was successful, False otherwise.
"""
try:
# Create a new stream to the peer using the identify/push protocol
stream = await host.new_stream(peer_id, [ID_PUSH])
async with limit:
try:
# Create a new stream to the peer using the identify/push protocol
stream = await host.new_stream(peer_id, [ID_PUSH])
# Create the identify message
identify_msg = _mk_identify_protobuf(host, observed_multiaddr)
response = identify_msg.SerializeToString()
# Create the identify message
identify_msg = _mk_identify_protobuf(host, observed_multiaddr)
response = identify_msg.SerializeToString()
# Send the identify message
await stream.write(response)
if use_varint_format:
# Send length-prefixed identify message
await stream.write(varint.encode_uvarint(len(response)))
await stream.write(response)
else:
# Send raw protobuf message
await stream.write(response)
# Close the stream
await stream.close()
# Wait for acknowledgment from the receiver with timeout
# This ensures the message was processed before closing
try:
with trio.move_on_after(1.0): # 1 second timeout
ack = await stream.read(2) # Read "OK" acknowledgment
if ack != b"OK":
logger.warning(
"Unexpected acknowledgment from peer %s: %s", peer_id, ack
)
except Exception as e:
logger.debug("No acknowledgment received from peer %s: %s", peer_id, e)
# Continue anyway, as the message might have been processed
logger.debug("Successfully pushed identify to peer %s", peer_id)
return True
except Exception as e:
logger.error("Error pushing identify to peer %s: %s", peer_id, e)
return False
# Close the stream after acknowledgment (or timeout)
await stream.close()
logger.debug("Successfully pushed identify to peer %s", peer_id)
return True
except Exception as e:
logger.error("Error pushing identify to peer %s: %s", peer_id, e)
return False
async def push_identify_to_peers(
host: IHost,
peer_ids: Optional[set[ID]] = None,
observed_multiaddr: Optional[Multiaddr] = None,
peer_ids: set[ID] | None = None,
observed_multiaddr: Multiaddr | None = None,
use_varint_format: bool = True,
) -> None:
"""
Push an identify message to multiple peers in parallel.
If peer_ids is None, push to all connected peers.
Args:
host: The libp2p host.
peer_ids: Set of peer IDs to push to (if None, push to all connected peers).
observed_multiaddr: The observed multiaddress (optional).
use_varint_format: True=length-prefixed, False=raw protobuf.
"""
if peer_ids is None:
# Get all connected peers
peer_ids = set(host.get_peerstore().peer_ids())
peer_ids = set(host.get_connected_peers())
# Create a single shared semaphore for concurrency control
limit = trio.Semaphore(CONCURRENCY_LIMIT)
# Push to each peer in parallel using a trio.Nursery
# TODO: Consider using a bounded nursery to limit concurrency
# and avoid overwhelming the network. This can be done by using
# trio.open_nursery(max_concurrent=10) or similar.
# For now, we will use an unbounded nursery for simplicity.
# limiting concurrent connections to CONCURRENCY_LIMIT
async with trio.open_nursery() as nursery:
for peer_id in peer_ids:
nursery.start_soon(push_identify_to_peer, host, peer_id, observed_multiaddr)
nursery.start_soon(
push_identify_to_peer,
host,
peer_id,
observed_multiaddr,
limit,
use_varint_format,
)

Some files were not shown because too many files have changed in this diff Show More