Commit Graph

66 Commits

Author SHA1 Message Date
aa7276c863 Implement closed_stream event handling and enable related tests (#834)
* Implement closed_stream event handling and enable related tests

* Fix linting issues and ensure all tests pass

* Add logging for exception in SwarmConn and create newsfragment for closed_stream feature
2025-08-13 16:19:53 -06:00
cb11f076c8 feat/606-enable-nat-traversal-via-hole-punching (#668)
* feat: base implementation of dcutr for hole-punching

* chore: removed circuit-relay imports from __init__

* feat: implemented dcutr protocol

* added test suite with mock setup

* Fix pre-commit hook issues in DCUtR implementation

* usages of CONNECT_TYPE and SYNC_TYPE have been replaced with HolePunch.Type.CONNECT and HolePunch.Type.SYNC

* added unit tests for dcutr and nat module and

* added multiaddr.get_peer_id() with proper DNS address handling and fixed method signature inconsistencies

* added assertions to verify DCUtR hole punch result in integration test

---------

Co-authored-by: Manu Sheel Gupta <manusheel.edu@gmail.com>
2025-08-07 19:00:16 -06:00
bdadec7519 ft. modernise py-libp2p (#618)
* fix pyproject.toml , add ruff

* rm lock

* make progress

* add poetry lock ignore

* fix type issues

* fix tcp type errors

* fix text example - type error - wrong args

* add setuptools to dev

* test ci

* fix docs build

* fix type issues for new_swarm & new_host

* fix types in gossipsub

* fix type issues in noise

* wip: factories

* revert factories

* fix more type issues

* more type fixes

* fix: add null checks for noise protocol initialization and key handling

* corrected argument-errors in peerId and Multiaddr in peer tests

* fix: Noice - remove redundant type casts in BaseNoiseMsgReadWriter

* fix: update test_notify.py to use SwarmFactory.create_batch_and_listen, fix type hints, and comment out ClosedStream assertions

* Fix type checks for pubsub module

Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>

* Fix type checks for pubsub module-tests

Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>

* noise: add checks for uninitialized protocol and key states in PatternXX

Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>

* pubsub: add None checks for optional fields in FloodSub and Pubsub

Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>

* Fix type hints and improve testing

Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>

* remove redundant checks

Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>

* fix build issues

* add optional to trio service

* fix types

* fix type errors

* Fix type errors

Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>

* fixed more-type checks in crypto and peer_data files

* wip: factories

* replaced union with optional

* fix: type-error in interp-utils and peerinfo

* replace pyright with pyrefly

* add pyrefly.toml

* wip: fix multiselect issues

* try typecheck

* base check

* mcache test fixes , typecheck ci update

* fix ci

* will this work

* minor fix

* use poetry

* fix wokflow

* use cache,fix err

* fix pyrefly.toml

* fix pyrefly.toml

* fix cache in ci

* deploy commit

* add main baseline

* update to v5

* improve typecheck ci (#14)

* fix typo

* remove holepunching code (#16)

* fix gossipsub typeerrors (#17)

* fix: ensure initiator user includes remote peer id in handshake (#15)

* fix ci (#19)

* typefix: custom_types | core/peerinfo/test_peer_info | io/abc | pubsub/floodsub | protocol_muxer/multiselect (#18)

* fix: Typefixes in PeerInfo  (#21)

* fix minor type issue (#22)

* fix type errors in pubsub (#24)

* fix: Minor typefixes in tests (#23)

* Fix failing tests for type-fixed test/pubsub (#8)

* move pyrefly & ruff to pyproject.toml & rm .project-template (#28)

* move the async_context file to tests/core

* move crypto test to crypto folder

* fix: some typefixes (#25)

* fix type errors

* fix type issues

* fix: update gRPC API usage in autonat_pb2_grpc.py (#31)

* md: typecheck ci

* rm comments

* clean up : from review suggestions

* use | None over Optional as per new python standards

* drop supporto for py3.9

* newsfragments

---------

Signed-off-by: sukhman <sukhmansinghsaluja@gmail.com>
Signed-off-by: varun-r-mallya <varunrmallya@gmail.com>
Co-authored-by: acul71 <luca.pisani@birdo.net>
Co-authored-by: kaneki003 <sakshamchauhan707@gmail.com>
Co-authored-by: sukhman <sukhmansinghsaluja@gmail.com>
Co-authored-by: varun-r-mallya <varunrmallya@gmail.com>
Co-authored-by: varunrmallya <100590632+varun-r-mallya@users.noreply.github.com>
Co-authored-by: lla-dane <abhinavagarwalla6@gmail.com>
Co-authored-by: Collins <ArtemisfowlX@protonmail.com>
Co-authored-by: Abhinav Agarwalla <120122716+lla-dane@users.noreply.github.com>
Co-authored-by: guha-rahul <52607971+guha-rahul@users.noreply.github.com>
Co-authored-by: Sukhman Singh <63765293+sukhman-sukh@users.noreply.github.com>
Co-authored-by: acul71 <34693171+acul71@users.noreply.github.com>
Co-authored-by: pacrob <5199899+pacrob@users.noreply.github.com>
2025-06-09 11:39:59 -06:00
4b1860766d Feat: Adding Yamux as default multiplexer, keeping Mplex as fallback (#538)
* feat: Replace mplex with yamux as default multiplexer in py-libp2p

* Retain Mplex alongside Yamux in new_swarm with messaging that Yamux is preferred

* moved !BBHII to a constant YAMUX_HEADER_FORMAT at the top of yamux.py with a comment explaining its structure

* renamed the news fragment to 534.feature.rst and updated the description

* renamed the news fragment to 534.feature.rst and updated the description

* added a docstring to clarify that Yamux does not support deadlines natively

* Remove the __main__ block entirely from test_yamux.py

* Replaced the print statements in test_yamux.py with logging.debug

* Added a comment linking to the spec for clarity

* Raise NotImplementedError in YamuxStream.set_deadline per review

* Add muxed_conn to YamuxStream and test deadline NotImplementedError

* Fix Yamux implementation to meet libp2p spec

* Fix None handling in YamuxStream.read and Yamux.read_stream

* Fix test_connected_peers.py to correctly handle peer connections

* fix: Ensure StreamReset is raised on read after local reset in yamux

* fix: Map MuxedStreamError to StreamClosed in NetStream.write for Yamux

* fix: Raise MuxedStreamReset in Yamux.read_stream for closed streams

* fix: Correct Yamux stream read behavior for NetStream tests

Fixed 	est_net_stream_read_after_remote_closed by updating NetStream.read to raise StreamEOF when the stream is remotely closed and no data is available, aligning with test expectations and Fixed 	est_net_stream_read_until_eof by modifying YamuxStream.read to block until the stream is closed (
ecv_closed=True) for
=-1 reads, ensuring data is only returned after remote closure.

* fix: Correct Yamux stream read behavior for NetStream tests

Fixed 	est_net_stream_read_after_remote_closed by updating NetStream.read to raise StreamEOF when the stream is remotely closed and no data is available, aligning with test expectations and Fixed 	est_net_stream_read_until_eof by modifying YamuxStream.read to block until the stream is closed (
ecv_closed=True) for
=-1 reads, ensuring data is only returned after remote closure.

* fix: raise StreamEOF when reading from closed stream with empty buffer

* fix: prioritize returning buffered data even after stream reset

* fix: prioritize returning buffered data even after stream reset

* fix: Ensure test_net_stream_read_after_remote_closed_and_reset passes in full suite

* fix: Add __init__.py to yamux module to fix documentation build

* fix: Add __init__.py to yamux module to fix documentation build

* fix: Add libp2p.stream_muxer.yamux to libp2p.stream_muxer.rst toctree

* fix: Correct title underline length in libp2p.stream_muxer.yamux.rst

* fix: Add a = so that is matches the libp2p.stream\_muxer.yamux length

* fix(tests): Resolve race condition in network notification test

* fix: fixing failing tests and examples with yamux and noise

* refactor: remove debug logging and improve x25519 tests

* fix: Add functionality for users to choose between Yamux and Mplex

* fix: increased trio sleep to 0.1 sec for slow environment

* feat: Add test for switching between Yamux and mplex

* refactor: move host fixtures to interop tests

* chore: Update __init__.py removing unused import

removed unused
```python
import os
import logging
```

* lint: fix import order

* fix: Resolve conftest.py conflict by removing trio test support

* fix: Resolve test skipping by keeping trio test support

* Fix: add a newline at end of the file

---------

Co-authored-by: acul71 <luca.pisani@birdo.net>
Co-authored-by: acul71 <34693171+acul71@users.noreply.github.com>
2025-05-22 14:01:51 -06:00
fabf2cefc4 feat: implement get_remote_address via delegation pattern 2025-03-24 08:02:21 -06:00
d7eab27564 refactored and moved all interfaces to abc.py (#504)
* refactored : host_interface

Co-authored-by: mystical-prog <jdgt.vd.0405@gmail.com>

* refactored : network_interface

Co-authored-by: mystical-prog <jdgt.vd.0405@gmail.com>

* refactored : notifee_interface

Co-authored-by: mystical-prog <jdgt.vd.0405@gmail.com>

* refactored : net_connection_interface

Co-authored-by: mystical-prog <jdgt.vd.0405@gmail.com>

* refactored: raw_connection_interface, secure_conn_interface and stream_muxer abc.py

* refactored: addrbook_interface

* refactored :peerdata_interface

Co-authored-by: mystical-prog <jdgt.vd.0405@gmail.com>

* refactored :peermetadata_interface

Co-authored-by: mystical-prog <jdgt.vd.0405@gmail.com>

* refactored :multiselect_client_interface

Co-authored-by: mystical-prog <jdgt.vd.0405@gmail.com>

* refactored :multiselect_communicator_interface

Co-authored-by: mystical-prog <jdgt.vd.0405@gmail.com>

* refactored :multiselect_muxer_interface

Co-authored-by: mystical-prog <jdgt.vd.0405@gmail.com>

* refactored :interfaces

Co-authored-by: mystical-prog <jdgt.vd.0405@gmail.com>

* refactored :security_transport_interface

Co-authored-by: mystical-prog <jdgt.vd.0405@gmail.com>

* refactored :listener_interface

Co-authored-by: mystical-prog <jdgt.vd.0405@gmail.com>

* moved all interfaces and typing files

Co-authored-by: mystical-prog <jdgt.vd.0405@gmail.com>

* fixed documentation and moved pubsub abc.py

Co-authored-by: Khwahish Patel <khwahish.p1@ahduni.edu.in>

* added exclude-members in custom_types docs

* added : newsfragment for moving all interfaces to libp2p.abc

---------

Co-authored-by: mystical-prog <jdgt.vd.0405@gmail.com>
Co-authored-by: Mystical <125946525+mystical-prog@users.noreply.github.com>
2025-02-21 16:01:45 -07:00
8787613e91 run lint with pyupgrade at py39-plus 2025-01-25 15:48:39 -07:00
8dda7b933e turn mypy checks back on, fix some errors 2024-04-27 10:12:39 -06:00
94483714a3 run lint and fix errors, except mypy 2024-02-19 16:05:58 -07:00
12cb0d9ac4 Swarm: change notify_xxx back to async func 2020-02-04 22:56:13 +08:00
13930ae718 SwarmConn: perform close right away
In `_handle_new_streams`, when the underlying muxed conn is unavailable,
close `SwarmConn` itself right away, to reset all the streams.
Therefore, the stream processed by `_handle_muxed_stream` are conscious
of the fact that they are reset. It allows a more graceful clean up.
2020-02-04 22:51:21 +08:00
f884bfa39e SwarmConn: don't access Swarm.manager
Open a local nursery instead.
2020-02-04 21:57:11 +08:00
3a91f114ab Swarm: add default_stream_handler
Advantage:
- To avoid `None` checks
- If users forget to register a stream handler for `Swarm`,
with the default stream handler, opened streams aren't removed
until `Swarm` finishes.
2020-02-04 17:05:53 +08:00
3fc60cb312 SwarmConn: iterate streams.copy in _cleanup
To avoid `RuntimeError` if `streams` is changed.
2020-02-04 17:04:28 +08:00
d483982acb SwarmConn: don't catch exceptions in handler 2020-02-04 15:10:49 +08:00
e57d01f360 PR feedback
- Use f-string
- Fix wrongly indented comments
- Add dep `trio-typing`
2020-01-28 15:48:41 +08:00
ddfbf9ffc8 Use raise from to reserve stacktrace 2020-01-26 23:54:29 +08:00
5b4b65faa8 Change default value of read()
From `n = -1` to `n = None`, to comply with trio API
2020-01-26 23:03:38 +08:00
eef241e70e Make Mplex and SwarmConn not Service
After second thoughts, they seem not a good candidate of `Service`.
The shutdown logic becomes simpler by making them not `Service`.
2020-01-07 21:50:03 +08:00
45eeb4fba3 Change notify_xxx to sync functions
Since we already have `Swarm.run_task`, we can just change notify
functions to sync.
2020-01-07 16:45:06 +08:00
fb0519129d Refine Mplex.close and SwarmConn.close
Ensure `close` cleans up things and cancel the service finally.
2019-12-17 15:50:55 +08:00
1929f307fb Fix all modules except for security 2019-12-06 17:06:37 +08:00
79fcdf3a02 Update tests in test_tcp.py
Besides, run `make format`
2019-12-01 16:26:16 +08:00
1e600ea7e0 Fix Mplex and Swarm 2019-11-29 19:09:56 +08:00
ec43c25b45 Rewrite factories, made some of the test running 2019-11-26 19:24:30 +08:00
a397ccdc04 makes test_mplex_stream.py::test_mplex_stream_read_write work 2019-11-26 15:27:06 +08:00
41ff884eef rewrite tcp reader/writer interface 2019-11-26 15:27:05 +08:00
97b3aca535 Fix:
Force context switch before canceling swarm connection tasks
2019-11-09 23:24:08 +08:00
ded3792924 Merge pull request #331 from dmuhs/fix/docs-format
Add automatic docstring formatting
2019-10-28 09:39:52 -06:00
bafdd8512d Enforce pre-summary newline in docstrings 2019-10-24 20:10:45 +02:00
b5eeceecbf Rename raw connection initiator flags 2019-10-24 19:28:42 +02:00
eef505f2d9 Add automatic docstring formatter and apply 2019-10-24 08:41:10 +02:00
92deae41dc Change SwarmConn.conn to muxed_conn 2019-09-23 15:46:50 +08:00
8d2415a404 Move calls to Notifee inside Swarm 2019-09-23 15:01:58 +08:00
6f8394e4bd Merge branch 'master' into fix/change-notifee-and-add-tests-for-swarm-conn-and-mplex 2019-09-21 18:34:12 +08:00
e44c2145cc Merge branch 'master' into fix/detection-of-close 2019-09-21 18:05:54 +08:00
89c127eff4 Merge branch 'master' into fix/refactor-mplex-swarm-host 2019-09-20 15:45:28 +08:00
7fc958e7be Add exception raised to docstring 2019-09-19 22:19:36 +08:00
a9ad37bc6f Add mplex tests and fix error in SwarmConn.close 2019-09-19 16:31:41 +08:00
d61327f5f9 Add tests for SwarmConn 2019-09-19 16:31:41 +08:00
5307c0506b Change IMuxedConn to INetConn in Notifee 2019-09-19 16:31:41 +08:00
e7304538da Add test for Swarm.close_peer 2019-09-19 16:31:12 +08:00
2d8e02b7eb Add detection for disconnections in mplex 2019-09-19 15:55:26 +08:00
7cf0495f37 Remove print 2019-09-19 15:38:38 +08:00
a7bc9fc358 Asynchronously handling the accepted stream. 2019-09-19 13:59:51 +08:00
6cb033fd1f Refactor multiselect out of Swarm to BasicHost 2019-09-19 13:59:50 +08:00
0bd213bbb7 Refactor mplex and start to add close detection 2019-09-19 13:56:05 +08:00
4cd5b77f10 Raise RawConnError in RawConnection 2019-09-16 18:37:00 +08:00
cb632fa509 Add RawConnError 2019-09-16 18:35:48 +08:00
8e8318aa5c Introduces IO abstractions apart from asyncio or those attached to IRawConnection
Also adds `msgio` utilities to mirror the Go implementation
2019-09-03 21:59:50 -07:00