Repository navigation
Description
Problem
Eight test fixtures bind a std::net::TcpListener and call set_nonblocking(true) on it, so the accept loop can poll a deadline. They then read the accepted stream as if it were blocking, relying on set_read_timeout.
The two platforms treat the accepted socket differently:
- On Linux,
accept(2)does not copyO_NONBLOCKfrom the listener, so the read waits up to its timeout. - On macOS it does, so the read returns
WouldBlockat once whenever the client's bytes have not arrived yet.
Under parallel test load the fixtures therefore fail on macOS with WouldBlock, or with "SDK did not contact fixture". Which tests fail changes from run to run. CI runs only on ubuntu-latest, so CI never shows this.
The probe below, run on macOS (Darwin 27.2.0), shows the difference. It accepts from a non-blocking listener, sets a 2 s read timeout and reads with no data sent. The read returns WouldBlock after 1.5 µs.
let listener = TcpListener::bind("127.0.0.1:0")?;
listener.set_nonblocking(true)?;
let _client = TcpStream::connect(listener.local_addr()?)?;
// accept() polled until a stream arrives
accepted.set_read_timeout(Some(Duration::from_secs(2)))?;
accepted.read(&mut [0_u8; 1]); // macOS: Err(WouldBlock) at onceAffected fixtures
| File | Added by |
|---|---|
crates/edgezero-store-aws/src/secrets_manager.rs |
#414 |
crates/edgezero-store-aws/src/dynamodb_tests.rs |
#414 |
crates/edgezero-adapter-axum/tests/native_aws_startup.rs |
#414 |
crates/edgezero-cli/tests/native_store_generated_proof.rs |
#414 |
crates/edgezero-cli/tests/dynamodb_generated_proof.rs |
#414 |
crates/edgezero-store-celld/src/client.rs |
#422 |
crates/edgezero-adapter-axum/tests/native_celld_startup.rs |
#422 |
crates/edgezero-cli/tests/celld_generated_proof.rs |
#422 |
Expected behavior
Each fixture puts every accepted std stream back into blocking mode, or reads through an API that waits, before it relies on read and write timeouts. The tests then behave the same on macOS as on Linux.
Also failing on macOS, cause not yet traced
Four production_hosting tests fail on macOS through tests/fixtures/native_proxy.rs (#409), and they also fail on #409's own head:
native_selected_families_conflicts_malformed_sets_gaps_and_exhaustionnative_source_bound_single_proxy_and_mapped_cidr_match_only_owned_peersnative_two_proxy_chain_stops_at_untrusted_hop_and_gateway_replaces_prefixnative_verified_tls_ingress_overwrites_public_assertions_and_rejects_wrong_name
They get an empty response, or a TLS UnexpectedEof. native_proxy.rs accepts through tokio, so the cause above does not explain them.
Acceptance
- On macOS (Apple Silicon), with the default test thread count, these all pass:
edgezero-store-awsandedgezero-store-celldwith all features,native_aws_startup,native_celld_startup, and the generated celld, DynamoDB and native-store proofs. - The four
production_hostingtests either pass on macOS, or are documented as Linux-only with the reason.
Found while building #495.
Activity