[ShawnMcKee]

Fixes #8305.

Problem

For HTTP third-party copies, the flow-id of the firefly markers a pool emits does not describe the TCP connection that carries the data:

• The remote endpoint is the address the door resolved from the host name in the transfer URL (RemoteTransferHandler, new InetSocketAddress(_destination.getHost(), port)). If the remote server redirects the request, or its name has several addresses, the marker names a host the data does not go to. The copy of the marker for the remote peer is sent to that host as well.
• For uploads, RemoteHttpTransferClient reported a connection after each request had completed. A redirected request completes at once, so the start marker described the connection to the redirecting server while the end marker described the data connection. Without a redirect, the start marker was only sent after all data had been sent.

Observed on an AGLT2 pool (11.2.10), with fireflies and TCP connections captured together:

Push to a remote dCache	Local port	Remote	Remote port
Real: first PUT, redirected	55138	door A	2881
Real: data connection	44034	a pool	21468
start marker	55138	door B	2881
end marker	44034	door B	2881

In 15 minutes of production traffic, 12 of 13 pulls had a marker whose remote endpoint differed from the connection. Details are in #8305.

Fix

• RemoteHttpTransferClient: for an upload, onConnected is called just before the first byte of the entity is written, instead of after each request. With expect-continue the entity is not written to a server that redirects the request, so only the data connection is reported. Downloads are unchanged: the callback already runs after the redirects have been followed.
• RemoteHttpDataTransferProtocol: the start marker takes its remote endpoint from the connection (RemoteHttpTransferClient.remoteAddress()), as it already does for the local endpoint. It falls back to the address in the protocol info.
• Mover#getRemoteEndpoint and MoverProtocol#getRemoteEndpoint are added, and DefaultPostTransferService uses them for the end marker, so both markers carry the same flow-id.

This follows the approach of the in-review door-side change (commit 01d10cb, "webdav: send SciTags fireflies for transfers proxied by the door"), which takes the endpoints from the connection and marks a push at the first write of the entity. The two changes conflict only in RemoteHttpTransferClientTest, where both add a test at the same place.

Behaviour change to be aware of

An upload that is rejected before any data is sent no longer triggers onConnected. It gets no start marker, the mover reports no local endpoint for it, and its end marker falls back to the derived local endpoint and the protocol-info address, as for other movers without endpoint information.

Testing

• Two new tests in RemoteHttpTransferClientTest: the connection is reported before the upload completes, and for an upload redirected by a server that does not send "100 Continue" (as a dCache door behaves) only the data connection is reported. Both fail without the change to RemoteHttpTransferClient.
• Ran the PR pipeline's unit-test job locally in maven:3.9.12-eclipse-temurin-21 (mvn -P code-coverage clean verify -DskipTests=false -Drun.slow.tests): 5174 tests, 0 failures, 0 errors.
• Not tested on a pool. The change is against master; the pool I can test on runs 11.2, where this code is structured differently. The mover and DefaultPostTransferService changes are covered only by compilation and the existing tests.

Signed-off-by: Shawn McKee smckee@umich.edu

🤖 Generated with Claude Code