[ShawnMcKee]

Fixes #8307.

Problem

A firefly ties a network flow to an experiment and an activity. Its flow-id must therefore name the TCP connection that carries the data.

For xrootd and http(s) transfers that a door redirects to a pool, it does not. NettyTransferService.executeMover() sends the start marker when the mover is created, before the client has connected to the pool, and takes the remote endpoint from the protocol info. That is the client's connection to the door. The client then connects to the pool from a different port:

Connection	Client end	dCache end	Carries
1. Control	client address, port B	door	open request, redirect
2. Data	client address, port D	pool, port C	the file
Marker today	address the door saw, port B	pool, port C

The marker carries one port from each connection and matches no flow. The end marker repeats the same endpoint. Details and measurements are in #8307.

Fix

• NettyTransferService: the start marker is no longer sent from executeMover(). NettyMoverChannel gains connected(remote, local). The first connection reported is recorded on the mover and announced with the start marker.
• HttpPoolRequestHandler calls it for GET and PUT, and XrootdPoolRequestHandler when a file is opened for reading or writing, with the addresses of the Netty channel.
• Mover#getRemoteEndpoint is added, and DefaultPostTransferService uses it for the end marker, so both markers of a transfer carry the same flow-id. This hunk is identical to the one in pool: report the data connection in firefly markers of HTTP-TPC transfers #8306, so the two changes merge in either order.
• Mover#isPassive is added and is true for NettyMover. For a passive mover that no client connected to, no end marker is sent: there was no flow and no start marker.

Behaviour changes to be aware of

• Timing. The start marker is sent when the client has connected to the pool, not before. The first packets of the connection therefore precede it. The client's port cannot be known earlier.
• Movers nobody connects to emit no markers. Before, they emitted a start marker for a flow that never came into existence.
• Further connections to the same mover (keep-alive requests, reconnects) are not announced. The markers describe the first connection.
• xrootd third-party copy with the pool as destination emits no markers. The file arrives over a connection that the pool's third-party client (TpcWriteDescriptor) opens to the source. The requesting client's connection carries no file data, so it is not reported. Before this change the start marker named the requesting client's connection to the door and the end marker named the source host from the request, neither of which is the data connection. I intend to follow up by having the third-party client report its own connection through the same connected() call.

Testing

• New NettyTransferServiceFlowMarkerTest (6 tests): the marker carries the endpoints of the reported connection, is sent once, is not sent before a connection is reported, and non-IP channels are ignored.

• Ran the PR pipeline's unit-test job locally in maven:3.9.12-eclipse-temurin-21 (mvn -P code-coverage clean verify -DskipTests=false -Drun.slow.tests): 5178 tests, 0 failures, 0 errors.

• Tested on a production pool. The commit cherry-picks cleanly onto 11.2.10. That build ran on an AGLT2 pool node (two pools). TCP connections arriving at the mover ports were captured for 10 minutes and compared with the endpoints the pools logged for each transfer:

  	Before	After
  Client transfers started	55	65
  Client port in the marker matches the real connection	0	65
  Client port differs	54	0
  No connection from the logged address	1	0

  After the change, 64 of the transfers were xrootd and 1 https. Start-side and end-side log lines carried the same flow-id in 67 of 67 transfers. Two external transfers (one xrootd read, one https write) sent fireflies; all four were captured on the wire and named a connection seen in the capture. Most transfers in both runs were on-site, for which the marker is suppressed but its endpoints are logged.

• Not exercised on the pool: a mover that no client connects to, xrootd third-party copy, and an xrootd client reconnecting to a mover. The DefaultPostTransferService change for unconnected movers is covered only by compilation.

Signed-off-by: Shawn McKee smckee@umich.edu

🤖 Generated with Claude Code