logging needs filtering/reduction about flux-core HOT 4 CLOSED

Here's an idea that might not make sense:

We might need something similar for a "simple rsh" implementation to handle stdout/err.
Imagine if the flux rsh RANKS COMMAND... frontend worked something like:

• generate a unique ID for the current run
• subscribe to "log" stream for ID
• send rsh.execute or similar command with json decsription of command+environment
• stderr/out "log" messages would be copied back to stderr/out of flux rsh command -- other log messages could be optionally displayed based on --verbose. Collapsed lines could optionally be expanded by flux rsh
• exit code(s) could come back as CMB replies, or perhaps specially formatted log messages

Does this make any sense? Maybe it doesn't make sense to derive the flux rsh protocol from the
log implementation, but instead think of a lower abstraction from which both rsh and log services
are derived?

from flux-core.

Could we just use the existing logging interface on the rshd end, e.g.

flux_log_set_facility (h, "rsh-%d", rsh_jobid);
flux_log (h, LOG_INFO, "%s", stdout_line)
flux_log (h, LOG_ERR, "%s", stderr_line)

Then we would just need a way for the rsh end to subscribe to messages sent to that facility. Are we OK with presuming that stdio will be consumed on rank 0? If so maybe part of the log design could be an ipc:// socket that all logs are published to, with PUB-SUB topic string derived from the facility. Then rsh could connect to the socket and subscribe to its particular rsh_jobid.

The flux-snoop utility works with a "snoop socket" in pretty much this way now.

from flux-core.

With the "reduction handle" improvements in pr #298, I was thinking perhaps this issue should be revisited. Since TIMEDWAIT is the obvious "flush policy" for compressing identical log messages, and flux_reduce_t requires the flux reactor for installing internal timer watchers, the fact that the broker still uses zloop is an impediment.

I've opened #320 to remind us to get off zloop in the broker.

from flux-core.

#320 is no longer a blocker, but this feels to me a bit like premature optimization and furthermore, is a fairly obvious possibility so I don't think needs an issue to remind us. Closing.

from flux-core.