would like tooling for finding log files on the current system

[davepacheco]

We have lots of log files:

• our main services each have their current SMF log file: nexus, DNS, etc.
• every zone has SMF log files for various services
• CockroachDB has its own set of log files somewhere else

Live logs are in the corresponding zones and you can find most of them with svcs -L (but not the CockroachDB ones). For each of these we also have the archived logs. But because these are in zones that themselves have unique names, even on a single-system deployment, it can be hard to find, say, "the current log file for Nexus" or especially "all the Nexus log files for timestamp T".

Here's an idea: say we ship a GZ tool called oxlog:

# prints the names of the latest Nexus log files from all Nexus zones
oxlog --latest nexus 
# prints the names of the archived Nexus log files from all Nexus zones
oxlog --archived nexus
# prints the names all Nexus logs (current and archived) from all Nexus zones
oxlog --latest --archived nexus
# prints the names of all Nexus log files for a specific zone
oxlog --latest --archived --zone ZONE_ID nexus

I'm thinking we'd use this similar to how we use svcs -L, like:

# Search all Nexus log files for a particular request id
grep SOME_REQ_ID $(oxlog --latest --archived nexus)

# Watch activity for all Nexus instances on the system
tail -f $(oxlog --latest nexus)

Maybe it could take timestamp filters too? That's trickier to implement though.

[sunshowers]

Ran into this during my own time doing dogfood:

Finding the log was a bit of a challenge, required running find /pool/ext -name 'oxide-sled-agent:default.log*' | xargs ls -al | less and looking for a file mtime of Nov 30 in the output (note times are in UTC, so if were doing this later in the day it could have been Dec 1).

[andrewjstone]

PR open

[andrewjstone]

4810 has since been merged. oxlog is present at /opt/oxide/oxlog/oxlog in the global zone.