The origin of his research was the question of how macOS logs every restart in detail since Catalina, even though the separate data volume is only made possible when the user logs in. If a Mac admin checks later, all logs still appear on the data volume. Oakley discovered that a background process called logd_helper is responsible for this: it collects the accumulated messages from memory and preboot volume to add them to the log database. The actual messages are generated by the logd daemon.
Tidy up by size
Instead of storing all messages in text files, the Unified Log uses a database format that consists of documents with the ending “tracev3”; These can be found in a folder structure under /var/db/diagnostics. The logd service regularly ensures that old stocks are deleted. This apparently follows clear rules: In the “Special” folder, the individual files are only a few kilobytes in size, while the objects in the “Persist” folder are all around 10.5 MB in size. The number of files in the Persist folder is always 50, so the total size of the files is around 530 MB. According to Oakley, the “Special” folders contain files with a size of approximately two MB. Overall, the size of the diagnostics folder remains one to two gigabytes.
macOS stores the log files in database format under /var/db/diagnostics.
Oakley’s log analysis utilities
Since the introduction of Unified Log, the “Console” program is no longer suitable for subsequently searching through all log files. You first have to open the app and click “Start” so that the full extent of the logs appears here. Oakley has developed a free utility called LogUI, which provides insight into the log inventory. In addition, he recently published an app called Logstitician, which visualizes the Unified Log statistics integrated into macOS (after copying their jsonl files to another location and opening them using Logstitician).
Howard Oakley’s Logstitican visualizes where most error messages came from in the past.