Constructing our own Console.app: A Custom OSLog facility viewer

Introduction: Apple’s Unified Logging System

Famous with developers, ‘printf debugging’ is one of the most common methods of trying to diagnose issues, usually done by using the builtin language function to log to standard output/input.

However Apple takes it a step further by having their own unique logging System called the Unified Logging System, which acts as a replacement for standard logging function. Instead of just outputting the message given, the Unified Logging System has a couple of features, including:

All of these factors combined result in an overall better, less-messy debugging experience, with an API almost identical to that of print/printf in most languages, for example:

#include <os/log.h>

os_log_t log = os_log_create(/* subsystem = */ "com.antoine.MyApp.UserInterface",
							 /* category = */ "UIHandling");
os_log(log, "Laying out user interface (file=%s)", #file);

or in Swift:

import OSLog

let logger = Logger(subsystem: "com.antoine.MyApp.UserInterface", category: "UIHandling")
logger.log("Laying out user interface (file=\(#file))")

When we call os_log or logger.log, our message gets logged to the Apple Unified Logging System.

On macOS, you can open a pre-installed System app (Console.app) to view all logs happening in real time:

All logs in real time

In the image, you can see a multitude of System log messages, with information such as the timestamp, process, and message etc. Tapping on an individual log message will reveal more detailed information about it.

Now that we have understood Apple’s Unified Logging System, let’s get to the part where we make our own OSLog viewer — simply put, let’s make our own Console.app (but for iOS!) that shows a stream of System Logs happening in real time.

Note: this article is meant to be a summary and writeup of using the OSLog streaming APIs, not a 1-to-1 guide on building an app that does this.

Creating our own OSLog Stream & Viewer App

Goals

First, to create our own OSLog Viewer app, lets establish some goals. The app should be able to:

Getting started

To start, we first need to figure out a way to actually connect to the stream of System logs and somehow receive them. Let’s find a System binary which does this on macOS and reverse it (you may be asking, why not Console.app? Because a Command Line Tool would have substantially less code to reverse through than a full fledged UI Application). Fortunately, there is a simple command line tool on macOS which does this, called log (/usr/bin/log). Upon reversing log, the source of it’s log stream is straightforward: A private System Framework called LoggingSupport.framework, using an Objective-C class called OSActivityStream.

OSActivitStream in IDA

log creates an instance of OSActivityStream, then assigns a delegate to OSActivityStream called StreamDelegate (very creatively named), and handles received logs with the - (BOOL) activityStream:(OSActivityStream *)stream results:(NSArray *)results method.

If we delve further and reverse LoggingSupport.framework, we can actually see that OSActivityStream is a wrapper class around C functions from the very same framework. To create the stream, it uses a C function, called os_activity_stream_for_pid. (See code below in the -[OSActivityStream startLocal] method)

if we google os_activity_stream_for_pid, quite a few results come up! In fact, Apple used (to use?) this private API publicly in LLVM here (Note: I didn’t know at the time of making my OSLog facility viewer a year ago, but it turns out that this API was actually documented before by Peter Steinberger).

Using what we’ve seen online as well reversed LoggingSupport.framework code, the steps to making a Stream seems to be as follows:

Now lets see how that actually looks in practice.

Crafting our Stream

To create our stream, we need to call os_activity_stream_for_pid, this function takes in 3 parameters:

Now that we know how to make our stream, let’s start writing some of the code for it:

// import this bridging header first: https://github.com/NSAntoine/Antoine/blob/b1be43a6ae235d6e3154c2f14336f4c2cd43fa59/Antoine/Backend/Bridge/ActivityStreamAPI.h

// callback to handle new messages
// os_activity_stream_block_t is defined as 
// typedef bool (^os_activity_stream_block_t)(os_activity_stream_entry_t _Nonnull entry, int error);
// or, in Swift, 
// (os_activity_stream_entry_t, Int) -> Bool

let messageHandler: os_activity_stream_block_t = { (entry, error) in
   print("Got new entry \(entry)") // print the entry as an example, we will elaborate and show useful details in the next section
   return true /* The callback requires we return a Boolean value, guessing this is for whether or not the Stream should continue */
}

let activityStream = os_activity_stream_for_pid(-1, 0, messageHandler)
os_activity_stream_resume(activityStream) /* start the Stream */

In the code above, we created our callback for receiving entries, creating the stream itself, and starting it. Now, we need to register for when major events (such as when the Stream starts, stops, or fails). We can do this with os_activity_stream_set_event_handler:

// declare messageHandler, etc
let activityStream = os_activity_stream_for_pid(-1, 0, messageHandler)

let majorEventBlock: os_activity_stream_event_block_t = { ourStream, majorEvent in 
   /* 
   we got a major event here, update our app UI accordingly 
   but for the sake of brevity, we will just print what happened
   */
   switch majorEvent {
	case 1:
		print("Stream started")
	case 2:
	   print("Stream stopped")
	case 3:
		print("Stream failed")
	default:
	   break
   }   
}

os_activity_stream_set_event_handler(activityStream, majorEventBlock)

/* start stream etc */

Now that we have crafted and started our Stream, we need to actually process the information of each OSLog entry that we receive, which we will do in the next section

Processing Entries

Now that we receive entries, it’s time for us to actually get the information out of them which we’d want to present, such as:

Recall earlier that we receive entries in the os_activity_stream_block_t. In that callback, we receive each entry as a pointer to a C struct called os_activity_stream_entry_s, unfortunately, accessing info such as the message seems to be unstable and sometimes crash when using the struct, so instead, to access the data we need, we will use an Objective-C wrapper class called OSActivityLogMessageEvent from the same framework, which we can initialize with the entry struct we have.

@interface OSActivityLogMessageEvent : NSObject

// Start inherited properties
@property (copy, nonatomic) NSString *eventMessage;

// The 3 properties below point to the process that outputted the OSLog message, even if it came from a library/framework that the process uses, it'll point to info abt the process
@property (readonly, copy, nonatomic) NSString *process;
@property (readonly, copy, nonatomic) NSString *processImagePath;
@property (readonly, nonatomic) pid_t processID;

// The sender is a bit more specific on *where* the message came, ie, if it came from a library/framework, it'll describe the library/framework
@property (readonly, copy, nonatomic) NSString *sender;
@property (readonly, copy, nonatomic) NSString *senderImagePath;

@property (readonly, copy, nonatomic) NSDate *timestamp;
// End inherited properties

// The 3 following properties, unlike the ones above (which are derived from superclass OSActivityEvent), 
// are exclusive to OSActivityLogMessageEvent.
@property (readonly, copy, nonatomic) NSString * _Nullable category;
@property (readonly, copy, nonatomic) NSString * _Nullable subsystem;
@property (readonly, nonatomic) unsigned char messageType;

-(instancetype _Nonnull)initWithEntry:(struct os_activity_stream_entry_s * _Nonnull)arg0 ; // Create instance with our struct pointer

@end

Now, after we’ve imported that class, lets go back to the original Stream code we wrote above, and instead, change the messageHandler callback to this:

// callback to handle new messages
let messageHandler: os_activity_stream_block_t = { (entry, error) in
   // Now, we can get useful details from the entry.
   let logEvent = OSActivityLogMessageEvent(entry: entry)
   print("Log event message: \(logEvent.eventMessage)")
   print("Log event process: \(logEvent.process)")
   print("Log event timestamp: \(logEvent.timestamp)")
   
   return true
}

Now, we have written everything we need for the OSLog streams, we can:

(Initial) Restrictions

If we were to import the Objective-C & C symbols into Swift, and then run the code that we have written on a jailbroken device, you’d find out that nothing is logged yet! This is due to a restriction on Apple’s part called Entitlements, which is an XML embedded into every binary on Darwin platforms that describes the capabilities of that binary. To fix our issue, we need to find the needed entitlements and sign our binary with them. Thankfully, we can do this easily by just viewing entitlements of Console.app:

❯ ldid -e /System/Applications/Utilities/Console.app/Contents/MacOS/Console
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>com.apple.private.logging.diagnostic</key>
	<true/>
	<key>com.apple.private.logging.stream</key>
	<true/>
</dict>
</plist>

So, it seems that we just need 2 keys: com.apple.private.logging.stream and com.apple.private.logging.diagnostic. If we sign our iOS binary with those entitlements on a jailbroken device, then we will see that it actually does start showing System logs!

<private> Data!

Remember how we mentioned earlier that you can redact private variables using OSLog? This means that when you’re debugging your application with a debugger like Xcode (using LLDB), you can see the value of these variables, however, when streaming messages any other way (such as with Console.app, or with our own app), the value of these variables is instead substituted by <private>. For example: Example of a message with private data

Unredacting the <private> values is actually a very simple task: all we have to do is call host_set_atm_diagnostic_flag with a certain flag, see:

func enableShowPrivateData() {
    let privateDataFlag: UInt32 = 1 << 24
    
	// Add priv data flag
	let kret = host_set_atm_diagnostic_flag(mach_host_self(), privateDataFlag)
	
    if kret != KERN_SUCCESS {
	    let error = String(cString: mach_error_string(kret))
        NSLog("\(#function): Failed to set enable private data flag, error: \(error))")
    }
}

And with that, voila! We can now un-redact <private> values in entries! (Thanks to daniel_levi for letting me know of this method, which Saagar Jha initially wrote). Unfortunately, this does not work anymore for macOS since it is locked behind an entitlement (com.apple.private.set-atm-diagnostic-flag), however, since we are in the context of Jailbroken iOS land, we can grant ourselves just about any entitlement we want.

(Initial) Interface

Now that we have complete functionality for our app, the next part is the interface. What I had in mind was a list interface which updates in real time (using UICollectionView), similar to the macOS Console.app. In the very beginning of development, the interface looked something like this:Initial user interface at the start of development As shown in the image, the first iteration of the UI tries to be descriptive yet concise, however there was one simple issue: on desktop, Console.app can show a variety of information with each cell, such as timestamp, process and message. However, on a phone, you have limited space horizontally and cannot fit in most of that information all at once. In the image above, 3 pieces of information is shown within each cell, being:

Clicking on a cell would bring up a View Controller that showed all information about the log entry, therefore, I felt as though the cells should only show the necessary information, and in the end, the final design for the list was this: Final design list

The list would have a toolbar with controls, such as pausing/resuming, filtering, scrolling all the way down, and clearing all logs. The cells contained 2 pieces of info this time: the text message, and the process name, with the process name being emphasized. Tapping on a cell would show this detailed View Controller

Information regarding Entry’s messageInformation regarding about stuff like Process, SenderInformation regarding stuff like Category, subsystem, time, etc

Conclusion

Now, we have created our OSLog streaming app with the desired functionality using the LoggingSupport.framework API. The complete source code of the project is on Github, with more features such as filtering entries by level, message, subsystem/category, etc.