Performance Primer

It's easy to achieve great performance with YapDatabase. You just need to understand the basics behind connections and transactions.

Connections & Concurrency

With YapDatabase you can create multiple connections. Each connection is thread-safe. But...

It is important to understand that thread-safe != concurrency. Concurrency comes through using multiple connections.

For example, consider the following bad code:

@implementation MyUITableViewController {
    YapDatabaseConnection *databaseConnection;
}

// ...
- (UITableViewCell *)tableView:(UITableView *)sender cellForRowAtIndexPath:(NSIndexPath *)indexPath
{
    __block BNBook *book = nil;
    [databaseConnection readWithBlock:^(YapDatabaseReadTransaction *transaction) {    
        book = [[transaction ext:@"order"] objectAtIndex:indexPath.row inGroup:@"books"];
    }];
    
    // create and configure cell using book...
    return cell;
}

- (void)missingBooksDidDownload:(NSArray *)books
{
    // BAD CODE!
    [databaseConnection asyncReadWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction) {
        
        for (BNBook *book in books) {
            [transaction setObject:book forKey:book.uuid];
        }
    }];
}

Why is this bad code? I'm doing an async operation to avoid disk IO on the main thread. What's the problem?

The above code will technically work, but the performance won't be what you want.

Every YapDatabaseConnection only supports a single transaction at a time.

Essentially, each YapDatabaseConnection has an internal serial queue. All transactions on the connection must go through the connection's serial queue. This includes both read-only transactions and read-write transactions. It also includes async transactions.

Thus, in the bad code example above, we are blocking the main thread by blocking the single databaseConnection. That is, we are doing an "expensive" (disk IO) read-write transaction on the 'databaseConnection', which will block the 'databaseConnection' from doing other transactions until it completes.

A read-write transaction on connectionA will block read-only transactions on connectionA until the read-write transaction completes.

To achieve concurrency we use multiple connections. Here's the good code:

@implementation MyUITableViewController {
    YapDatabaseConnection *uiConnection;
    YapDatabaseConnection *bgConnection;
}

// ...
- (UITableViewCell *)tableView:(UITableView *)sender cellForRowAtIndexPath:(NSIndexPath *)indexPath
{
    __block BNBook *book = nil;
    [uiConnection readWithBlock:^(YapDatabaseReadTransaction *transaction) {    
        book = [[transaction ext:@"order"] objectAtIndex:indexPath.row inGroup:@"books"];
    }];
    
    // create and configure cell using book...
    return cell;
}

- (void)missingBooksDidDownload:(NSArray *)books
{
    // GOOD CODE.
    [bgConnection asyncReadWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction) {
        
        for (BNBook *book in books) {
            [transaction setObject:book forKey:book.uuid];
        }
    }];
}

Connections & Cost

You can achieve concurrency by using multiple connections. And it's just so darn easy to create a connection that it becomes easy to forgot that connections aren't free.

Here's some more bad code:

- (UITableViewCell *)tableView:(UITableView *)sender cellForRowAtIndexPath:(NSIndexPath *)indexPath
{
    // HORRIBLE CODE!
    __block BNBook *book = nil;
    [[database newConnection] readWithBlock:^(YapDatabaseReadTransaction *transaction) {    
        book = [[transaction ext:@"order"] objectAtIndex:indexPath.row inGroup:@"books"];
    }];
    
    // create and configure cell using book...
    return cell;
}

- (void)missingBooksDidDownload:(NSArray *)books
{
    // BAD CODE!
    [[database newConnection] asyncReadWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction) {
        // ...
    }];
}

- (void)didDownloadUpdatedBook:(BNBook *)book
{
    // BAD CODE!
    [[database newConnection] asyncReadWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction) {
        // ...
    }];
}

- (void)didUpdateTop20List:(NSArray *)books
{
    // BAD CODE!
    [[database newConnection] asyncReadWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction) {
        // ...
    }];
}

You should consider connections to be relatively heavy weight objects.

OK, truth be told they're not really that heavy weight. I'm just trying to scare you. Because in terms of performance, you get a lot of bang for your buck if you recycle your connections.

Many of the performance optimizations within the YapDatabase architecture are within connections. But you only get these optimizations if your connection sticks around for awhile.

One example of this is the cache. Each YapDatabaseConnection maintains its own cache. So if you have a connection that sticks around for awhile, it will automatically start populating its cache with the objects you're accessing. This is particularly beneficial for situations like fetching objects for a UITableView. Because it means that scrolling around in a UITableView is going to be hitting the cache almost the entire time. And hitting the cache is fast. Really really fast.

Another example is pre-compiled sqlite statements. In order to execute a statement in SQL, the sqlite engine has to parse the SQL text statement into a byte-code program. This is called compiling the SQL statement. And the resulting "compiled statement" is explicitly tied to a single sqlite connection. So the very first time you call setObject:forKey: the YapDatabaseConnection has to ask sqlite to compile the routine. The second time you call setObject:forKey: the connection can recycle the pre-compiled statement. Thus subsequent transactions on the same connection skip this overhead and inherit a little performance boost.

Additionally, maintaining a persistent connection allows you to configure it to match your needs.

Here's the good code:

@implementation MyTableViewController {
    YapDatabaseConnection *uiConnection;
    YapDatabaseConnection *bgConnection;
}

- (void)viewDidLoad
{
    uiConnection = [[AppDelegate database] newConnection];
    bgConnection = [[AppDelegate database] newConnection];
    
    uiConnection.objectCacheLimit = 500; // increase object cache size
    uiConnection.metadataCacheEnabled = NO; // not using metadata on this connection

    bgConnection.objectCacheEnabled = NO; // don't need cache for write-only connection
    bcConnection.metadataCacheEnabled = NO; // don't need cache for write-only connection
}

- (UITableViewCell *)tableView:(UITableView *)sender cellForRowAtIndexPath:(NSIndexPath *)indexPath
{
    // Good code.
    __block BNBook *book = nil;
    [uiConnection readWithBlock:^(YapDatabaseReadTransaction *transaction) {    
        book = [[transaction ext:@"order"] objectAtIndex:indexPath.row inGroup:@"books"];
    }];
    
    // create and configure cell using book...
    return cell;
}

- (void)missingBooksDidDownload:(NSArray *)books
{
    // Good code.
    [bgConnection asyncReadWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction) {
        // ...
    }];
}

- (void)didDownloadUpdatedBook:(BNBook *)book
{
    // Good code.
    [bgConnection asyncReadWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction) {
        // ...
    }];
}

- (void)didUpdateTop20List:(NSArray *)books
{
    // Good code.
    [bgConnection asyncReadWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction) {
        // ...
    }];
}

Read-Only vs Read-Write Transactions

A read-only transaction is fundamentally different from a read-write transaction.

As mentioned above, each YapDatabaseConnection can only perform a single transaction at a time. Concurrency comes from using multiple connections. However...

A database can only perform a single read-write transaction at a time.

This is an inherit limitation of sqlite. And it means that even if you have multiple YapDatabaseConnection's, all your readWrite transactions will execute in a serial fashion.

Recall that each YapDatabaseConnection has a serial queue, and that all transactions on that connection go through the connection's serial queue. In a similar fashion, YapDatabase has a serial queue for read-write transactions, and all read-write transactions (regardless of connection) must go through this "global" serial queue.

Never use a read-write transaction if a read-only transaction will do.

Consider the following bad code:

@implementation MyUITableViewController {
    YapDatabaseConnection *uiConnection;
    YapDatabaseConnection *bgConnection;
}

// ...
- (UITableViewCell *)tableView:(UITableView *)sender cellForRowAtIndexPath:(NSIndexPath *)indexPath
{
    // BAD CODE !!!
    __block BNBook *book = nil;
    [uiConnection readWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction) {    
        book = [[transaction ext:@"order"] objectAtIndex:indexPath.row inGroup:@"books"];
    }];
    
    // create and configure cell using book...
    return cell;
}

- (void)missingBooksDidDownload:(NSArray *)books
{
    [bgConnection asyncReadWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction) {
        
        for (BNBook *book in books) {
            [transaction setObject:book forKey:book.uuid];
        }
    }];
}

Even though we have multiple connections, we are still blocking the main thread because we're using an unnecessary read-write transaction.

The great thing about a read-only transaction on connectionA is that it can execute in parallel with a read-write transaction on connectionB.

Thus the following "best practice" is recommended to achieve great performance, and to prevent ever blocking the main thread:

Use a dedicated connection for the main thread
Do not use this connection anywhere but on your main thread
Do not execute any readWrite transactions with this connection
Only execute read-only transactions with this connection
Create separate YapDatabaseConnection(s) for readWrite operations
Use these separate connections to do your readWrite transactions

Remember, these are just performance tips. It's perfectly legal to do a readWrite transaction on the main thread. YapDatabase doesn't care. The philosophy behind these tips is simply to avoid expensive disk writes on the main thread, as a matter of general best practice.

The fix for the bad code example is easy. We were almost doing everything correct.

The good code:

@implementation MyUITableViewController {
    YapDatabaseConnection *uiConnection;
    YapDatabaseConnection *bgConnection;
}

// ...
- (UITableViewCell *)tableView:(UITableView *)sender cellForRowAtIndexPath:(NSIndexPath *)indexPath
{
    // Good code.
    __block BNBook *book = nil;
    [uiConnection readWithBlock:^(YapDatabaseReadTransaction *transaction) {    
        book = [[transaction ext:@"order"] objectAtIndex:indexPath.row inGroup:@"books"];
    }];
    
    // create and configure cell using book...
    return cell;
}

- (void)missingBooksDidDownload:(NSArray *)books
{
    [bgConnection asyncReadWriteWithBlock:^(YapDatabaseReadWriteTransaction *transaction) {
        
        for (BNBook *book in books) {
            [transaction setObject:book forKey:book.uuid];
        }
    }];
}

Armed with these concepts you can easily achieve concurrency and performance. And (just as important) you can avoid blocking your main thread.

Once you have these concepts down cold, you'll be ready to move onto the next optimizations:

Configuring your dedicated main-thread connection to use LongLivedReadTransactions.
And reducing UI updates by using YapDatabaseModifiedNotification.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Performance Primer

Connections & Concurrency

Connections & Cost

Read-Only vs Read-Write Transactions

Clone this wiki locally