Unexpected behaviour/result from TestableSubscriber

[rustproofFish]

I'm tinkering with persistence middleware for an app which uses Redux-like architecture using Realm as a backend. Results from queries on the database should be emitted by a Combine Publisher.

Although Realm now supports Combine and provides FrozenCollection types to manage threading and concurrency-related errors, I've been advised to use DTOs (structs) to make things as predictable as possible. I've written a test which utilises Entwine to ensure that the conversion of the Realm FrozenCollection to the DTO behaves as expected but the TestableSubscriber receives an empty array at time index 0 when it should in fact receive three inputs (initial empty array at t=0, test set of 3 elements at t = 100 and a second test set of 3 elements at t = 200).

I've reverted to testing using an Expectation as well as debugging the Publisher using print() and get the expected output (6 elements emitted in total, conversion to DTO working OK) so am working on the assumption that I am doing something wrong although I guess there is the outside possibility there's a bug?

Currently using Xcode12, iOS14. Code as follows:

Middleware (SUT)

import Foundation
import Combine

import RealmSwift

final class PersistenceMiddleware {
    private var cancellables = Set<AnyCancellable>()
    private var subject = CurrentValueSubject<[ToDo.DTO], Never>([])
    
    func allToDos(in realm: Realm = try! Realm()) -> AnyPublisher<[ToDo.DTO], Never> {
        realm.objects(ToDo.self)
            .collectionPublisher
//            .print()
            .assertNoFailure()
            .freeze()
            .map { item in
                item.map { $0.convertToDTO() }
            }
//            .print()
            .receive(on: DispatchQueue.main)
            .subscribe(subject)
            .store(in: &cancellables)
            
        return subject.eraseToAnyPublisher()
    }
    
    
    deinit {
        print("Deinit")
        cancellables = []
    }
}

Test class

import XCTest
import Combine

import RealmSwift
import EntwineTest

@testable import SwiftRex_ToDo_Persisted

class SwiftRex_ToDo_PersistedTests: XCTestCase {
    private var realm: Realm?
    
    private var testSet1: [ToDo] {
        [
            ToDo(name: "Mow lawn"),
            ToDo(name: "Wash car"),
            ToDo(name: "Clean windows")
        ]
    }
    
    private var testSet2: [ToDo] {
        [
            ToDo(name: "Walk dog"),
            ToDo(name: "Cook dinner"),
            ToDo(name: "Pay bills")
        ]
    }
    
    override func setUpWithError() throws {
        // Put setup code here. This method is called before the invocation of each test method in the class.
        realm = try! Realm(configuration: Realm.Configuration(inMemoryIdentifier: UUID().uuidString))
    }
    
    override func tearDownWithError() throws {
        // Put teardown code here. This method is called after the invocation of each test method in the class.
        realm = nil
    }
    
    func testCorrectNumberOfObjectsStoredInRealm() {
        realm!.addForTesting(objects: testSet1)
        XCTAssertEqual(realm!.objects(ToDo.self).count, 3)
        
        realm!.addForTesting(objects: testSet2)
        XCTAssertEqual(realm!.objects(ToDo.self).count, 6)
    }
    
    func testMiddlewarePublisherUsingEntwine() {
        let middleware = PersistenceMiddleware()
        let scheduler = TestScheduler()
        let subscriber = scheduler.createTestableSubscriber([ToDo.DTO].self, Never.self)

        middleware.allToDos(in: self.realm!)
            .subscribe(subscriber)
        
        scheduler.schedule(after: 100) { self.realm!.addForTesting(objects: self.testSet1) }
        scheduler.schedule(after: 200) { self.realm!.addForTesting(objects: self.testSet2) }

        scheduler.resume()

        print("\(subscriber.recordedOutput)")
    }
    
    func testMiddlewarePublisherUsingExpectation() {
        let middleware = PersistenceMiddleware()
        var cancellables = Set<AnyCancellable>()
        let receivedValues = expectation(description: "received expected number of published objects")
        
        middleware.allToDos(in: realm!)
            .sink { result in
                if result.count == 6 {
                    NSLog(result.debugDescription)
                    receivedValues.fulfill()
                }
            }
            .store(in: &cancellables)

        realm!.addForTesting(objects: testSet1)
        realm!.addForTesting(objects: testSet2)

        waitForExpectations(timeout: 1, handler: nil)
    }
    
}

Console output
Test Suite 'All tests' started at 2020-10-03 15:49:39.166
Test Suite 'SwiftRex-ToDo-PersistedTests.xctest' started at 2020-10-03 15:49:39.167
Test Suite 'SwiftRex_ToDo_PersistedTests' started at 2020-10-03 15:49:39.167
Test Case '-[SwiftRex_ToDo_PersistedTests.SwiftRex_ToDo_PersistedTests testCorrectNumberOfObjectsStoredInRealm]' started.
2020-10-03 15:49:39.182753+0100 SwiftRex-ToDo-Persisted[14502:643475] Version 5.4.7 of Realm is now available: https://github.com/realm/realm-cocoa/blob/v5.4.7/CHANGELOG.md
2020-10-03 15:49:39.231129+0100 SwiftRex-ToDo-Persisted[14502:643297] Setup
2020-10-03 15:49:39.232989+0100 SwiftRex-ToDo-Persisted[14502:643297] Tear down/n
Test Case '-[SwiftRex_ToDo_PersistedTests.SwiftRex_ToDo_PersistedTests testCorrectNumberOfObjectsStoredInRealm]' passed (0.067 seconds).
Test Case '-[SwiftRex_ToDo_PersistedTests.SwiftRex_ToDo_PersistedTests testMiddlewarePublisherUsingEntwine]' started.
2020-10-03 15:49:39.262702+0100 SwiftRex-ToDo-Persisted[14502:643297] Setup
2020-10-03 15:49:39.264956+0100 SwiftRex-ToDo-Persisted[14502:643474] [] nw_protocol_get_quic_image_block_invoke dlopen libquic failed
TestSequence<Array, Never>(contents: [(0, .subscribe), (0, .input([]))])
Deinit
2020-10-03 15:49:39.267422+0100 SwiftRex-ToDo-Persisted[14502:643297] Tear down/n
Test Case '-[SwiftRex_ToDo_PersistedTests.SwiftRex_ToDo_PersistedTests testMiddlewarePublisherUsingEntwine]' passed (0.033 seconds).
Test Case '-[SwiftRex_ToDo_PersistedTests.SwiftRex_ToDo_PersistedTests testMiddlewarePublisherUsingExpectation]' started.
2020-10-03 15:49:39.272248+0100 SwiftRex-ToDo-Persisted[14502:643297] Setup
2020-10-03 15:49:39.274820+0100 SwiftRex-ToDo-Persisted[14502:643297] [SwiftRex_ToDo_Persisted.ToDo.DTO(id: 0, name: "Mow lawn"), SwiftRex_ToDo_Persisted.ToDo.DTO(id: 1, name: "Wash car"), SwiftRex_ToDo_Persisted.ToDo.DTO(id: 2, name: "Clean windows"), SwiftRex_ToDo_Persisted.ToDo.DTO(id: 3, name: "Walk dog"), SwiftRex_ToDo_Persisted.ToDo.DTO(id: 4, name: "Cook dinner"), SwiftRex_ToDo_Persisted.ToDo.DTO(id: 5, name: "Pay bills")]
Deinit
2020-10-03 15:49:39.275235+0100 SwiftRex-ToDo-Persisted[14502:643297] Tear down/n
Test Case '-[SwiftRex_ToDo_PersistedTests.SwiftRex_ToDo_PersistedTests testMiddlewarePublisherUsingExpectation]' passed (0.008 seconds).
Test Suite 'SwiftRex_ToDo_PersistedTests' passed at 2020-10-03 15:49:39.277.
Executed 3 tests, with 0 failures (0 unexpected) in 0.108 (0.109) seconds
Test Suite 'SwiftRex-ToDo-PersistedTests.xctest' passed at 2020-10-03 15:49:39.277.
Executed 3 tests, with 0 failures (0 unexpected) in 0.108 (0.110) seconds
Test Suite 'All tests' passed at 2020-10-03 15:49:39.292.
Executed 3 tests, with 0 failures (0 unexpected) in 0.108 (0.126) seconds

[tcldr]

Hi @rustproofFish , thanks for the report. The issue is that EntwineTest isn't really suited for these kind of cross-boundary/integration tests yet, but more for unit tests where you want to create conditions that you have absolute control over.

There's two ways you could approach this:

1. Rather than interface with Realm directly you simulate a publisher that produces and behaves the way you would expect the actual publisher you're subscribing to would behave using TestablePublisher. This is similar to the idea of a mock object. So maybe you can use a TestablePublisher to output your Realm FrozenCollections? This also allow you to test failure conditions, delays, precise orderings, etc. Essentially you're simulating the way Realm feasibly could behave in a controlled, repeatable way – without risk of side effects.
2. If you really want to write an integration test (cross API boundaries) then I'm afraid that's waiting on resolution of the enhancement request documented at Issue Add toBlocking operator like RxSwift #4. In the mean time there's https://github.com/groue/CombineExpectations by @groue which behaves more like you're... expecting.

I go into a bit more detail on #28 and the enhancement request is issue #4. It's clearly something there is demand for so it's definitely on the radar.

[rustproofFish]

Thanks for the explanation - all makes sense. I'll keep an eye out for the enhancement request. Cheers :-)