codomains-eval.tex

\section{Codomains: Case Studies and Evaluation}
\label{sec:codomains-eval}

I explore programming with codomains through a series of application case
studies that highlight different scenarios and objectives, and propose methods
for measuring system performance and assessing behavioral correctness.


\subsection{Case Studies}

I explore three application case studies that differ in my expectation of the
amount of refactoring required to support a heterogeneous trust setting through
the use of codomains, starting with nearly transparent modifications, to
writing an application from scratch using codomains.


\parhead{Webserver}
%
My goal is to run a webserver on a third-party machine, and use codomains to
isolate the website's private TLS key.
%
My expectation is that this feature requires minimal application changes,
and that such changes may be transparent to the application if a taint-tracking
monitor can implicitly switch domains on behalf of the application during
private key accesses.


I evaluate the request throughput and latency of the webserver running in a
standard Linux environment, and compare against a configuration of the
webserver running with codomains.
%
I also compare the codomain version of the webserver to 
static partitioning schemes~\cite{eleos,glamdring,privtrans} that achieve
the same isolation.
%
When comparing to static partitioning schemes, I investigate not only the
performance differences, but also differences in the boundaries that
each scheme determines.
%
The benchmark affords multiple variants, such as investigating how codomains
scale with multiple tenants (multiple websites multiplexed by the same
webserver), how the system performs when the webserver itself contains private
data that must be cloaked from tenants, how well different server models (such
as multi-process and multi-threaded) interact with codomains, and how
host-codomains compare to enclave-codomains.


\parhead{Federated Databases}
%
I assume a \emph{federated database system}---a system of many
autonomous databases that share a schema and unite to appear as a single
engine for querying. 
%
I assume the federation spans mutually distrustful sources that are unable to 
aggregate their data on a neutral third-party.
%
My goal is to allow queries that preserve the privacy of each party's data by
modifying the database engine so that operators that compute solely on one
party's data execute on that party's database, and operators that compute on
data from multiple distrustful parties execute within an enclave-codomain.
%
As my concern is concealing the query's computation, I assume that the
system may use orthogonal approaches, such as query filters or different
privacy techniques, to guard against malicious queries.


My expectation is to isolate code modifications to the query planner and
executor, and that the database
applications themselves (the queries) remain unchanged. 
%
To that end, from the user's perspective, the system should behave
exactly like a conventional federated database where one submits SQL and
receives query results. 
%
Although the database system may require security policy at startup, which the
query planner would use in modeling the flow of sensitive data throughout its
operator tree, the user should not have to reason about how to combine the data
of each party.


I envision testing the system on synthetic medical data, and simulating a
federated system of partnering hospitals.
%
I imagine benchmarking and evaluating a set of representative clinical queries
that compute aggregate counts of disease comorbidity and treatment outcomes,
and comparing the query latencies to that of a monolithic database that
aggregates the data of all parties.



% TODO: what sort of data and queries are you going to explore?


%My goal is to modify the database's query planner so that relational operators
%that compute on  data from two distrustful parties execute within an
%enclave-codomain.


\parhead{Federated Learning}
%
I implement a suite of machine learning algorithms in a federated setting using
codomains, and compare performance against an existing federated framework,
such as TensorFlow-federated, as well as non-federated frameworks (which
assume all parties upload data to a single trusted party).
%
With federated learning, the result may be the global model parameters (such
as the weights of a deep neural network), derived from the secure aggregation of the
parameters from each party's locally-trained model.
%
In particular, each party locally trains in a party-specific codomain, and the
secure aggregation occurs in a shared enclave-codomain.


\subsection{Micro-benchmarks}

I want to isolate and measure the cost
of each facet of codomains---thread migration, taint tracking, the DSM
protocol---through a series of micro-benchmarks.
%
For such purposes, I envision developing a small set of multi-party programs
drawn from the literature, such as:

\begin{widelist}
\item \textbf{Timestamp signing server:} Returns a signature over the current
time to each client; I assume the server's key must be isolated from the
server's host.
%
\item \textbf{Richest protocol:} Computes the richest principal.
%
\item \textbf{Private set intersection:} Compute the intersection of each
    principal's data set, or the cardinality of that intersection.
\item \textbf{GPS protocol:} For each participating principal,
computes the peer that is nearest to their location; each principal learns
their nearest neighbor without learning anyone's exact location.
%
\item \textbf{Auction protocol:} Computes the high bidder among a set of
participating principals, as well as the second-highest bid, which is revealed
to everyone; only the auction holder learns the winning bidder.
\end{widelist}

For the timestamp signing server, I investigate how closely a
packet capture of the thread migration between codomains approximates
that of an alternative RPC-based implementation, such as keyless SSL\@.
%
In particular, how much traffic and how many round-trips does a codomain switch
incur, what is the purpose of each round-trip, and what is the corresponding
ratio of goodput to throughput?
%
Moreover, is each migration identical?


Closely related to the previous question, I seek to measure the latency of
migration and the related operations.
%
Direct overhead costs include network latency, while indirect costs include the
operations of checkpointing and restoring threads, as well as any TCP queuing
delays incurred by the source host while its thread is paused, waiting for
a return migration.


\subsection{Correctness}

I want to assert correctness of data cloaking and resource coherence.
%
For asserting correctness of data cloaking, I envision developing an external
program that dumps each process's memory and scans for a given byte string,
and running this program periodically and in tandem with the micro-benchmarks.
%
Similarly, asserting resource coherence may entail developing an external
manager that records a snapshot of the distributed system's global state, and
analyzes the state for consistency invariants.
%
Such a manager may also be useful in deadlock detection.
%
Finally, in order to test that codomains properly validate execution state when
restoring a thread, I imagine a set of fault-injection tests whereby one
codomain attempts to switch to another at an invalid state.