diff --git a/ExecutionBroker.tex b/ExecutionBroker.tex index b6fb256..a29bcc3 100644 --- a/ExecutionBroker.tex +++ b/ExecutionBroker.tex @@ -71,18 +71,14 @@ \newcommand{\executablething}[1] {\textit{executable~thing#1}} \newcommand{\excutabletask} {\textit{executable} task} -\newcommand{\execplan} {\textit{plan}} -\newcommand{\execoffer} {\textit{offer}} -\newcommand{\workerjob} {\textit{job}} +\newcommand{\execoffer}[1] {\textit{offer#1}} +\newcommand{\workerjob}[1] {\textit{session#1}} \newcommand{\teardown} {tear-down} \newcommand{\cpu}[1] {CPU#1} \newcommand{\gpu}[1] {GPU#1} \newcommand{\nvidiagpu} {NVIDIA~AD104~GPU} -\newcommand{\job} {\textit{job}} -\newcommand{\task} {task} - \newcommand{\scalable} {scalable} % TODO add a citation for the YAML specification. @@ -407,7 +403,7 @@ \subsection{Execution broker} \execbrokerclass{} services. Each \execbrokerclass{} service evaluates the request and responds with a top level \codeword{YES|NO} answer, and if -the answer is \codeword{YES}, a list of one or more offers describing how +the answer is \codeword{YES}, a list of one or more \execoffer{s} describing how the requested session could be executed on that platform. %\begin{lstlisting}[] @@ -417,9 +413,9 @@ \subsection{Execution broker} \includegraphics[width=0.9\textwidth]{diagrams/request-offers.pdf} -Each offer contains some metadata about the offer itself, +Each \execoffer{} contains some metadata about the \execoffer{} itself, including an identifier and an expiry time, -followed by an updated version of the session description +followed by a description of the \workerjob{} including details of when and how it would be executed. \begin{lstlisting}[] @@ -449,10 +445,10 @@ \subsection{Execution broker} .... \end{lstlisting} -The client can then ask its user to choose which of the offers best fits their requirements +The client can then ask its user to choose which of the \execoffer{s} best fits their requirements and sends a message to the \execbrokerclass{} accepting that offer. -In response, the \execbrokerclass{} will update the status of the offer to \codeword{ACCEPTED} and return a URL +In response, the \execbrokerclass{} will update the status of the \execoffer{} to \codeword{ACCEPTED} and return a URL pointing to a \workerjob{} in a \execworkerclass{} service, enabling the client to monitor the \workerjob{} status while it is executing. @@ -463,11 +459,11 @@ \subsection{Execution broker} \includegraphics[width=0.9\textwidth]{diagrams/accept-offer.pdf} -Once the client has accepted an offer, the conversation between the client and the \execbrokerclass{} -is finished. Once one offer has been accepted, the \execbrokerclass{} will cancel the other offers and +Once the client has accepted an \execoffer{}, the conversation between the client and the \execbrokerclass{} +is finished. Once one \execoffer{} has been accepted, the \execbrokerclass{} will cancel the other \execoffer{s} and release their resources. -The client does not need to cancel the offers made by the other \execbrokerclass{} services. -Offers are only valid for a limited period of time and they will expire naturally when they reach +The client does not need to cancel the \execoffer{s} made by the other \execbrokerclass{} services. +\execoffer{s} are only valid for a limited period of time and they will expire naturally when they reach the end of their lifetime. How the \execbrokerclass{} and \execworkerclass{} services are linked is up to the implementation architecture. @@ -511,7 +507,7 @@ \subsection{Execution worker} the \workerjob{} is \codeword{READY} in time for the \codeword{starttime} declared in the \execoffer{}. -If it will take 2 hours to transfer the input \dataset{} specified in the \execplan{} +If it will take 2 hours to transfer the input \dataset{} from cold storage to live storage co-located with the compute resources, then the \execworkerclass{} needs to start the \codeword{SETUP} phase at least 2 hours before the \codeword{starttime} declared in the \execoffer{}. @@ -529,7 +525,7 @@ \subsection{Execution worker} the \execworkerclass{} will change the \workerjob{} \codeword{phase} to \codeword{TEARDOWN} and begin the process of releasing the resources. -If the \execplan{} declared some persistent storage that will last beyond the end of the \workerjob{}, +If the \workerjob{} includes some persistent storage that should last beyond the end of the \workerjob{}, then part of the \teardown{} process may involve transferring results from the \workerjob{} onto the persistent storage before the local storage is released. @@ -888,8 +884,8 @@ \subsection{Minimum and maximum} to discover the best platform to execute the task. The client requests the minimun resources it needs, -and each service responds with a set of offers which specify the maximum -level of resources they can offer. +and each service responds with a set of \execoffer{s} which specify the maximum +level of resources they are able to provide. For example, if a platform is able to provide double the compute resources, 16 \cpu{} cores and 32G of memory, @@ -915,13 +911,13 @@ \subsection{Minimum and maximum} max: 32 \end{lstlisting} -This response represents an offer to start with a minimum of 8 \cpu{} cores and 16G of memory +This response represents an \execoffer{} to start with a minimum of 8 \cpu{} cores and 16G of memory as requested, with the option to use a maximum of 16 \cpu{} cores and 32G of memory if needed. -The client may receive different offers from different platforms and can pass this information -on the user to allow them to choose the offer that best fits their use case. +The client may receive different \execoffer{s} from different platforms and can pass this information +on the user to allow them to choose the \execoffer{} that best fits their use case. The notebook they have chose may specify a minimum of 8 \cpu{} cores and 16G of memory, -but an offer of twice the resources allows the user more scope for experimenting with +but an \execoffer{} of twice the resources allows the user more scope for experimenting with more data or more complex algorithms. This \scalable{} compute resource represents something like a \kubernetes{} platform where the @@ -930,7 +926,7 @@ \subsection{Minimum and maximum} This is slightly different to a platform like \openstack{} which allocates resources in specific blocks, defined by the set of \textit{'flavors'} available on that particular platform. If the smallest flavor of virtual machine available on the platform has 16 \cpu{} cores and 24G of memory, -then the service can represent that by setting the minimum values in its offer to match available resources. +then the service can represent that by setting the minimum values in its \execoffer{} to match available resources. \begin{lstlisting}[] # ExecutionBroker service response. @@ -952,9 +948,9 @@ \subsection{Minimum and maximum} max: 24 \end{lstlisting} -This response represents an offer to start with a fixed allocation of 16 \cpu{} cores and 24G of memory. +This response represents an \execoffer{} to start with a fixed allocation of 16 \cpu{} cores and 24G of memory. -An \execbrokerclass{} MAY NOT make an offer with less than the minimum resources requested. +An \execbrokerclass{} MAY NOT make an \execoffer{} with less than the minimum resources requested. For example, if an \openstack{} platform only has a virtual machine flavor with 1 \cpu{} core and 2G of memory, then it MAY NOT offer this resource as it is less than the requested minimum. @@ -980,8 +976,8 @@ \subsection{Storage resources} There are two types of internal storage resources: \begin{itemize} - \item Ephemeral storage available for the duration of the \job{}, created when the \job{} starts and released when the \job{} is completed. - \item Persistent storage that exists beyond the lifetime of the \job{}, created before the \job{} starts and remaining after the \job{} has completed. + \item Ephemeral storage available for the duration of the \workerjob{}, created when the \workerjob{} starts and released when the \workerjob{} is completed. + \item Persistent storage that exists beyond the lifetime of the \workerjob{}, created before the \workerjob{} starts and remaining after the \workerjob{} has completed. \end{itemize} There are two levels of persistent storage: @@ -1131,13 +1127,13 @@ \subsection{Storage resources} the client is asking the \execbrokerclass{} service to take care of allocating the storage and managing its lifecycle. It is up to the \execbrokerclass{} service to decide where to store the data and -how make it accessible after the \job{} has completed. +how make it accessible after the \workerjob{} has completed. For example, a science platform may have a \rucio{} storage system co-located with the compute platform which it uses to store user generated data. -In which case the \execbrokerclass{} service would respond with an offer that +In which case the \execbrokerclass{} service would respond with an \execoffer{} that stores the results in the \rucio{} system and provides details of how the user -can access it after the \job{} has completed. +can access it after the \workerjob{} has completed. % Type URLs % https://www.purl.org/ivoa.net/storage-types/rucio-storage @@ -1179,18 +1175,18 @@ \subsection{Storage resources} and in response the service is offering up to 220G available for 5 days stored in a \rucio{} system co-located close to the compute platform. It is up to the client to check if it can access the particular \rucio{} system -described in the response before it accepts the offer. +described in the response before it accepts the \execoffer{}. -Making an offer with the \codeword{lifecycle} set to \codeword{managed} and the +Making an \execoffer{} with the \codeword{lifecycle} set to \codeword{managed} and the \codeword{lifetime.max} set to \codeword{P5D} means that the service will manage the lifecycle. -The storage will be available for 5 days after the \job{} completes and then it +The storage will be available for 5 days after the \workerjob{} completes and then it will be deleted automatically. The client doesn't need to worry about tidying up afterwards. It is important to note that at this point in time the storage is proposed, but not yet allocated. -The persistent storage is only allocated if the client accepts this particular offer. -This allows an \execbrokerclass{} service to make multiple offers with different storage options, +The persistent storage is only allocated if the client accepts this particular \execoffer{}. +This allows an \execbrokerclass{} service to make multiple \execoffer{s} with different storage options, allowing the client to select and accept the one that best fits its use case. The same \datamodel{} can be used the other way around as well. @@ -1231,7 +1227,7 @@ \subsection{Storage resources} provided by the client. If it can access the \vospace{} location, then the \execbrokerclass{} MAY respond -with an offer, otherwise if it can't access the \vospace{} location for whatever +with an \execoffer{}, otherwise if it can't access the \vospace{} location for whatever reason the \execbrokerclass{} MUST respond with \codeword{NO}. Note that in this example, the client has specified the lifecycle as \codeword{unmanaged}, @@ -1263,7 +1259,7 @@ \subsection{Storage resources} In this example, the client is specifying an external \vospace{} location to store the data, but it is asking the \execbrokerclass{} service to manage the lifecycle, creating the location -in \vospace{} at the start of the \job{} and deleting it 2 days after the \job{} completes. +in \vospace{} at the start of the \workerjob{} and deleting it 2 days after the \workerjob{} completes. This negotiation of who is responsible for creating and deleting storage resources enables a client to put together a workflow of interconnected steps, with the @@ -1275,7 +1271,7 @@ \section{Authentication} \label{authentication} The client may also want to check whether the user account making the request -has sufficient access rights and resource quota needed to execute a \task{}. +has sufficient access rights and resource quota needed to execute it. This is equivalent to asking: \textit{"Do I have permission to use these resources to run this \jupyternotebook{}?"} @@ -1285,7 +1281,7 @@ \section{Authentication} The implicit method is for the client to authenticate to the \execbrokerclass{} service as normal and the service will use the identity from the request headers to check if the -user has sufficient access rights and resource quota to execute the \job{}. +user has sufficient access rights and resource quota. If the \webservice{} call to the \execbrokerclass{} service is authenticated using OpenIDConnect @@ -1338,7 +1334,7 @@ \section{Authentication} This pattern allows the client to supply multiple authentication methods in the request. The \execbrokerclass{} service can select the authentication methods it accepts from those in the request and include the name and -type of the methods in its offer along with enough non-secret +type of the methods in its \execoffer{} along with enough non-secret information to identify the authenticated identity. For example, if the client supplies both basic and token authentication @@ -1366,7 +1362,7 @@ \section{Authentication} If the \execbrokerclass{} service only accepts the token authentication, it should skip the basic authentication and only include a reference to the token -based authentication in its offer. +based authentication in its \execoffer{}. \begin{lstlisting}[] # ExecutionBroker server response. @@ -1389,19 +1385,19 @@ \section{Authentication} Response - NO \end{lstlisting} -The result is that an offer made by an \execbrokerclass{} service should only include details -of the authenticated identities that are allowed to use the offer. +The result is that an \execoffer{} made by an \execbrokerclass{} service should only include details +of the authenticated identities that are allowed to use the \execoffer{}. If the original question is equivalent to: \textit{"Do \textbf{these identities} have sufficient access rights and quota to run this \jupyternotebook{}?"} Then the response from the \execbrokerclass{} service is equivalent to: -\textit{"This offer asserts that \textbf{these identities} have sufficient access rights and quota to run this \jupyternotebook{}?"} +\textit{"This \execoffer{} asserts that \textbf{these identities} have sufficient access rights and quota to run this \jupyternotebook{}?"} -The client MUST use one of the authentication methods listed in the offer when -contacting the \execbrokerclass{} service to accept the offer and start the \job{}. +The client MUST use one of the authentication methods listed in the \execoffer{} when +contacting the \execbrokerclass{} service to accept the \execoffer{} and start the \workerjob{}. The client MUST continue to use the same authentication method when making subsequent -requests to the \execworkerclass{} service to access the \job{} status and results. +requests to the \execworkerclass{} service to access the \workerjob{} status and results. The \executionbroker{} specification includes an initial set of authentication methods corresponding to the methods defined in the @@ -1425,26 +1421,26 @@ \section{Date and time} \label{date-time} The \codeword{datetime} part of the \datamodel{} enables the client and server to have a -conversation about when a \job{} can be executed. +conversation about when a \workerjob{} can be executed. -The client can specify one or more time periods when it would like to start the \job{}, -and the minimum duration that it thinks would be needed to complete the \job{}. +The client can specify one or more time periods when it would like to start the \workerjob{}, +and the minimum duration that it thinks would be needed to complete the \workerjob{}. -The \execbrokerclass{} service may respond with one or more offers that specify when the \job{} -would start and the maximum duration that the \job{} would be allowed to consume. -It is then up to the client to select which of the offers best suits its use case. +The \execbrokerclass{} service may respond with one or more \execoffer{s} that specify when the \workerjob{} +would start and the maximum duration that the \workerjob{} would be allowed to consume. +It is then up to the client to select which of the \execoffer{s} best suits its use case. -The start time for a \job{} is expressed as an array of time intervals, as defined by +The start time for a \workerjob{} is expressed as an array of time intervals, as defined by ISO 8601 \citep{std:iso8601}. Specifically, type 1 or type 2 intervals (start/end and start/duration), excluding type 3 and 4 intervals (duration/end and duration only) and excluding repeats\footurl{https://www.iso.org/iso-8601-date-and-time-format.html}\footurl{https://en.wikipedia.org/wiki/ISO_8601}. Note that the duration part of the interval applies to the start time, specifying a -time range during which the \job{} may start. +time range during which the \workerjob{} may start. If no duration is specified, this means an absolute start time; -e.g. the \job{} SHOULD start at 11:30 on 14 August. +e.g. the \workerjob{} SHOULD start at 11:30 on 14 August. \begin{lstlisting}[] # ExecutionBroker client request. .... @@ -1452,9 +1448,9 @@ \section{Date and time} - start: "2023-08-14T11:30Z" \end{lstlisting} -If the start and end are specified, this means the \job{} SHOULD start somewhere between +If the start and end are specified, this means the \workerjob{} SHOULD start somewhere between the start and end values; -e.g. the \job{} SHOULD start between 11:30 and 12:00 on 14 August. +e.g. the \workerjob{} SHOULD start between 11:30 and 12:00 on 14 August. \begin{lstlisting}[] # ExecutionBroker client request. .... @@ -1462,9 +1458,9 @@ \section{Date and time} - start: "2023-08-14T11:30Z/T12:00Z" \end{lstlisting} -If a duration is specified, this means the \job{} SHOULD start somewhere between +If a duration is specified, this means the \workerjob{} SHOULD start somewhere between start and the start plus duration; -e.g. the \job{} SHOULD start between 11:30 and 12:00 on 14 August. +e.g. the \workerjob{} SHOULD start between 11:30 and 12:00 on 14 August. \begin{lstlisting}[] # ExecutionBroker client request. .... @@ -1472,16 +1468,16 @@ \section{Date and time} - start: "2023-08-14T11:30Z/PT30M" \end{lstlisting} -The \execbrokerclass{} service SHOULD respond with one or more offers that start within +The \execbrokerclass{} service SHOULD respond with one or more \execoffer{s} that start within the ranges specified in the request. -The start times in the offers MAY be more precise than the start times in the request, +The start times in the \execoffer{s} MAY be more precise than the start times in the request, but they MUST all occur within at least one of the ranges specified in the request. The client MAY also specify the maximum and minimum execution duration, expressed as time periods as defined by ISO 8601. For example, for the unattended batch mode execution of an \dockercontainer, the user might not be concerned about -when their \job{} starts, but they may want to specify the minimum duration needed to complete the task. +when their \workerjob{} starts, but they may want to specify the minimum duration needed to complete the task. In which case, the client may simply request a minimum duration of 1 hour. \begin{lstlisting}[] @@ -1492,7 +1488,7 @@ \section{Date and time} min: "P1H" \end{lstlisting} -The \execbrokerclass{} service MAY respond with offers that start at different times and +The \execbrokerclass{} service MAY respond with \execoffer{s} that start at different times and set different values for the maximum duration. It MAY offer a maximum duration of 1 hour in the morning, starting at 11:00. @@ -1519,7 +1515,7 @@ \section{Date and time} \end{lstlisting} Different values for start time and duration can be combined with different values for the -compute resources to make a range of different offers to the client. +compute resources to make a range of different \execoffer{s} to the client. For example, if a client asks for 2 cores and 2G of memory for 1 hour sometime on 14 August: \begin{lstlisting}[] @@ -1537,9 +1533,9 @@ \section{Date and time} min: "P1H" \end{lstlisting} -The \execbrokerclass{} service may respond with 2 offers, +The \execbrokerclass{} service may respond with 2 \execoffer{s}, the minimum 2 cores and 2G of memory for 1 hour starting at 11:30, -and a larger offer of up to 8 cores and 8Gb of memory for up to 4 hours +and a larger \execoffer{} of up to 8 cores and 8Gb of memory for up to 4 hours starting between 22:00 and 23:00. \begin{lstlisting}[] @@ -1580,14 +1576,14 @@ \section{Date and time} max: "P4H" \end{lstlisting} -It is then up to the client to decide which offer better suits their use case. -Accept the limited offer in the morning, or accept the more generous offer with +It is then up to the client to decide which \execoffer{} better suits their use case. +Accept the limited \execoffer{} in the morning, or accept the more generous \execoffer{} with more resources and more time later in the day. If an \execbrokerclass{} service is offering more than one option for the \codeword{datetime} -section it MUST make separate offers for each different option. +section it MUST make separate \execoffer{s} for each different option. Even if all of the other parameters are the same, e.g. compute and storage resources, the -\execbrokerclass{} service MUST NOT include more than one time slot in the same offer. +\execbrokerclass{} service MUST NOT include more than one time slot in the same \execoffer{}. Technically the \datamodel{} allows an array of values for the \codeword{datetime} section, but this would impose unnecessary complexity on the client for no real gain in user experience. @@ -1608,8 +1604,8 @@ \section{Federated architecture} A project or organization may deploy several of these low level services, providing a range of different capabilities. -Given a new \job{} to execute, a client can simply poll each of the services to see which one will -make an offer to execute it. +Given a new \workerjob{} to execute, a client can simply poll each of the services to see which ones are +able to execute it. The client does not need to have any prior knowledge about the services apart from their endpoint address. @@ -1621,13 +1617,13 @@ \section{Federated architecture} This service would be configured with a list of the local task specific services and act as an aggregator service sitting between the client and the low level services. This aggregator service would forward a copy of each request it receives to each of the low level services in its list and -then aggregate the offers from the individual responses into a single response that is sent back to the client. +then aggregate the \execoffer{s} from the individual responses into a single response that is sent back to the client. A simple aggregator service would just forward every request to all the low level services below it, regardless of what the request contained. A more complex aggregator service may have some prior knowledge about what types of task or compute resources -each of the lower level services were able to offer, enabling it to make more informed decisions about +each of the lower level services are able to provide, enabling it to make more informed decisions about which low level serviecs to send the requests to. The aggregator service may also have an understanding of the location of \dataset{s} within the organization and @@ -1806,7 +1802,7 @@ \subsection{ExecutionBroker} The main \execbrokerclass{} \webservice{} implements two interfaces; a \codeword{POST} method for sending requests, and a \codeword{GET/POST} -interface for accessing offers and updating their status. +interface for accessing \execoffer{s} and updating their status. \subsubsection{Request method} \label{execution-planner-request} @@ -1897,30 +1893,30 @@ \subsubsection{Request method} twice MAY return different results. \begin{itemize} - \item The list of offers in a response depends on the resources available at the point when the request is made. - \item Sending a request generates a new list of offers. - Those offers will contain references to resources that are reserved for the lifetime of the \codeword{offer}. + \item The list of \execoffer{s} in a response depends on the resources available at the point when the request is made. + \item Sending a request generates a new list of \execoffer{s}. + Those \execoffer{s} will contain references to resources that are reserved for the lifetime of the \codeword{offer}. \item If a second request is made soon after the first request, - then the offers made in response to the first request may still be active. - In which case, the resources listed in those offers may still be reserved, + then the \execoffer{s} made in response to the first request may still be active. + In which case, the resources listed in those \execoffer{s} may still be reserved, reducing the resources available for the second request. \item If a second request is made some time after the first, then - other users may have requested and accepted offers reducing the resources available for the second request, + other users may have requested and accepted \execoffer{s} reducing the resources available for the second request, or the platform may have completed some tasks, increasing the resources available for the second request. \end{itemize} \subsubsection{Offer GET} \label{execution-planner-offer-get} -The \codeword{offer} \codeword{GET} method requests the current state of an offer. -The identifier for the offer is encoded in the URL of the HTTP request by appending the +The \codeword{offer} \codeword{GET} method requests the current state of an \execoffer{}. +The identifier for the \execoffer{} is encoded in the URL of the HTTP request by appending the identifier to the path of the HTP request URL. \begin{lstlisting}[] http[s]://foo.example.org/offers/{identifier} \end{lstlisting} -The \datamodel{} for the content of the \codeword{offer} response is described in section +The \datamodel{} for the content of the response is described in section \ref{datamodel-offer}. An \execbrokerclass{} service MUST be able to provide both \yaml{} and \json{} serializations @@ -1928,7 +1924,7 @@ \subsubsection{Offer GET} The serialization is determined by the \codeword{Accept} header in the HTTP request. \begin{itemize} \item \codeword{application/yaml} for \yaml{}. - \item \codeword{application/json} for \json. + \item \codeword{application/json} for \json{}. \item If the serialization is not declared, the default is assumed to be \yaml{}. \item If the both \yaml{} and \json{} are declared, the default is assumed to be \yaml{}. \end{itemize} @@ -1960,7 +1956,7 @@ \subsubsection{Offer POST} The \codeword{offer} \codeword{POST} method provides the client with a method for updating the status of an \codeword{offer}. -The identifier for the offer is encoded in the URL of the HTTP request by appending the +The identifier for the \execoffer{} is encoded in the URL of the HTTP request by appending the identifier to the path of the HTP request URL. \begin{lstlisting}[] @@ -2023,16 +2019,16 @@ \subsubsection{Offer POST} \subsection{ExecutionWorker} \label{execution-worker-spec} -Accepting an offer from an \execbrokerclass{} service creates a \job{} on an associated \execworkerclass{}, -with status \codeword{PENDING} until the \job{} is started automatically by the \execworkerclass{}. +Accepting an \execoffer{} from an \execbrokerclass{} service creates a \workerjob{} on an associated \execworkerclass{}, +with status \codeword{PENDING} until the \workerjob{} is started automatically by the \execworkerclass{}. -The offer from the \execbrokerclass{} will contain the URL \job{} in the \execworkerclass{}. +The \execoffer{} from the \execbrokerclass{} will contain the URL \workerjob{} in the \execworkerclass{}. \begin{itemize} - \item \execbrokerclass{} is responsible for creating and starting the \job{}. + \item \execbrokerclass{} is responsible for creating and starting the \workerjob{}. \item Client can use a HEAD request to check network access at anytime. - \item Client can use a GET request to check the status of the \job{} at anytime. - \item Client can cancel the \job{} at anytime. + \item Client can use a GET request to check the status of the \workerjob{} at anytime. + \item Client can cancel the \workerjob{} at anytime. \end{itemize} \pagebreak @@ -2115,7 +2111,7 @@ \subsection{\execbrokerclass{} request} \begin{itemize} \item The main body of an \execbrokerclass{} \codeword{request}. \item The main body of each \codeword{offer} in an \execbrokerclass{} response. - \item The main body of an \execworkerclass{} \job{} record. + \item The main body of an \execworkerclass{} \workerjob{} record. \end{itemize} \subsubsection{Executable} @@ -2162,7 +2158,7 @@ \subsubsection{Jupyter notebook} \end{itemize} \item An \codeword{endpoint} URL that points to the live notebook platform. \begin{itemize} - \item The \codeword{endpoint} URL is set by the \execworkerclass{} once the \job{} is \codeword{READY}. + \item The \codeword{endpoint} URL is set by the \execworkerclass{} once the \workerjob{} is \codeword{READY}. \end{itemize} \end{itemize} @@ -2202,7 +2198,7 @@ \subsubsection{Zeppelin notebook} \end{itemize} \item An \codeword{endpoint} URL that points to the live notebook platform. \begin{itemize} - \item The \codeword{endpoint} URL is set by the \execworkerclass{} once the \job{} is \codeword{READY}. + \item The \codeword{endpoint} URL is set by the \execworkerclass{} once the \workerjob{} is \codeword{READY}. \end{itemize} \end{itemize} @@ -2465,12 +2461,12 @@ \subsection{Response} \subsubsection{Positive response} \label{datamodel-positive-response} A positive (yes, this platform can execute the task) \execbrokerclass{} service response contains \codeword{result} -set to \codeword{YES}, and a list of offers. +set to \codeword{YES}, and a list of \execoffer{s}. \subsubsection{Offer} \label{datamodel-offer} -Each offer contains a unique identifier a status value and an expiry time for the offer +Each \execoffer{} contains a unique identifier a status value and an expiry time for the \execoffer{} followed by an updated copy of the \codeword{request} with details filled in by the \execbrokerclass{} service. @@ -2522,17 +2518,17 @@ \subsubsection{Negative response} %TODO more examples of reasons ... -\subsection{Job} -\label{datamodel-job} +\subsection{Session} +\label{datamodel-session} -An \execworkerclass{} \workerjob{} record consists of a unique identifier for the job, +An \execworkerclass{} \workerjob{} record consists of a unique identifier for the \workerjob{}, and a \codeword{phase} indicating what execution stage the \workerjob{} has reached. Followed by an updated copy of the request \codeword{body} from the \execbrokerclass{} -offer with details filled in by the \execworkerclass{} as the \workerjob{} is executed. +\execoffer{} with details filled in by the \execworkerclass{} as the \workerjob{} is executed. \begin{lstlisting}[] job: - # A unique identifier for this job + # A unique identifier for this session ident: !!str # The current execution phase. phase: !!str