Updated December 2014
This codelab is an introduction to Google Cloud Endpoints, the technology that enables the publication of RESTful APIs which can easily be consumed by Android applications. We will start from an existing mobile Android application and connect it to a brand new backend powered by Cloud Endpoints. All development are done with the Android Studio IDE.
The original github repository for this codelab is here: https://github.com/GoogleCloudPlatform/endpoints-codelab-android/
- Android Endpoints "Todo.txt" CodeLab
- Codelab Requirements
- Introduction
- Step 0 - Create your own project in the Google Developers Console
- Step 1 - Get the Android application source code
- Step 2 - Create the Endpoints backend project
- Step 3 - Implement reading and writing tasks to the Datastore
- Step 4 - Modify the android application to use the new backend. Run. Test.
- Step 5 - Escape to the command line and deploy to production!
- Step 6 - Improve the application!
- Appendix - Using the APIs Explorer
- Appendix - Looking around the App Engine Console
- A reasonably-powered laptop with plenty of RAM
- please avoid ultrabooks! sorry, also no chromebooks ATM
- A Java Development Kit (JDK, not JRE), version 7 (Java 7)
- Latest Android Studio (tested with 1.0)
- The Android Emulator
- The AVD should be created with:
- Android 4.4.x API Level 19+
- Hardware Keyboard Present
- SD Card present, 200MiB
- The AVD should be created with:
- Android SDK Manager (accessible from Tools > Android) configured with:
- Android SDK Tools (tested with version 23.0.5)
- Android SDK Platform-tools (tested with version 21)
- Android SDK Build-tools 21.1.1
- Android Support Repository (tested with version 9)
- Android 4.4.2 (API 19+) or above
- The Android Emulator
- A reasonably good Internet connection as the tools will download artifacts on first builds (and you'll probably want to deploy your work to the Cloud).
Todo.txt is an Android application available from the Play Store that stores tasks in the Cloud. It's simple to understand and use, has great user ratings and, best of all, is open sourced with full code available on GitHub at https://github.com/ginatrapani/todo.txt-android.
In this codelab we will use a single development environment (Android Studio) to build a brand new Google Cloud Platform backend to store user tasks, thus replacing the application's existing Cloud provider.
All coding in this codelab, client and backend, will be done in Java within Google App Engine's free tier and using Google Cloud Endpoints.
Please read this short intro to Android Studio, the IDE used throughout the codelab. Check out also this brief intro to Google Cloud Endpoints, the key underlying technology for this codelab.
Google Cloud DataStore is another key technology used in this codelab and is used for storing user-generated data from the Android application once it has made it to your cloud application via the Endpoints technology. This Datastore is a fully-managed data storage service, no need to start, stop, patch or administer this highly-scalable NoSQL database - it is provided to you the developer as a service.
As you can see on this diagram, we'll be building an App Engine application which will store the user tasks in Google's Cloud Datastore and communicate with the Android application via Endpoints. This offers optimal flexibility for defining your own endpoints, implementing business logic and interacting with the rest of the Google Cloud Platform services.
Note: if you're looking for a solution that involves no server-side coding, check out Firebase.
Using this new backend will require making small changes to the current Android application.
To make to calls to our RESTful endpoints, we can rely on generated client libraries which will help us make calls from the Android application as if they we local. No JSON parsing or HTTP-level coding required!
Let's get going!
Point your browser to the Google Developers Console and create a new project (login first if required):
Use the suggested Project ID (it is unique and available) or create your own. You'll need this to deploy your backend after it's created using Android Studio so remember what you enter. Once created, this Google Cloud Platform project will enable you to manage all the services the Cloud Platform has to offer. In this lab, we'll use App Engine and Cloud Datastore and we'll create a new backend project from scratch.
The Android application source code is available in this GitHub repository as a Gradle project You can get the source code on your machine by running the following command:
git clone https://github.com/GoogleCloudPlatform/endpoints-codelab-android
Alternatively, you can download this [.zip archive] (https://github.com/GoogleCloudPlatform/endpoints-codelab-android/archive/master.zip). This is the slightly modified version of the Todo.txt application code found on GitHub, enough to start coding to add Google Cloud Platform as a backend.
Open Android Studio and Import Non-Android Studio project
(as
shown in the screenshot below) the code as a new project.
Remember, this needs to use Java 7. Check the JDK Location in your File > Project Structure.
This should trigger a successful (Gradle) build:
- If the build fails, make sure you have the Support Repository installed (see last paragraph of http://developer.android.com/sdk/installing/studio.html for details).
- If you are asked to setup an Android SDK ("Project SDK is not defined"), simply click on the "Setup SDK" link and select the SDK without configuring it. This codelab has been tested with API level 19.
From here we'll create, build and test the backend before we make our way back to the Android client to hook it up to this new backend.
Android Studio has built-in support for Google Cloud Endpoints (as part of the IDE's support for App Engine backend templates)
To add a backend to the current Android project, simply go to: File > New Module or right-click on your "endpoints-codelab-project" in the Project side pane and choose New > Module and choose "Google Cloud Module".
IMPORTANT: Make sure you select the "App Engine Java Endpoints Module" template and enter the following values in the next screen:
- Module Name: todoTxtBackend
- Package Name: com.google.todotxt.backend
- Client module: todoTxtTouch (com.todotxt.todotxttouch)
This creates a new backend Gradle module for our Android Studio project
(using the Gradle App Engine
Plugin)
and adds it as an additional dependency in the settings.gradle
file. It
also adds the required dependencies for the generated client libraries to the
app's build.gradle
file. The generated code is pretty straight-forward with
an object model for the data manipulated by the Endpoint, and the actual
Endpoint implementation:
@Api(name = "myApi", version = "v1",
namespace = @ApiNamespace(ownerDomain = "backend.todotxt.google.com",
ownerName = "backend.todotxt.google.com",
packagePath = ""))
public class MyEndpoint {
@ApiMethod(name = "sayHi")
public MyBean sayHi(@Named("name") String name) {
MyBean response = new MyBean();
response.setData("Hi, " + name);
return response;
}
}
An HTTP POST request to myApi/v1/sayHi/Test
will be routed to the sayHi()
method and "Test"
will be mapped to the name
attribute.
The convention here is that sending data to the Endpoint implies that the
server state will change and thus only POST requests will be mapped to this
method. You can be explicit about which HTTP verb is used (in particular to
distinguishing between a PUT and a POST) by specifying the optional
httpMethod
attribute of @ApiMethod
(HttpMethod.POST
in this case).
- All Cloud Endpoints annotations and attributes are documented here: https://developers.google.com/appengine/docs/java/endpoints/annotations
- Parameter types and return values are documented here: https://developers.google.com/appengine/docs/java/endpoints/paramreturn_types
Without making any changes for the time being, simply start the development app server (a full App Engine local environment) by pressing the green "Play" button after selecting todoTxtBackend as the current module:
If you see a red cross on the module, this means that either:
- you do not have a Java 7 setting for the project. You'll need to update this in: File > Project Structure > SDK Location > JDK location.
- you have to build the module first to bring in all the dependencies (such as the App Engine SDK). Simply Build > Make module 'todoTxtBackend'.
Once the development app server has started you should see this log message:
INFO: Module instance default is running at http://localhost:8080/
com.google.appengine.tools.development.AbstractModule startup
INFO: The admin console is running at http://localhost:8080/_ah/admin
com.google.appengine.tools.development.DevAppServerImpl doStart
INFO: Dev App Server is now running
You can then point your browser to http://localhost:8080 to exercise the sample Endpoints functionality or more interestingly go to http://localhost:8080/_ah/api/explorer to use the Google API Explorer tool which you may already be familiar with if you've used in the past any public Google API.
See the Appendix - Using the APIs Explorer at the end of this document for further details.
Once done testing the default Endpoint, stop the local development app server (use the red square in the log window or Run > Stop).
At this point we have the basic Endpoints infrastructure in place but we still need to store tasks in a persistent Datastore and wire up the Android application to use this new backend.
Let's start with some small refactoring:
- Rename the
MyBean
class toTaskBean
(Refactor > Rename) - Add a
Long id
attribute to theTaskBean
class along with getter and setter (Code > Generate > Getter and Setter)
This is the modified TaskBean class:
public class TaskBean {
private Long id;
public Long getId() { return id; }
public void setId(Long id) { this.id = id; }
private String myData;
public String getData() { return myData; }
public void setData(String data) { myData = data; }
}
In MyEndpoint.java
, change the name of the Endpoint from "myApi"
to
"TaskApi"
. Notice how a tooltip indicates that this name is not a recommended
choice (names should not be capitalized). Consequently, change it to "taskApi"
:
This suggestion is one of a set of Endpoints-related inspections in Android Studio which you can find in the project properties:
Remove the default sayHi()
method and add the three methods discussed below
with their associated annotations (you can either copy/paste from this document
or use the MyEndpoint.java
file located in the
snippets/
directory
of the archive for this codelab):
The following storeTask
method will be invoked when the following HTTP
request is received:
POST https://<project_id>.appspot.com/_ah/api/taskApi/v1/storeTask
Note how the @Api
and @ApiMethod
attributes are used to build this URI.
The mapping from a POST request is implicit again here because we're passing
the method a parameter (of type TaskBean
). This parameter is exchanged on
the wire in json format (as a simple pair of data
and id
attribute values)
and marshalled in and out of Java directly by Cloud Endpoints.
@ApiMethod(name = "storeTask")
public void storeTask(TaskBean taskBean) {
DatastoreService datastoreService = DatastoreServiceFactory.getDatastoreService();
Transaction txn = datastoreService.beginTransaction();
try {
Key taskBeanParentKey = KeyFactory.createKey("TaskBeanParent", "todo.txt");
Entity taskEntity = new Entity("TaskBean", taskBean.getId(), taskBeanParentKey);
taskEntity.setProperty("data", taskBean.getData());
datastoreService.put(taskEntity);
txn.commit();
} finally {
if (txn.isActive()) {
txn.rollback();
}
}
}
This next getTasks()
method is invoked when this HTTP request is received:
GET https://<project_id>.appspot.com/_ah/api/taskApi/v1/taskbeancollection
This time we're responding to a GET operation because the method takes no
parameter and starts with get
and thus it should be safe to assume that all
such request will be idempotent (i.e. it will not change the server state).
@ApiMethod(name = "getTasks")
public List<TaskBean> getTasks() {
DatastoreService datastoreService = DatastoreServiceFactory.getDatastoreService();
Key taskBeanParentKey = KeyFactory.createKey("TaskBeanParent", "todo.txt");
Query query = new Query(taskBeanParentKey);
List<Entity> results = datastoreService.prepare(query).asList(FetchOptions.Builder.withDefaults());
ArrayList<TaskBean> taskBeans = new ArrayList<TaskBean>();
for (Entity result : results) {
TaskBean taskBean = new TaskBean();
taskBean.setId(result.getKey().getId());
taskBean.setData((String) result.getProperty("data"));
taskBeans.add(taskBean);
}
return taskBeans;
}
This third and final clearTasks()
method is called on this HTTP request:
POST https://<project_id>.appspot.com/_ah/api/taskApi/v1/clearTasks
@ApiMethod(name = "clearTasks")
public void clearTasks() {
DatastoreService datastoreService = DatastoreServiceFactory.getDatastoreService();
Transaction txn = datastoreService.beginTransaction();
try {
Key taskBeanParentKey = KeyFactory.createKey("TaskBeanParent", "todo.txt");
Query query = new Query(taskBeanParentKey);
List<Entity> results = datastoreService.prepare(query)
.asList(FetchOptions.Builder.withDefaults());
for (Entity result : results) {
datastoreService.delete(result.getKey());
}
txn.commit();
} finally {
if (txn.isActive()) { txn.rollback(); }
}
}
This method clearly does modify the server state and thus its name does not
start with get
. An HTTP POST is required to call this method. Another
common RESTful request is the use of HTTP PUT but in our case we're never
modifying existing data. If needed, here are the appropriate import statements
for the entire endpoints class:
import com.google.api.server.spi.config.Api;
import com.google.api.server.spi.config.ApiMethod;
import com.google.api.server.spi.config.ApiNamespace;
import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.FetchOptions;
import com.google.appengine.api.datastore.Key;
import com.google.appengine.api.datastore.KeyFactory;
import com.google.appengine.api.datastore.Query;
import com.google.appengine.api.datastore.Transaction;
import java.util.ArrayList;
import java.util.List;
These three methods all manipulate instances of TaskBean
to store, read
and delete tasks from the Datastore. Getting a hold of the Datastore is as
easy as calling a factory method. TaskBean
instances need to be turned
into Entity instances with a hard-coded "todo.txt"
key and appropriate
attributes before they can be read from or wrote into the Cloud Datastore.
Writing and deleting happens within the boundaries of a transaction.
At this point we have a fully functional backend exposing three methods to store, read and delete tasks in the highly-scalable Cloud Datastore, all through the Cloud Endpoints technology.
While exposing standard RESTful interfaces makes it possible to access them from an Android application using HTTP/JSON calls, the Cloud Endpoints technology is able to create client libraries (in our case a Java library) to enable a much easier implementation on the client-side with high-level abstractions such as Java classes rather than the underlying JSON and HTTP concepts.
To generate these client libraries, we can simply re-build the backend by
navigating to Build > Make Module 'todoTxtBackend'. Once the build
finishes, the generated client libraries will be placed in
todoTxtBackend/build/libs/todoTxtBackend-endpoints-android.jar
archive.
To start calling these libraries, let's first check that when creating the
backend Android Studio properly added the appropriate dependency to the Android
client module. Check in todoTxtTouch/build.gradle
for the following :
dependencies {
...
compile project(path: ':todoTxtBackend', configuration: 'android-endpoints')
}
The complete build.gradle
file for the todoTxtTouch
module is located in
the snippets/
directory of the codelab archive but
you shouldn't have to change or add anything.
Your Android client application is now ready to start using the Endpoints client library to store tasks in our Google Cloud backend!
Here's how we can do it. First, navigate to the
com/todotxt/todotxttouch/task/TaskBagImpl.java
file in the todoTxtTouch
module (it may be one of the tabs already open) and navigate to line 215
(using Navigate > Line should get you to /* REMOTE APIS */
).
This is the interesting part: the pushToRemote
and pullToRemote
methods
implement the actual communication with the backend which we want to set to
use our new Cloud Endpoints-powered backend.
Rather than making changes to this existing class we'll create a new
subclass of TaskBagImpl
called EndpointsTaskBagImpl
(full source
here):
package com.todotxt.todotxttouch.task;
public class EndpointsTaskBagImpl extends TaskBagImpl {
final TaskApi taskApiService;
// Constructor
public EndpointsTaskBagImpl (TodoPreferences preferences, LocalTaskRepository localRepository) {
super(preferences, localRepository, null);
TaskApi.Builder builder = new TaskApi.Builder(AndroidHttp.newCompatibleTransport(),
new AndroidJsonFactory(), null)
.setRootUrl("http://10.0.2.2:8080/_ah/api/")
.setGoogleClientRequestInitializer( new GoogleClientRequestInitializer() {
@Override
public void initialize(AbstractGoogleClientRequest<?> abstractGoogleClientRequest)
throws IOException {
abstractGoogleClientRequest.setDisableGZipContent(true);
}
}
);
taskApiService = builder.build();
} // end of constructor, other methods to follow in this class...
This constructor above sets up a development environment by initializing the
TaskApi
instance to enable testing with the local development app server and
the local Android emulator. As we'll see later, the initialization for the
production code is somewhat simpler.
If you were to run it on a real device (not on the emulator), here are the required changes:
- edit the
todoTxtBackend
run configuration to listen on 0.0.0.0 (default is 127.0.0.1)- replace 10.0.2.2 with the LAN ip address of the laptop
The other two methods for the class are the new overridden pushToRemote
and
pullFromRemote
methods. These methods use the taskApiService
and instances
of TaskBean
which are synchronized with the local application task repository.
Here is the first method (you do not need to type this one) pushing local tasks
to our Google Cloud backend via a simple call to
taskApiService.storeTask(taskBean).execute()
.
@Override
public synchronized void pushToRemote (boolean overridePreference, boolean overwrite) {
try {
ArrayList<String> taskStrList =
TaskIo.loadTasksStrFromFile(LocalFileTaskRepository.TODO_TXT_FILE);
taskApiService.clearTasks().execute();
long id = 1;
for (String taskStr : taskStrList) {
TaskBean taskBean = new TaskBean();
taskBean.setData(taskStr);
taskBean.setId(id++);
taskApiService.storeTask(taskBean).execute();
}
lastSync = new Date();
} catch (IOException e) {
Log.e(EndpointsTaskBagImpl.class.getSimpleName(),
"Error when storing tasks", e);
}
}
Finally, the second method reads all the tasks stored in our Google Cloud
backend via a call to taskApiService.getTasks().execute().getItems()
and
writes them to local storage.
@Override
public synchronized void pullFromRemote(boolean overridePreference) {
try {
// Remote Call
List<TaskBean> remoteTasks = taskApiService.getTasks().execute().getItems();
if (remoteTasks != null) {
ArrayList<Task> taskList = new ArrayList<Task>();
for (TaskBean taskBean : remoteTasks) {
taskList.add(new Task(taskBean.getId(), taskBean.getData()));
}
store(taskList);
reload();
lastSync = new Date();
}
} catch (IOException e) {
Log.e(EndpointsTaskBagImpl.class.getSimpleName(), "Error when loading tasks", e);
}
}
Finally, here is the full list of import statements for EndpointsTaskBagImpl
:
import android.util.Log;
import com.google.api.client.extensions.android.http.AndroidHttp;
import com.google.api.client.extensions.android.json.AndroidJsonFactory;
import com.google.api.client.googleapis.services.AbstractGoogleClientRequest;
import com.google.api.client.googleapis.services.GoogleClientRequestInitializer;
import com.google.todotxt.backend.taskApi.TaskApi;
import com.google.todotxt.backend.taskApi.model.TaskBean;
import com.todotxt.todotxttouch.TodoPreferences;
import com.todotxt.todotxttouch.util.TaskIo;
import java.io.IOException;
import java.util.Date;
import java.util.ArrayList;
import java.util.List;
Note that the entire code for EndpointsTaskBagImpl.java
is located in the
snippets/
directory of the codelab
archive.
The very last step is to switch the Android client to use this new subclass
implementation. This is done in the TaskBagFactory
class. Simply replace:
return new TaskBagImpl(sharedPreferences, localFileTaskRepository, null);
with:
return new EndpointsTaskBagImpl(sharedPreferences,localFileTaskRepository);
We can now run the entire architecture locally!
- First, make sure the backend is running: select the
todoTxtBackend
module in the dropdown and run the server. - Second, select the
todoTxtTouch
module and press run. If you don't have the emulator running, this should bring up this dialog:
Click OK and wait for the emulator to start.
If you don't have any virtual device defined, simply bring up the AVD manager and create one with Android 4.4.2 API (Level 19) or later.
Once started, you can start using the application to add tasks, mark them as completed, and delete them :
Use the API Explorer available at
http://localhost:8080/_ah/api/explorer
to invoke getTasks()
with no parameters and make sure tasks are indeed
stored in your Datastore:
While we've been comfortably sitting in an IDE throughout the development of both the client and the server sides of this codelab, everything was actually handled under the covers by the Gradle App Engine Plugin which means we can easily escape to a command-line or to another tool (such as a continuous integration server).
While we could use the IDE's built-in Terminal (Tools > Open Terminal...
)
or even deploy to the Cloud straight from Android Studio (full details here: cloud.google.com/mobile/app_engine/run_test_deploy)
we are going to use an external terminal to deploy to App Engine.
But first, open appengine-web.xml
and set the Project ID to the name you used
when creating the Google Cloud Platform project at the very beginning of this
codelab (the one that has no space in the name) :
Now, navigate to the build.gradle
file at the root of the todoTxtBackend
module
and make sure the appengine
section contains this authorization attribute
which will allow Gradle to propagate your Google Cloud OAuth2 credentials to
deploy the backend to Google App Engine:
appengine {
...
appcfg {
oauth2 = true
}
...
}
Now open a terminal window and set the directory to the root of the project (select the top node and Edit > Copy Path).
On the command line type the following command to (build if necessary and) deploy the Endpoints backend application to Google App Engine:
$ ./gradlew todoTxtBackend:appengineUpdate
:todoTxtBackend:appengineDownloadSdk
:todoTxtBackend:compileJava UP-TO-DATE
...
:todoTxtBackend:appengineUpdate
Reading application configuration data...
...
INFO: Successfully processed todo.txt-android/todoTxtBackend/build/exploded-app/WEB-INF/appengine-web.xml
INFO: Successfully processed todo.txt-android/todoTxtBackend/build/exploded-app/WEB-INF/web.xml
Beginning interaction for module default...
0% Created staging directory at: '/var/folders/00/17tt8000h01000cxqpsvm004zb9/T/appcfg68248664257210145.tmp'
5% Scanning for jsp files.
20% Scanning files on local disk.
25% Initiating update.
28% Cloning 2 static files.
31% Cloning 13 application files.
52% Uploaded 1 files.
61% Uploaded 2 files.
68% Uploaded 3 files.
73% Uploaded 4 files.
77% Uploaded 5 files.
80% Uploaded 6 files.
82% Initializing precompilation...
84% Sending batch containing 6 file(s) totaling 16KB.
90% Deploying new version.
95% Will check again in 1 seconds.
98% Will check again in 2 seconds.
99% Will check again in 4 seconds.
99% Closing update: new version is ready to start serving.
99% Uploading index definitions.
Update for module default completed successfully.
Success.
Cleaning up temporary files for module default...
BUILD SUCCESSFUL
Total time: 24.6 secs
- If you are not yet authenticated with your project, the deploy via gradle should redirect you to a web page to set up authorization and generate a long String. This string should be pasted in the shell window (even if there is no explicit prompt, most likely on this line :
> Building 88% > :todoTxtBackend:appengineUpdate
.- If deployment fails with
invalid_grant
with no opportunity to set up authorization, consider adding temporarilynoCookies = true
to theappcfg
closure within theappengine
section of thebuild.gradle
config file discussed previously.- The Gradle App Engine Plugin is documented here: https://github.com/GoogleCloudPlatform/gradle-appengine-plugin and has many more details if needed.
Once the backend is successfully deployed, you can go back to the Developers Console (console.developers.google.com) and check the dashboard, current application version, logs, etc.
On the Android side you can now simplify the initialization of the
taskAPIService
by removing the root URL and request initializer used for
testing against the local development app server.
In
EndpointsTaskBagImpl.java
, this is now how we create the TaskAPI.Builder
object:
TaskApi.Builder builder =
new TaskApi.Builder(AndroidHttp.newCompatibleTransport(), new AndroidJsonFactory(), null);
Rebuilding the Android application will pick up the changes made to the backend (it's no longer local to your machine but rather lives in the Cloud) so you don't need to explicitly re-install the Endpoints client libraries. If needed the Gradle App Engine Plugin offers tasks to install, export, and document endpoints. This would work well if the same developer is not building both the client and the backend.
Run the Android client one more time and add a new task.
The current synchronisation implementation will fetch tasks from local storage so unless you explicitly ask for a sync in the application menu or add a new task, the Datastore and the application will not be in sync.
As you can see in the screenshot above a query of TaskBean
entities in the
Cloud Datastore returns the list of tasks in our new Android Todo.txt
application!
Believe it or not, this application is not quite perfect! :)
Here are a few ideas to enhance the code and the user experience:
- Add Endpoints authentication by re-using the Android logged-in user credentials and thus preventing anyone to access your Endpoints. Check out this documentation: https://developers.google.com/appengine/docs/java/endpoints/auth
- Use memcache to drastically improve performance of your application under load. See this Memcache overview: https://developers.google.com/appengine/docs/java/memcache/
- Implement better sync logic with some level of incremental synchronisation.
You are also encouraged to use a real Android device, either through USB debugging or by generating a signed APK (In Android Studio: Build > Generate Signed APK...) and installing it on a phone or tablet.
If you'd like to watch a 30-minute presentation of an earlier version of this codelab with live coding of all the steps, simply look at this YouTube video (recorded in San Francisco in Mars 2014).
This concludes this codelab: Thanks for your time and we hope this was valuable to you!
Now go build something great! :)
Check out this world's shortest introduction to the API Explorer.
Curious about the console? Here's a super quick overview of the App Engine Developer Console.