diff --git a/README.md b/README.md index 1e53457..e502daa 100644 --- a/README.md +++ b/README.md @@ -1,22 +1,314 @@ # knfun -Knative kn client demo +Knfun is a set of Knative micro-services / micro-functions to illustrate the end-to-end developer experience. + +The first demo is to use three functions in a live settings. The following diagram is a sketch of this demo + +![Demo sketch](docs/demo1-sketch.png) + +## Functions + +1. *TwitterFn* search function (via [Twitter's API](https://developer.twitter.com/en/docs)) + _in_: hashtags or string to search + _in_: count (max number of tweets) + _out_: recent tweets with images + +2. *WatsonFn* visual recognition image classifier function (via IBM's [Watson APIs](https://cloud.ibm.com/apidocs/visual-recognition/visual-recognition-v3)) + _in_: image URLs + _out_: image features (class) with confidence (score) + +3. *SummaryFn* function + _in_: search string and twitterFn and WatsonFn + _in_: count (max number of tweets) + _out_: HTML page displaying summary + +Of course, both the *TwitterFn* and *WatsonFn* require credentials to execute. This means that the keys for both the *TwitterFn* and *WatsonFn* are also required, however, to simplify the discussion, they are ommited. # Setup +This repository is self-contained except for the following dependencies which you should have before getting started. + +1. Go version go1.12.x or later. Go to [Golang download](https://golang.org/dl/) page to get the version for your OS. +2. [Docker engine](https://docs.docker.com/engine/installation/) for your machine. Specifically ensure you can execute the `docker` executable at the command line. +3. [Knative](https://knative.dev/) cluster. Depending on your Kubernetes cluster provider you can find different means for [getting Knative installed](https://knative.dev/docs/install/) onto your cluster. +4. [Knative's `kn` client](https://github.com/knative/client). Once you have your Knative cluster, follow the steps to build `kn` or get one from the released builds. + +Once these four dependencies are met, continue below with steps to get credentials, build, test, and deploy the functions for the demos. + +# Credentials + +As mentioned above, you need to get credentials for both the [Twitter API](https://developer.twitter.com/en/docs) and the [IBM Watson API](https://cloud.ibm.com/apidocs/visual-recognition/visual-recognition-v3). Once you do, then you should be able to follow the steps below. + +To facilitate using and passing these secrets in the commands, I recommend setting environment variables for each string for your shell. Feel free to use other means but in the description below I am assuming that the secret values are set to the environment variables named. + ## Twitter +Once you have access to the Twitter API, you will have four different keys (as strings). They are: + +1. Twitter API key - TWITTER_API_KEY +2. Twitter API secret key - TWITTER_API_SECRE_KEY +3. Twitter Access Token - TWTTER_ACCESS_TOKEN +4. Twitter Access Token secret - TWITTER_ACCESS_TOKEN_SECRET + +To set these on your shell, do the following replacing the content in `<>` with your key value + +```bash +export TWITTER_API_key= +export TWITTER_API_SECRET_KEY= +export TWITTER_ACCESS_TOKEN= +export TWITTER_ACCESS_TOKEN_SECRET= +``` ## Watson -`go get -u github.com/watson-developer-cloud/go-sdk/...` +Access to the IBM Watson API requires one secret and two constant values. They are: + +1. Watson API key - WATSON_API_KEY +2. Watson API URL - WATSON_API_URL +3. Watson APU Version - WATSON_API_VERSION + +```bash +export WATSON_API_KEY= +export WATSON_API_URL=https://gateway.watsonplatform.net/visual-recognition/api +export WATSON_API_VERSION=2018-03-19 +``` # Build -`go build -o twitter-fn funcs/twitter/main.go` +After cloning the repository. You can build all functions with the following command: + +```bash +./hack/build.sh +πŸ•ΈοΈ Update +🧽 Format +βš–οΈ License +πŸ“– Docs +🚧 Compile +──────────────────────────────────────────── +success +``` -`go build -o watson-fn funcs/watson/main.go` +The result is that you should now have three executables: `twitter-fn`, `watson-fn`, and `summary-fun`. + +```bash +ls +LICENSE docs go.mod hack twitter-fn watson-fn +README.md funcs go.sum summary-fn vendor +``` + +These are designed as both CLI and server functions that you can test locally and deploy on Knative. # Test +Let's first explore how to test the functions locally. + +## TwitterFn + +For each function you can run them locally as a CLI to get immediate response or as a server and use your browser to see the response. The following will display recent tweets that have the word `NBA` + +```bash +./twitter-fn search NBA -c 20 -o text \ + --twitter-api-key $TWITTER_API_KEY \ + --twitter-api-secret-key $TWITTER_API_SECRET_KEY \ + --twitter-access-token $TWITTER_ACCESS_TOKEN \ + --twitter-access-token-secret $TWITTER_ACCESS_TOKEN_SECRET \ +``` + +To run this as a server and see JSON output on your browser or with curl, do the following: + +```bash +./twitter-fn search NBA -c 20 -o text -S -p 8080 \ + --twitter-api-key $TWITTER_API_KEY \ + --twitter-api-secret-key $TWITTER_API_SECRET_KEY \ + --twitter-access-token $TWITTER_ACCESS_TOKEN \ + --twitter-access-token-secret $TWITTER_ACCESS_TOKEN_SECRET \ +``` + +Then open your browser at `http://localhost:8080` or do `curl http://localhost:8080`. + +To see what other options are available for the `twitter-fn` `search` function get the CLI help with: `./twitter-fn search --help` + +## WatsonFn + +Similarly for the `watson-fn` function you can test locally with any image for which you have the URL with the following. + + +```bash +./watson-fn vr classify http://... -o text \ + --watson-api-key $WATSON_API_KEY \ + --watson-api-url $WATSON_API_URL \ + --watson-api-version $WATSON_API_VERSION \ +``` + +To run this as a server and see JSON output on your browser or with curl, do the following: + +```bash +./twitter-fn search NBA -c 20 -o text -S -p 8080 \ + --twitter-api-key $TWITTER_API_KEY \ + --twitter-api-secret-key $TWITTER_API_SECRET_KEY \ + --twitter-access-token $TWITTER_ACCESS_TOKEN \ + --twitter-access-token-secret $TWITTER_ACCESS_TOKEN_SECRET \ +``` + +## SummaryFn + +Finally, the `summary-fn` function can be tested locally after running the `twitter-fn` and `watson-fn` servers. For instance, respectively at ports `8080` and `8081`. + + +```bash +./summary-fn NBA -o text -c 10 -o text \ + --twitter-fn-url http://localhost:8080 \ + --watson-fn-url http://localhost:8081 \ +``` + +To run this as a server and see output on your browser or with curl, do the following: + +```bash +./summary-fn NBA -o text -c 10 -o text -S -p 8082 \ + --twitter-fn-url http://localhost:8080 \ + --watson-fn-url http://localhost:8081 \ +``` + +Open your browser at `http://localhost:8082` or `curl http://localhost:8082` + +## Credentials config + +You can avoid passing all the credentials everytime by creating a file named `.knfun.yaml` in your home directory and adding the credential values. The follwing command will create that file and set the values from the environment variables we discussed above. + +```bash +touch ~/.knfun.yaml +cat <> ~/.knfun.yaml +twitter-api-key: $TWITTER_API_KEY +twitter-api-secret-key: $TWITTER_API_SECRET_KEY +twitter-access-token: $TWITTER_ACCESS_TOKEN +twitter-access-token-secret: $TWITTER_ACCESS_TOKEN_KEY + +# watson-fn +watson-api-key: $WATSON_API_KEY +watson-api-url: https://gateway.watsonplatform.net/visual-recognition/api +watson-api-version: 2018-03-19 +EOF +``` + +Once this config `.knfun.yaml` is present in your home directory, execution of the `twitter-fn` and `watson-fn` will pick up the key values automatically. So instead of invoking: + +```bash +./watson-fn vr classify http://pbs.twimg.com/media/EHpWVAvWoAEfVzO.jpg -o json \ + --watson-api-key $WATSON_API_KEY \ + --watson-api-url $WATSON_API_URL \ + --watson-api-version $WATSON_API_VERSION \ +``` + +You can simply do: + +```bash +./watson-fn vr classify http://pbs.twimg.com/media/EHpWVAvWoAEfVzO.jpg -o json +``` + +# Deploy + +In order to deploy into a Knative cluster, you must first create Docker images and publish them into a repository. The `./hack/deploy.sh --docker-images` and `./hack/deploy.sh --docker-push` will respectively create Docker images and push them in your Docker Hub for you. You simply need to make sure the `docker` executable is visible to your shell and that the environment variable `DOCKER_USERNAME` is set to your Docker Hub user ID. + +```bash +./hack/build.sh --docker-images +🚧 🐳 build images + 🚧 🐳 twitter-fn +Sending build context to Docker daemon 55.42MB +... +... +Successfully tagged drmax/twitter-fn:latest + 🚧 🐳 watson-fn +... +Successfully tagged drmax/watson-fn:latest + 🚧 🐳 summary-fn +... +Successfully tagged drmax/summary-fn:latest +``` + +And when publishing the images. + +```bash +./hack/build.sh --docker-push +🐳 push images + 🐳 twitter-fn +The push refers to repository [docker.io/drmax/twitter-fn] +... + 🐳 watson-fn +The push refers to repository [docker.io/drmax/watson-fn] +... + 🐳 summary-fn +... +``` + +Your images should not be avaible at: + +1. `docker.io/${DOCKER_USERNAME}/twitter-fn` +2. `docker.io/${DOCKER_USERNAME}/watson-fn` +3. `docker.io/${DOCKER_USERNAME}/summaryr-fn` + +You can use these images to deploy them into your Knative cluster with the `kn` CLI. + # Run + +Deploying into Knative means you have a Knative cluster ready and your `$KUBECONFIG` is set or that the `~/.kube/config` file is pointing to the cluster's Kubernetes configuration file. + +## Twitter + +```bash +./kn service create twitter-fn \ + --env TWITTER_API_KEY=$TWITTER_API_KEY \ + --env TWITTER_API_SECRET_KEY=$TWITTER_API_SECRET_KEY \ + --env TWITTER_ACCESS_TOKEN=$TWITTER_ACCESS_TOKEN \ + --env TWITTER_ACCESS_TOKEN_SECRET=$TWITTER_ACCESS_TOKEN_SECRET \ + --image docker.io/drmax/twitter-fn:latest \ +Creating service 'twitter-fn' in namespace 'default': + + 0.496s Configuration "twitter-fn" is waiting for a Revision to become ready. + 0.521s ... + 6.986s ... + 7.148s ... + 7.268s Ready to serve. + +Service 'twitter-fn' created with latest revision 'twitter-fn-njnks-1' and URL: +http://twitter-fn.knative-cluster.us-south.containers.cloud.ibm.com +``` + +## Watson + +```bash +./kn service create watson-fn \ + --env WATSON_API_KEY=$WATSON_API_KEY \ + --env WATSON_API_URL=$WATSON_API_URL \ + --env WATSON_API_VERSION=$WATSON_API_VERSION \ + --image docker.io/drmax/watson-fn:latest \ +Creating service 'watson-fn' in namespace 'default': + + 0.197s Configuration "watson-fn" is waiting for a Revision to become ready. + 7.643s ... + 7.752s ... + 7.870s Ready to serve. + +Service 'watson-fn' created with latest revision 'watson-fn-dnfxc-1' and URL: +http://watson-fn.default.knative-cluster.us-south.containers.cloud.ibm.com +``` + +## Summary + +You need to then set the `TWITTER_FN_URL` and `WATSON_FN_URL` to the URL that `kn` output for the respective function. For instance: + +```bash +export TWITTER_FN_URL=twitter-fn.knative-cluster.us-south.containers.cloud.ibm.com +export WATSON_FN_URL=watson-fn.knative-cluster.us-south.containers.cloud.ibm.com +``` + +```bash +./kn service create summary-fn \ + --env TWITTER_FN_URL=$TWITTER_FN_URL \ + --env WATSON_FN_URL=$WATSON_FN_URL \ + --image docker.io/drmax/summary-fn:latest \ +... +``` + + +You can test the `summary-fn` function by going to the generated function URL with your browser or with `curl`. + diff --git a/docs/demo1-sketch.png b/docs/demo1-sketch.png new file mode 100644 index 0000000..9aadbd7 Binary files /dev/null and b/docs/demo1-sketch.png differ