-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
680872d
commit 8351b40
Showing
2 changed files
with
296 additions
and
4 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,22 +1,314 @@ | ||
# knfun | ||
|
||
Knative kn client demo | ||
Knfun is a set of Knative micro-services / micro-functions to illustrate the end-to-end developer experience. | ||
|
||
The first demo is to use three functions in a live settings. The following diagram is a sketch of this demo | ||
|
||
![Demo sketch](docs/demo1-sketch.png) | ||
|
||
## Functions | ||
|
||
1. *TwitterFn* search function (via [Twitter's API](https://developer.twitter.com/en/docs)) | ||
_in_: hashtags or string to search | ||
_in_: count (max number of tweets) | ||
_out_: recent tweets with images | ||
|
||
2. *WatsonFn* visual recognition image classifier function (via IBM's [Watson APIs](https://cloud.ibm.com/apidocs/visual-recognition/visual-recognition-v3)) | ||
_in_: image URLs | ||
_out_: image features (class) with confidence (score) | ||
|
||
3. *SummaryFn* function | ||
_in_: search string and twitterFn and WatsonFn | ||
_in_: count (max number of tweets) | ||
_out_: HTML page displaying summary | ||
|
||
Of course, both the *TwitterFn* and *WatsonFn* require credentials to execute. This means that the keys for both the *TwitterFn* and *WatsonFn* are also required, however, to simplify the discussion, they are ommited. | ||
|
||
# Setup | ||
|
||
This repository is self-contained except for the following dependencies which you should have before getting started. | ||
|
||
1. Go version go1.12.x or later. Go to [Golang download](https://golang.org/dl/) page to get the version for your OS. | ||
2. [Docker engine](https://docs.docker.com/engine/installation/) for your machine. Specifically ensure you can execute the `docker` executable at the command line. | ||
3. [Knative](https://knative.dev/) cluster. Depending on your Kubernetes cluster provider you can find different means for [getting Knative installed](https://knative.dev/docs/install/) onto your cluster. | ||
4. [Knative's `kn` client](https://github.com/knative/client). Once you have your Knative cluster, follow the steps to build `kn` or get one from the released builds. | ||
|
||
Once these four dependencies are met, continue below with steps to get credentials, build, test, and deploy the functions for the demos. | ||
|
||
# Credentials | ||
|
||
As mentioned above, you need to get credentials for both the [Twitter API](https://developer.twitter.com/en/docs) and the [IBM Watson API](https://cloud.ibm.com/apidocs/visual-recognition/visual-recognition-v3). Once you do, then you should be able to follow the steps below. | ||
|
||
To facilitate using and passing these secrets in the commands, I recommend setting environment variables for each string for your shell. Feel free to use other means but in the description below I am assuming that the secret values are set to the environment variables named. | ||
|
||
|
||
Once you have access to the Twitter API, you will have four different keys (as strings). They are: | ||
|
||
1. Twitter API key - TWITTER_API_KEY | ||
2. Twitter API secret key - TWITTER_API_SECRE_KEY | ||
3. Twitter Access Token - TWTTER_ACCESS_TOKEN | ||
4. Twitter Access Token secret - TWITTER_ACCESS_TOKEN_SECRET | ||
|
||
To set these on your shell, do the following replacing the content in `<>` with your key value | ||
|
||
```bash | ||
export TWITTER_API_key=<your Twitter API key value here> | ||
export TWITTER_API_SECRET_KEY=<your Twitter API secret key value here> | ||
export TWITTER_ACCESS_TOKEN=<your Twitter access token value here> | ||
export TWITTER_ACCESS_TOKEN_SECRET=<your Twitter access token secret value here> | ||
``` | ||
|
||
## Watson | ||
|
||
`go get -u github.com/watson-developer-cloud/go-sdk/...` | ||
Access to the IBM Watson API requires one secret and two constant values. They are: | ||
|
||
1. Watson API key - WATSON_API_KEY | ||
2. Watson API URL - WATSON_API_URL | ||
3. Watson APU Version - WATSON_API_VERSION | ||
|
||
```bash | ||
export WATSON_API_KEY=<your Watson API key value here> | ||
export WATSON_API_URL=https://gateway.watsonplatform.net/visual-recognition/api | ||
export WATSON_API_VERSION=2018-03-19 | ||
``` | ||
|
||
# Build | ||
|
||
`go build -o twitter-fn funcs/twitter/main.go` | ||
After cloning the repository. You can build all functions with the following command: | ||
|
||
```bash | ||
./hack/build.sh | ||
🕸️ Update | ||
🧽 Format | ||
⚖️ License | ||
📖 Docs | ||
🚧 Compile | ||
──────────────────────────────────────────── | ||
success | ||
``` | ||
|
||
`go build -o watson-fn funcs/watson/main.go` | ||
The result is that you should now have three executables: `twitter-fn`, `watson-fn`, and `summary-fun`. | ||
|
||
```bash | ||
ls | ||
LICENSE docs go.mod hack twitter-fn watson-fn | ||
README.md funcs go.sum summary-fn vendor | ||
``` | ||
|
||
These are designed as both CLI and server functions that you can test locally and deploy on Knative. | ||
|
||
# Test | ||
|
||
Let's first explore how to test the functions locally. | ||
|
||
## TwitterFn | ||
|
||
For each function you can run them locally as a CLI to get immediate response or as a server and use your browser to see the response. The following will display recent tweets that have the word `NBA` | ||
|
||
```bash | ||
./twitter-fn search NBA -c 20 -o text \ | ||
--twitter-api-key $TWITTER_API_KEY \ | ||
--twitter-api-secret-key $TWITTER_API_SECRET_KEY \ | ||
--twitter-access-token $TWITTER_ACCESS_TOKEN \ | ||
--twitter-access-token-secret $TWITTER_ACCESS_TOKEN_SECRET \ | ||
``` | ||
|
||
To run this as a server and see JSON output on your browser or with curl, do the following: | ||
|
||
```bash | ||
./twitter-fn search NBA -c 20 -o text -S -p 8080 \ | ||
--twitter-api-key $TWITTER_API_KEY \ | ||
--twitter-api-secret-key $TWITTER_API_SECRET_KEY \ | ||
--twitter-access-token $TWITTER_ACCESS_TOKEN \ | ||
--twitter-access-token-secret $TWITTER_ACCESS_TOKEN_SECRET \ | ||
``` | ||
|
||
Then open your browser at `http://localhost:8080` or do `curl http://localhost:8080`. | ||
|
||
To see what other options are available for the `twitter-fn` `search` function get the CLI help with: `./twitter-fn search --help` | ||
|
||
## WatsonFn | ||
|
||
Similarly for the `watson-fn` function you can test locally with any image for which you have the URL with the following. | ||
|
||
|
||
```bash | ||
./watson-fn vr classify http://... -o text \ | ||
--watson-api-key $WATSON_API_KEY \ | ||
--watson-api-url $WATSON_API_URL \ | ||
--watson-api-version $WATSON_API_VERSION \ | ||
``` | ||
|
||
To run this as a server and see JSON output on your browser or with curl, do the following: | ||
|
||
```bash | ||
./twitter-fn search NBA -c 20 -o text -S -p 8080 \ | ||
--twitter-api-key $TWITTER_API_KEY \ | ||
--twitter-api-secret-key $TWITTER_API_SECRET_KEY \ | ||
--twitter-access-token $TWITTER_ACCESS_TOKEN \ | ||
--twitter-access-token-secret $TWITTER_ACCESS_TOKEN_SECRET \ | ||
``` | ||
|
||
## SummaryFn | ||
|
||
Finally, the `summary-fn` function can be tested locally after running the `twitter-fn` and `watson-fn` servers. For instance, respectively at ports `8080` and `8081`. | ||
|
||
|
||
```bash | ||
./summary-fn NBA -o text -c 10 -o text \ | ||
--twitter-fn-url http://localhost:8080 \ | ||
--watson-fn-url http://localhost:8081 \ | ||
``` | ||
|
||
To run this as a server and see output on your browser or with curl, do the following: | ||
|
||
```bash | ||
./summary-fn NBA -o text -c 10 -o text -S -p 8082 \ | ||
--twitter-fn-url http://localhost:8080 \ | ||
--watson-fn-url http://localhost:8081 \ | ||
``` | ||
|
||
Open your browser at `http://localhost:8082` or `curl http://localhost:8082` | ||
|
||
## Credentials config | ||
|
||
You can avoid passing all the credentials everytime by creating a file named `.knfun.yaml` in your home directory and adding the credential values. The follwing command will create that file and set the values from the environment variables we discussed above. | ||
|
||
```bash | ||
touch ~/.knfun.yaml | ||
cat <<EOF >> ~/.knfun.yaml | ||
twitter-api-key: $TWITTER_API_KEY | ||
twitter-api-secret-key: $TWITTER_API_SECRET_KEY | ||
twitter-access-token: $TWITTER_ACCESS_TOKEN | ||
twitter-access-token-secret: $TWITTER_ACCESS_TOKEN_KEY | ||
# watson-fn | ||
watson-api-key: $WATSON_API_KEY | ||
watson-api-url: https://gateway.watsonplatform.net/visual-recognition/api | ||
watson-api-version: 2018-03-19 | ||
EOF | ||
``` | ||
|
||
Once this config `.knfun.yaml` is present in your home directory, execution of the `twitter-fn` and `watson-fn` will pick up the key values automatically. So instead of invoking: | ||
|
||
```bash | ||
./watson-fn vr classify http://pbs.twimg.com/media/EHpWVAvWoAEfVzO.jpg -o json \ | ||
--watson-api-key $WATSON_API_KEY \ | ||
--watson-api-url $WATSON_API_URL \ | ||
--watson-api-version $WATSON_API_VERSION \ | ||
``` | ||
|
||
You can simply do: | ||
|
||
```bash | ||
./watson-fn vr classify http://pbs.twimg.com/media/EHpWVAvWoAEfVzO.jpg -o json | ||
``` | ||
|
||
# Deploy | ||
|
||
In order to deploy into a Knative cluster, you must first create Docker images and publish them into a repository. The `./hack/deploy.sh --docker-images` and `./hack/deploy.sh --docker-push` will respectively create Docker images and push them in your Docker Hub for you. You simply need to make sure the `docker` executable is visible to your shell and that the environment variable `DOCKER_USERNAME` is set to your Docker Hub user ID. | ||
|
||
```bash | ||
./hack/build.sh --docker-images | ||
🚧 🐳 build images | ||
🚧 🐳 twitter-fn | ||
Sending build context to Docker daemon 55.42MB | ||
... | ||
... | ||
Successfully tagged drmax/twitter-fn:latest | ||
🚧 🐳 watson-fn | ||
... | ||
Successfully tagged drmax/watson-fn:latest | ||
🚧 🐳 summary-fn | ||
... | ||
Successfully tagged drmax/summary-fn:latest | ||
``` | ||
|
||
And when publishing the images. | ||
|
||
```bash | ||
./hack/build.sh --docker-push | ||
🐳 push images | ||
🐳 twitter-fn | ||
The push refers to repository [docker.io/drmax/twitter-fn] | ||
... | ||
🐳 watson-fn | ||
The push refers to repository [docker.io/drmax/watson-fn] | ||
... | ||
🐳 summary-fn | ||
... | ||
``` | ||
|
||
Your images should not be avaible at: | ||
|
||
1. `docker.io/${DOCKER_USERNAME}/twitter-fn` | ||
2. `docker.io/${DOCKER_USERNAME}/watson-fn` | ||
3. `docker.io/${DOCKER_USERNAME}/summaryr-fn` | ||
|
||
You can use these images to deploy them into your Knative cluster with the `kn` CLI. | ||
|
||
# Run | ||
|
||
Deploying into Knative means you have a Knative cluster ready and your `$KUBECONFIG` is set or that the `~/.kube/config` file is pointing to the cluster's Kubernetes configuration file. | ||
|
||
|
||
```bash | ||
./kn service create twitter-fn \ | ||
--env TWITTER_API_KEY=$TWITTER_API_KEY \ | ||
--env TWITTER_API_SECRET_KEY=$TWITTER_API_SECRET_KEY \ | ||
--env TWITTER_ACCESS_TOKEN=$TWITTER_ACCESS_TOKEN \ | ||
--env TWITTER_ACCESS_TOKEN_SECRET=$TWITTER_ACCESS_TOKEN_SECRET \ | ||
--image docker.io/drmax/twitter-fn:latest \ | ||
Creating service 'twitter-fn' in namespace 'default': | ||
|
||
0.496s Configuration "twitter-fn" is waiting for a Revision to become ready. | ||
0.521s ... | ||
6.986s ... | ||
7.148s ... | ||
7.268s Ready to serve. | ||
|
||
Service 'twitter-fn' created with latest revision 'twitter-fn-njnks-1' and URL: | ||
http://twitter-fn.knative-cluster.us-south.containers.cloud.ibm.com | ||
``` | ||
|
||
## Watson | ||
|
||
```bash | ||
./kn service create watson-fn \ | ||
--env WATSON_API_KEY=$WATSON_API_KEY \ | ||
--env WATSON_API_URL=$WATSON_API_URL \ | ||
--env WATSON_API_VERSION=$WATSON_API_VERSION \ | ||
--image docker.io/drmax/watson-fn:latest \ | ||
Creating service 'watson-fn' in namespace 'default': | ||
|
||
0.197s Configuration "watson-fn" is waiting for a Revision to become ready. | ||
7.643s ... | ||
7.752s ... | ||
7.870s Ready to serve. | ||
|
||
Service 'watson-fn' created with latest revision 'watson-fn-dnfxc-1' and URL: | ||
http://watson-fn.default.knative-cluster.us-south.containers.cloud.ibm.com | ||
``` | ||
|
||
## Summary | ||
|
||
You need to then set the `TWITTER_FN_URL` and `WATSON_FN_URL` to the URL that `kn` output for the respective function. For instance: | ||
|
||
```bash | ||
export TWITTER_FN_URL=twitter-fn.knative-cluster.us-south.containers.cloud.ibm.com | ||
export WATSON_FN_URL=watson-fn.knative-cluster.us-south.containers.cloud.ibm.com | ||
``` | ||
|
||
```bash | ||
./kn service create summary-fn \ | ||
--env TWITTER_FN_URL=$TWITTER_FN_URL \ | ||
--env WATSON_FN_URL=$WATSON_FN_URL \ | ||
--image docker.io/drmax/summary-fn:latest \ | ||
... | ||
``` | ||
|
||
|
||
You can test the `summary-fn` function by going to the generated function URL with your browser or with `curl`. | ||
|
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.