Initial Documentation Commit

.github/workflows/deploy.yml

@@ -0,0 +1,44 @@
+name: Deploy to GitHub Pages
+
+on:
+  push:
+    branches:
+      - docsite
+    # Review gh actions docs if you want to further define triggers, paths, etc
+    # https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#on
+
+permissions:
+  contents: write
+
+jobs:
+  deploy:
+    name: Deploy to GitHub Pages
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 18
+          cache: npm
+          cache-dependency-path: ./package-lock.json
+      - name: Install dependencies
+        run: npm ci
+      - name: Build website
+        run: npm run build
+      # Popular action to deploy to GitHub Pages:
+      # Docs: https://github.com/peaceiris/actions-gh-pages#%EF%B8%8F-docusaurus
+      - name: Deploy to GitHub Pages
+        uses: peaceiris/actions-gh-pages@v3
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          # Build output to publish to the `gh-pages` branch:
+          publish_dir: ./build
+          # The following lines assign commit authorship to the official
+          # GH-Actions bot for deploys to `gh-pages` branch:
+          # https://github.com/actions/checkout/issues/13#issuecomment-724415212
+          # The GH actions bot is used by default if you didn't specify the two fields.
+          # You can swap them out with your own user credentials.
+          user_name: github-actions[bot]
+          user_email: 41898282+github-actions[bot]@users.noreply.github.com

.github/workflows/test-deploy.yml

@@ -0,0 +1,27 @@
+name: Test deployment
+
+on:
+  pull_request:
+    branches:
+      - docsite
+    # Review gh actions docs if you want to further define triggers, paths, etc
+    # https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#on
+
+jobs:
+  test-deploy:
+    name: Test deployment
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 18
+          cache: npm
+          cache-dependency-path: ./package-lock.json
+
+      - name: Install dependencies
+        run: npm ci
+      - name: Test build website
+        run: npm run build

.gitignore

@@ -0,0 +1,20 @@
+# Dependencies
+/node_modules
+
+# Production
+/build
+
+# Generated files
+.docusaurus
+.cache-loader
+
+# Misc
+.DS_Store
+.env.local
+.env.development.local
+.env.test.local
+.env.production.local
+
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*

README.md

@@ -0,0 +1,16 @@
+# Website
+
+This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.
+
+### Local Development
+First install npm packages:
+```
+$ npm install
+```
+
+To bring up the service, run:
+```
+$ npm run start
+```
+
+This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.

babel.config.js

@@ -0,0 +1,3 @@
+module.exports = {
+  presets: [require.resolve('@docusaurus/core/lib/babel/preset')],
+};

blog/2023-07-19-introduction-oh.mdx

@@ -0,0 +1,19 @@
+---
+slug: introduction-oh
+title: Taking Charge of Tables, Introducing OpenHouse for Big Data Management
+authors:
+  name: Sumedh Sakdeo
+  title: OpenHouse @ LinkedIn, ex-Head of Data Lyft Self Driving Division
+  url: https://www.linkedin.com/in/sumedhsakdeo/
+  image_url: https://github.com/sumedhsakdeo.png
+tags: [openhouse, introduction, big data]
+hide_reading_time: true
+---
+
+import Link from '@docusaurus/Link';
+
+<Link
+    className='button button--primary button--small' 
+    to="https://www.linkedin.com/blog/engineering/data-management/taking-charge-of-tables--introducing-openhouse-for-big-data-mana">
+    <small> Read More </small>
+</Link>

development/CONTRIBUTING.md

@@ -0,0 +1,24 @@
+# Build OpenHouse
+## Prerequisite
+- Java 1.8 or above
+- Scala 2.12 or above
+
+## Steps 
+Simply run `./gradlew clean build`.
+
+
+# Using IDE
+## IntelliJ
+There are a few subprojects, like `:client:tableclient` that contains no code before build and include generated
+source as part of source files during build runtime. There are also classes imports generated source as libraries.
+To ensure they are properly indexed by IDE, you need to:
+- Run a full build after fresh repository checkout.
+- Open the OpenHouse project from the root `build.gradle` file in the repo.
+
+### Fixing missing classes errors
+Intellij doesn't work nicely with shadow plugin, this is a known issue tracked here: [IntelliJ IDEA unable to resolve classes of project dependency shadow jar #264](https://github.com/johnrengelman/shadow/issues/264)
+
+Modules such as `:integrations:spark:openhouse-spark-itest` `:integrations:java:openhouse-java-itest` face this issue.
+
+To fix this issue please follow instructions as follows: [define-library](https://www.jetbrains.com/help/idea/library.html#define-library)
+![](ide-setup-for-shadow-jars.gif)

development/client-code-generation.md

@@ -0,0 +1,41 @@
+## Workflow
+# Produce OpenApi Spec for the server
+To enable a server to produce OpenApi spec, just include the following code block in its build.gradle:
+```
+plugin {
+  id 'com.github.johnrengelman.processes' version '0.5.0'
+  id 'org.springdoc.openapi-gradle-plugin' version '1.3.3'
+  id 'openhouse.service-specgen-convention'
+}
+```
+This will create the spec file in `<Root>/build/<Service>/specs/`.
+
+# Generate client for the server
+Each generated client should have its corresponding module. These modules are placed in `:client`, for example
+`:client:hts`.  Just add a build.gradle with following code block:
+```
+plugins {
+  id 'java'
+  id 'openhouse.client-codegen-convention'
+  id 'openhouse.springboot-conventions'
+}
+
+ext {
+  codeGenForService = ":services:housetables"
+}
+
+apply from: "${project(':client:common').file("codegen.build.gradle")}"
+```
+Note: the service should be enabled to produce spec file.
+
+## FAQ
+### The service controller was changed on my local build, but the generated client doesn't reflect it. WTD?
+OpenApi gradle plugin doesn't reproduce the spec file when code change happens. Because of this, the spec doesn't contain
+the latest Api details. We can re-trigger the spec generation by doing a `:clean` on the server module.
+
+Client codegen will be automatically triggered if it detects a spec file change.
+
+Steps involved for `:hts` would be:
+```
+./gradlew :service:housetables:clean :client:hts:build 
+```

development/container-images.md

@@ -0,0 +1,24 @@
+# Container Images
+
+- [Container Images](#container-images)
+  - [Build container images locally](#build-and-run-container-images-locally)
+  - [Prepare MP to produce container images](#prepare-mp-to-produce-container-images)
+  - [Monitoring Metrics]()
+
+## Build and Run container images locally
+
+Dockerfile for each of the services are made available. Below are the example commands for Tables service
+defined by tables-service.Dockerfile.
+
+```  
+    # Build Container Image for Tables Service
+    docker build [--build-arg JAR_FILE=$JAR_FILE | --tag $REPOSITORY:$TAG] --file tables-service.Dockerfile .
+    
+    # Run the Container Image for Tables Services
+    docker run [--env OPENHOUSE_CLUSTER_CONFIG_PATH=$[FILE_PATH]] [--interactive --tty | --detach] -p [$HOST_IP]$HOST_PORT:8080 $REPOSITORY:$TAG
+```
+
+## Monitoring Metrics
+
+Services can be configured to report metrics for operational monitoring using prometheus. OpenHouse uses prometheus
+as the time-series database for storing metrics.