-
Notifications
You must be signed in to change notification settings - Fork 26
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #225 from treeverse/glue-iceberg-demo-v1.0
Added Iceberg integration notebooks which work with AWS Glue
- Loading branch information
Showing
5 changed files
with
1,267 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
# Integration of lakeFS with Glue Notebooks and Iceberg | ||
|
||
Start by ⭐️ starring [lakeFS open source](https://go.lakefs.io/oreilly-course) project. | ||
|
||
This repository includes following Glue notebooks: | ||
|
||
1. iceberg-books: | ||
* Use Case: Isolated Dev/Test Environments | ||
* This interactive notebook demonstrate integration of lakeFS with [Glue Notebooks](https://docs.aws.amazon.com/glue/latest/dg/notebook-getting-started.html) and [Iceberg](https://docs.lakefs.io/integrations/iceberg.html) with a basic example. | ||
|
||
2. iceberg-lakefs-nyc: | ||
* Use Case: Isolated Dev/Test Environments | ||
* This interactive notebook demonstrate integration of lakeFS with Glue Notebooks and Iceberg for [NYC Film Permits](https://data.cityofnewyork.us/City-Government/Film-Permits/tg4x-b46p/about_data) example. | ||
|
||
|
||
## Prerequisites | ||
* lakeFS installed and running in your AWS environment or in the lakeFS Cloud. If you don't have lakeFS already running then either use [lakeFS Cloud](https://lakefs.cloud/) which provides free lakeFS server on-demand with a single click or [Deploy lakeFS on AWS](https://docs.lakefs.io/howto/deploy/aws.html) doc. | ||
|
||
|
||
## Setup | ||
|
||
Download these notebooks from GitHub and upload these notebooks as Jupyter notebooks in your AWS Glue Studio. | ||
|
||
## Demo Instructions | ||
|
||
Open the notebook in Glue Studio and follow the instructions. |
Oops, something went wrong.