Skip to content

Commit

Permalink
Merge pull request #84 from AI4WA/feature/i-52-documentations
Browse files Browse the repository at this point in the history
update docs index
  • Loading branch information
PascalSun authored Jun 3, 2024
2 parents 44e8c7c + c895cd2 commit c1cb88d
Show file tree
Hide file tree
Showing 2 changed files with 34 additions and 15 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ How we construct this unified multimodal knowledge graph step by step:

---

## Setup and Developement
## Setup and Development

```bash
python3 -m venv venv
Expand Down
47 changes: 33 additions & 14 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,37 @@
# [Welcome to Docs2KG](https://ai4wa.com)

**An Open Source Framework for Transforming Unstructured Data into Unified Knowledge Graph**
**Unified Knowledge Graph Construction from Heterogeneous Documents Assisted by Large
Language Models**

![PyPI](https://img.shields.io/pypi/v/Docs2KG)
[![Demo](https://img.shields.io/badge/Demo-Available-blue)](https://docs2kg.ai4wa.com/Video/)
![Lint](https://github.com/AI4WA/Docs2KG/actions/workflows/lint.yml/badge.svg)
![Documentation](https://github.com/AI4WA/Docs2KG/actions/workflows/docs.yml/badge.svg)
![Status](https://img.shields.io/badge/Status-Work%20in%20Progress-yellow)

## Installation

We have published the package to PyPi, you can install it via:

```bash
pip install Docs2KG
```

---

## Motivation

Three pillars of the LLM applications in our opinion:

- Data
- RAG
- LLM

Most of the tools in the market nowadays are focusing on the Retrieval Augmented Generation (RAG) pipelines or
Most of the tools in the market nowadays are focusing on the **Retrieval Augmented Generation (RAG) pipelines** or
How to get Large Language Models (LLMs) to run locally.

Typical tools include: Ollama, LangChain, LLamaIndex, etc.

However, to make sure the wider community can benefit from the latest research, we need to first solve the data problem.

The Wider community includes personal users, small business, and even large enterprises.
Expand All @@ -19,38 +40,36 @@ form, and distributed in different places.

So the first challenges will be:

- How can we easily process the unstructured data into a centralized place?
- What is the best way to organize the data within the centralized place?
- **How can we easily process the unstructured data into a centralized place?**
- **What is the best way to organize the data within the centralized place?**

## Proposed Solution

This package is a proposed solution to the above challenges.

- We developed the tool for the wider community to easily process the unstructured data into a centralized place.
- We proposed a way to organize the data within the centralized place, via a Unified Knowledge Graph altogether with
semi-structured data.
- We proposed a way to organize the data within the centralized place, via a Unified Multimodal Knowledge Graph
altogether with semi-structured data.

### Overall Architecture

The overall workflow will be:

![img.png](images/Docs2KG.jpg)

## System Architecture
### Implemented System Architecture

![img.png](images/Modules.jpg)

### PDF
### Unified Multimodal Knowledge Graph

The Main data under this category is PDF.

So we need to be able to extract the text and images from the PDF.

## Unified Multimodal Knowledge Graph
How we construct this unified multimodal knowledge graph step by step:

![img.png](images/KGConstruction.jpg)

## Development Setup
---

## Setup and Development

```bash
python3 -m venv venv
Expand Down

0 comments on commit c1cb88d

Please sign in to comment.