Skip to content

Commit

Permalink
1. Ingore 'dist' directory
Browse files Browse the repository at this point in the history
2. Update docs
  • Loading branch information
chenhjia committed Nov 29, 2019
1 parent ce3d643 commit 99869a8
Show file tree
Hide file tree
Showing 7 changed files with 30 additions and 29 deletions.
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,5 @@ out/
*.iws
__pycache__/
*.pyc
logs
logs
dist/
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)

English | [中文](docs/zh_CN/ch1/README.md)
English | [中文](docs/zh_CN/ch1)

## Overview
Qualitis is a one-stop data quality management platform that supports quality verification, notification, and management for various datasource. It is used to solve various data quality problems caused by data processing.
Expand All @@ -11,10 +11,10 @@ At the same time, Qualitis provides enterprise-level features of financial-level

## Features
- **Define Data Quality Model**
Supports data quality model below:
Supports data quality model below:
1.Model of single table.
2.Model of multi-table.
3.Custom model.
2.Model of multi-table.
3.Custom model.
<br /> At the same time, Qualitis presets multiple data quality verification templates, including common checks such as null check, blank check, number check, enumeration check and so on, which simplifies data quality model definition.

- **Data Quality Model Scheduling**
Expand All @@ -31,7 +31,7 @@ Supports abnormal data storage, which can quickly locate problems.

- **Workflow**
Supports workflow
Workflow needs [DSS](https://github.com/WeBankFinTech/Linkis).
Workflow needs [DataSphereStudio](https://github.com/WeBankFinTech/DataSphereStudio).

- **Administrator Console**
Administrator console provided.
Expand Down
4 changes: 2 additions & 2 deletions docs/en_US/ch1/QuickDeploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Hadoop (2.7.2)
Hive (1.2.1)
Zookeeper (3.4.9)
Linkis (0.9.0), Spark engine required.[How to install Linkis?](https://github.com/WeBankFinTech/Linkis)
DSS (0.5.0) Optional. If you want to use workflow, it it required. [How to install DSS?](https://github.com/WeBankFinTech/DSS)
DataSphereStudio (0.5.0) Optional. If you want to use workflow, it it required. [How to install DataSphereStudio?](https://github.com/WeBankFinTech/DataSphereStudio)

## 2. Download
[Download](https://github.com/WeBankFinTech/Qualitis/releases)
Expand Down Expand Up @@ -95,7 +95,7 @@ Here is an example:

Tips:
Workflow default supported.
You need to install [DSS](https://github.com/WeBankFinTech/DSS) to use workflow.
You need to install [DataSphereStudio](https://github.com/WeBankFinTech/DataSphereStudio) to use workflow.
Enabling workflow does not affect the normal use of Qualitis.
If you want to close it, you can do as follow:
```
Expand Down
4 changes: 2 additions & 2 deletions docs/en_US/ch1/User Manual.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
## 1. Introduction
Qualitis is a one-stop data quality management platform that supports quality verification, notification, and management for various datasource. It is used to solve various data quality problems caused by data processing.

Based on Spring Boot, Qualitis submits quality model task to [Linkis](https://github.com/WeBankFinTech/Linkis). It provides functions such as data quality model construction, data quality model execution, data quality verification, reports of data quality generation and so on.
Based on Spring Boot, Qualitis analysis quality model by [Linkis](https://github.com/WeBankFinTech/Linkis). It provides functions such as data quality model construction, timing execution, trigger execution, data quality verification, reports of data quality generation and so on.

At the same time, Qualitis provides enterprise-level features of financial-level resource isolation, management and access control. It is also guaranteed working well under high-concurrency, high-performance and high-availability scenarios.
At the same time, Qualitis provides enterprise-level features of financial-level resource isolation, management, access control and collaborative edition. It is also guaranteed working well under high-concurrency, high-performance and high-availability scenarios.

## 2. Login
![登录](../../../images/en_US/ch1/Login.png)
Expand Down
6 changes: 3 additions & 3 deletions docs/zh_CN/ch1/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
[![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)

[English](../../../README.md) | 中文
[English](../../../) | 中文

## 引言
Qualitis是一个支持多种异构数据源的质量校验、通知、管理服务的一站式数据质量管理平台,用于解决业务系统运行、数据中心建设及数据治理过程中的各种数据质量问题。

Qualitis基于Spring Boot开发,依赖于Linkis进行数据计算,提供数据质量模型构建,数据质量模型执行,数据质量任务管理,异常数据发现保存以及数据质量报表生成等功能。并提供了金融级数据质量模型资源隔离,资源管控,权限隔离等企业特性,具备高并发,高性能,高可用的大数据质量管理能力。
Qualitis基于Spring Boot,依赖于Linkis进行数据计算,提供数据质量模型构建,数据质量模型执行,数据质量任务管理,异常数据发现保存以及数据质量报表生成等功能。并提供了金融级数据质量模型资源隔离,资源管控,权限隔离等企业特性,具备高并发,高性能,高可用的大数据质量管理能力。

## 核心特点
- **数据质量模型定义**
Expand All @@ -29,7 +29,7 @@ Qualitis基于Spring Boot开发,依赖于Linkis进行数据计算,提供数

- **支持工作流**
支持在工作流当中进行数据质量校验
工作流必装[DSS](https://github.com/WeBankFinTech/Linkis).
工作流必装[DataSphereStudio](https://github.com/WeBankFinTech/Linkis).

- **管理员控制台**
提供管理员控制台界面,支持人员管理,权限管理,权限管理,元数据管理等管理功能。
Expand Down
4 changes: 2 additions & 2 deletions docs/zh_CN/ch1/快速搭建手册.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Hadoop (2.7.2)
Hive (1.2.1)
Zookeeper (3.4.9)
Linkis(0.9.0), 必装Spark引擎。[如何安装Linkis](https://github.com/WeBankFinTech/Linkis)
DSS (0.5.0) 可选. 如果你想使用工作流,必装DSS [如何安装DSS?](https://github.com/WeBankFinTech/DSS)
DataSphereStudio (0.5.0) 可选. 如果你想使用工作流,必装DataSphereStudio [如何安装DataSphereStudio?](https://github.com/WeBankFinTech/DataSphereStudio)

## 二、安装包下载
[下载](https://github.com/WeBankFinTech/Qualitis/releases)
Expand Down Expand Up @@ -94,7 +94,7 @@ Hive数据库密码(用于连接HiveMetaStore的MySQL数据库)
---

Tips:
工作流已默认开启,工作流需要安装DSS[https://github.com/WeBankFinTech/Linkis]
工作流已默认开启,工作流需要安装DataSphereStudio[https://github.com/WeBankFinTech/DataSphereStudio]
开启工作流不影响Qualitis正常使用。
如需关闭,可修改配置文件
```
Expand Down
28 changes: 14 additions & 14 deletions docs/zh_CN/ch1/接入工作流指南.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
# 接入工作流指南

## 一、介绍
DSS定位为闭环涵盖数据应用的全流程,在统一的UI下,以工作流式的图形化拖拽开发体验。Qualitis可以接入DSS,成为其中的一个工作流节点。
DataSphereStudio定位为闭环涵盖数据应用的全流程,在统一的UI下,以工作流式的图形化拖拽开发体验。Qualitis可以接入DataSphereStudio,成为其中的一个工作流节点。

Qualitis接入DSS,会有以下变动。
Qualitis接入DataSphereStudio,会有以下变动。

#### 项目同步
Qualitis中工作流项目与DSS中工程打通并同步
若DSS中创建工程,则Qualitis中会同步创建工程,删除修改亦然。
以DSS中工程为主
Qualitis中工作流项目与DataSphereStudio中工程打通并同步
若DataSphereStudio中创建工程,则Qualitis中会同步创建工程,删除修改亦然。
以DataSphereStudio中工程为主

#### 用户自动创建
由于DSS内嵌Qualitis的页面,并且DSS和Qualitis中登录系统不同
**为了保证DSS中能够打开Qualitis的页面,当DSS中触发打开Qualitis页面时,若登录DSS用户在Qualitis端不存在,则Qualitis会自动创建该用户,且该用户的密码与用户名相同。**
由于DataSphereStudio内嵌Qualitis的页面,并且DataSphereStudio和Qualitis中登录系统不同
**为了保证DataSphereStudio中能够打开Qualitis的页面,当DataSphereStudio中触发打开Qualitis页面时,若登录DataSphereStudio用户在Qualitis端不存在,则Qualitis会自动创建该用户,且该用户的密码与用户名相同。**

#### DSS工作流节点与Qualitis规则组一一对应
一个DSS Qualitis工作流节点与Qualitis中的一组规则对应。
当在DSS中拖出一个工作流节点,并创建出规则时,会在Qualitis中创建一组规则与之对应。
#### DataSphereStudio工作流节点与Qualitis规则组一一对应
一个DataSphereStudio Qualitis工作流节点与Qualitis中的一组规则对应。
当在DataSphereStudio中拖出一个工作流节点,并创建出规则时,会在Qualitis中创建一组规则与之对应。


## 二、接入步骤
Expand All @@ -27,9 +27,9 @@ vim conf/application.yml
```
并修改workflow.enable=true

### DSS修改
DSS接入外部系统原理可看文章[TODO]()
**1. DSS服务端修改**
### DataSphereStudio修改
DataSphereStudio接入外部系统原理可看文章:[第三方系统接入DSS指南](https://github.com/WeBankFinTech/DataSphereStudio/blob/master/docs/zh_CN/ch4/%E7%AC%AC%E4%B8%89%E6%96%B9%E7%B3%BB%E7%BB%9F%E6%8E%A5%E5%85%A5DSS%E6%8C%87%E5%8D%97.md)
**1. DataSphereStudio服务端修改**
##### 数据库修改

修改dss_application表,增加如下记录,该表用于增加首页以及菜单页面的跳转逻辑
Expand All @@ -56,7 +56,7 @@ DSS接入外部系统原理可看文章[TODO]()

并创建dss-server/appjoints/qualitis/appjoint.properties以及linkis-appjoint-entrance/appjoints/qualitis/appjoint.properties文件(空文件)。

**2. DSS客户端修改(已默认修改,如无变动,可跳过该步骤)**
**2. DataSphereStudio客户端修改(已默认修改,如无变动,可跳过该步骤)**
2.1 增加节点类型
修改src/js/service/nodeType.js文件,增加Qualitis节点类型

Expand Down

0 comments on commit 99869a8

Please sign in to comment.