Skip to content

Commit

Permalink
updated README
Browse files Browse the repository at this point in the history
  • Loading branch information
tikazyq committed May 22, 2022
1 parent 4f875cd commit f479f3d
Show file tree
Hide file tree
Showing 4 changed files with 16 additions and 135 deletions.
26 changes: 4 additions & 22 deletions README-zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,25 +30,11 @@

基于Golang的分布式爬虫管理平台支持PythonNodeJSGoJavaPHP等多种编程语言以及多种爬虫框架

[查看演示 Demo](https://demo-pro.crawlab.cn) | [文档](https://docs.crawlab.cn) | [文档 (v0.6-beta)](https://docs-next.crawlab.cn)
[查看演示 Demo](https://demo-pro.crawlab.cn) | [文档](https://docs.crawlab.cn/zh/)

## 安装

三种方式:
1. [Docker](http://docs.crawlab.cn/zh/Installation/Docker.html)(推荐)
2. [直接部署](http://docs.crawlab.cn/zh/Installation/Direct.html)(了解内核)
3. [Kubernetes](http://docs.crawlab.cn/zh/Installation/Kubernetes.html) (多节点部署)

### 要求Docker
- Docker 18.03+
- MongoDB 3.6+
- Docker Compose 1.24+ (可选但推荐)

### 要求直接部署
- Go 1.15+
- Node 12.20+
- MongoDB 3.6+
- [SeaweedFS](https://github.com/chrislusf/seaweedfs) 2.59+
您可以参考这个[安装指南](https://docs.crawlab.cn/zh/guide/installation)。

## 快速开始

Expand Down Expand Up @@ -109,7 +95,7 @@ services:
- master
mongo:
image: mongo:latest
image: mongo:4.2
container_name: crawlab_example_mongo
restart: always
```
Expand All @@ -120,11 +106,7 @@ services:
docker-compose up -d
```
Docker部署的详情,请见[相关文档](https://tikazyq.github.io/crawlab-docs/Installation/Docker.html)。
### 直接部署
请参考[相关文档](https://tikazyq.github.io/crawlab-docs/Installation/Direct.html)。
Docker部署的详情,请见[相关文档](https://docs.crawlab.cn/zh/guide/installation/docker.html)。
## 截图
Expand Down
24 changes: 5 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,25 +30,11 @@

Golang-based distributed web crawler management platform, supporting various languages including Python, NodeJS, Go, Java, PHP and various web crawler frameworks including Scrapy, Puppeteer, Selenium.

[Demo](https://demo-pro.crawlab.cn) | [Documentation](https://docs.crawlab.cn) | [Documentation (v0.6-beta)](http://docs-next.crawlab.cn)
[Demo](https://demo-pro.crawlab.cn) | [Documentation](https://docs.crawlab.cn/en/)

## Installation

Three methods:
1. [Docker](http://docs.crawlab.cn/en/Installation/Docker.html) (Recommended)
2. [Direct Deploy](http://docs.crawlab.cn/en/Installation/Direct.html) (Check Internal Kernel)
3. [Kubernetes](http://docs.crawlab.cn/en/Installation/Kubernetes.html) (Multi-Node Deployment)

### Pre-requisite (Docker)
- Docker 18.03+
- MongoDB 3.6+
- Docker Compose 1.24+ (optional but recommended)

### Pre-requisite (Direct Deploy)
- Go 1.15+
- Node 12.20+
- MongoDB 3.6+
- [SeaweedFS](https://github.com/chrislusf/seaweedfs) 2.59+
You can follow the [installation guide](https://docs.crawlab.cn/en/guide/installation/).

## Quick Start

Expand All @@ -60,7 +46,7 @@ cd examples/docker/basic
docker-compose up -d
```

Next, you can look into the `docker-compose.yml` (with detailed config params) and the [Documentation (Chinese)](http://docs.crawlab.cn) for further information.
Next, you can look into the `docker-compose.yml` (with detailed config params) and the [Documentation](http://docs.crawlab.cn/en/) for further information.

## Run

Expand Down Expand Up @@ -110,7 +96,7 @@ services:
- master

mongo:
image: mongo:latest
image: mongo:4.2
container_name: crawlab_example_mongo
restart: always
```
Expand All @@ -121,7 +107,7 @@ Then execute the command below, and Crawlab Master and Worker Nodes + MongoDB wi
docker-compose up -d
```
For Docker Deployment details, please refer to [relevant documentation](https://tikazyq.github.io/crawlab-docs/Installation/Docker.html).
For Docker Deployment details, please refer to [relevant documentation](https://docs.crawlab.cn/en/guide/installation/docker.html).
## Screenshot
Expand Down
85 changes: 0 additions & 85 deletions docker-compose.local.yml

This file was deleted.

16 changes: 7 additions & 9 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,33 +4,31 @@ services:
image: crawlabteam/crawlab:latest
container_name: crawlab_master
environment:
CRAWLAB_BASE_URL: crawlab
CRAWLAB_SERVER_MASTER: Y
CRAWLAB_NODE_MASTER: Y
CRAWLAB_MONGO_HOST: mongo
ports:
- "8080:8080" # frontend port mapping 前端端口映射
depends_on:
- mongo
# volumes:
# - "/var/crawlab/log:/var/logs/crawlab" # log persistent 日志持久化
# - "/opt/crawlab/master:/data" # data persistent 持久化数据
worker:
image: crawlabteam/crawlab:latest
container_name: worker
container_name: crawlab_worker
environment:
CRAWLAB_SERVER_MASTER: "N"
CRAWLAB_NODE_MASTER: "N"
CRAWLAB_MONGO_HOST: "mongo"
# CRAWLAB_REDIS_ADDRESS: "redis"
depends_on:
- mongo
# volumes:
# - "/var/crawlab/log:/var/logs/crawlab" # log persistent 日志持久化
# - "/opt/crawlab/worker:/data" # data persistent 持久化数据
mongo:
image: mongo:latest
image: mongo:4.2
#restart: always
# environment:
# MONGO_INITDB_ROOT_USERNAME: username
# MONGO_INITDB_ROOT_PASSWORD: password
# volumes:
# - "/opt/crawlab/mongo/data/db:/data/db" # make data persistent 持久化
# - "/opt/crawlab/mongo/data/db:/data/db" # data persistent 持久化数据
# ports:
# - "27017:27017" # expose port to host machine 暴露接口到宿主机

0 comments on commit f479f3d

Please sign in to comment.