Skip to content

Commit

Permalink
[add] from new mac
Browse files Browse the repository at this point in the history
  • Loading branch information
mulyack2 committed Apr 28, 2024
1 parent b83a792 commit 1d311b9
Show file tree
Hide file tree
Showing 8 changed files with 165 additions and 0 deletions.
49 changes: 49 additions & 0 deletions _posts/bigdata/kafka/2024-04-20-esp.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
title: ESP(Event Streaming Platform)에 대한 이해
date: 2022-11-08
categories: [Bigdata, Kafka]
tags: [Bigdata, Kafka]
---

## ESP의 개념

Event Streaming Platform

- Event Streaming
- 이벤트 소스와 이벤트 대상간의 지속적인 이벤트 전송
- 하나의 이벤트 소스에 하나의 이벤트 대상이 지정된다면 ETL로 할 수 있다.
- 다양한 이벤트가 있고, 이벤트 소스와 이벤트 대상이 다를 때 ESP를 통해 이벤트 관리

### ETL

![image](/assets/img/_posts/bigdata/kafka/etl.png)

### Without ESP

![image](/assets/img/_posts/bigdata/kafka/without_esp.png)

### With ESP

![image](/assets/img/_posts/bigdata/kafka/with_esp.png)

## ESP의 구성

### 1. Broker

이벤트를 수신, 소비하는 브로커

#### Ingestor : 이벤트 수신

#### Processor : 이벤트 처리

직렬화, 역직렬화 / 암호화, 비암호화 / 압축

#### Consumption : 이벤트 배포

### 2. Storage

수신된 이벤트를 저장하는 스토리지

### 3. Analytic and Query Engine

이벤트를 분석, 쿼리하는 엔진
28 changes: 28 additions & 0 deletions _posts/bigdata/kafka/2024-04-20-kafka.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
title: Kafka에 대한 이해
date: 2022-11-08
categories: [Bigdata, Kafka]
tags: [Bigdata, Kafka]
---

## Kafka

### Architecture

분산된 클라이언트, 서버 형태

kafka 브로커

- 여러개의 관련 서버가 있는 클러스터
- 이벤트 수신, 저장 배포 하는 전용 서버
- 브로커는 ZooKeeper라는 분산 시스템에서 관리됨
- 브로커가 효율적이고 협력적으로 업무 수행
- TCP기반의 통신을 통해 클라이언트와 서버 간에 데이터를 교환

### Example

```plaintext
[tweets in JSON] -> [Twitter Producer] -> [twitter_topic]
[twitter_topic] -> [Twitter Comsumer] -> [tweets in JSON]
[tweets in JSON] -> [DB Writer] -> [RDBMS]
```
38 changes: 38 additions & 0 deletions _posts/think/2024-04-20-chmod.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
title: chmod를 확실하게 이해하자
date: 2022-11-08
categories: [Think, Linux]
tags: [Linux]
---

## 권한

권한의 종류
- r : read
- w : write
- x : execute

## 유저

유저의 종류
- u : user (본인)
- g : group (그룹)
- o : others (다른유저)
- a : all (전체)

## 간단한 예시

```plaintext
drwxr-xr-x⠀staff⠀64⠀Apr 24 19:03:26⠀ ⠀my_dir/
-rw-r--r--⠀staff⠀0 ⠀Apr 24 19:03:32⠀ ⠀my_file_01
-rw-r--r--⠀staff⠀0 ⠀Apr 24 19:03:43⠀ ⠀my_file_02
-rw-r--r--⠀staff⠀0 ⠀Apr 24 19:03:44⠀ ⠀my_file_03
0123456789
권한의 해석
0 : is_directory (d / -)
1 : User : read (r / -)
2 : User : write (w / -)
3 : User : execute (e / -)
```
42 changes: 42 additions & 0 deletions _posts/think/2024-04-20-linux.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
title: 리눅스를 살펴보자
date: 2022-11-08
categories: [Think, Linux]
tags: [Linux]
---

## Five Layers of Linux

1. UI(User Interface)
1. Application
1. Operating System
1. Kernel
1. Hard Ware

---

## Shell / Terminal

1. User
1. Terminal
1. Shell / OS / Kernel
1. Hard Ware

---

## Debian / Redhat

### Debian

`apt based`

### Redhat

`yum based`

---

## 권한

- drwx : \[directory]\[read]\[write]\[execute]
- -rwx : \[file]\[read]\[write]\[execute]
8 changes: 8 additions & 0 deletions _posts/write/2023-06-01-after_sk_data_course.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
title: 국비지원교육 및 기업연계프로젝트를 마치며
date: 2023-06-01
categories: [writes]
tags: [_All, _Writes]
---

##
Binary file added assets/img/_posts/bigdata/kafka/etl.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/img/_posts/bigdata/kafka/with_esp.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/img/_posts/bigdata/kafka/without_esp.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 1d311b9

Please sign in to comment.