Skip to content

Commit

Permalink
Merge pull request #53 from RTradeLtd/fixes
Browse files Browse the repository at this point in the history
Remove References To Upstream
  • Loading branch information
bonedaddy authored Apr 20, 2020
2 parents a85f85c + f3bbb75 commit 552d1e0
Show file tree
Hide file tree
Showing 12 changed files with 22 additions and 16 deletions.
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ ENV GO111MODULE on

RUN \
apk add --no-cache git && \
git clone https://github.com/minio/minio && cd minio && \
git clone https://github.com/RTradeLtd/s3x && cd minio && \
go install -v -ldflags "$(go run buildscripts/gen-ldflags.go)"

FROM alpine:3.10
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile.arm.release
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ ENV GO111MODULE on

RUN \
apk add --no-cache git 'curl>7.61.0' && \
git clone https://github.com/minio/minio && \
git clone https://github.com/RTradeLtd/s3x && \
curl -L https://github.com/balena-io/qemu/releases/download/v3.0.0%2Bresin/qemu-3.0.0+resin-arm.tar.gz | tar zxvf - -C . && mv qemu-3.0.0+resin-arm/qemu-arm-static .

FROM arm32v7/alpine:3.10
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile.arm64.release
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ ENV GO111MODULE on

RUN \
apk add --no-cache git 'curl>7.61.0' && \
git clone https://github.com/minio/minio && \
git clone https://github.com/RTradeLtd/s3x && \
curl -L https://github.com/balena-io/qemu/releases/download/v3.0.0%2Bresin/qemu-3.0.0+resin-arm.tar.gz | tar zxvf - -C . && mv qemu-3.0.0+resin-arm/qemu-arm-static .

FROM arm64v8/alpine:3.10
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile.release
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ ENV GO111MODULE on

RUN \
apk add --no-cache git && \
git clone https://github.com/minio/minio
git clone https://github.com/RTradeLtd/s3x

FROM alpine:3.10

Expand Down
4 changes: 2 additions & 2 deletions Dockerfile.simpleci
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
#-------------------------------------------------------------
FROM golang:1.13

COPY . /go/src/github.com/minio/minio
WORKDIR /go/src/github.com/minio/minio
COPY . /go/src/github.com/RTradeLtd/s3x
WORKDIR /go/src/github.com/RTradeLtd/s3x

RUN apt-get update && apt-get install -y jq
ENV GO111MODULE=on
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ GOARCH := $(shell go env GOARCH)
GOOS := $(shell go env GOOS)

VERSION ?= $(shell git describe --tags)
TAG ?= "minio/minio:$(VERSION)"
TAG ?= "rtradetech/s3x:$(VERSION)"

BUILD_LDFLAGS := '$(LDFLAGS)'

Expand Down
10 changes: 8 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,13 +104,19 @@ In order to enable the speed needed by S3 applications we used TemporalX to prov
* Compared to go-ipfs and existing IPFS implementations it is **[fast](https://medium.com/temporal-cloud/temporalx-vs-go-ipfs-official-node-benchmarks-8457037a77cf)**
* Built in data replication

The default setting of s3x is to use a publicly available TemporalX server, however this is only sufficient for demo/trial purposes and in production you'll want to be running a TemporalX server yourself. To reduce network delays, and latency decreasing performance of s3x, it is recommended that you run TemporalX on the same box as you are running s3x, however this is only recommendation and it is totally fine to use a remote TemporalX node.

## Storage

All data, that is all the objects, buckets, metadata, is stored on IPFS. Because IPFS uses hashes, and S3 uses arbitrary names for the buckets and objects, we perform bookkeeping with a "ledger". The actual "on-ipfs" structure of the objects, and buckets themselves is defined via protocol buffers, saved as an IPLD "raw" object type. They contain the data if there, as well as object and bucket metadata.

## Ledger
## Ledger Store

The "ledger store" is an internal book keeper responsible for keeping a map of bucket and objects names, as well as the latest version of their IPFS CID. The ledger is what bridges TemporalX/IPFS to S3/MinIO, and is solely responsible for this `name->hash` map. All object/bucket data is stored on IPFS, with the only "off-IPFS" being this `name->hash` map, sometimes called "ledger store metadata". The default implementation is done using a badger v2 datastore using `dgraph-io/badger` sufficient for running single s3x node installations.

There is a second available option for this ledger store that enables highly available s3x setupts with multiple nodes. By using a CRDT datastore ontop of a badger datastore, updates to the ledger state can be broadcasted between any s3x instance configured to listen to the same topic. One thing to note is that the performance of this will be slightly worse than the pure badger ledger store.

The "ledger" is an internal book keeper responsible for keeping track of the latest IPFS CID's that belong to each and every object, and bucket stored, and is currently implemented as a `dgraph-io/badger/v2` key-value datastore.
Please note that this doesn't give high availability for the actual bucket/object data. For this you would want to use TemporalX's built-in replication system.

# Kubernetes

Expand Down
2 changes: 1 addition & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Whenever there is a security update you just need to upgrade to the latest versi

## Reporting a Vulnerability

All security bugs in [minio/minio](https://github,com/minio/minio) (or other minio/* repositories)
All security bugs in [RTradeLtd/s3x](https://github,com/RTradeLtd/s3x) (or other minio/* repositories)
should be reported by email to [email protected]. Your email will be acknowledged within 48 hours,
and you'll receive a more detailed response to your email within 72 hours indicating the next steps
in handling your report.
Expand Down
4 changes: 2 additions & 2 deletions browser/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,14 @@
},
"repository": {
"type": "git",
"url": "https://github.com/minio/minio"
"url": "https://github.com/RTradeLtd/s3x"
},
"author": "MinIO Inc",
"license": "Apache-2.0",
"bugs": {
"url": "https://github.com/RTradeLtd/s3x/issues"
},
"homepage": "https://github.com/minio/minio",
"homepage": "https://github.com/RTradeLtd/s3x",
"devDependencies": {
"async": "^1.5.2",
"babel-cli": "^6.26.0",
Expand Down
2 changes: 1 addition & 1 deletion kubernetes_local.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ spec:
claimName: workdir
containers:
- name: s3x
image: rtradetech/s3x-test:latest
image: rtradetech/s3x:latest
args: ["gateway", "s3x", "--temporalx.endpoint", "localhost:9090", "--temporalx.insecure"]
env:
- name: MINIO_ACCESS_KEY
Expand Down
2 changes: 1 addition & 1 deletion kubernetes_remote.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ spec:
claimName: workdir
containers:
- name: s3x
image: rtradetech/s3x-test:latest
image: rtradetech/s3x:latest
args: ["gateway", "s3x", "--temporalx.insecure"]
env:
- name: MINIO_ACCESS_KEY
Expand Down
4 changes: 2 additions & 2 deletions minio.spec
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
%define tag RELEASE.2017-04-25T01-27-49Z
%define subver %(echo %{tag} | sed -e 's/[^0-9]//g')
# git fetch https://github.com/minio/minio.git refs/tags/RELEASE.2017-02-16T01-47-30Z
# git fetch https://github.com/RTradeLtd/s3x.git refs/tags/RELEASE.2017-02-16T01-47-30Z
# git rev-list -n 1 FETCH_HEAD
%define commitid 83abb310b4ce3a0dfc6d7faf78e33cb6f9132cfe
Summary: Cloud Storage Server.
Expand All @@ -21,7 +21,7 @@ BuildRoot: %{tmpdir}/%{name}-%{version}-root-%(id -u -n)
## Go related tags.
%define gobuild(o:) go build -ldflags "${LDFLAGS:-}" %{?**};
%define gopath %{_libdir}/golang
%define import_path github.com/minio/minio
%define import_path github.com/RTradeLtd/s3x

%description
MinIO is an object storage server released under Apache License v2.0.
Expand Down

0 comments on commit 552d1e0

Please sign in to comment.