Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore/broswerfs deprecated issue 127 #93

Draft
wants to merge 65 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
20f1fdf
Update license, package.json, and add basic notice to the Readme (#3)
wilwade Apr 27, 2021
5d03d1b
Bloom r us (#2)
enddynayn May 5, 2021
5547ad4
GitHub Actions NPM Release (#5)
wilwade May 6, 2021
c887f10
Setup package.json to support the dist directory instead of just incl…
wilwade May 10, 2021
7d3e6dc
Build before publishing (#7)
wilwade May 10, 2021
ce24b50
chore(README): update to include bloom filter (#9)
enddynayn May 11, 2021
4099f37
use xxhash-wasm (#8)
shannonwells May 12, 2021
2ccc4eb
Update package org from unfinishedlabs -> dsnp (#12)
wilwade May 18, 2021
a8ddc96
Chore/pre release 0.1.0 (#13)
wilwade Jun 7, 2021
a4f425a
Don't publish types for now (#14)
wilwade Jun 7, 2021
ba60652
feat(fetch): add browser/node http client (#15)
enddynayn Jun 18, 2021
b35942b
Parquetjs browser support (#17)
shannonwells Jun 30, 2021
f605856
fix bug caused by side-effect of change in xxhasher. regen package-lo…
shannonwells Jul 1, 2021
ac8ebb2
update BSON due to vulnerability, minor fixes (#20)
shannonwells Jul 1, 2021
81e38e4
add cjs and ejm builds (#21)
acruikshank Jul 13, 2021
98871a2
Clean up dependencies and remove lodash (#22)
wilwade Aug 17, 2021
44d9f3c
Added test cases for types and fixed bigint precision (#24)
waylandli Oct 21, 2021
a82975b
Chore/typescript conversion/27 (#33)
waylandli Nov 30, 2021
7c7bbb9
updated typescript and esbuild. Fixed typescript issue (#35)
waylandli Dec 2, 2021
6fb5fcd
Convert lib/schema.js to typescript 1/3 (#37)
waylandli Feb 22, 2022
f1bee25
Converting shred.js to typescript 2/3 (#40)
waylandli Feb 24, 2022
14bbb97
Convertinging util.js to typescript 3/3 (#39)
waylandli Feb 25, 2022
ce1f98b
Reader conversion (#44)
waylandli Mar 16, 2022
eb029bd
[Issue #41] - Slimmed down FixedTFramedTransport class (#46)
dopatraman Mar 16, 2022
d3b2089
Removed redundant reference to callback (#48)
dopatraman Apr 12, 2022
a1bc22e
fix for typed array BYTE_ARRAY (#23) (#49)
aramikm Apr 13, 2022
4bd382e
removed force32 (#51)
aramikm Apr 19, 2022
91ac192
Removed ts-ignored lines (#50)
dopatraman Apr 19, 2022
8f30383
Add test coverage for Buffer Reader (#52)
dopatraman Apr 22, 2022
b4ca349
Support async iteration in the reader (#55)
wilwade Apr 26, 2022
33c80fe
Export the types into the package (#54)
wilwade Apr 26, 2022
0b027a3
Removed async declaration (#59)
dopatraman Apr 29, 2022
5a94d60
Chore/ts writer take2 (#60)
mehtaishita May 10, 2022
555af72
Upstream bug fixes for RLE encoding (#61)
dopatraman May 10, 2022
cadf486
Performance improvements using a cursor instead of a shift (#57)
wilwade May 10, 2022
2f150a0
Update packages (#63)
wilwade May 10, 2022
b8bfc07
Remove baseUrl config option (#65)
wilwade May 11, 2022
c86f490
Chore/fix entry file (#66)
wilwade May 11, 2022
d382d0f
Types Cleanup (#67)
wilwade May 11, 2022
73cf70d
Better handling of millis and micros (#68)
wilwade May 12, 2022
a2cd4ff
Bug/browser types bug #70 (#71)
wilwade Jun 22, 2022
a62db08
Fix typo in comment (#74)
JasonYeMSFT Dec 13, 2022
a011a2e
Feature - collect and report multiple field errors (#75)
dgaudet Mar 8, 2023
2c733b5
Add ability to read decimal columns (#79)
dgaudet Apr 26, 2023
b5698e4
Feature: Parquet Schema from JSON Schema (#82) with
wilwade May 25, 2023
17cb5ed
update to node 16, FIX HASHER BUG (#84)
shannonwells May 25, 2023
fa1865b
Ensure Buffer objects are returned by compression functions (#88)
JasonYeMSFT Jun 21, 2023
19f3ffa
Decimal Writer Support (#90)
wilwade Jun 23, 2023
43732c5
Add null pages and boundary order (Fixes #92) (#94)
wilwade Jul 13, 2023
ac5257d
Feature: Timestamp support for JSONSchema generated schemas (#95)
noxify Jul 25, 2023
c07e7e8
Add support to byte array decimal fields (#97)
YECHUNAN Aug 14, 2023
bda4e3f
upgrade to nodejs 18.18.2 (#101)
shannonwells Nov 17, 2023
19707ef
Support bloom filters in lists/nested columns (#105)
shannonwells Nov 28, 2023
e0f1ebd
Update docs and add simple test for readme encoding examples (#107)
wilwade Jan 9, 2024
0a42955
Use the Double primative for JSON Schema "number" type (#111)
mpotter Jan 11, 2024
6fdb9da
Update index.ts to support RLE_DICTIONARY (#112)
saritvakrat Jan 18, 2024
8d34ac1
Reference Tests and Breaking Change: Optional nullable fields are now…
wilwade Jan 19, 2024
2622ff1
allow typeLength to come from opts.column when decoding FIXED_LEN_BYT…
j4ys0n Jan 19, 2024
117e5a5
Feat/support aws s3 v3 (#115)
shannonwells Jan 30, 2024
91fc71f
Update Deprecated Function Calls (#118)
wilwade Feb 10, 2024
3de7eea
fix: Fix incorrect primitive type detection (#122)
JasonYeMSFT Mar 13, 2024
5db9db4
Swap brotli dependency (#123)
wilwade Mar 14, 2024
2435994
Dependency updates (#128)
wilwade May 14, 2024
6396d26
Linting and Formatting (#133)
wilwade Jul 1, 2024
d6b64e0
chore(*): replace browserfs for zenfs
enddynayn Jul 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .babelrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
module.exports = {
sourceType: 'unambiguous',
plugins: ['babel-plugin-add-module-exports'],
presets: [
[
'@babel/preset-env',
{
loose: true,
modules: 'auto',
useBuiltIns: 'entry', // to ensure regeneratorRuntime is defined; see bootstrap.js
corejs: 3, // use w/ "useBuiltIns", defaults=2, must match what is in package.json
// "targets": "> 0.25%, not dead"
},
],
'@babel/preset-typescript',
],
};
29 changes: 29 additions & 0 deletions .github/ISSUE_TEMPLATE/BUG_Issue.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
name: Bug Report
about: Report an issue found in @dsnp/parquetjs
title: ""
labels: ""
assignees: ""
---

Thanks for reporting an issue!

### Steps to reproduce

Tell us how to reproduce this issue.

### Expected behaviour

Tell us what should happen

### Actual behaviour

Tell us what happens instead

### Any logs, error output, etc?

...

### Any other comments?

...
2 changes: 2 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# yaml-language-server: $schema=https://json.schemastore.org/github-issue-config.json
blank_issues_enabled: true
22 changes: 22 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Problem

problem statement
Solution
========
What I/we did to solve this problem

with @pairperson1

## Change summary:

- Tidy, well formulated commit message
- Another great commit message
- Something else I/we did

## Steps to Verify:

1. A setup step / beginning state
1. What to do next
1. Any other instructions
1. Expected behavior
1. Suggestions for testing
30 changes: 30 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# yaml-language-server: $schema=https://json.schemastore.org/github-workflow.json
name: Tests CI
on:
push:
branches: [main]
pull_request:
branches: ['**']
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Use Node.js
uses: actions/setup-node@v4
with:
node-version: 18

- name: Install 💾
run: npm ci

- name: Lint
run: npm run lint

- name: Test
run: npm run test

- name: Build & Publish Dry Run
run: npm publish --dry-run
30 changes: 30 additions & 0 deletions .github/workflows/publish-next.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# yaml-language-server: $schema=https://json.schemastore.org/github-workflow.json
name: Publish NPM @next
on:
push:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Use Node.js
uses: actions/setup-node@v4
with:
node-version: 18
registry-url: 'https://registry.npmjs.org'

- name: Install 💾
run: npm ci

- name: Version ⬆️
run: npm version --new-version "v0.0.0-${FULL_SHA:0:6}" --no-git-tag-version
env:
FULL_SHA: ${{ github.sha }}

- name: Publish @next 🚂
run: npm publish --tag next --access public
env:
NODE_AUTH_TOKEN: ${{secrets.NODE_AUTH_TOKEN}}
28 changes: 28 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# yaml-language-server: $schema=https://json.schemastore.org/github-workflow.json
name: Release Package
on:
release:
types: [released]
jobs:
publish-to-npm:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Use Node.js
uses: actions/setup-node@v4
with:
node-version: 18
registry-url: 'https://registry.npmjs.org'

- name: Install 💾
run: npm ci

- name: Version ⬆️
run: npm version --new-version ${{ github.event.release.tag_name }} --no-git-tag-version

- name: Publish 🚂
run: npm publish --tag latest
env:
NODE_AUTH_TOKEN: ${{secrets.NODE_AUTH_TOKEN}}
9 changes: 6 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
.DS_STORE
node_modules
*.parquet
npm-debug.log
package-lock.json
.nyc_output

!test/test-files/*.parquet
dist
!test/test-files/*.parquet
!test/reference-test/files/*.parquet
examples/server/package-lock.json
test/browser/*.js
4 changes: 4 additions & 0 deletions .mocharc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"$schema": "https://json.schemastore.org/mocharc.json",
"require": "tsx"
}
1 change: 1 addition & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
gen-nodejs/
8 changes: 8 additions & 0 deletions .prettierrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"trailingComma": "es5",
"tabWidth": 2,
"semi": true,
"singleQuote": true,
"useTabs": false,
"printWidth": 120
}
1 change: 1 addition & 0 deletions .tool-versions
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
nodejs 18.18.2
11 changes: 0 additions & 11 deletions .travis.yml

This file was deleted.

4 changes: 3 additions & 1 deletion LICENSE → LICENSE.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
Copyright (c) 2017 ironSource Ltd.
Copyright for portions of the project are (c) 2017 ironSource Ltd. https://github.com/ironSource/parquetjs
Copyright for portions of the project are https://github.com/ZJONSSON/parquetjs
Copyright for portions of the project are https://github.com/LibertyDSNP/parquetjs

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in the
Expand Down
Loading