Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Basic Support for Arrays #175

Closed
4 tasks done
KennyOliver opened this issue Jan 3, 2025 · 2 comments · Fixed by #183 or #184
Closed
4 tasks done

Implement Basic Support for Arrays #175

KennyOliver opened this issue Jan 3, 2025 · 2 comments · Fixed by #183 or #184
Assignees
Labels
documentation Improvements or additions to documentation enhancement New feature or request

Comments

@KennyOliver
Copy link
Owner

KennyOliver commented Jan 3, 2025

Implement Basic Support for Arrays

Description

I need to implement support for arrays to enable robust & versatile data handling. Arrays should act as ordered collections of elements, allowing for functional operations and seamless integration with the functional programming paradigm. With FlavorLang, I think that arrays should be treated as literals, with various functions provided in the standard library for handling them.

Features to Implement

1. Basic Operations

  • Array Creation: Support array literals using [1, 2, 3] syntax.
  • Element Access: Enable access via zero-based indexing (e.g., array[0]).
  • Length Retrieval: Provide length(array) in the standard library to return the size of the array.

2. Array Manipulation

  • Appending: append(array, element) — Adds an element to the end.
  • Prepending: prepend(array, element) — Adds an element to the start.
  • Insertion: insert(array, index, element) — Adds an element to the array after the specified index.
  • Concatenation: array1 + array2 — Combines two arrays.
  • Slicing: Enable slicing with [start:end:step] syntax (e.g., array[1:3], array[::-1]).

3. Transformation

  • Reversing: array[::-1] — Reverses the array.

4. Error Handling

  • Invalid Access: Handle out-of-bound errors gracefully (e.g., a[10]).
  • Invalid Step: Throw errors for invalid steps in slices (e.g., a[0:4:0]).
  • Immutability: Enforce that const arrays cannot be modified.

Future Feature Ideas

1. Functional Operations

  • Mapping: map(array, function) — Applies a function to each element.
  • Filtering: filter(array, function) — Returns elements that satisfy a predicate.
  • Reducing: reduce(array, function) — Combines elements into a single value.

2. Search & Indexing

  • Element Index: index_of(array, element) — Finds the index of the first matching element.
  • Contains: contains(array, element) — Checks if an array contains a specific element.

3. Transformation

  • Sorting: sort(array, compareFunction?) — Sorts elements in order.
  • Sum, Min, Max: Built-in functions like sum(array), min(array), max(array).

Potential Syntax

  • Array Literals: [1, 2, 3]
  • Immutable Operations: Operations like a + b or append(array, element) return new arrays.

Tasks

  • 1. Implement the array data structure & basic operations (e.g., creation, length, access).
  • 3. Support slicing ([start:end:step]) with error handling for invalid cases.
  • 5. Handle errors for invalid accesses or modification of const arrays.
  • 6. Write comprehensive unit tests to cover all scenarios.

References / Inspiration

  • Python lists for slicing & negative indexing.
  • Functional array methods in TypeScript (Array.prototype.map, etc.).
  • Rust’s Vec for immutability & deep copying.
  • Examples from other languages like Swift, Ruby, etc.

Discussion Points / Considerations

  1. Should FlavorLang arrays support heterogeneous types or enforce homogeneity?
  2. Do I need lazy evaluation for operations like map for performance reasons?
  3. What things should I consider in regard to trade-offs for immutable operations (e.g., performance, memory usage)?
@KennyOliver KennyOliver added documentation Improvements or additions to documentation enhancement New feature or request labels Jan 3, 2025
@KennyOliver KennyOliver self-assigned this Jan 3, 2025
@KennyOliver
Copy link
Owner Author

KennyOliver commented Jan 6, 2025

Proposed Arrays Design

I've thought about the design for my arrays and have come to this!

Syntax Design

Note

All array operations do not mutate original array to reduce confusion.

Operation Syntax Explanation
Element Access array[5] Accesses element at index 5.
Append array[^+] Appends an element to end.
Prepend array[+^] Prepends an element to beginning.
Remove Last Element array[^-] Removes last element.
Remove First Element array[-^] Removes first element.
Slicing array[0:5] Slices from index 0 (incl.) to 5 (excl.).
array[::-1] Reverses using step of -1.
array[3:] Slices from index 3 (incl.) to end.
array[:8] Slices from start to index 8 (excl.).
array[3:8:2] Slices from index 3 (incl.) to 8 (excl.), skipping every 2nd element.
array[8:3:-2] Slices from index 8 (incl.) to 3 (excl.), skipping every 2nd element (reverse).
array[3::-1] Slices from index 3 (incl.) to start (reversed).
array[:8:-1] Slices from end towards index 8 (excl., reversed).
array[::2] Slices entire array, skipping every 2nd element.
array[::-3] Reverses entire array, skipping every 3rd element.

Reviewing the Discussion Points

  1. Should FlavorLang arrays support heterogeneous types or enforce homogeneity?

    I am opting for heterogeneous types (allowing a mix of types) like Python.

  2. Do I need lazy evaluation for operations like map for performance reasons?

    Circle back to this later in the future.

  3. What things should I consider in regard to trade-offs for immutable operations (e.g., performance, memory usage)?

    Ditto.

KennyOliver added a commit that referenced this issue Jan 6, 2025
KennyOliver added a commit that referenced this issue Jan 6, 2025
KennyOliver added a commit that referenced this issue Jan 6, 2025
KennyOliver added a commit that referenced this issue Jan 6, 2025
- Rearranged sequence of logic to fix this, since `-` would indicate a number previously

#175
KennyOliver added a commit that referenced this issue Jan 6, 2025
KennyOliver added a commit that referenced this issue Jan 6, 2025
- Restructured `assignment` node in `ASTNode` to have `lhs` & `rhs`

#175
KennyOliver added a commit that referenced this issue Jan 6, 2025
- Looking through the debugging output for the lexer, I realized that negative numbers were being tokenized as `-` followed by the positive version of the number.
- Updated `scan_array` to detect and correctly handle negative numbers (`-`) within array slices, ensuring that `-1` is tokenized as a single `TOKEN_INTEGER` instead of separate tokens for `-` and `1`.
- Modified `scan_number` to construct lexemes for negative numbers properly, including the `-` sign.
- Added conditions to differentiate between negative numbers and array operators (`^-`, `-^`) based on context.
- Included debug statements to verify lexeme capture during tokenization.
- Verified fixes with test cases for array slicing scenarios, including `[::-1]`, `[-5:-1:2]`, and mixed positive/negative indices.
- These changes now resolves issues where the lexer misinterpreted `-` in slices, leading to parsing errors like "Expected expression (found `-`)."

#175
KennyOliver added a commit that referenced this issue Jan 7, 2025
KennyOliver added a commit that referenced this issue Jan 7, 2025
KennyOliver added a commit that referenced this issue Jan 7, 2025
KennyOliver added a commit that referenced this issue Jan 7, 2025
KennyOliver added a commit that referenced this issue Jan 7, 2025
- Refactored `ASTNode` to separate fields for distinct node types, preventing memory conflicts.
- Updated parser functions to use correct `ASTNodeType` for declarations and assignments.
- Fixed crash in `print_ast` caused by uninitialized or overlapping fields.
- Ensured proper memory initialization using calloc for zero-initialized nodes.
- Enhanced debugging and error handling for better resilience against invalid memory access.

#175
KennyOliver added a commit that referenced this issue Jan 8, 2025
KennyOliver added a commit that referenced this issue Jan 8, 2025
@KennyOliver KennyOliver linked a pull request Jan 8, 2025 that will close this issue
KennyOliver added a commit that referenced this issue Jan 8, 2025
@KennyOliver KennyOliver reopened this Jan 8, 2025
@KennyOliver
Copy link
Owner Author

Reopening — GitHub auto closed this issue because the linked branch was merged, but I'm still working on it.

KennyOliver added a commit that referenced this issue Jan 8, 2025
KennyOliver added a commit that referenced this issue Jan 8, 2025
KennyOliver added a commit that referenced this issue Jan 8, 2025
KennyOliver added a commit that referenced this issue Jan 8, 2025
KennyOliver added a commit that referenced this issue Jan 8, 2025
- Can be used to run all tests in sequence

#175
KennyOliver added a commit that referenced this issue Jan 8, 2025
KennyOliver added a commit that referenced this issue Jan 8, 2025
KennyOliver added a commit that referenced this issue Jan 8, 2025
- Running `run_tests.sh` may give something this: `flavor(77629,0x1f3df8840) malloc: nano zone abandoned due to inability to reserve vm space.`.
- This only occurs only with this iterative script, but I don't get these warnings/errors/notices with `all.flv`, so this seems to be a 'non-issue'; at least for now.

#175
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
1 participant