Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #586 follow-up: Too little indentation if named arguments come after a newline #596

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions src/dfmt/formatter.d
Original file line number Diff line number Diff line change
Expand Up @@ -1804,6 +1804,12 @@ private:
assert(l2 != -1, "Recent '{' is not found despite being in struct initializer");
indentLevel = l2 + 1;
}
else if (canFind(astInformation.namedArgumentColonLocations, tokens[nextNonComment(1)].index))
{
immutable l2 = indents.indentToMostRecent(tok!"(");
assert(l2 != -1, "Recent '(' is not found despite being in named function argument");
indentLevel = l2 + 1;
}
else if ((config.dfmt_compact_labeled_statements == OptionalBoolean.f
|| !isBlockHeader(2) || peek2Is(tok!"if")) && !indents.topIs(tok!"]"))
{
Expand Down Expand Up @@ -2316,6 +2322,25 @@ const pure @safe @nogc:
return previousTokenEndLineNo < tokens[index].line;
}

/++
+ Get the index of the next token that isn't a comment starting from
+ current index + n.
+ If n is negative, searches backwards.
+ If n = 0, returns index.
+ Params:
+ n = Offset to index where search begins. Negative values search backwards.
+ Returns:
+ Index of next token that isn't a comment or `size_t.max` if no such
+ token exists,
+/
size_t nextNonComment(int n = 1)
{
size_t i = index + n;
while (n != 0 && i < tokens.length && tokens[i].type == tok!"comment")
i = n > 0 ? i + 1 : i - 1;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you run this on tokens == [tok!"...", tok!"comment"] with start index = 0 and n = 1, this will return 2 and cause a range error above in your usage (just in general if the token stream ends with comments and you start searching on it)

Starting on 0 and searching negative will immediately go out of bounds and return size_t.max or a little bit below that.

I think you should make the function always return size_t.max for non-found tokens (and document it that way), since otherwise out-of-bounds tokens are too hard to check for and require knowledge about the tokens length, which isn't necessary for an API like this.

In your call site above, make sure you check the return value of this to not be size_t.max then. Right now it looks like that branch could easily cause a range error if the code ends in comments. (remember the array access is called also when it's not a named argument, since it itself is how it checks if it is a named argument or not, so the syntax may be arbitrary here) - however it's not that easy to get, I haven't been able to do it with a few tests right now yet. Just to defend against range errors I want you to test for it with a function that can generate one anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the slow response

Don't get me wrong, I don't want to argue and will implement it how you like, but I think you might've missed something (or maybe I am, it's been a while). Please consider this 1. regarding the range error (= out of bounds access?), 2. regarding return value of the new function:


1.
Caller is in line Line 1807 and has surrounding code like this (most importantly Line 1794):

if (hasCurrent)
{
  if (...)
  else if (currentIs(tok!"identifier") && peekIs(tok!":")) // Line 1794,
  {                                                        // peekIs ignores comments when invoked like this.
    if (...)
    else if (canFind(astInformation.namedArgumentColonLocations, tokens[nextNonComment(1)].index)) // Line 1807
  }
}

The if-condition in line 1794, that is true for the block in which tokens[nextNonComment(1)] is run, guarantees us that the tokens array looks at least something like this (block below). So your example (if I understand it correctly) where the function is invoked with only trailing comments in the array isn't possible:

[..., tok!"identifier", {arbitrary number of tok!"comment"}, tok!":", ...]
      ^current                                               ^guaranteed to exist by line 1794,
                                                              found by nextNonComment(1)

I assume the range error that you see as a possibility is that one, otherwise iirc tokens[nextNonComment(1)].index is the position in the file and canFind should be either true or false, so I don't see the need for additional access checks (or a reasonable way to implement any, peekIs finding a non-comment should be the appropriate check).

For the same reason I think this is a moot point since while the tok!":" may not be related to named args, it only needs to exist for the array access to not throw:

(remember the array access is called also when it's not a named argument, since it itself is how it checks if it is a named argument or not, so the syntax may be arbitrary here) - however it's not that easy to get, I haven't been able to do it with a few tests right now yet. Just to defend against range errors I want you to test for it with a function that can generate one anyway.

If I am not mistaken, there is currently no way to write a testcase that fails with an out of bounds error for tokens[nextNonComment(1)], even when writing the tokens array manually and then invoking the surrounding function.


2.
I changed the return value when no token is found from nextNonComment(), but I am not convinced this is a good thing: the function is a private member of struct TokenFormatter so anyone calling it also has access to tokens.length (I don't understand how out-of-bounds tokens would be hard to check for when that's accessible).
With the explicit return of size_t.max (for a variable expecting an index) I can see how it encourages to test for returnedIndex == size_t.max instead of returnedIndex < tokens.length which I assume can easily have much worse consequences if another error is at play.

return i < tokens.length ? i : size_t.max;
}

/// Bugs: not unicode correct
size_t tokenEndLine(const Token t)
{
Expand Down
7 changes: 7 additions & 0 deletions tests/allman/issue0586.d.ref
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,10 @@ void main()

temp(v1: () { S s = S(i: 5); return s.i; }, v2: 1);
}

void g()
{
tmp(namedArg1: "abc abc abc abc abc abc abc abc abc abc abc abc abc abc",
namedArg2: "abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc",
namedArg3: "abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc");
}
7 changes: 7 additions & 0 deletions tests/allman/issue0586_keep_line_breaks.d.ref
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
void test()
{
return Struct(
foo: field.foo,
bar: field.bar,
baz: field.baz);
}
7 changes: 7 additions & 0 deletions tests/issue0586.d
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,10 @@ void main()

temp(v1: () { S s = S(i: 5); return s.i; }, v2: 1);
}

void g()
{
tmp(namedArg1: "abc abc abc abc abc abc abc abc abc abc abc abc abc abc",
namedArg2: "abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc",
namedArg3: "abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc");
}
1 change: 1 addition & 0 deletions tests/issue0586_keep_line_breaks.args
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
--keep_line_breaks true
7 changes: 7 additions & 0 deletions tests/issue0586_keep_line_breaks.d
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
void test()
{
return Struct(
foo: field.foo,
bar: field.bar,
baz: field.baz);
}
7 changes: 7 additions & 0 deletions tests/knr/issue0586.d.ref
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,10 @@ void main()

temp(v1: () { S s = S(i: 5); return s.i; }, v2: 1);
}

void g()
{
tmp(namedArg1: "abc abc abc abc abc abc abc abc abc abc abc abc abc abc",
namedArg2: "abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc",
namedArg3: "abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc");
}
7 changes: 7 additions & 0 deletions tests/knr/issue0586_keep_line_breaks.d.ref
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
void test()
{
return Struct(
foo: field.foo,
bar: field.bar,
baz: field.baz);
}
6 changes: 6 additions & 0 deletions tests/otbs/issue0586.d.ref
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,9 @@ void main() {

temp(v1: () { S s = S(i: 5); return s.i; }, v2: 1);
}

void g() {
tmp(namedArg1: "abc abc abc abc abc abc abc abc abc abc abc abc abc abc",
namedArg2: "abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc",
namedArg3: "abc abc abc abc abc abc abc abc abc abc abc abc abc abc abc");
}
6 changes: 6 additions & 0 deletions tests/otbs/issue0586_keep_line_breaks.d.ref
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
void test() {
return Struct(
foo: field.foo,
bar: field.bar,
baz: field.baz);
}
Loading