ghstack-source-id: 855519648a4bf2dced501f96e6de1b9b164d85ad
Pull Request resolved: llvm#424

This is currently missing and Debug builds are failing without it.

… dep

ghstack-source-id: 0706d6bb81b5b8eefb04146719b4443aedb29ab1
Pull Request resolved: llvm#427

One more step towards completing try/catch.

…rface

This is prep work for introducing cir.try_call inside cir.try scopes.

One more incremental step towards try/catch: properly use cir.try_call instead
of regular cir.call when within a cir.try region.

)

This patch adds a new `volatile` tag to the following operations to
distinguish volatile loads and stores from normal loads and stores:

- `cir.load`
- `cir.store`
- `cir.get_bitfield`
- `cir.set_bitfield`

Besides, this patch also updates the CodeGen and LLVMIR lowering code to
start emitting CIR and LLVMIR operations with volatile flag.

This is part 3 of implementing vector types and vector operations in
ClangIR, issue llvm#284.

Create new operation `cir.vec.cmp` which implements the relational
comparison operators (`== != < > <= >=`) on vector types. A new
operation was created rather than reusing `cir.cmp` because the result
is a vector of a signed intergral type, not a `bool`.

Add CodeGen and Lowering tests for vector comparisons.

Fix the floating-point comparison predicate when lowering to LLVM. To
handle NaN values correctly, the comparisons need to be ordered rather
than unordered. (Except for `!=`, which needs to be unordered.) For
example, "ueq" was changed to "oeq".

Compilation of the following test
```
void foo6(A* a1) {
  A a2 = (*a1);
}
```
fails with.
```
NYI
UNREACHABLE executed at /home/huawei/cir/repo/llvm-project/clang/lib/CIR/CodeGen/CIRGenExprAgg.cpp:175!
```
Commit adds required visitor and fixes the issue.

In the original codegen a new type is created for the base class, while
in CIR we were rewriting the type being processed (due tp misused
pointers). This PR fix this, and also makes CIR codegen even with the
original one.

This is a first PR for variable length array support. There are one (or
more :) ) ahead.

Basically, we already did lot's of preliminary job in order to land VLA
in CIR in llvm#367 llvm#346 llvm#340. So now we add initial VLA support itself.

Most of the changes are taken from the original codegen, so there is
nothing to be scary of)

I added just one test, and basically that's all we can test right now.
Later, I will add more, from the original codegen tests.

Support \_\_extension\_\_ keyword in CIRGen

Support for BoolAttr in isNullValue

This PR adds support for section("$name") attribute

Support missing zero initialization of Bools

)

Compiling the given c-code

```
void foo() {
  int      i [2][1] = { { 1 }, { 0 } };
  long int li[2][1] = { { 1 }, { 0 } };
  float    fl[2][1] = { { 1 }, { 0 } };
  double   d [2][1] = { { 1 }, { 0 } };
}
```

leads to compilation error

```
unknown element in ConstArrayAttr
UNREACHABLE executed at /home/huawei/cir/repo/van/llvm-project/clang/lib/CIR/Lowering/DirectToLLVM/LowerToLLVM.cpp:951!
```

PR implements conversion the cir.zero attr to dense constant and fixed
this error.

This PR introduces CIR CodeGen support for `dynamic_cast`.

The full feature set of `dynamic_cast` is not fully implemented in this
PR as it's already pretty large. This PR only include support for
downcasting and sidecasting a pointer or reference. `dynamic_cast<void
*>` is not yet implemented.

Initial support for the following builtins:
```
__builtin_expect
__builtin_expect_with_probability
__builtin_unpredictable
```
This PR supports codegen for this builtins on "-O0" compilation
pipeline.

Support for ConditionalOperator inside the if condition stmt

This patch adds CIRGen for downcasting a pointer to the complete object
through `dynamic_cast<void *>`.

Together with llvm#426 , the full functionality of `dynamic_cast` should be
supported in CIRGen after this PR merges.

This PR adds support for constant arrays with trailing zeros.

The original `CodeGen` does the following: once a constant array contain
trailing zeros, a struct with two members is generated: initialized
elements and `zeroinitializer` for the remaining part. And depending on
some conditions, `memset` or `memcpy` are emitted. In the latter case a
global const array is created.
Well, we may go this way, but it requires us to implement
[features](https://github.com/llvm/clangir/blob/main/clang/lib/CIR/CodeGen/CIRGenDecl.cpp#L182)
that are not implemented yet.

Another option is to add one more parameter to the `constArrayAttr` and
utilize it during the lowering. So far I chose this way, but if you have
any doubts, we can discuss here. So we just emit constant array as
usually and once there are trailing zeros, lower this arrray (i.e. an
attribute) as a value.

I added a couple of tests and will add more, once we agree on the
approach. So far I marked the PR as a draft one.

…nters (llvm#400)

This also adds a missing check whether the pointer returned from
`memchr` is null and changes the result to `last` in that case.

Emit the false-branch of the consteval if statement, if any.

Originally, the location associated with a function is checked to be an
`mlir::FileLineColLoc` before the function is lowered to an LLVMIR
FuncOp. However, runtime function declarations do not have such
locations. This patch further allows `mlir::UnknownLoc` to be associated
with a function.

This PR adds a support for const structs with bitfields.
Now only global structs are supported, the support of the local ones can
be added more or less easily - there is one ugly thing need to be done
though)

So .. what is all about.
First of all - as usually, I'm sorry for the big PR. But it's hard to
break it down to peaces. The good news is that in the same time it's a
copy-pasta from the original codegen, no surprises here. Basically, the
most hard place to read is `ConstantAggregateBuilder::addBits` copied
with minimum of changes.

The main problem - and frankly speaking I have no idea why it's done
this way in the original codegen - is that the data layout is different
for such structures, I mean literally another type is used. For
instance, the code:
```
struct T {
  int X : 15;
  int Y : 6;
  unsigned Z : 9;
  int W;
};

struct T GV = { 1, 5, 256, -1};
```
is represented in LLVM IR (with no CIR enabled) as:

```
%struct.T = type { i32, i32 }
%struct.Inner = type { i8, i32 }

@gv = dso_local global { i8, i8, i8, i8, i32 } ...
```
i.e. the global var `GV` is looks like a struct of single bytes (up to
the last field, which is not a btfield).
And my guess is that we want to have the same behavior in CIR. So we do.

The main problem is that we have to treat the same data differently -
and this is why one additional `bitcast` is needed when we create a
global var. Actually, there was a comment there - and I really wonder
where it came from. But anyways, I don't really like this and don't see
any good workaround here. Well, maybe we may add a kind of map in order
to store the correspondence between types and do a bitcast more wisely.
The same is true for the const structs with bitfields defined locally.

The minimal bug repro:
```
#include <stdbool.h>
#include <stdint.h>
void bar() {
  bool x = true;
  uint8_t y = (uint8_t)x;
}
```
Fails on verification stage:
```
loc("repro.c":5:24): error: integer width of the output type is smaller or equal to the integer width of the input type
fatal error: error in backend: The pass manager failed to lower CIR to LLVMIR dialect!
```
The problem is that in some cases lowering from CIR emits the invalid
zext operation. PR fixes this issue by emitting the llvm.bitcast instead
of llvm.zext in such cases.

…tchOp

The change is taken from original codegen.

The issue is that the CIR codegen assumes that function pointer is
always result of cir.load op.
But it isn't true because the funcion pointer may be result of other
operations (f.e cir.call).

This PR adds support for global compound literals.
The implementation is almost the same as in original codegen. But the
original codegen can reuse the value of emitted compound literal global
variable in case then the init expression of new variable and this
variable are the same.
It's easy to implement this feature. But I can't find any test-case then
this feature will be applied. So I decided to ignore this optimization
opportunity to avoid mistakes.

Here is the next step in VLA support.
Basically, these changes handle different expressions, like `int
(*a[5])[n]` or `sizeof(a[n])`.

I took tests from the original `codegen` - they don't check anything,
just verify we don't fail.

There is still an issue with a proper cleanup - there are cases when
`stack_save` doesn't dominate a corresponded `stack_restore`. For
example in the next example:

```
void test(unsigned x) {
  while (1) {
    char a[x];
    if (x > 5)
      break;
    ++x;
  }
}
```
Look like `break` here doesn't lead to `stack_restore`. But I would say
this is less related to VLA, though probably I need to fix this as well.

This PR fixes the issue connected with folding a simple boolean
expresion pattern (f.e. `0 && RHS = 0`).
The problem is that the scalar expression emitter always creates a
`cir.bool` attribute as a result of expression. But in some cases the
result expression should be a `cir.int` attr.

… start/end functionality

The change is taken from the original llvm codegen.

Originally those are only used for debug info generation, so get a bit more
specific on what's missing here.

When replacing the no-proto functions with it's real definition, codegen
assumes that only `cir.call` operation may use the replaced function.
Such behaviour leads to compilation error because of the
`cir.get_global` op can also use the function to get pointer to
function.
This PR adds handle the case with `cir.get_global` operation and fixes
the issue.

…RAII

This PR adds missing case to lowerCirAttrAsValue.

…r.try

The final destination here is to support cir.try_calls that are not within a
`try {}` statement in C++.  This only affect untested paths that will
assert a bit later than before, testcase coming soon.

The second part of the job started in llvm#412 , now about local structures.

As it was mentioned previously, sometimes the layout for structures with
bit fields inited with constants differ from the originally created in
`CIRRecordLayoutBuilder` and it cause `storeOp` verification fail due to
different structure type was used to allocation.
This PR fix it.
An example:
```
typedef struct {
  int a : 4;
  int b : 5;
  int c;
} D;

void bar () {
  D d = {1,2,3};
}
```

Well, I can't say I'm proud of these changes - it seems like a type
safety violation, but looks like it's the best we can do here.

The original codegen doesn't have this problem at all, there is just a
`memcpy` there, I provide LLVM IR just for reference:

```
%struct.D = type { i16, i32 }

@__const.bar.d = private unnamed_addr constant { i8, i8, i32 } { i8 33, i8 0, i32 3 }, align 4

; Function Attrs: noinline nounwind optnone uwtable
define dso_local void @bar() #0 {
entry:
  %d = alloca %struct.D, align 4
  call void @llvm.memcpy.p0.p0.i64(ptr align 4 %d, ptr align 4 @__const.bar.d, i64 8, i1 false)
  ret void
}
```

In llvm#426 we confirmed that CIR needs a `cir.unreachable` operation to
mark unreachable program points
[(discussion)](llvm#426 (comment)).
This PR adds it.

This PR adds support for evaluating constants in member exprs.
The change is taken from original codegen.

- Add it to functions but not yet on calls.
- Add more skeleton for tagging function attributes.
- Testcases

One more incremental step towards cir.try_call outside cir.try scopes.

The next step in inline-assembly support: we add instruction operands!
Nothing interesting, just some copy-pasta from the `codegen` with some
sort of simplifications for now.

Well, I'm not sure `functional-type` is the best way to print operands
though it's used in mlir's `InlineAsmOp`. But anyways, maybe you have a
better idea.

There are two or three steps ahead, so we are not that far from being
able to run something!

One more tiny step!
This a tiny PR that adds clobbers to constraint string.
Note, that `~{dirflag},~{fpsr},~{flags}` is a
[X86](https://github.com/llvm/clangir/blob/main/clang/lib/Basic/Targets/X86.h#L281)
dependent clobbers.

Basically, the next things remain:
- lowering
- store the results of the `cir.asm`

…llvm#471)

* Pointers to CIR types that do not have converters (e.g. cir.struct)
cause crashes due to passing null types to construct mlir::MemRefType.

* This commit adds checks for pointers and alloca lowering to fail
gracefully if the underlying type can not be converted.

As in original codegen this PR uses the do-while loop to initialize the
array elements with the filler expression.
But unlike the original codegen we allocates the temporary variable on
stack. Allocation is necessary to store the pointer to the first
uniinitialized element.

This PR fixes a bug caused by `IntType` size limitations in CIR (and by
some magic of numbers as well).

As you know, we need to create a storage for bit fields that usually
contain several of them. There next code fails with `IntType` size check
which exceeds 64 bits.
```
typedef struct {
    uint8_t a;
    uint8_t b;
    uint8_t c;

    int d: 2;
    int e: 2;
    int f: 4;
    int g: 25;
    int h: 3;
    int i: 4;
    int j: 3;
    int k: 8;
    int l: 14;
} D;

void foo() {
    D d;
}
```
Note, if we remove first three fields (or even one) everything will be
fine even without this fix, because
[this](https://github.com/llvm/clangir/blob/main/clang/lib/CIR/CodeGen/CIRRecordLayoutBuilder.cpp#L553)
check won't pass. The bug is kind of hard to reproduce and I would say
it's a rare case. I mean the problem is not only in the number of bit
fields that form together something bigger than 64 bits.

### Several details

Well, while iterating over the bit fields in some struct type, we need
to stop accumulating bit fields in one storage and start to do the same
in another one. Basically, we operate with `Tail` and `StartBitOffset`
where the former is an offset of the next field. And once `Tail -
StartBitOffset >= 64` we say that it's not possible to create a storage
of such size due to `IntType` size limitation. Sounds reasonable.

But it can be a case when we can not afford to take the next field
because its `Tail` in turn leads to a storage of the size bigger then
64. Thus, we want to check it as well. From the implementation point of
view I added one more check to the `IsBetterAsSingleFieldRun` in order
to have all these checks for size in a single place. And the check I
mentioned before were saving us from hitting this issue.

This PR adds a dedicated `cir.float` type for representing
floating-point types. There are several issues linked to this PR: llvm#5,
llvm#78, and llvm#90.

This PR fixes a couple of  NIY features for bit fields.

Basically, such expressions with bit fields `x->a++` and `x->a |= 42`
are supported now.

The main problem is `UnOp` verification - now it can be defined both by
`loadOp` and by `GetBitfieldOp` or even by `CastOp`. So shame on me, I
removed a test from `invalid.cir`

…#467)

CIR codegen always casts the no-proto function pointer to `FuncOp`. But
the function pointer may be result of cir operations (f.e. `cir.load`).
As a result in such cases the function pointer sets to `nullptr`. That
leads to compilation error.
So this PR removes the unecessary cast to 'FuncOp' and resolves the
issue.

This patch adds initial support for the pointer-to-data-member type.
Specifically, this commit includes:

- New ops, types, and attributes:
- CodeGen for pointer-to-data-member types and values
  - Lower C++ pointer-to-member type
  - Lower C++ expression `&C::D`
  - Lower C++ expression `c.*p` and `c->*p`

This patch only includes an initial support. The following stuff related
to pointer-to-member types are not supported yet:

- Pointer to member function;
- Conversion from `T Base::*` to `T Derived::*`;
- LLVMIR lowering.

As discussed in pull llvm#401 , The present `UnimplementedFeature` class is
made for the CIRGen submodule and we need similar facilities for code
under `clang/lib/CIR/Dialect/IR`. This NFC patch adds a new
`CIRDialectUnimplementedFeature` class that provides unimplemented
feature guards for CIR dialect code.

This PR adds lowering for `cir.asm`.

Also, two flags were added to the `cir.asm` : `hasSideEffects` and
`isStackAligned` in order to match with the llvm dialect.

Also, I added several simple tests for lowering.

I'm not sure but most likely the next PR will be the last one in this
story about assembly support )

This patch changes the emission of implicit returns from functions whose
return type is not `void`. Instead of emitting `cir.return`, this PR
aligns to the original clang CodeGen and emits a `cir.unreachable`
operation.

Related issue: llvm#457 .

This PR adds CIRGen support for the following built-in bit operations:

  - `__builtin_ffs{,l,ll,g}`
  - `__builtin_clz{,l,ll,g}`
  - `__builtin_ctz{,l,ll,g}`
  - `__builtin_clrsb{,l,ll,g}`
  - `__builtin_popcount{,l,ll,g}`
  - `__builtin_parity{,l,ll,g}`

This PR adds a new operation, `cir.bits`, to represent such bit
operations on the input integers. LLVMIR lowering support is not
included in this PR.

> [!NOTE]
> As a side note, C++20 adds the `<bit>` header which includes some bit
operation functions with similar functionalities to the built-in
functions mentioned above. However, these standard library functions
have slightly different semantics than the built-in ones and this PR
does not include support for these standard library functions. Support
for these functions may be added later, or amended into this PR if the
reviewers request so.

Co-authored-by: Bruno Cardoso Lopes <[email protected]>

This is part 4 of implementing vector types and vector operations in
ClangIR, issue llvm#284. This change has three small additions.

Implement a "vector splat" conversion, which converts a scalar into
vector, initializing all the elements of the vector with the scalar.

Implement incomplete initialization of a vector, where the number of
explicit initializers is less than the number of elements in the vector.
The rest of the elements are implicitly zero initialized.

Implement conversions between different vector types. The language rules
require that the two types be the same size (in bytes, not necessarily
in the number of elements). These conversions are always implemented
with a bitcast.

The first two changes only required changes to the AST -> ClangIR code
gen. There are no changes to the ClangIR dialect, so no changes to the
LLVM lowering were needed.

The third part only required a change to a validation rule. The code to
implement a vector bitcast was already present. The compiler just needed
to stop rejecting it as invalid ClangIR.

This PR adds the new `cir.expect` opcode which is similar to llvm.expect
intricnsics.
Codegen of `__builtin_expect` emits `cir.expect` opcode. Then
`cir.expect` will be lowered to `llvm.expect` intrinsic.

When implementing __builtin_expect I faced with minor issue.
CIR lowering of `if` often emits the lllvm IR with redundant cast
instructions. Like this:
```
%0 = call i64 @llvm.expect.i64(i64 %any, i64 1), !dbg !13
%1 = icmp ne i64 %0, 0
%2 = zext i1 %0 to i8   // redundant
%3 = trunc i8 %1 to i1 // redundant
br i1 %3, label %l1, label %l2
```
But the llvm pass `LowerExpectIntrinsicPass` (that should replace
`llvm.expect` with branch metadata) performs only simple
pattern-matching. And it can't handle this zext/trunc intructions. So
this pass in such cases just removes the `llvm.expect` without updating
a branch metadata.
In this reason this PR also avoid emitting the redundant zext/cast
instruction sequence.

Support `offset` expression in case when we can evaluate offset
expression as integer.

…m#494)

Change is taken from original llvm codegen

This PR adds the `cir.trap` operation, which corresponds to the
`__builtin_trap` builtin function. When executed, the operation
terminates the program abnormally in an implementation-defined manner.

This PR also includes CIRGen and LLVM lowering support for the new
operation.

Remove check since paths that aren't related to ClangIR shouldn't be affected
at all, even if the flag is ON. Also, this conservative error message wasn't
tested.

Implement the vector version of the ternary (`?:`) operator. This is a
separate MLIR op than the regular `?:` operator because the vector
version is not short-circuiting and always evaluates all its arguments.

…lvm#501)

This PR adds the LLVMIR lowering support for CIR bit operations.

For `cir.bit.clz`, `cir.bit.ctz`, and `cir.bit.popcount`, they can be
lowered directly to LLVM intrinsic calls to `@llvm.ctlz`, `@llvm.cttz`,
and `@llvm.ctpop`, respectively.

For the other three bit operations, namely `cir.bit.clrsb`,
`cir.bit.ffs`, and `cir.bit.parity`, they are lowered to a sequence of
LLVM IR instructions that implements their functionalities. This
lowering scheme is also used by the original clang CodeGen.

CIRGenFunction::buildFromMemory can handle the `cir.bool` values. So we
no longer need to emit the `NIY` error here.

This is the next step in inline assembly support and it's more like a
service PR and mostly dedicated to the in/out argument types.

Also, operand attributes are added and it's the last change in the
`cir.asm` operation afaik. But I would wait untill the next PR,
which will contain more examples and maybe will help us to get more
readable format for the operation.
Note, that we have to add an attribute for each operand - because the
lowering of the llvm dialect to LLVM IR iterates over them in the same
order.

The next PR will be last one (so far) in the series of PRs dedicated to
the inline assembly support. It will add storing of the results.

Three small changes, all cleanup or refactoring in nature.

1. Fix the assemblyFormat for all the vector operations in the ClangIR
dialect so that vector types appear in ClangIR as `!cir.vector<type x
n>` instead of as just `<type x n>`. When I first created the vector
ops, I forgot to use `qualified` as necessary when writing out types.
This change fixes that. There is no change in behavior, but there is a
change to the text version of ClangIR, which required changing the
ClangIR expected results and ClangIR inputs in the tests.

2. Create a new `cir.vec.splat` operation and use that for "vector
splat", i.e. a conversion from a scalar to a vector. A "vector splat"
conversion had been implemented with `cir.vec.create` before. This
change results in different ClangIR and different LLVM IR, which again
required updating the tests, but no noticeable change in compiler
behavior.

3. Create an `IntegerVector` type constraint, which requires that the
given type be a vector whose element type is an integer. It can be any
integral type, and the vector can be of any size. Use the new type
constraint in the definition of `cir.vec.ternary`, whose condition
operand must be an `IntegerVector`. Remove the integral type check from
`VecTernaryOp::verify`, since doing the check there is now redundant.
The only possibly visible change is to the text of an error message when
validation of `cir.vec.ternary` fails. The expected output of a
validation test was updated with the new message.

This PR intends to fix some problems with packed structures support, so
the llvm#473 will work.

Basically, the main problem for the packed structures support is an
absence of arbitrary sized integers in CIR. Well, one workaround is to
use `mlir::IntegerType` for that, but it's kind of wrong way (please
correct me if I'm wrong). Another way is to introduce this type in CIR.
So far I suggest this way: instead of arbitrary sized integers we will
create an array of bytes for bitfield storages whenever they doesn't fit
into the CIR `IntType`.

Well, the original codegen creates storages with alignment 8 - so it can
be `i24` storage type for instance. Previously, we just created storages
that could be represented as CIR `IntType`: 8, 16, 32, 64. And it was
working before I came up with a necessity to support packed structures.
At first glance it's not a problem - just add `determinePacked` method
from the original codegen and that's it. But it turned out that this
method _infers_ the fact if a structure is packed or not. It doesn't use
the AST attribute for that as one could think - it works with offsets
and alignments of fields. Thus, we either need to invent our own way to
determine packed structures (which is error prone and maybe not doable
at all) or try to use the existing one. Also, we go closer to the
original lllvm's data layout in this case.

1) I had to move the lowering details from the `LoweringPrepare` to the
`LowerToLLVM`, because it's not possible to do a `load` from the array
of bytes to the integer type - and it's ok in llvm dialect. Thus, all
the math operations can be expressed without any problems. Basically the
most of the diff you see is because of the changes in the lowering. The
remaining part is more or less easy to read.
2) There are minor changes in `CIRRecordLayoutBuilder` - as described
above, we use may generate an array of bytes as a storage.
3) Some cosmetic changes in `CIRGenExpr` - since we don't want to infer
the storage type again and just use the one stored in the
`CIRGenBitFieldInfo`.
4) Helpers are introduced in the lowering - but nothing hard - just
shifts and logical ops.
5) I removed `bitfield-ops` test - because now the test cases covered
there are all in `bitfields.c` and `bitfields.cpp` .

So ... This is still a suggestion, though I believe it's a good one. So
you are welcome to discuss, suggest another ways to solve the problem
and etc.

Still missing lowering support, which will come next.

This doesn't change existing functionality, for existing crashes related to
atomics we just hit asserts a bit further now, but no support added just yet.
Next set of commits will introduce functionlity with testcases.

Just like previous commit, add more infra pieces, still NFC since all relevant
testcases hit asserts, just a bit deeper.

This PR adds a support for packed structures.

Basically, now both `pragma pack(...)` and
`__attribute__((aligned(...)))` should work.
The only problem is that `getAlignment` is not a total one - I fix only
a couple of issues I faced with - for struct types and arrays.

PR adds support for initialization of unions. The change is copy-pasted
from the original CodeGen.

This PR adds support for `__builtin_prefetch`. CIRGen of this builtin
emits the new 'cir.prefetch' opcode. Then `cir.prefetch` lowers to
`llvm.prefetch` intrinsic.

Co-authored-by: Bruno Cardoso Lopes <[email protected]>

This PR adds MemRead/MemWrite markers to the `GetBitfieldOp` and
`SetBitfieldOp` (as discussed in llvm#487)
Also, minor renaming in the `SetBitfieldOp`

---------

Co-authored-by: Bruno Cardoso Lopes <[email protected]>

This PR adds support for `__builtin_constant_p`.
Implementation introduces the new `cr.is_constant` opcode to it during
the codegeneration of builtin.
Codegeneration is taken from the original llvm codegen.

This patch adds CIRGen support for the C++20 three-way comparison
operator `<=>`. The binary operator is directly lowered to existing CIR
operations.

Most of the changes are tests.