Boot-time configurable heap constants

[wenyuzhao]

This CL refactors the vm_layout_constants related code and changes all the related compile-time constants to run-time constants, so that VM bindings can configure them during boot time based on dynamic user inputs.

This is a necessary refactoring to support compressed pointers. For most of the VMs (e.g. OpenJDK), compressed pointers are enabled at run-time based on command line arguments, and VM bindings should be able to enable compressed pointer heap dynamically during the VM boot process. Such a process usually involves dynamically configuring heap ranges, space memory layout, and choosing a different VMMap implementation [1]. Thus all vm_layout_constants, including heap constants, should be made as boot-time configurable, instead of compile-time constants.

This CL also makes risc-v's 39-bit heap support easier, as it now should only involve adding a new set of vm_layout_constants.

[1] This is already done by #732

---

Benchmark results: Almost no performance impact

http://squirrel.anu.edu.au/plotty-public/wenyuz/v8/p/NkK9YG

[caizixian]

@wenyuzhao can you suppress some warnings on x86?

[caizixian]

V8 binding tests will fail until #472 is merged

[qinsoon]

Can you run performance tests for the PR?

I am wondering if we would need a feature like runtime_vm_layout. Without the feature, we can just have a const VMLayoutConstants.

[wenyuzhao]

Can you run performance tests for the PR?

will do

I am wondering if we would need a feature like runtime_vm_layout. Without the feature, we can just have a const VMLayoutConstants.

Yes, this is possible. Let's wait for the performance test results and see if it is worth doing so.

[qinsoon]

I suggest adding some tests that only run on 64 bits architecture in DummyVM (vmbindings/dummyvm/src/tests): set a custom VM layout for the compressed pointer layout, initialize MMTk, create a mutator, and finish one allocation. The test should run for all plans (Add // GITHUB-CI: MMTK_PLAN=all at the top of the test file)

I added some tests for VMLayout. Some are failing. Feel free to modify the tests if they are wrong, or ignore some tests. I just added the tests to see what is working, and what is not.

[wenyuzhao]

I suggest adding some tests that only run on 64 bits architecture in DummyVM (vmbindings/dummyvm/src/tests): set a custom VM layout for the compressed pointer layout, initialize MMTk, create a mutator, and finish one allocation. The test should run for all plans (Add // GITHUB-CI: MMTK_PLAN=all at the top of the test file)

I added some tests for VMLayout. Some are failing. Feel free to modify the tests if they are wrong, or ignore some tests. I just added the tests to see what is working, and what is not.

Interesting. I can't even pass all the dummyvm tests on the master branch

https://gist.github.com/wenyuzhao/ed9614dfec7b1fdac7bee8664b547e46

Is MMTK_PLAN=NoGC cargo test the correct command to run dummyvm tests?

[qinsoon]

I suggest adding some tests that only run on 64 bits architecture in DummyVM (vmbindings/dummyvm/src/tests): set a custom VM layout for the compressed pointer layout, initialize MMTk, create a mutator, and finish one allocation. The test should run for all plans (Add // GITHUB-CI: MMTK_PLAN=all at the top of the test file)

I added some tests for VMLayout. Some are failing. Feel free to modify the tests if they are wrong, or ignore some tests. I just added the tests to see what is working, and what is not.

Interesting. I can't even pass all the dummyvm tests on the master branch

https://gist.github.com/wenyuzhao/ed9614dfec7b1fdac7bee8664b547e46

Is MMTK_PLAN=NoGC cargo test the correct command to run dummyvm tests?

No. Each test module (file) is executed separately:

mmtk-core/vmbindings/dummyvm/src/tests/mod.rs

Line 1 in 3cfdf75

// NOTE: Since the dummyvm uses a global MMTK instance,

You can do MMTK_PLAN=NoGC cargo test --features features_you_need -- test_module_name

[qinsoon]

Maybe try a different heap layout for the failing test on macos, or simply ignore that test on macOS. I think either is okay.

[caizixian]

The address is too low for 64-bit macOS https://stackoverflow.com/questions/46916112/osx-ld-why-does-pagezero-size-default-to-4gb-on-64b-osx

[wenyuzhao]

Maybe try a different heap layout for the failing test on macos, or simply ignore that test on macOS. I think either is okay.

Switched to a different address on macos. Now all the checks are passed.

[qinsoon]

Looks good. Thanks for the PR. Let's merge this after the current release (0.19) which should be done today.