Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(wasm-gen): avoid infinite wait-wake for loose syscalls #3615

Merged
merged 41 commits into from
Feb 7, 2024
Merged
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
4fa944b
feat(wasm-gen): avoid infinite wait-wake for loose syscalls
StackOverflowExcept1on Dec 21, 2023
56db883
compare with i32.const 1
StackOverflowExcept1on Dec 28, 2023
37a2ebe
Merge remote-tracking branch 'origin/master' into av/wasm-gen-deadlocks
StackOverflowExcept1on Jan 7, 2024
8daa513
Merge remote-tracking branch 'origin/master' into av/wasm-gen-deadlocks
StackOverflowExcept1on Jan 7, 2024
957d229
fix tests
StackOverflowExcept1on Jan 7, 2024
62c2418
[si-skip] more efficient memory allocation
StackOverflowExcept1on Jan 7, 2024
bc2f951
Merge remote-tracking branch 'origin/master' into av/wasm-gen-deadlocks
StackOverflowExcept1on Jan 16, 2024
b708f2e
use % == 0, add extra check for init_called
StackOverflowExcept1on Jan 16, 2024
d5cc590
resolve some discussions
StackOverflowExcept1on Jan 18, 2024
06c9f61
fix bug with MutSizedBufferStart
StackOverflowExcept1on Jan 18, 2024
5dbbbd0
reserve 256 bytes on the last page
StackOverflowExcept1on Jan 18, 2024
196bb79
add reserved_memory_size to wasm-smith
StackOverflowExcept1on Jan 20, 2024
c116cae
inject_critical_gas_limit: write to const ptr instead of last memory …
StackOverflowExcept1on Jan 20, 2024
805fa95
share memory layout between syscalls and instructions
StackOverflowExcept1on Jan 21, 2024
bac5ab1
fix clippy
StackOverflowExcept1on Jan 21, 2024
2a2e61e
add runtime assert when reserved memory exceeded
StackOverflowExcept1on Jan 21, 2024
8691738
rename waiting_frequency to waiting_probability
StackOverflowExcept1on Jan 23, 2024
6e1b731
Merge remote-tracking branch 'origin/master' into av/wasm-gen-deadlocks
StackOverflowExcept1on Jan 23, 2024
27e53dd
add simple test to check avoid wait in init()
StackOverflowExcept1on Jan 24, 2024
62afff2
hardcode memory_offset_choices
StackOverflowExcept1on Jan 28, 2024
10eb254
update dep
StackOverflowExcept1on Jan 28, 2024
450201c
Merge remote-tracking branch 'origin/master' into av/wasm-gen-deadlocks
StackOverflowExcept1on Jan 28, 2024
a74c7d1
update dep to gear-stable
StackOverflowExcept1on Jan 28, 2024
0c14085
replace if let with .expect
StackOverflowExcept1on Jan 28, 2024
6fbeb3f
move memory_size_{bytes, pages} under callers
StackOverflowExcept1on Jan 28, 2024
29c5edd
extract code new method limit_infinite_waits
StackOverflowExcept1on Jan 28, 2024
28193ca
move some stuff to const params
StackOverflowExcept1on Jan 28, 2024
7ec0f15
add new memory_layout module
StackOverflowExcept1on Jan 28, 2024
4c4478d
pass mem_import_gen_proof to the entry point generator for better typ…
StackOverflowExcept1on Jan 28, 2024
7bc1ecb
[skip-ci] add docs for memory layout
StackOverflowExcept1on Jan 28, 2024
c2f26f9
use NonZeroU32
StackOverflowExcept1on Feb 1, 2024
6ca3e0a
Merge remote-tracking branch 'origin/master' into av/wasm-gen-deadlocks
StackOverflowExcept1on Feb 1, 2024
5f3c29e
fix tests
StackOverflowExcept1on Feb 1, 2024
7c47b8d
move memory layout to wasm module
StackOverflowExcept1on Feb 1, 2024
fc444a6
add GearWasmGeneratorWithMemory newtype
StackOverflowExcept1on Feb 5, 2024
ccaea29
Merge remote-tracking branch 'origin/master' into av/wasm-gen-deadlocks
StackOverflowExcept1on Feb 5, 2024
fa20ca2
rename newtype to EntryPointsGeneratorInstantiator
StackOverflowExcept1on Feb 5, 2024
fcaeac6
remove mem_imports_gen_proof from SyscallsImportsGeneratorInstantiator
StackOverflowExcept1on Feb 7, 2024
9addce9
fix
StackOverflowExcept1on Feb 7, 2024
0217157
[skip-ci] fix docs
StackOverflowExcept1on Feb 7, 2024
1f12ff6
Merge remote-tracking branch 'origin/master' into av/wasm-gen-deadlocks
StackOverflowExcept1on Feb 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 11 additions & 0 deletions utils/wasm-gen/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,11 @@ impl ConfigsBundle for (GearWasmGeneratorConfig, SelectableParams) {
pub struct StandardGearWasmConfigsBundle {
/// Externalities to be logged.
pub log_info: Option<String>,
/// Probability of wait syscalls.
///
/// For example, if this parameter is 4, wait syscalls will be invoked
/// with probability 1/4.
pub waiting_probability: Option<u32>,
/// Flag which signals whether recursions must be removed.
pub remove_recursion: bool,
/// If the limit is set to `Some(_)`, programs will try to stop execution
Expand All @@ -156,6 +161,7 @@ impl Default for StandardGearWasmConfigsBundle {
fn default() -> Self {
Self {
log_info: Some("StandardGearWasmConfigsBundle".into()),
waiting_probability: Some(4),
breathx marked this conversation as resolved.
Show resolved Hide resolved
remove_recursion: false,
critical_gas_limit: Some(1_000_000),
injection_types: SyscallsInjectionTypes::all_once(),
Expand All @@ -171,6 +177,7 @@ impl ConfigsBundle for StandardGearWasmConfigsBundle {
fn into_parts(self) -> (GearWasmGeneratorConfig, SelectableParams) {
let StandardGearWasmConfigsBundle {
log_info,
waiting_probability,
remove_recursion,
critical_gas_limit,
injection_types,
Expand All @@ -186,6 +193,10 @@ impl ConfigsBundle for StandardGearWasmConfigsBundle {
if let Some(log_info) = log_info {
syscalls_config_builder = syscalls_config_builder.with_log_info(log_info);
}
if let Some(waiting_probability) = waiting_probability {
syscalls_config_builder =
syscalls_config_builder.with_waiting_probability(waiting_probability);
}
syscalls_config_builder = syscalls_config_builder.with_params_config(params_config);

let memory_pages_config = MemoryPagesConfig {
Expand Down
12 changes: 8 additions & 4 deletions utils/wasm-gen/src/config/module.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@

use std::num::NonZeroUsize;

use crate::MemoryLayout;
use arbitrary::{Arbitrary, Result, Unstructured};
pub use wasm_smith::InstructionKind;
use wasm_smith::{InstructionKind::*, InstructionKinds, SwarmConfig};
Expand Down Expand Up @@ -81,6 +82,8 @@ impl From<(SelectableParams, ArbitraryParams)> for WasmModuleConfig {
min_data_segments,
max_types,
min_types,
memory_offset_choices,
reserved_memory_size,
} = ConstantParams::default();

let SelectableParams {
Expand Down Expand Up @@ -109,7 +112,6 @@ impl From<(SelectableParams, ArbitraryParams)> for WasmModuleConfig {
max_type_size,
max_values,
memory_max_size_required,
memory_offset_choices,
min_element_segments,
min_elements,
min_globals,
Expand Down Expand Up @@ -169,6 +171,7 @@ impl From<(SelectableParams, ArbitraryParams)> for WasmModuleConfig {
min_uleb_size,
multi_value_enabled,
reference_types_enabled,
reserved_memory_size,
tail_call_enabled,
relaxed_simd_enabled,
saturating_float_to_int_enabled,
Expand Down Expand Up @@ -205,7 +208,6 @@ pub struct ArbitraryParams {
max_type_size: u32,
max_values: usize,
memory_max_size_required: bool,
memory_offset_choices: (u32, u32, u32),
min_element_segments: usize,
min_elements: usize,
min_globals: usize,
Expand Down Expand Up @@ -237,7 +239,6 @@ impl Arbitrary<'_> for ArbitraryParams {
max_type_size,
max_values,
memory_max_size_required,
memory_offset_choices,
min_element_segments,
min_elements,
min_globals,
Expand Down Expand Up @@ -267,7 +268,6 @@ impl Arbitrary<'_> for ArbitraryParams {
max_type_size,
max_values,
memory_max_size_required,
memory_offset_choices,
techraed marked this conversation as resolved.
Show resolved Hide resolved
min_element_segments,
min_elements,
min_globals,
Expand Down Expand Up @@ -312,6 +312,8 @@ pub struct ConstantParams {
float_enabled: bool,
memory_grow_enabled: bool,
min_types: usize,
memory_offset_choices: (u32, u32, u32),
reserved_memory_size: Option<u64>,
}

impl Default for ConstantParams {
Expand Down Expand Up @@ -343,6 +345,8 @@ impl Default for ConstantParams {
min_data_segments: 0,
max_types: 100,
min_types: 5,
memory_offset_choices: (75, 25, 0),
reserved_memory_size: Some(MemoryLayout::RESERVED_MEMORY_SIZE as u64),
}
}
}
Expand Down
14 changes: 14 additions & 0 deletions utils/wasm-gen/src/config/syscalls.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ impl SyscallsConfigBuilder {
precise_syscalls_config: PreciseSyscallsConfig::default(),
error_processing_config: ErrorProcessingConfig::None,
log_info: None,
waiting_probability: None,
})
}

Expand Down Expand Up @@ -93,6 +94,13 @@ impl SyscallsConfigBuilder {
self
}

/// Set probability of wait syscalls.
pub fn with_waiting_probability(mut self, waiting_probability: u32) -> Self {
self.0.waiting_probability = Some(waiting_probability);

self
}

/// Setup fallible syscalls error processing options.
pub fn with_error_processing_config(mut self, config: ErrorProcessingConfig) -> Self {
self.0.error_processing_config = config;
Expand All @@ -114,6 +122,7 @@ pub struct SyscallsConfig {
precise_syscalls_config: PreciseSyscallsConfig,
error_processing_config: ErrorProcessingConfig,
log_info: Option<String>,
waiting_probability: Option<u32>,
StackOverflowExcept1on marked this conversation as resolved.
Show resolved Hide resolved
}

impl SyscallsConfig {
Expand Down Expand Up @@ -143,4 +152,9 @@ impl SyscallsConfig {
pub fn error_processing_config(&self) -> &ErrorProcessingConfig {
&self.error_processing_config
}

/// Get probability of wait syscalls.
pub fn waiting_probability(&self) -> Option<u32> {
self.waiting_probability
}
}
19 changes: 14 additions & 5 deletions utils/wasm-gen/src/generator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -124,8 +124,9 @@ impl<'a, 'b> GearWasmGenerator<'a, 'b> {
let (disabled_mem_gen, frozen_gear_wasm_gen, mem_imports_gen_proof) =
self.generate_memory_export();

let (disabled_ep_gen, frozen_gear_wasm_gen, ep_gen_proof) =
Self::from((disabled_mem_gen, frozen_gear_wasm_gen)).generate_entry_points()?;
let (disabled_ep_gen, frozen_gear_wasm_gen, ep_gen_proof, mem_imports_gen_proof) =
Self::from((disabled_mem_gen, frozen_gear_wasm_gen))
.generate_entry_points(mem_imports_gen_proof)?;

let (disabled_syscalls_invocator, frozen_gear_wasm_gen) =
Self::from((disabled_ep_gen, frozen_gear_wasm_gen))
Expand Down Expand Up @@ -173,16 +174,24 @@ impl<'a, 'b> GearWasmGenerator<'a, 'b> {
/// Generate gear wasm gentry points using entry points generator.
pub fn generate_entry_points(
self,
mem_import_gen_proof: MemoryImportGenerationProof,
) -> Result<(
DisabledEntryPointsGenerator<'a, 'b>,
FrozenGearWasmGenerator<'a, 'b>,
GearEntryPointGenerationProof,
MemoryImportGenerationProof,
)> {
let (ep_gen, frozen_gear_wasm_gen): (EntryPointsGenerator, FrozenGearWasmGenerator) =
self.into();
let (disabled_ep_gen, ep_gen_proof) = ep_gen.generate_entry_points()?;

Ok((disabled_ep_gen, frozen_gear_wasm_gen, ep_gen_proof))
let (disabled_ep_gen, ep_gen_proof, mem_import_gen_proof) =
ep_gen.generate_entry_points(mem_import_gen_proof)?;

Ok((
disabled_ep_gen,
frozen_gear_wasm_gen,
ep_gen_proof,
mem_import_gen_proof,
))
}

/// Generate syscalls using syscalls module generators.
Expand Down
41 changes: 37 additions & 4 deletions utils/wasm-gen/src/generator/entry_points.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,12 @@
//! Gear wasm entry points generator module.

use crate::{
generator::{CallIndexes, FrozenGearWasmGenerator, GearWasmGenerator, ModuleWithCallIndexes},
EntryPointsSet, WasmModule,
generator::{
CallIndexes, FrozenGearWasmGenerator, GearWasmGenerator, MemoryImportGenerationProof,
ModuleWithCallIndexes,
},
wasm::{PageCount as WasmPageCount, WasmModule},
EntryPointsSet, MemoryLayout,
};
use arbitrary::{Result, Unstructured};
use gear_wasm_instrument::parity_wasm::{
Expand Down Expand Up @@ -99,9 +103,11 @@ impl<'a, 'b> EntryPointsGenerator<'a, 'b> {
/// Returns disabled entry points generator and a proof that all entry points from config were generated.
pub fn generate_entry_points(
mut self,
mem_import_gen_proof: MemoryImportGenerationProof,
techraed marked this conversation as resolved.
Show resolved Hide resolved
) -> Result<(
DisabledEntryPointsGenerator<'a, 'b>,
GearEntryPointGenerationProof,
MemoryImportGenerationProof,
)> {
log::trace!("Generating gear entry points");

Expand All @@ -117,7 +123,11 @@ impl<'a, 'b> EntryPointsGenerator<'a, 'b> {
self.generate_export("handle_reply")?;
}

Ok((self.disable(), GearEntryPointGenerationProof(())))
Ok((
self.disable(),
GearEntryPointGenerationProof(()),
mem_import_gen_proof,
))
}

/// Generates an export function with a `name`.
Expand Down Expand Up @@ -165,7 +175,8 @@ impl<'a, 'b> EntryPointsGenerator<'a, 'b> {
});

let export_body_instructions =
self.generate_export_body(export_body_call_idx, export_body_call_func_type)?;
self.generate_export_body(name, export_body_call_idx, export_body_call_func_type)?;

self.module.with(|module| {
let module = builder::from_module(module)
.function()
Expand Down Expand Up @@ -194,6 +205,7 @@ impl<'a, 'b> EntryPointsGenerator<'a, 'b> {
/// Generates body of the export function.
fn generate_export_body(
&mut self,
name: &str,
export_body_call_idx: usize,
export_body_call_func_type: FunctionType,
) -> Result<Vec<Instruction>> {
Expand All @@ -211,6 +223,27 @@ impl<'a, 'b> EntryPointsGenerator<'a, 'b> {
}
res.push(Instruction::Call(export_body_call_idx as u32));
res.extend(results.iter().map(|_| Instruction::Drop));

// after initializing the program, we will write about this in a special pointer
if name == "init" {
let memory_size_pages = self
.module
.initial_mem_size()
.expect("generator is instantiated with a mem import generation proof");
let mem_size = Into::<WasmPageCount>::into(memory_size_pages).memory_size();

let MemoryLayout {
init_called_ptr, ..
} = MemoryLayout::from(mem_size);

res.extend_from_slice(&[
// *init_called_ptr = true
Instruction::I32Const(init_called_ptr),
Instruction::I32Const(1),
Instruction::I32Store8(0, 0),
]);
}

res.push(Instruction::End);

Ok(res)
Expand Down
8 changes: 8 additions & 0 deletions utils/wasm-gen/src/generator/syscalls.rs
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,14 @@ impl InvocableSyscall {
})
}

/// Returns `true` for wait syscalls.
fn is_wait_syscall(&self) -> bool {
use InvocableSyscall::*;
use SyscallName::*;

matches!(self, Loose(Wait | WaitFor | WaitUpTo))
}

/// Checks whether syscall is error-prone either by returning error indicating value
/// or by providing error pointer as a syscall param.
///
Expand Down
13 changes: 7 additions & 6 deletions utils/wasm-gen/src/generator/syscalls/imports.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ use crate::{
GearWasmGenerator, MemoryImportGenerationProof, ModuleWithCallIndexes,
},
wasm::{PageCount as WasmPageCount, WasmModule},
InvocableSyscall, SyscallInjectionType, SyscallsConfig,
InvocableSyscall, MemoryLayout, SyscallInjectionType, SyscallsConfig,
};
use arbitrary::{Error as ArbitraryError, Result, Unstructured};
use gear_wasm_instrument::{
Expand Down Expand Up @@ -325,8 +325,8 @@ impl<'a, 'b> SyscallsImportsGenerator<'a, 'b> {
}

impl<'a, 'b> SyscallsImportsGenerator<'a, 'b> {
/// The amount of memory used to create a precise syscall.
const PRECISE_SYSCALL_MEMORY_SIZE: u32 = 100;
/// The amount of reserved memory used to create a precise syscall.
const PRECISE_SYSCALL_RESERVED_MEMORY_SIZE: u32 = 128;

/// Generates a function which calls "properly" the `gr_reservation_send`.
fn generate_send_from_reservation(
Expand Down Expand Up @@ -799,12 +799,13 @@ impl<'a, 'b> SyscallsImportsGenerator<'a, 'b> {

/// Reserves enough memory build precise syscall.
fn reserve_memory(&self) -> i32 {
self.memory_size_in_bytes()
.saturating_sub(Self::PRECISE_SYSCALL_MEMORY_SIZE) as i32
self.memory_size_bytes()
.saturating_sub(MemoryLayout::RESERVED_MEMORY_SIZE)
.saturating_sub(Self::PRECISE_SYSCALL_RESERVED_MEMORY_SIZE) as i32
StackOverflowExcept1on marked this conversation as resolved.
Show resolved Hide resolved
}

/// Returns the size of the memory in bytes that can be used to build precise syscall.
fn memory_size_in_bytes(&self) -> u32 {
fn memory_size_bytes(&self) -> u32 {
let initial_mem_size: WasmPageCount = self
.module
.initial_mem_size()
Expand Down
Loading
Loading