Chapter 6.2. Memory Leakage Detection
Rust employs a novel ownership-based resource management model to facilitate automated deallocation during compile time. However, sometimes developers may interventionally drives it into mannualy drop mode and which is prone to memory leak.
rCanary is a static model checker to detect leaks across the semi-automated boundary.
We support the detection of the following two types of issues:
- Orphan Object: An orphan object is the heap item wrapped by the smart pointer ManuallyDrop.
fn main() { let mut buf = Box::new("buffer"); // heap item ’buf’ becomes an orphan object let ptr = Box::into_raw(buf); // leak by missing free operation on ’ptr’ // unsafe { drop_in_place(ptr); } }
- Proxy type: A proxy type is a compound type having at least one field that stores an orphan object.
struct Proxy<T> { ptr: * mut T, } impl<T> Drop for Proxy<T> { fn drop(&mut self) { // user should manually free the field ’ptr’ // unsafe { drop_in_place(self.ptr); } } fn main() { let mut buf = Box::new("buffer"); // heap item ’buf’ becomes an orphan object let ptr = &mut * ManuallyDrop::new(buf) as * mut _; let proxy = Proxy { ptr }; // leak by missing free ’proxy.ptr’ in drop } }
What's behind rCanary?
Canary is a component of RAPx, and the overall architecture is as shown in the following diagram:
It can generate SMT-Lib2 format constraints for Rust MIR and is implemented as a Cargo component. We design an encoder to abstract data with heap allocation and formalize a refined leak-free memory model based on boolean satisfiability.
#![allow(unused)] fn main() { pub fn start_analyzer(tcx: TyCtxt, config: RapConfig) { let rcx_boxed = Box::new(RapGlobalCtxt::new(tcx, config)); let rcx = Box::leak(rcx_boxed); let _rcanary: Option<rCanary> = if callback.is_rcanary_enabled() { let mut rcx = rCanary::new(tcx); rcx.start(); Some(rcx) } else { None }; } pub fn start(&mut self) { let rcx_boxed = Box::new(rCanary::new(self.tcx)); let rcx = Box::leak(rcx_boxed); TypeAnalysis::new(rcx).start(); FlowAnalysis::new(rcx).start(); } }
The start_analyzer function defines all the analysis processes of rCanary, with two important steps being TypeAnalysis and FlowAnalysis, corresponding to the ADT-DEF Analysis, constraint construction, and constraint solving described in the paper.
Running rCanary
Invocation
Before using rCanary, please make sure that the Z3 solver is installed in your operating system.
If not, run: brew install z3 or apt-get install z3, with a minimum version of 4.10 for Z3.
Running rCanary within RAPx is very simple, just enter the sysroot (root directory) of a Cargo program and run:
cargo rapx -mleak
Note: Analysis will be terminated if the directory or toolchain version is not
nightly-2024-06-30.
Additional Configure Arguments
rCanary also provides several optional environment variables output sections for intermediate results.
ADTRES- Print the results of type analysis, including the type definition and the analysis tuple.
Z3GOAL- Emit the Z3 formula of the given function, it is in the SMT-Lib2 format.
ICXSLICE- Set Verbose to print the middle metadata for rCANARY debug.
Note: These parameters may change due to version migration.
Running Results on PoC
For the test case, we chose the same PoC as in the paper to facilitate user understanding and testing.
fn main() { let mut buf = Box::new("buffer"); // heap item ’buf’ becomes an orphan object let ptr = Box::into_raw(buf); // leak by missing free operation on ’ptr’ // unsafe { drop_in_place(ptr); } }
Running cargo rapx -mleak:
22:10:39|RAP|WARN|: Memory Leak detected in function main warning: Memory Leak detected. --> src/main.rs:3:16 | 1 | fn main() { 2 | let buf = Box::new("buffer"); 3 | let _ptr = Box::into_raw(buf); | ------------------ Memory Leak Candidates. 4 | }
ADTRES
#![allow(unused)] fn main() { core::fmt::rt::Placeholder [(Unowned, [])] std::ptr::Unique<T/#0> [(Owned, [false])] std::convert::Infallible [] std::marker::PhantomData<T/#0> [(Unowned, [false])] std::boxed::Box<T/#0, A/#1> [(Owned, [false, true])] std::alloc::Global [(Unowned, [])] std::alloc::Layout [(Unowned, [])] std::mem::ManuallyDrop<T/#0> [(Unowned, [true])] std::result::Result<T/#0, E/#1> [(Unowned, [true, false]), (Unowned, [false, true])] std::alloc::AllocError [(Unowned, [])] core::fmt::rt::ArgumentType<'a/#0> [(Unowned, [false]), (Unowned, [false])] core::fmt::rt::Count [(Unowned, []), (Unowned, []), (Unowned, [])] std::fmt::Arguments<'a/#0> [(Unowned, [false])] std::ptr::NonNull<T/#0> [(Unowned, [false])] std::ptr::Alignment [(Unowned, [])] core::fmt::rt::Alignment [(Unowned, []), (Unowned, []), (Unowned, []), (Unowned, [])] std::ops::ControlFlow<B/#0, C/#1> [(Unowned, [false, true]), (Unowned, [true, false])] core::fmt::rt::Argument<'a/#0> [(Unowned, [false])] std::option::Option<T/#0> [(Unowned, [false]), (Unowned, [true])] std::ptr::alignment::AlignmentEnum [(Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, []), (Unowned, [])] }
Z3GOAL
#![allow(unused)] fn main() { (goal |CONSTRAINTS: T 0| (= |0_0_1_ctor_fn| #b01) |CONSTRAINTS: S 1 2| (= |1_2_1| #b00) (= |1_2_3_ctor_asgn| |0_0_1_ctor_fn|) |CONSTRAINTS: T 1| (= |1_0_3_drop_all| (bvand |1_2_3_ctor_asgn| #b10)) (= |1_0_2_ctor_fn| #b1) |CONSTRAINTS: S 2 1| |CONSTRAINTS: T 2| (= |2_0_1_return| |1_2_1|) (= |2_0_1_return| #b00) (= |2_0_2_return| |1_0_2_ctor_fn|) (= |2_0_2_return| #b0) (= |2_0_3_return| |1_0_3_drop_all|) (= |2_0_3_return| #b00)) }
ICXSLICE
#![allow(unused)] fn main() { IcxSlice in Terminator: 0: _1 = std::boxed::Box::<&str>::new(const "buffer") -> [return: bb1, unwind continue] IcxSliceForBlock [Taint { set: {} }, Taint { set: {} }, Taint { set: {} }, Taint { set: {} }] [0, 2, 0, 0] [Declared, Init(|0_0_1_ctor_fn|), Declared, Declared] [[], [Owned, Unowned], [], []] [TyWithIndex(None), TyWithIndex(Some((2, std::boxed::Box<&'{erased} str, std::alloc::Global>, None, true))), TyWithIndex(None), TyWithIndex(None)] IcxSlice in Assign: 1 2: Assign((_3, move _1)) IcxSliceForBlock [Taint { set: {} }, Taint { set: {} }, Taint { set: {} }, Taint { set: {} }] [0, 2, 0, 2] [Declared, Init(|1_2_1|), Declared, Init(|1_2_3_ctor_asgn|)] [[], [Owned, Unowned], [], [Owned, Unowned]] [TyWithIndex(None), TyWithIndex(Some((2, std::boxed::Box<&'{erased} str, std::alloc::Global>, None, true))), TyWithIndex(None), TyWithIndex(Some((2, std::boxed::Box<&'{erased} str, std::alloc::Global>, None, true)))] IcxSlice in Terminator: 1: _2 = std::boxed::Box::<&str>::into_raw(move _3) -> [return: bb2, unwind: bb3] IcxSliceForBlock [Taint { set: {} }, Taint { set: {} }, Taint { set: {TyWithIndex(Some((2, std::boxed::Box<&'{erased} str, std::alloc::Global>, None, true)))} }, Taint { set: {} }] [0, 2, 1, 2] [Declared, Init(|1_2_1|), Init(|1_0_2_ctor_fn|), Init(|1_0_3_drop_all|)] [[], [Owned, Unowned], [Unowned], [Owned, Unowned]] [TyWithIndex(None), TyWithIndex(Some((2, std::boxed::Box<&'{erased} str, std::alloc::Global>, None, true))), TyWithIndex(Some((1, *mut &'{erased} str, None, true))), TyWithIndex(Some((2, std::boxed::Box<&'{erased} str, std::alloc::Global>, None, true)))] IcxSlice in Assign: 2 1: Assign((_0, const ())) IcxSliceForBlock [Taint { set: {} }, Taint { set: {} }, Taint { set: {TyWithIndex(Some((2, std::boxed::Box<&'{erased} str, std::alloc::Global>, None, true)))} }, Taint { set: {} }] [0, 2, 1, 2] [Declared, Init(|1_2_1|), Init(|1_0_2_ctor_fn|), Init(|1_0_3_drop_all|)] [[], [Owned, Unowned], [Unowned], [Owned, Unowned]] [TyWithIndex(None), TyWithIndex(Some((2, std::boxed::Box<&'{erased} str, std::alloc::Global>, None, true))), TyWithIndex(Some((1, *mut &'{erased} str, None, true))), TyWithIndex(Some((2, std::boxed::Box<&'{erased} str, std::alloc::Global>, None, true)))] IcxSlice in Terminator: 2: return IcxSliceForBlock [Taint { set: {} }, Taint { set: {} }, Taint { set: {TyWithIndex(Some((2, std::boxed::Box<&'{erased} str, std::alloc::Global>, None, true)))} }, Taint { set: {} }] [0, 2, 1, 2] [Declared, Init(|1_2_1|), Init(|1_0_2_ctor_fn|), Init(|1_0_3_drop_all|)] [[], [Owned, Unowned], [Unowned], [Owned, Unowned]] [TyWithIndex(None), TyWithIndex(Some((2, std::boxed::Box<&'{erased} str, std::alloc::Global>, None, true))), TyWithIndex(Some((1, *mut &'{erased} str, None, true))), TyWithIndex(Some((2, std::boxed::Box<&'{erased} str, std::alloc::Global>, None, true)))] }
Reference
The feature is based on our rCanary work, which was published in TSE
@article{cui2024rcanary,
title={rCanary: rCanary: Detecting memory leaks across semi-automated memory management boundary in Rust},
author={Mohan Cui, Hongliang Tian, Hui Xu, and Yangfan Zhou},
journal={IEEE Transactions on Software Engineering},
year={2024},
}