Memory Strategies – The Merits of (Un)safe • 2022 • Conference Guide • Handmade Seattle

0:00Opening Titles

0:43Abner Coimbre: Welcome to the podcast on memory management and safety

2:07AC: Introduce our moderator Allen Webster

2:50AC: Introduce Evan Ovadia, creator of Vale,¹ representing memory safety

3:13AC: Introduce Ryan Fleury, promoting the merits of unsafety

3:24AC: Introduce John Austin, founder of Pontoco,² remaining more neutral

4:03AC: Allen, take it away

4:15Allen Webster: Introductions

4:40Evan Ovadia: Introduction: Creator of Vale,³ work on games, servers, apps and languages

5:07John Austin: Introduction: Founder and lead engineer of Pontoco,⁴ interested in the pragmatics of languages

6:03Ryan Fleury: Introduction: Working on the debugger project at Epic Games, lifelong game maker, coming from the memory unsafe angle

6:52AW: Introduction: Meta-interested in seeing programmers get better at having productive conversations on contentious topics

7:23AW: Laying out the basics: 1) What is memory management?

7:50RF: Memory management as the problem-space of organising memory re-use, grouping and arrangement

8:41AW: Does that definition make sense to everyone else?

8:49EO: That lines up pretty perfectly with my understanding

8:52JA: Memory management considerations at the application-level

9:44AW: So Ryan's definition is at the lowest level, with memory management taking on additional meaning in the context of different languages?

10:08JA: Yeah

10:25AW: Laying out the basics: 2) What is memory safety?

10:35EO: Memory safety as ways to avoid undefined behaviour

11:44AW: Does everyone understand that definition?

11:49RF: Narrow down memory safety to the use-after-free and double-free area of undefined behaviour

12:22EO: Include integer overflow in the scope of memory safety

12:37AW: Summarise memory safety as ways to address bugs arising from memory-related undefined behaviours

13:06JA: Associate memory safety with security

13:48EO: Angles of memory safety: 1) Bounds-checking

14:16EO: Angles of memory safety: 2) Privacy

14:47AW: How important is memory safety?

15:25EO: Memory safety importance is tied to the domain

16:30RF: Memory safety as a proxy to other bugs

17:26JA: What do you mean by "proxy"?

17:37RF: Memory safety's solving of a subset of problems, all of which actually require robust logic

18:50JA: Yeah, you can solve your memory problems but not necessarily your actual bugs?

19:00RF: Even security exploits

19:08JA: The difficulty of determining when we're merely shifting bugs

19:28JA: Valuing memory safety for its ease of identifying and handling bugs

21:05EO: Memory safety making problems less scary

21:31JA: Memory safety depending on the domain

21:47AW: Achieving memory safety: What is the borrow checker in Rust?

22:17JA: Rust's borrow checker

24:18EO: Do you consider the single-ownership aspect to be part of the borrow checker?

24:23JA: In terms of ownership of the values themselves as opposed to the reference model?

24:29EO: Yeah

24:30JA: The ownership comes from the reference model

24:43EO: I feel like they're intertwined

24:49JA: Multi-ownership with pointers

25:06EO: Single-ownership in Rust and C

26:04JA: That's a very good point

26:09AW: How much of this has runtime performance implications?

26:20JA: Rust's borrow checker performance implications: 1) None, operating at compile-time

26:48JA: Rust's borrow checker performance implications: 2) Its force to write your code differently

27:31AW: So it's not like there's a bit that gets passed around to figure out who owns something?

27:40JA: With single-ownership there is only one place that the data resides

28:04EO: You can think of it like a bit that travels with the data, but at compile-time

28:21JA: I feel like it's simpler than that, though

28:50RF: Developing personal programming rules

30:00RF: If you moved to C, to what degree would the lack of the borrow checker be a problem for you?

30:49EO: Bringing a pure functional style from Rust to C, but leaving behind Rust's trouble with intrusive data structures

31:55RF: I use intrusive structures in C a lot

32:04AW: Can we define "intrusive structures"?

32:10RF: Intrusive structure as a struct that points to a member of its own type

32:59AW: Is that what "intrusive data structure" means to you, Evan

33:03EO: Clarify that Rust can handle tree-shaped intrusive data structures

33:19AW: Would a singly-linked list be a special-case of a tree and, therefore, work?

33:26EO: Yes

33:27AW: Why doesn't Rust's borrow checker like intrusive data structures?

33:32EO: Understanding the borrow checker's trouble with doubly-linked lists due to them needing multiple mutable references to the same node

34:30AW: So I couldn't remove a node because I couldn't fix up the nodes on either side?

34:37EO: Yeah, you can reach the node through multiple immutable references, but can't modify anything

34:47JA: Weird Rust libraries that allow doubly-linked lists

35:10EO: The restrictive borrow checker's inspiration GhostCell⁵

35:57JA: GhostCell⁶ is innovative, but only solving a problem that Rust itself creates

36:35EO: Getting around the borrow checker as a national sport, e.g. Generational indices

37:15RF: Graceful failure with generational handles in C

38:21EO: Generational indices appearing in Windows XP

38:50RF: I see

38:51AW: Could we define generational indices?

38:59EO: Generational index used to say "I am the nth spaceship to inhabit this slot in the array"

40:30AW: Unilaterally disallow memory safety from games, for the delightful bugs when pointing to wrong entities

41:14EO: Generational indices handle problems that memory safety cannot

42:01JA: The borrow checker by default wouldn't let you access an old spaceship, would it?

42:13EO: In C, using a fixed array…

42:20JA: Right, if you're getting around the borrow checker by using a flat array of values

42:30AW: With Ryan coming to and taking the performance hit of generational indices, unencumbered by Rust's borrow checker, it's not really a performance hit if it's the correct solution anyway

43:01RF: The performance of generational indices

43:43RF: Thoughts on forests of malloc() and free() – the programming of which Rust's memory safety features seem designed to aid – being wrong in the first place

45:03JA: Layers of performance

46:26JA: Plug 'Hard Mode Rust'⁷

47:06EO: Using techniques from 'Hard Mode Rust'⁸ in C gives you stronger safety guarantees

47:58EO: Custom allocators in 'Hard Mode Rust'⁹ – pool and temporary arenas – yielding memory safety

50:37JA: Up-front allocation only being workable in certain domains

51:52AW: How is a pool allocator implemented?

52:09EO: Implementing a pool allocator

52:53AW: Does that make sense to everyone else?

52:55JA: Yeah

52:58AW: Just making sure

53:02JA: Seeing terms thrown around

53:14AW: What does "arena allocator" mean?

53:22RF: Implementing an arena (or bump) allocator

55:41RF: Implementing a growable pool atop an arena

56:30AW: Does everyone else follow all those ideas?

56:37JA: Admire the virtual memory table

57:36JA: Effectively reimplementing malloc()

58:14EO: The general-purpose nature of malloc(), and speed of arenas

58:54JA: Layering on free lists brings you closer to what malloc() does anyway

59:22EO: Custom allocators tracking the size of allocations

59:53JA: Liking custom allocators' tracking of allocation sizes

1:00:10EO: Excitement for Vale,¹⁰ Zig¹¹ and Odin¹² natively supporting custom allocators

1:01:10AW: How can a novel custom allocator be faster than malloc()

1:01:42JA: More interested in "how can you make a faster general-purpose allocator?"

1:02:18AW: In Unix libc is kind of the OS

1:02:23JA: malloc() can't know as much about the program as a custom allocator

1:02:30EO: malloc() and free() prioritise simplicity and stability and, in so doing, sacrifice performance-helping information

1:03:06RF: Making an arena the ground basis for composing memory allocators

1:06:22AW: So you don't just want to have a custom allocator, but to have a different allocator from malloc() and free() as the base?

1:06:41RF: Thoughts on malloc() and free() locking you into the most generic form of allocator

1:07:32AW: As in a statically known size?

1:07:39RF: Implementing a pool atop an arena

1:07:53JA: Ergonomics of memory allocation

1:08:16EO: Zig¹³ and Odin¹⁴ let you pass in an allocator

1:08:29JA: Not wanting the ergonomics of passing in an allocator

1:08:46EO: Are you saying, Ryan, that you can write code better if you know the specific allocator you're working with?

1:08:59RF: Being okay with Odin's¹⁵ context system, and considering custom allocation atop arenas to be simpler

1:09:49JA: The benefit of not having to call free() with arenas

1:10:07AW: Evan, did you follow that train of thought?

1:10:16EO: The trade-off of writing performant and simple code against a specific allocator, but having to remember to call free() with an arbitrary allocator

1:10:47AW: Summarise Ryan's arena-based proposal in the context of library development

1:12:14RF: Why arena-based allocation is easier, with a quote from Raymond Chen's article 'An amusing story about a practical use of the null garbage collector'¹⁶

1:15:19EO: Finding arenas to be more special-purpose, for temporary allocation and non-temporary fixed data

1:16:10AW: We have a contradiction

1:16:22JA: Program lifetimes that may struggle with the arena model

1:17:56RF: Arena lifetime example: Per-request arena for a web server

1:18:18JA: Thoughts on an asynchronous task runner possibly not having the space to allocate a page per task

1:18:44RF: For tasks smaller than a page, why not just use the stack?

1:19:04JA: The stack might disappear in an asynchronous task runner if you go to sleep

1:19:28RF: Implementing a pool that composes with an arena

1:22:40JA: So your opinion is total, that all programs should be written with arenas?

1:22:57EO: Benefits of arenas, e.g. Basil¹⁷

1:23:58AW: That'd be the opposite of the ECS dream

1:24:03EO: Maybe 50% of all dynamic allocation could use arenas

1:24:17JA: The challenge of unravelling arenas

1:24:40AW: What is that 50% difference between Ryan and Evan's perspectives?

1:25:07EO: I think you could use the "compose atop arenas" system, just not a bump allocator, for everything

1:25:27RF: Composing a pool atop an arena, providing the arena to, say, EntityAlloc() and EntityFree() functions

1:28:06AW: Does that 50% difference remain, Evan?

1:28:20EO: Clarifying question: Can this one function be used in situations where you're using either a bump or pool allocator?

1:28:32RF: Which function?

1:28:35EO: The one that's calling EntityAlloc() and EntityFree()

1:28:39RF: Not either / or, the idea is that you pass both an arena and a pool's free list

1:29:23JA: Counter example: …

1:29:33AW: Pause to savour the disagreement

1:29:47EO: Worry about accidental agreement

1:29:53JA: Counter example: A program that accepts millions of arbitrary-length requests

1:31:01RF: So this program is receiving potentially 10 million requests per second each with arbitrary-length input?

1:31:25JA: Maybe it's generating arbitrary-length strings

1:31:45RF: Lay out the one-request-per-second pitch

1:32:38JA: So you've up-front allocated enough memory to fit one string of arbitrary length?

1:32:43RF: Are you asking how arena-based allocation could deal with the sheer rate of arbitrary-length requests coming in?

1:33:01JA: And, worst case, most users have small requests, with only the last 500,000 dishing up 1TB strings

1:33:19RF: Well, 1TB is pretty nuts

1:33:28JA: Yeah, let's say up to 1GB, but they're quickly coming in

1:33:33RF: Server architecture may support spawning enough address spaces to solve this problem

1:33:51AW: I don't think what's relevant to the question is the fact that physical memory will give out at some point

1:34:06JA: If virtual allocating a bunch of space for each request, they will also run out

1:34:21RF: Approaching John's counter example problem: Multiple thread pools for different-sized requests, rate-limiting them

1:36:17JA: Agree with the code simplicity of arenas, but they have knock-on effects

1:37:10RF: I'd be surprised if malloc() [sic] was going to be less performant

1:37:17AW: The spec of that problem was: "a server that implements malloc()"

1:38:09JA: Enjoy that summary of the hypothetical problem

1:38:30AW: Summarise Ryan's case that arenas can cover many use cases, and John's case that lifetimes can be too chaotic for arenas

1:39:31AW: What could make you all reevaluate your thinking on memory safety and arena-based allocation?

1:40:35EO: Memory safety reevaluation: Scepticism that the borrow-checker is universally useful

1:41:23EO: Arena reevaluation: Enjoying the concept of an output arena

1:42:16AW: That's great. John?

1:42:21JA: Memory safety reevaluation: Improved ergonomics over Rust

1:44:26AW: It sounds like you see benefits to other styles, but the ergonomics are too costly?

1:44:44JA: I need to write lots of code that's fast enough for the use-case

1:45:12AW: That makes perfect sense

1:45:17EO: Is Rust's improved development speed attributable to the borrow checker or the type system?

1:45:41JA: Love Rust for smart API decisions, type system and sensibleness

1:46:25AW: Ryan?

1:46:34RF: Memory safety reevaluation: Welcoming compile-time checks if implemented more simply

1:50:11EO: Note of hope: 1) Languages selectively learning from Rust, e.g. Val¹⁸ with its mutable value semantics model and cloning

1:51:02EO: Note of hope: 2) Composability of memory safety features, e.g. Vale's¹⁹ region borrow checker, and Cone²⁰ as Rust with reference counting and garbage collection

1:51:46RF: Yearn for simplification to everything but the language, e.g. file formats, debug info

1:52:54JA: Love Rust's tooling

1:53:03AW: Time for plugs

1:53:23EO: Plug Vale²¹

1:53:29JA: Plug personal²² and Pontoco²³ websites

1:53:45RF: Plug blog²⁴ and the article 'Untangling Lifetimes: The Arena Allocator'²⁵

1:54:01AW: Thanks, everybody and Abner

1:54:05Closing Titles

Conference Guide

Keyboard Navigation

Global Keys

Menu toggling

In-Menu and Index Controls

Quotes and References Menus and Index

Quotes, References and Credits Menus

Filter Menu

Filter and Link Menus

Credits Menu