4016 stories
·
4 followers

Google's Rust Style Guide

1 Share

Google Style Guides

This document is a style guide for the usage of the Rust programming language at Google. In this context, “style” refers to more than just formatting; it includes guidance on API and implementation design.

Rust is used in many places, including google3, Android, ChromeOS, Fuchsia, and Chrome, and we’d like for Rust usage at Google to be consistent both within and across codebases. In addition to bringing consistency to our Rust code, style guidance also helps establish a standard of quality.

Background

Scope

The current scope of authority of the rules outlined in this guide:

  • In Android, Chromium, and other codebases: This style guide has been developed in consultation with other codebases, and they intend to start applying it. These codebases may have their own rules about how strictly this is expected to be applied. Expect more information here as these codebases adopt this guide.

Goals of the style guide

By and large, we have this document for the same reasons we have a C++ style guide.

Most style rules come from one or more of the goals listed below.

Style rules should prioritize readability and maintainability over writeability

We expect our code to be around for long periods of time, and we consider a little bit of extra effort applied during code writing to be worth it in the long run if the style rule improves readability.

Style rules should favor better automation

Large-Scale Changes (LSCs) are common at Google, and consistency helps make LSCs easier to carry out.

Style rules should be consistent with the wider Rust ecosystem where possible

The broader Rust ecosystem already has a pretty good amount of consistency in style, and we should rely on that where possible to reduce surprise and make it less work to learn Google’s Rust style.

Style rules should pull their weight

Every Rust developer and reviewer at Google has to keep these style rules in mind: as such, we should be careful to include rules that have a tangible benefit.

Style rules may help avoid minor style disagreements

There are many cases where reasonable people may disagree about a style, and we expect that disagreement to show up often on CLs. Having a style rule helps avoid having to rehash this each time. Rules in this category that do not have other benefits should be kept to a minimum.

Follow public Rust style rules

Follow the official public Rust style rules to the extent they don’t conflict with this style guide.

A “conflict” would be a case where this style guide suggests a pattern that is discouraged, or counter to the primary encouraged pattern in the Rust style guide.

Why does the guidance in this style guide deviate from regular Rust style rules?

Most style guides have three goals: consistency, readability, and encouraging good API design. Regardless of the specific choices made, it helps everyone to have these choices made beforehand to limit bikeshedding.

This style guide shares those goals, but it also has an additional goal not seen as often: smoothing the way for Large-Scale Changes (LSCs). Large codebases like many of the ones at Google occasionally need to run codebase-spanning refactorings that are assisted by tooling: for this to work well, the code should be something tooling has an easy time working with. Consistency helps for this, but so does having deliberate refactor-friendly patterns.

Apply API design rules in context

When applying rules that relate to API design, you should distinguish two types of code:

  • Library code

    • Intended for use by other projects (applications or other libraries).
    • Solves a relatively general problem.
  • Application code

    • Intended for use in a single application.
    • Solves a more specific problem.

For library code, style guide rules that concern API design are binding.

For application code, the rules on API design are a strong recommendation. As you apply them, keep in mind that since application code is written to solve a specific problem, it should generally favor simplicity and directness over broad reusability. Avoid premature generalization and abstraction; focus on what is required to meet the immediate need.

Code organization

Terminology:

  • A crate is a unit of compilation. A crate always contains a nameless Rust module, the “root module”.

  • A Rust module is a unit of code organization within a crate. Rust modules are used to control visibility of names. Modules can be nested. They can also be used to split code across files and directories.

The following sections convey that in Chromium, it is preferable to not make use of all possibilities of splitting modules across files.

Prefer small crates [chrome]

Prefer small crates in Chromium; ideally, but not necessarily, a single file (focusing on one abstraction). If your code can be split up into small crates, do so.

A crate corresponds to a single rust_static_library (chrome) rule, which may contain one or more .rs files.

Additional considerations:

  • The root module is allowed to re-export public APIs from leaf submodules, making submodules an implementation detail of the crate (as users don’t need to know about these submodules).

  • This rule about small crates applies equally to crates with and without submodules. First, evaluate whether the APIs need to be in the same crate, and only then consider whether submodules could help organize the APIs.

  • If you decide to use submodules, follow the expected file structure.

Larger crates are allowed when trait impl coherence rules force multiple types and traits to be included in one library, or if you’re building a large foundational library.

Rationale

Fine-grained library targets are recommended in Chromium due to the massive size of our codebase, non-local effects of adding dependencies in the build graph, and to improve performance of parallel compilation and build caching.

Use flat file layout for rust_library sources [chrome]

In Chromium .rs source files belonging to one rust_library (one Rust crate) must reside in a single directory.

Don’t use subdirectories to organize files that belong to a single crate. If organizing one crate’s files into subdirectories feels necessary, this is a strong indication that the crate has grown too large and should be split.

This crate layout is intentionally different from the conventional Cargo directory structure typically used in open-source repositories.

Exception: Rust code in //third_party that is also built with Cargo should follow the conventional Cargo directory structure.

Rationale

The approach in Chromium is driven by the following factors:

  • Based on the preference for small crates, most rust_library targets will only contain one source file.
  • Because of the above point, creating a directory for each rust_library target would result in most directories containing only one source file.
  • Multi-file rust_library targets should not create dedicated directories for submodules either; this ensures a consistent approach.

Name source files and leaf modules after the primary user-defined item

When a leaf module has a primary user-defined item (e.g., Foo), the module should be named after that item (e.g., mod foo and foo.rs).

Leaf modules that are not primarily about any particular abstraction can be named as desired.

Rationale

This makes it easy to identify the content/purpose of a source file or leaf module at a glance.

Don’t implement APIs in non-leaf modules

The only items in non-leaf modules should be module declarations (not definitions) and re-exports.

Note that the definition of leaf modules allows the following sub-modules: mod test and cross-language binding modules like #[cxx::bridge].

Exceptions:

  • Having a small amount of core “ties things together” implementation code in a non-leaf module may be fine. The goal is to not have a lot of code in these modules.

Rationale

This makes it easier to find things, and avoids mixing structure and contents.

Don’t use glob imports

Don’t use glob imports, except in the following situations where you may optionally do so at your own discretion:

  • use super::* for inline modules
  • use enum::* in method bodies of enums, and functions that are primarily dealing with that enum
  • pub use submodule::* for non-leaf modules
  • C++ interop: Since the same namespace (in C++) is a submodule in different Rust crates (in Rust), you may merge the namespaces into a private copy using glob imports. For example:

    mod absl {
        pub use status::absl::*;
        pub use strings::absl::*;
    }
    

    In this hypothetical example, you can use absl::Status and absl::StrCat from Rust, the same way you would from C++, instead of spelling them as status::absl::Status and strings::absl::StrCat.

    This must only be used for merging C++ namespaces, and the resulting merged module must not be public.

If using third-party crates that encourage a prelude, it is up to the reviewer’s discretion to allow it.

This also means that defining preludes should be avoided.

Rationale

Glob imports increase toil for code janitors running LSCs, and make it harder to know where a type comes from in contexts without xrefs.

A use statement should import items from one module

This rule corresponds to the rustfmt setting imports_granularity = Module.

Basically, there should not be any ::’s inside the {..} block of a use statement. For example:

use foo::Foo;
use foo::bar::Bar;
use foo::bar::Baz;
use foo::Foo;
use foo::bar::{Bar, Baz};
use foo::{Foo, bar::Bar};

Prefer locally-scoped trait imports

Import traits within function bodies, unless the trait’s methods are used extensively throughout the file.

fn foo(x: u8) {
    // Import the trait within a function, not at the module scope.
    use std::str::FromStr;

    let string: String = x.parse();
}

Qualify calls to top-level functions with a module name

To use top-level functions defined in other modules or crates, import their module and call them with modulename::functionname(). For example:

use std::mem;

let x = mem::transmute(y);
use std::mem::transmute;

let x = transmute(y);

Exceptions:

  • When importing functions with sufficiently disambiguated names, e.g., functions from C++ (where it is typical to call top-level (free) functions directly).
  • In Chromium, when importing functions from small, single-purpose first-party crates.

  • When the module name itself is prone to collisions. For example, when the module name is a C++ namespace name, and multiple crates define the same submodules.

Rationale

Many Rust codebases consistently keep a style such that unqualified lowercase names are assumed local, so foo() can be assumed to be from the same module. This rule largely exists to avoid reviewer disagreements, the benefit is mostly consistency and a minor edge on reading code without xrefs.

Type names should be imported and used unqualified

By default, types and traits should be referenced in code without using their module path, e.g., code should use Regex, not regex::Regex, with a use regex::Regex at the top of the file or potentially in the relevant function.

Exceptions:

  • In cases where there are ambiguities, it is acceptable to directly reference types with common names like io::Error. It is also acceptable to rename (use std::io::Error as IoError).
  • In cases where you have a large “group” of types, e.g. the hir module in rustc which contains a large number of HIR types.
  • In macros and generated code; where requiring a use is counterproductive.
  • In code with lots of dependencies, where importing everything would lead to a very large import block with little benefit.

Rationale

This rule largely exists to avoid reviewer disagreements.

Use the appropriate import mechanism [chrome]

  • Chrome Use chromium::import!().

Code formatting and aesthetics

Use the appropriate formatting tool

Use the appropriate formatting tool (e.g., git cl format in Chromium).

Configuration (for style guide authors)

A codebase may choose to not apply the following rustfmt options if they care about building on stable and the option is unavailable on their stable Rust:

  • Maximum of 100 columns per line: max_width = 100

  • Usage of lowercase alphabets in hex literals (0xab123cdef as opposed to 0xAB123CDEF): hex_literal_case = Lower

  • Macro matcher formatting: format_macro_matchers = true

  • Usage of escapes for invisible characters in literals. E.g., if you wish to have text containing a Zero-Width joiner, use \u{200D}.

Opting out of automatic formatting

You may opt out of automatic formatting using #[rustfmt::skip], but only in the following situations:

  • Manual alignment – If you have a situation-specific reason to have a snippet of code line up in a particular way (e.g., multiple similar lines of code that you want to remain aligned for readability, or a matrix-constructing function).
  • Generated code (ideally, use a genrule instead).

Important: Formatting should not be skipped for personal style choices.

Ownership and borrowing

Avoid Rc and Arc

Prefer unique ownership and borrowing over Rc and Arc for managing memory.

Exception:

  • Use Rc and Arc when the alternative ownership and borrowing structure would be unreasonably complex to write, understand, or maintain. This is a judgment call to be discussed during code review.

Rationale

Generally, Rust allows representing object graphs without using shared ownership smart pointers. These pointers add runtime overhead and their own unique complexity. Engineers coming from other languages might tend to use Rc and Arc too quickly.

Prefer borrows over deep copies

Use .clone() when you genuinely need a deep copy of an object.

Avoid using .clone() as a substitute for borrowing. Design code that works naturally with Rust’s ownership and borrowing rules. However, if the borrowing relationships become excessively complex, .clone() can be a pragmatic choice.

Consider the performance needs of the code while choosing between cloning and borrowing.

Note: This advice does not apply to cloning Rc or Arc pointers. Cloning reference-counted pointers does not create a deep copy, and is an expected way to use them.

Rationale

Deep copies can be expensive to make due to memory allocation and data copying.

Function inputs and outputs

Accept flexible references for immutable data

When functions accept borrowed, immutable data, use a reference type that affords the most flexibility. Pass references to the data itself instead of references to smart pointers, containers, or wrappers.

Use Instead of Why?
&T &Box<T> Borrow the data inside the box
&[T] &Vec<T> Accept any slice instead of specific containers
&str &String Accept a string slice instead of references to owned strings
Option<&T> &Option<T> Borrow the data inside the Option

Exceptions:

  • When expecting non-linear character data such as Rope/Cord.

For context, see Clippy lints borrowed_box and ptr_arg.

Rationale

Accepting reference types that don’t mention specific smart pointers, containers, or wrappers makes functions more versatile and performant: callers can pass data held in a variety of ways without copying it first.

Iterators vs. owned collections in return types

Prefer returning iterators (either concrete types or impl Iterator) where possible over returning Vec<T>.

Note: In application code, returning a Vec<T> is fine if it simplifies the code and if flexibility and performance advantages of iterators are not clearly needed.

Exceptions:

  • Most callers will need to construct an owned vector.
  • The function’s implementation naturally constructs a vector.
  • The returned value will likely need to be randomly indexed.
  • The algorithm works better as a for loop and the forced allocation doesn’t matter too much.
  • The function’s intended purpose is creating a new owned collection.
  • The function is returning a pre-existing vector.

Rationale

Highly reusable library code generally should strive to provide more flexible and performant APIs. Returning an iterator allows callers to choose how to consume the data: collect it into a vector, process it in a loop, or chain further iterator adapters. Some of these options allow users to avoid unnecessary allocations.

In application code, the usage patterns for a function are often easier to predict. If it is clear that performance is not a concern, or that data would be collected into a vector anyway, returning a vector makes for a simpler API.

Iterators vs. slices in function parameters

Highly reusable library code should accept iterators or impl IntoIterator if it only needs to iterate over the input.

Note: In application code, accepting a slice &[T] is fine if it simplifies the code and if flexibility and performance advantages of iterators are not clearly needed.

Rationale

Highly reusable library code generally should strive to provide more flexible and performant APIs. Functions that accept iterators offer callers:

  • More flexibility (data can be supplied from any collection)
  • Efficiency (data does not need to be copied into a contiguous piece of memory)
  • Composability (the function works better with iterator chains)

In application code, the usage patterns for a function are often easier to predict. If it is clear that performance is not a concern, or that callers would always have a vector anyway, accepting a slice makes for a simpler API.

Accepting Into

When designing library APIs that accept a T but users are likely to have various types, consider accepting impl Into<T> instead of T if call-site ergonomics are paramount.

Tip: This should be balanced against code bloat concerns; it is not a requirement to use lots of Into<T> in your code.

Note: Application code should almost never use impl Into<T> parameters.

Rationale

Using impl Into<T> allows a function to directly accept various arguments of various compatible types. For example, a function accepting impl Into<String> could be called with a String or a &str literal directly.

The tradeoff here is balancing the convenience of avoiding a .into() at the callsite against the cost of generics (more complex API signature, increased code size, sometimes even type inference difficulties).

Prefer enum over bool in APIs

Use enum function parameters instead of bool in cases where the purpose of the parameter is not obvious.

If you do use bool, consider using an inline comment to document the purpose of the bool in an argument list.

Reporting errors

Choose the error reporting mechanism that fits your API contract: Option, Result, or panic!.

Use Option for simple failures where the cause is obvious and the caller would not benefit from further details. This is best for standard, anticipated outcomes like a missing value or a parsing mismatch.

// Good: The cause of the failure is obvious (integer overflow). No further
// details could be reasonably provided.
fn checked_add(a: u32, b: u32) -> Option<u32> { ... }

Use Result for recoverable failures where the caller needs context to handle the error. These are often “unavoidable” errors caused by external factors (like I/O or network) that even correctly implemented code cannot prevent with checks ahead of time. Nevertheless, the caller can reasonably anticipate the kinds of errors (a file-related function can fail with an I/O error). The error details allow the caller to distinguish failure modes and recover appropriately.

// Good: The caller needs to know *why* it failed (NotFound vs PermissionDenied).
fn open_file(path: &Path) -> Result<File, io::Error> { ... }

Use panic! to report bugs and violated invariants. These errors are avoidable: the program is missing a check, and is calling an operation without fulfilling a precondition. These defects must be fixed in the code, not handled at runtime, although coarse-grained recovery might be possible with catch_unwind.

// Good: Panics if the invariant (index must be in bounds) is violated.
// This indicates a bug in the caller's logic.
impl<T> MyVec<T> {
    pub fn remove(&mut self, index: usize) -> T {
        if index >= self.len() {
            panic!("index out of bounds");
        }
        // ...
    }
}

Note: Some codebases may choose to avoid panics entirely, and require even broken invariants to be reported via Result.

Exceptions:

  • Developer-facing binaries may use panics for non-fatal errors.
  • Tests may use panics for assertions.

Types

Newtypes versus type aliases

Use a newtype instead of a type alias in the following cases:

  • If the type you are defining has notable semantic differences.
  • For types that warrant an API of their own.
  • If you want the flexibility to change the underlying type later, especially if you don’t control all usages of your type.
  • For type safety, if it’s easy for the types to get mixed up. For example, a Date type may use newtypes to distinguish Day, Month, and Year numbers in its internal calculations.

Prefer named fields over tuples and non-newtype tuple structs

Prefer types with named fields over tuples, as tuple indexes (.0, .1, …) quickly become confusing.

Exceptions:

  • Newtype tuple structs (tuple structs with only one field).
  • In very local contexts where there is unlikely to be any confusion.

Traits

Add trait implementations when they become necessary

As a general rule, low-level library code should anticipate users’ needs proactively, while application code should add trait impls as needed.

Implement known standard library traits instead of custom methods

If a method matches the purpose and behavior of a standard library trait, implement that trait. For example, instead of writing fn from_str(&str) -> Self on Foo, you should impl FromStr for Foo.

It is acceptable to have both the method and the trait impl for discoverability reasons.

For context, see Clippy lint should_implement_trait (list of traits).

Use #[derive()] if possible

Manual implementations should be avoided if they can be expressed as a #[derive()].

Generics versus dyn Trait

In libraries, prefer accepting generics (T: Trait or impl Trait) in function parameters over dyn Trait.

Exceptions:

  • Cases where code size matters.

Rationale

  • Efficiency

    • Additional indirection: Creating a dyn Trait requires setting up a fat pointer, and a vtable lookup is required for every call of a function on the trait.
    • If a function takes a dyn Trait, the compiler can’t perform certain optimizations that it could if the function was generic, such as inlining calls to functions of the trait.
  • Some traits are not dyn compatible. For example, a trait is not dyn compatible if it has generic functions, associated functions without a self parameter, or functions that return Self.

When to define a trait versus an inherent implementation

Define a trait only for the following purposes:

  • To enable generic programming (to use T: Trait in constraints of generic functions and types).

  • To accept or return an impl Trait or dyn Trait.

  • To define an extension trait (but see also the caveats in the section on extension traits).

In all other cases where you want to implement functionality for a type, use an inherent implementation.

Don’t use traits merely to organize code.

Extension traits

An extension trait is a trait that adds methods to a type from a different crate.

What differentiates an extension trait from other traits is that its sole purpose is to add methods to the type; it is not intended for use in generics (T: Trait, impl Trait) or dynamic dispatch (dyn Trait). To indicate this, the name of an extension trait should carry the suffix ~Ext (for example, ForeignTypeExt).

In general, the team that owns the original type should also own the extension trait.

Exceptions:

  • Extension methods that are expected to become well known and provide significant value, for example, in foundational libraries / frameworks.

Note: This rule is specifically about extension traits. It is relatively common to define a trait as an input trait (e.g., DateLike) and implement it on a bunch of foreign types.

Examples

extern crate foreign_crate;
use foreign_crate::ForeignType;

trait ForeignTypeExt {
  fn additional_method(&self);
}

impl ForeignTypeExt for ForeignType {
  fn additional_method(&self) { ... }
}

It is also possible to define an extension trait for every type that implements some foreign trait. Effectively, this is equivalent to adding default methods to the foreign trait:

extern crate foreign_crate;
use foreign_crate::ForeignTrait;

trait ForeignTraitExt {
  fn additional_method(&self);
}

impl<T: ForeignTrait> ForeignTraitExt for T {
  fn additional_method(&self) { ... }
}

Rationale

When the same team owns the extension trait and the type being extended, this helps avoid method name conflicts between the base type and the extension trait or between two extension traits.

Specific guidance for important traits

  • Types with public fields: Consider implementing traits implemented by all subfields.

  • Error types: Implement the Copy trait whenever possible. Copy types are simpler to manage and propagate because they can be passed by value without moving ownership. An example of a case where this is not possible is if the error must contain non-Copy fields like a String.

  • Copy: Implement whenever it is reasonably clear that the type will continue to be Copy in the future.

  • Clone: Most types should implement Clone unless they represent some external resource.

  • Default: Implement if the type implements new() or otherwise has a default state. For context, see Clippy lint new_without_default.

  • PartialEq / Eq: If you want users to use the == operator, prefer implementing PartialEq over Eq. When you do implement Eq, be aware of the additionally required reflexivity (details: <a href="https://doc.rust-lang.org/std/cmp/trait.Eq.html" rel="nofollow">https://doc.rust-lang.org/std/cmp/trait.Eq.html</a>).

  • PartialOrd / Ord: If you want users to use the < or > operators, prefer implementing PartialOrd over Ord. When you do implement Ord, be aware that it requires total ordering (details: <a href="https://doc.rust-lang.org/std/cmp/trait.Ord.html" rel="nofollow">https://doc.rust-lang.org/std/cmp/trait.Ord.html</a>).

  • Map keys

    • If you think it is likely that a type will be used as a map key or otherwise cached, implement Hash and Eq.
    • If, in addition, the type has a clear total ordering, implement Ord (and any traits it needs).

Pair important From impls with explicit constructor methods

If a type has an important From impl that is expected to be one of the primary ways of constructing it, add an inherent ::new(), ::new_from_foo()¸ or ::from_foo() method.

Options structs should implement Default

Structs with all-public fields that are primarily used as a “bag of options” for a function should implement Default.

Rationale

This enables convenient use of struct update syntax (Struct { a: 0, ..Default::default()} ).

Concurrency

Handling poison

Never reach inside a poisoned mutex to get at its insides, no matter how tempting the use case may seem.

Don’t use async / .await [chrome]

Don’t use async / .await in Chromium.

Don’t start new threads through the Rust standard library [chrome]

In Chromium, prefer to use sequences instead of threads.

Use fuchsia_sync::Mutex [Fuchsia]

Use Fuchsia’s mutex type, rather than one from std::sync or parking_lot.

Macros

When considering how to express an API, prefer in order:

  • Regular Rust functions, types, traits, objects, and modules. (Avoid macros, code generation, and metaprogramming where possible).

  • macro_rules!-style macros.

  • Other code generation strategies (e.g. genrule)

  • proc_macro macros.

When you decide to use a macro or code generation, please weigh the benefits against the costs:

  • IDEs, Code Search and other developer tools will work less well on code that uses macros.
  • Macros can create dialects of the language that make it harder for engineers to work across the codebase.
  • Large scale changes are more difficult and expensive to apply.

Rationale

Prefer non-macro facilities: Macros can create dialects of the language that make it harder for engineers to work across the codebase. When a reader sees a usage of a macro, they can make fewer assumptions about that code than if it didn’t use a macro. Macros are harder to write correctly than regular Rust code.

Use macro_rules! for simple code generation: Since Rust macros operate on the AST, they can be less fragile than textual code generation.

Naming

Naming methods

Follow the Rust standard library style for naming methods. See also the guidance on naming conventions in the Comprehensive Rust course.

Specific considerations:

  • Use prefixes such as as_...(), to_...(), or into_...() with the appropriate self convention. See also this section of the Rust API Guidelines and the Clippy lint wrong_self_convention.

Naming lifetimes

Lifetimes should have descriptive names, especially type lifetimes.

Exceptions:

  • Type lifetimes that are extremely generic abstractions like Cow.
  • It is fine to use anonymous lifetimes ('_) where they work.
  • Single-character lifetimes are okay when they are simply expressing the relationship between a method’s input and output, or needed for a local higher-ranked lifetime bounds (for<'a>). Up to two such local lifetimes are fine, if more are needed, we strongly advise you to use descriptive names.

Naming generic parameters

Use a short name for your generic parameters (either single or multi character), related to the kind of thing they represent. Single-letter generics are okay, but the letter chosen should be somewhat indicative of the object type, ideally taken from the most important trait bound.

T or U are fine when the generics are in a relatively unbound context; i.e. it is an API intended for use with most types, like Vec<T>.

Naming extension traits

When naming an extension trait, give the name an ~Ext suffix (for example, ForeignTypeExt).

Naming initialization functions

Follow these guidelines when naming initialization functions:

  • Constructors: Use new(), or new_with_foo() if there is more than one. If the constructor returns a Result, use try_new().

  • Value-to-value conversions: Use from_foo() with an associated From impl. See here for details on when an operation can be considered to be a value-to-value conversion.

Avoid abbreviations

Don’t use abbreviations (especially acronyms and initialisms) that someone outside your project is unlikely to know. Don’t abbreviate by deleting letters within a word.

See also the Google C++ Style Guide.

Exceptions:

  • Common abbreviations found in Wikipedia.
  • Abbreviations that are unlikely to be confusing to people outside of your project, for example IpAddr. You and your reviewer should make this determination. In general, the preference should still be not to abbreviate.

When and how to write documentation

Add documentation comments to public APIs, potentially using the missing_doc lint to help with this.

Best practices for documentation comments:

  • Consider including examples: It is not necessary for every function to have examples, but enough functionality should be covered in examples somewhere to give an idea of how the crate works.

  • Always document panic cases for functions which may panic due to malformed input.

  • Clearly document invariants for unsafe functions and traits.

  • Use capitalization and punctuation.

  • Don’t explain the basics of the programming language. For example, don’t do the following:

    struct MockFileSystem { ... }
    
    /// A mock implementation of the FileSystem trait, intended to be used in unit tests.
    impl FileSystem for MockFileSystem
    

Exceptions:

  • Comments should say something that the code does not already say. If there’s nothing insightful to say, it is better to omit the comment than to repeat the function name in a trivial way. For example, don’t do the following (it’s fine to allow the lint in that case):

    /// Removes an element.
    pub fn remove_element() -> Element {}
    
  • It is generally okay to omit documentation on individual enum variants, especially if their name is self-explanatory.

How to format documentation comments

Follow these guidelines when writing documentation comments:

  • Use /// comments to document items following the comment, such as functions, structs, enums, traits, and type aliases.
  • Use //! comments to document the module or the library itself. These comments should be placed at the top of the file.
  • Follow the rules laid out in the official Rust style guide on doc comments.
  • Use Markdown syntax.
  • Use ordinary comment syntax (//) for non-documentation comments, such as implementation comments, TODOs that are not interesting for users, commented out source code, etc.
  • Prefer to document things in a straightforward manner, using /// and //! comments, rather than generating documentation from macros. This makes the documentation directly readable when viewing or modifying the source code.

It is acceptable, and even encouraged, to mix doc comments (/// and //!) and ordinary comments (//). If you have one thing you wish to document publicly, and another thing you want to note as an implementation comment, then use both! For example:

/// Finds the first `FooItem` with the given `name`, or `None` if not present.
// TODO: crbug.com/12345 - `name` should be &str, not String.
pub fn get_named(items: &[FooItem], name: String) -> Option<&FooItem> {...}

Rationale

Many, though not all, languages have a different commenting convention (such as extra * characters) for doc comments as opposed to implementation comments. Rust and Python are notable for not just having different commenting conventions, but an actual different syntax entirely. This has concrete benefits to the Rust ecosystem:

  • Doc comments can be consumed by tooling, including documentation generators like rustdoc, IDE features, and Kythe.
  • Doc comments have a standard structure and formatting rules (Markdown), allowing for rich text in those contexts.
  • Errors in applying doc comment syntax (e.g., applying it to an expression instead of a documentable item) are linted against, catching documentation errors early. If there are no lint errors, the doc comments are guaranteed to be visible to documentation tooling, removing a degree of flakiness and unpredictability from the documentation process.
  • Doc comments are a part of the AST, and so can be produced and consumed by macros, including both macro_rules! macros and procedural macros.

Other Rust features

Unsafe code

Don’t write unsafe code unless there is no way to implement the same functionality with safe code.

If you do have to write unsafe code:

  • Encapsulate unsafe code in a safe abstraction and isolate it from business logic as much as possible. Ideally, unsafe code should be in a separate crate with few dependencies beyond the standard library.
  • Add a safety comment starting with SAFETY: that explains why the code is correct.
  • Add a technical justification for unsafe code. Depending on context, this can be a justification for an individual unsafe block or a more general justification that covers an entire file or build target. The justification should answer these questions:
    • Why is this code not written in an equivalent safe way?
    • If the reason is performance, are there benchmarks?

No unstable features

Don’t use unstable features.

Exceptions:

  • Android Embedded programming may require the use of certain unstable features. While review will be required to enable them (to help search for alternatives), features absolutely required to build an artifact correctly will generally be approved.
  • Chrome Style arbiters may blanket-approve a small list of features for the entire codebase. Typically, only features that have a clear path to stabilization will be approved.
  • Fuchsia See guidance here.

Getters and setters

Writing and naming getters and setters should follow these guidelines:

  • foo() for a getter that returns a T by value.
  • foo_mut() for a getter that returns a &mut T.
  • take_foo() for a getter that “clears” the value (by setting it to None or some other default state) and returns the old value.
  • set_foo() for a setter that takes a T by value.
  • with_foo() / with_mut_foo() for an accessor that operates via a closure (see details below). Such an accessor is invoked as object.with_foo(|foo| frob(foo)).
  • replace_foo() for a setter that returns the old value.

Examples

impl Stuff {
   fn foo(&self) -> &Foo { ...  }
   fn foo_mut(&mut self) -> &mut Foo { ... }

   /// Sets "baz" property to `None` and returns the old value.
   fn take_baz(&mut self) -> Option<Baz> { ... }

   fn set_foo(&mut self, foo: Foo) { ... }

   /// Calls `f` with a borrow of the "bar" property.
   /// Returns the return value of `f`.
   fn with_bar<R>(&self, f: impl FnOnce(&Bar) -> R) -> R { ... }

   /// Calls `f` with a mutable borrow of the "bar" property.
   /// Returns the return value of `f`.
   fn with_mut_bar<R>(&self, f: impl FnOnce(&mut i32) -> R) -> R { ... }

   /// Sets "foo" property to `foo` and returns the old value.
   fn replace_foo(&mut self, foo: Foo) -> Foo  { ... }
}

More on accessors that accept closures

Accessors that accept a closure (.with_foo()) are typically invoked as object.with_foo(|foo| do_something_with(foo)). They can be a good choice in the following situations:

  • When the property is contained in an interior mutability wrapper (e.g. Mutex or RefCell). These mechanisms need to constrain access to the property to a scope, and the closure provides this scope. This is usually preferable to directly returning the RAII object that implements the scope (e.g. MutexGuard<T>) because the latter would leak an implementation detail.

  • When follow-up work or validation is needed after a property is changed and a set_foo() accessor cannot be used (for example because the property’s type is expensive to clone or cannot be cloned at all).

A with_foo() accessor will often allow the closure to return a value and forward it by returning it from with_foo().

If you need to provide two closure-accepting accessors, one for read-only access and one for mutable access, it is common to name them with_foo() and with_mut_foo(). Otherwise, with_foo() is often also used for mutating accessors.

Implement expected API sets together

If you implement len(), also implement is_empty(). For context, see Clippy lint len_without_is_empty.

Do not access items marked #[doc(hidden)] that you don’t own

Treat items marked #[doc(hidden)] as private. Don’t access them unless you own the code.

Use const functions responsibly

Mark a public function as const only if you expect it to remain const in the future (or can commit to helping clients migrate when it changes).

Rationale

It’s a breaking change to turn a const function into a regular function.

Avoid overloading for non-mathematical purposes

Use operator overloads for their original mathematical/logical purpose, not just to make the code more concise (as “syntax sugar”).

Apply #[non_exhaustive] to publicly-exposed enums

If an enum is part of a public API used by other teams, mark it #[non_exhaustive].

Exceptions:

  • The enum will never change. For example:
    • Enums describing an existing ontology (e.g., enum Color { Red, Green, Blue }).
    • Enums with two states that could be Booleans (e.g., enum Visibility { Visible, Hidden })
  • The enum may change in the future, but you are committed to fixing resulting breakages via an LSC.

For context, see Clippy lint exhaustive_enums.

Rationale

Marking an enum #[non_exhaustive] forces users to include a wildcard arm (_ => ...) in match statements. This ensures code continues to compile when new variants are added to the enum later.

This rule is primarily important for enums used across team boundaries, because the scope of the breakage could be significant. Enums used only a small number of times can’t create a large breakage and thus may ignore this rule.

Apply #[non_exhaustive] to public structs with public fields

Structs with public fields that are part of a public API used by other teams should be marked #[non_exhaustive].

Exceptions:

  • The struct fields are unlikely to change, e.g., because they describe an existing ontology (e.g., a Rectangle type) or the reviewer can otherwise be confident that they will not change.
  • The struct fields may change in the future, but you are committed to fixing resulting breakages via an LSC.
  • You want to enable your users to use functional record update (FRU) struct initialization syntax (Person { name: "foo", ..Default::default() }).

Rationale

Adding a new field (even a private one) to a struct with all public fields is a breaking change, because callers can no longer name all fields in a struct literal.

A #[non_exhaustive] struct can’t be constructed with the struct literal syntax, except for the code in the same crate. This allows new private fields to be added without breaking users.

This rule is primarily important for structs used across team boundaries, because the scope of the breakage could be significant. Structs used only a small number of times can’t create a large breakage and thus may ignore this rule.

#[must_use] on functions

Use #[must_use] on functions in these cases:

  • The caller cannot use the function correctly if they ignore the return value. This includes pure functions (functions without side effects), such as HashMap::get().

  • Any function whose return value represents an error or otherwise critical information about the function’s behavior. If the return type is already annotated #[must_use] (which is true of Result, for example), the function does not need to be annotated #[must_use].

#[must_use] on types

Use #[must_use] on types intended to be returned by functions that would otherwise be annotated #[must_use] according to the rules above.

Examples:

  • The type is an error type (similar to Result or StatusOr).

  • The type represents a lazily evaluated computation. In other words, the function call doesn’t do anything by itself, and the return value needs to be accessed to cause some desired side effect.

Do not run code before main()

Do not cause code to run before main(), for example by using techniques such as the ctor crate.

Implementation concerns

Don’t overuse type inference

Add enough explicit types so that someone familiar with the codebase can figure out what the inferred types are.

Prefer to add the types to variable declarations, rather than as turbofishes. If the compiler forces you to add a turbofish, it is generally fine to add a _ inference marker.

Specific situations in which you should probably add types:

  • When calling methods with a generic return type, such as .into() or .parse().
  • For the result of complex iterator chains.

Prefer pattern matching over check-then-unwrap

Avoid potentially panicking operations when the input data guarantees they cannot panic. For example, instead of using Option::unwrap on a value known to be valid, use pattern matching to prove correctness to the compiler.

// Good: The match arm proves to the compiler that `x` is `Some`.
// A panic is obviously not possible.
if let Some(value) = option {
    use_value(value);
}
// Bad: Correctness of `unwrap` relies on a runtime check that `option` is `Some`.
if option.is_some() {
    use_value(option.unwrap());
}

Prefer expect over unwrap

If a panic is necessary because an invariant cannot be expressed in the type system, prefer expect over unwrap to document the reason.

Don’t omit patterns on non_exhaustive enums when matching

Prefer to list all enum variants in match statements over non_exhaustive enums.

If you don’t care about specific variants, place them in the wildcard arm. If you only care about specific variants, use an if let or a let … else {}.

Examples:

#[non_exhaustive]
pub enum Shape {
    Circle,
    Square,
    Triangle,
}
fn process_shape(shape: Shape) {
  match shape {
    Shape::Circle => println!("It's a circle"),
    Shape::Square => println!("It's a square"),
    // Triangle and any future variants don't require any actions.
    Shape::Triangle | _ => {}
  }
}
fn process_shape(shape: Shape) {
  if let Shape::Circle = shape {
    println!("It's a circle"),
  }
}
fn process_shape(shape: Shape) {
  match shape {
    Shape::Circle => println!("It's a circle"),
    _ => {}
  }
}

Exceptions:

  • You only care about specific variants and match is cleaner than if let.

This is covered by the built-in lint non_exhaustive_omitted_patterns which you can enable on your code.

Use context strings to clarify assertions

While some assertions are self-explanatory, many benefit from additional context that clarifies why an assertion is expected to hold, or what caused it to be violated. Use the optional format string argument for assert! to convey this context.

You should always consider adding context, but particularly in the following situations:

  • If you’re thinking of adding a comment to the assertion. Use a format string instead: It has the benefit that it will appear in logs.
  • If you expect failing assertions to be seen by users. In this case, a human-readable explanation is more helpful than a Rust expression.
  • If logging a value helps in understanding why the assertion failed.
  • To clarify whether a failing assertion means that
    • The client misused an API (i.e., they should change their use of the API), or
    • The implementation contains a bug (i.e., the client should file a bug report).

Examples

Explaining why an assertion should hold

    assert!(
        values.is_sorted(),
        "`values` is populated from a `PriorityQueue`, so we expect it to be sorted"
    );

Logging a value to clarify why the assertion failed

assert!(refund > 0, "Refund must be positive, got {refund:?});

Flagging misuse of an API

impl Database {
    ...

    fn execute_query(&self, query: &str) {
        assert!(self.stream.is_some(), "Call connect() before executing queries");
    }

    ...
}

Don’t use the context string merely to repeat the condition

assert(!values.is_sorted(), "values should always be sorted");

Don’t use iterator combinator chains when direct iteration is clearer

Avoid using combinator chains in ways that hurt readability. Specifically:

  • Avoid long and complex combinator chains.
  • Don’t construct combinator chains in the header of a for loop. Instead, assign the combinator chain to a variable, then consume that variable in the for loop.
  • Avoid Iterator::for_each unless you’re convinced you have a good reason. A for loop is usually easier to read.

Combinators are preferable for shorter chains and for cases where the combinator itself provides a lot of logic that would otherwise have to be written out in the corresponding for loop.

Examples

Complex combinator chains

Here’s an example of a combinator chain that is hard to read because it involves conditionals and mutates external state:

let mut error_count = 0;
let _processed_items: Vec<_> = items
    .into_iter()
    .map(transformation_that_may_fail)
    .map(|result| {
        if result.is_err() {
            error_count += 1;
        }
        result
    })
    .filter_map(Result::ok)
    .collect();

The corresponding for loop is easier to read:

let mut error_count = 0;
let mut processed_items = Vec::with_capacity(items.len());

for item in items {
    match transformation_that_may_fail(item) {
        Ok(val) => processed_items.push(val),
        Err(_) => error_count += 1,
    }
}

Combinator chains in for loops

Avoid constructing a combinator chain in the header of a for loop:

for bug in bug_descriptions
    .iter()
    .filter_map(|description| parse_bug_description(description).ok())
    .filter(|bug| bug.priority <= Priority::P1)
{
    assign_bug(bug);
}

Instead, assign the combinator chain to a variable. This reduces clutter, and the variable name documents what the combinator chain does:

let urgent_bugs = bug_descriptions
    .iter()
    .filter_map(|description| parse_bug_description(description).ok())
    .filter(|bug| bug.priority <= Priority::P1);

for bug in urgent_bugs {
    assign_bug(bug);
}

When to prefer a combinator chain

Here’s an example where a combinator chain is preferable:

let threshold_exceeded = values.iter().any(|&v| v > threshold);

The corresponding for loop is harder to read and reinvents the logic contained in any():

let mut threshold_exceeded = false;
for v in values {
  if v > threshold {
    threshold_exceeded = true;
    break;
  }
}

Don’t call panic::catch_unwind

Use of panic::catch_unwind is prohibited in first-party Rust code.

Exceptions:

  • It is acceptable as a defense in depth mechanism as long as it is never relied upon for any properties. A codebase that is already attempting to avoid panics may choose to wrap critical entry points with a catch_unwind recovery mechanism just in case a panic slips through. Code authors must convince the reviewer that the codebase is doing most of what it can to avoid panics. This exception only applies to code that has the possibility of being run with panic=unwind (so this does not apply to Chrome, Android, Fuchsia).

Rationale

Rust code may be used in many environments, and panic::catch_unwind will only function correctly in environments with panic=unwind. While the final panic disposition of prod binaries has not yet been determined, Chrome, Android, and Fuchsia all require panic=abort.

Prefer format! / write! for concatenation and substitution

Prefer format!() and write!() to other alternatives in the following situations:

  • To concatenate more than two strings. For example, use format!("{a}{b}{c}") instead of a + b + c.
  • To perform substitutions on a literal string (instead of using str::replace).

Rationale

  • format! / write! require fewer allocations than concatenating strings using +.
  • Prefer standard format strings to inventing your own templating syntax.

Use the right per-codebase test framework

Most codebases use their own specific test framework rather than #[test].

  • Chrome Use rust_gtest_interop. This makes it possible to integrate Rust tests into C++ test binaries.

  • Fuchsia Use #[fuchsia::test]. See here for more details.

Read the whole story
emrox
19 minutes ago
reply
Hamburg, Germany
Share this story
Delete

Joshua Idehen – Mum Does the Washing

1 Comment

The world according to your mum doing the washing

Capitalism
Your mum does the washing, you pay her a dollar
You get her to do your mate's washing, your mate pays you fifty dollars

Communism
Your mum does the washing, you do the washing
Every night, you salute a picture of your dad

Socialism
Your mum does the washing, you do the cooking
Everybody is happy, in theory

Feudalism
Your mum does the washing and pays you tax

Colonialism
You barge into your mum's room
Claim you discovered your mum's room
Dump all your dirty clothes on the floor

Fascism
Your mum does the washing
And you are obsessed with her nose

Liberalism
You watch your mum do the washing
And you feel really, really bad
"Something must be done" you say
Something may or may not get done

Libertarianism
Your mum does the washing
You believe you did the washing

Religion
Your mum does the washing
You thank god

Atheism
Your mum does the washing
You make a twelve-part YouTube video demanding peer-reviewed evidence
That she did, in fact, do the washing

Zionism
You shoved your mum into the washing machine
And the spinning made her dizzy
And that dizziness made her vomit
And you point to that vomit and call it antisemitism

Americanism
Your mum does the washing
It's in the constitution
End of discussion

Mansplaining
Your mum does the washing
You tell her how best to do the washing
You have never done the washing

Same-sex marriage
Your mums do the washing

Patriarchy
Your mum doesn't exist
The washing is mysteriously done

TERFism
Your mum doesn't want your other mum doing the washing
Because of like, genitals and stuff

Polling
There is a seventy percent chance
You will do the washing on Saturday night
Come Sunday morning
It was one-hundred percent done by your mum

Africanism
You go to Benny's house
Benny's mum does the washing
You ask your mum why she doesn't do the washing
You do not survive the night

Feminism
Your mum insists you grow up and do your own washing

White feminism
Your mum hired a woman of color to do the washing

Male feminism
The one time I did the washing
I told everyone I did the washing
I blogged about it
Bragged about it
Took a selfie
Insta story
Went on TV
Won an Oscar
Went on Fortnite
Did a dance
I am an ally
Hashtag metoo

Egalitarianism
The one time you did the washing
Is proof it's all equal and no one needs feminism anymore

Hip-hop
Everyday I'm hustling
Everyday I'm hustling
When I bring the basket
Mama puts the washing in

Narcissism
You look good in the clothes your mum washed

Surrealism
The washing does your mum

Read the whole story
emrox
1 hour ago
reply
I heard that one on a mixset yesterday. Really made me laugh (you can read it there, but listening might be more fun)
Hamburg, Germany
Share this story
Delete

Setting up SSH keys on macOS with Touch ID

1 Share

Blog post social card

In the first part of this post I’ll show a simple, secure way to create an SSH key on Mac (it’s 3 commands, including support for GitHub). In the second part I’ll dig deeper and explain what’s actually happening for the curious reader.

Macs have a built-in so-called Secure Enclave, where we can create “identities” that simply cannot be stolen (unless maybe you’ve made enemies with a nation state). Those identities are protected with Touch ID and can be used for SSH authentication. There are some great benefits about using this over traditional SSH key files:

  • The private material cannot be extracted, is never shown, cannot be stolen,
  • You get prompted whenever the identity is used,
  • No key files (like ~/.ssh/id_ed25519.pub and friends) littering the file system; and there’s no risk of mistakenly deleting them either.

This extra security will be in effect whenever you use SSH, including git-over-ssh like GitHub. It’s great that advanced security tech is available by default on some of the most widely distributed computers in the world! A few years ago stuff like this could only be found at CERN and/or in Dan Brown novels. Let’s see how it can be set up on your consumer-grade Mac.

It is essential that the human interface be designed for ease of use, so that users routinely and automatically apply the protection mechanisms correctly.

  • Saltzer and Schroeder, “The Protection of Information in Computer Systems”

Quick Setup

We’ll create a secure identity on the Secure Enclave, then tell SSH how to use it for authentication, and finally add the key to GitHub.

Step 1: create the identity that you’ll use on this Mac:

sc_auth create-ctk-identity -l ssh-key -k p-256-ne -t bio # run once to create identity

You’ll be prompted to touch “Touch ID”. Your identity is now created and saved securely (parameters are explained in the deep dive below).

Step 2: tell SSH to use the identity (when prompted for a PIN, just hit enter):

ssh-add -K -S /usr/lib/ssh-keychain.dylib # run after a reboot

From here on, SSH will know to use the identity from the Secure Enclave when connecting via SSH; it does not have a good memory so you’ll need to remind it after a reboot.

Step 3: Whenever you need to register your key to a new server (like GitHub), run this command:

ssh-add -L # run when you need the public key

If multiple identities are listed, use the line that starts with sk-ecdsa-sha2-nistp256.... It’s a shareable handle for the identity, and this is the SSH key that GitHub will ask about.

image

The output of ssh-add -L can be added as a GitHub SSH Key

That’s all you need for a super secure setup! The new identity can now be used with SSH, and you’ll be prompted for Touch ID during authentication:

git push # triggers Touch ID prompt

No one can steal your identity (well, you can’t even give it away willingly since it can’t be extracted from the Secure Enclave), but with great powers etc: if your Mac dies, so does the identity, and you’ll need to create a new one (and re-register it where necessary, like GitHub).

image

You’ll get a Touch ID prompt when git tries to authenticate with SSH

If you’re curious about how this works, read on!

Under the hood

We’ll now try to understand what happens under the hood. We’ll start with the actual identity creation in the Secure Enclave, then look at how SSH is allowed to interface with the Secure Enclave with ssh-keychain.dylib, and then try to see what happens in a complete SSH sign-in flow.

This is the result of reading posts (like this great investigation by arianvp), manual pages, and experimentation. A lot of this is undocumented, and this is the minimum information I needed to form a mental model of what’s happening and convince myself that it does, indeed, work.

Some of the command outputs below have been trimmed for legibility, I invite you to run them locally if you want the full experience!

Creating an identity on the Secure Enclave

Identities are managed with the sc_auth command (where sc_ stands for Smart Card). In Apple-speak, they are “CryptoTokenKit” identities, or CTK identities for short. Why not Smart Card Identities? What is a Smart Card? No idea! There’s a lot of weird Apple vocabulary, but in practice they’re just private keys stored on the Secure Enclave:

$ sc_auth list-ctk-identities
Key Type Public Key Hash                          Prot Label
p-256-ne 458F712F281B53C683F23FC02E69D44DAFDAA3DE bio  ssh-key

If you followed along in the Quick Setup, you should have exactly one identity here, that was created when you ran the sc_auth create-ctk-identity command. Let’s look at the create-ctk-identity command in more detail:

sc_auth create-ctk-identity -l <label> -k <key type> -t <protection>

When using the p-256-ne key type, this command creates the identity on the Secure Enclave — I’m guessing by first gathering some entropy, then generating some private key material and allocating whatever it needs for bookkeeping and ensuring it can only be accessed with your fingerprint.

The label is freeform and just a note attached to the key. The key type is a cryptographic setting (the elliptic curve used). At the time of writing, the following values are accepted: p-256|p-384|p-521|p-256-ne|p-384-ne. The -ne suffix means “Non-Extractable”, meaning the key is generated by the enclave itself and cannot be extracted from the Secure Enclave. For SSH we only need signing, which is done by (and on) the Secure Enclave itself, and at no point do we need (or want) to export the private key. The p-384 curve is not supported by (Open)SSH, so this narrows it down to p-256-ne.

Finally, protection can be either bio or none. When set to bio, using the key requires biometrics (like Touch ID). With none, the key still cannot be extracted (i.e. stolen) but programs would be able to silently use SSH; I recommend using bio, but YMMV; if you get annoyed by the Touch ID prompts whenever you git pull, you might want to use none here.

Now that we have some elusive identity on the Secure Enclave, let’s see how we can tell SSH to use it for authentication.

Translating between SSH and the Secure Enclave

There’s a bit of impedance mismatch, and ssh doesn’t know how to use the identity by default:

$ ssh -T <a href="mailto:git@github.com">git@github.com</a>
git@github.com: Permission denied (publickey).

The SSH agent cannot use the Secure Enclave identities as-is, but Apple has us covered with a small translation layer in the form of ssh-keychain.dylib, a dlopen-able library available natively on all recent macOS versions:

$ ssh-add -K -S /usr/lib/ssh-keychain.dylib
...
Resident identity added: ECDSA-SK SHA256:QZ+Cx6/AkS9sek9INtsXt6t7wuHOtA555CLJu8a8V7w

Here we’re relying on ssh-add’s built-in support for FIDO-style authenticator via the -K parameter. This lets the SSH agent know that it shouldn’t read a standard private key (for instance ~/.ssh/id_ed25519) but that it should expect the keys to be stored “externally” (for a lack of better words). With -K alone, ssh-add would look for public keys on USB devices (think YubiKey); though we can customize this behavior by registering a “security key provider” with -S <provider>. This provider should be a stateless dynamically-loadable library, which is exactly what /usr/lib/ssh-keychain.dylib is. The interface is not documented (at least not in a way I was able to meaningfully process), but we can have a look at what ssh-keychain.dylib exposes:

$ dyld_info -exports /usr/lib/ssh-keychain.dylib
/usr/lib/ssh-keychain.dylib:
    -exports:
        offset      symbol
        0x00001A2C  _C_GetFunctionList
        0x00000F30  _sk_api_version
        0x00000F38  _sk_enroll
        0x0000136C  _sk_load_resident_keys
        0x00000F40  _sk_sign

Most likely, _sk_load_resident_keys is meant to return a list of (public keys of) identities, and _sk_sign will sign a payload using a specified identity (here, resident means “residing on e.g. the Secure Enclave”).

We’ve seen how the identities can be created on the Secure Enclave, and how they can, in theory, be used by the SSH agent. Let’s put this to the test and see what happens when an SSH connection is created.

SSH Sign-in flow with the ssh-keychain provider

Let’s see what we need for authenticating to a remote SSH server. At all times, the SSH agent needs (1) a list of (public) keys that can be used to authenticate, and (2) a way to prove to a server that it has the corresponding private key for any of those keys.

When ssh-add is first called with -S <provider>, the _sk_load_resident_keys routine is called, which returns the public keys of all resident keys on the FIDO-style authenticator. Those are stored in the SSH agent (alongside their corresponding provider, namely ssh-keychain.dylib) and will later be offered as candidates to any remote SSH server we try connecting to. (Also now that the SSH agent knows about the Secure Enclave identity we created earlier, we can view its SSH-style public key with ssh-add -L.)

Let’s say an SSH connection is initiated: the remote accepts one of those public keys offered by the agent and “challenges” the client to prove its identity. The agent calls _sk_sign to create a signature with the private key (matching the accepted public key) and the challenge (roughly speaking). This will be forwarded to the Secure Enclave, prompt Touch ID, and if approved, will return a signature for the challenge. The signature is forwarded back to the server, which can confirm we — the client — have the corresponding private key.

Let’s see what this looks like in practice:

$ ssh -T <a href="mailto:git@github.com">git@github.com</a>
git@github.com: Permission denied (publickey).
$ ssh-add -K -S /usr/lib/ssh-keychain.dylib
Enter PIN for authenticator:
Resident identity added: ...
$ ssh -vvv -T <a href="mailto:git@github.com">git@github.com</a>
debug1: get_agent_identities: agent returned 1 keys
debug1: Offering public key: ssh: ECDSA-SK SHA256:QZ+Cx6/AkS9sek9INtsXt6t7wuHOtA555CLJu8a8V7w authenticator agent
debug1: Server accepts key: ssh: ECDSA-SK SHA256:QZ+Cx6/AkS9sek9INtsXt6t7wuHOtA555CLJu8a8V7w authenticator agent
debug3: sign_and_send_pubkey: signing using <a href="mailto:sk-ecdsa-sha2-nistp256@openssh.com">sk-ecdsa-sha2-nistp256@openssh.com</a> SHA256:QZ+Cx6/AkS9sek9INtsXt6t7wuHOtA555CLJu8a8V7w
<at this point, Touch ID prompt appears>
Authenticated to <a href="http://github.com" rel="nofollow">github.com</a> ([20.250.119.67]:22) using "publickey".
Hi nmattia! You've successfully authenticated, but GitHub does not provide shell access.

The ssh debug logs show that one key is offered by the agent, accepted, and then used to create a signature. After the server has verified the signature, authentication is granted (although GitHub does not provide shell access).

One note: whenever we register the identity with the SSH agent, ssh-add prompts us for a PIN. What gives? Why does entering an empty PIN work, isn’t this mighty unsafe‽

Well, no, it’s not unsafe! When we tell ssh-add to use a FIDO-style authenticator, I suspect its default behavior is to ask for a PIN as it assumes we’ll be using a USB, YubiKey-style authenticator; only we shortcut the whole thing with ssh-keychain.dylib (and the security comes not from a PIN, but from the soft skin under your fingertips; much cooler). Not great in terms of usability, but I guess it makes sense. (I have no idea what would happen if you have more than one FIDO authenticator; go support me on GitHub and I’ll buy a collection of YubiKeys to experiment).

Conclusion

Let’s recap: we can use a somewhat cryptic, poorly documented “Smart Card” tool to create a “CryptoTokenKit” identity on the “Secure Enclave”. Then, by using the poorly documented ssh-keychain.dylib, we can use said identity to generate signatures that Apple’s ssh can use. There’s no guarantee that all this missing documentation wouldn’t say “Do not use this; terribly unsafe in practice”, but hey: all this security talk by Apple, they wouldn’t lie to us, right? *spooky*

Joke aside, it’s pretty cool to have sci-fi style security modules baked into most modern devices; the more people start using them and pushing for openness, the better the support from the companies will be. We can have nice things!

Again, a lot of this was derived from this gist; highly recommend having a read and potentially asking questions in the discussion.

Let me knoworjoin the discussion if you enjoyed this article! You can also subscribe to receive updates.


Here's more on the topic of macOS:

Read the whole story
emrox
1 day ago
reply
Hamburg, Germany
Share this story
Delete

The Future of CSS ✨

1 Share

Every year, browser vendors collaborate via the Interop project to synchronize their efforts on new HTML/CSS/JS features. This year, there’s a ton of super exciting features being considered! In this blog post, we’ll explore the CSS of the future, to get a glimpse of what we’ll soon be able to do.



Read the whole story
emrox
1 day ago
reply
Hamburg, Germany
Share this story
Delete

The End of Rationing

1 Share

s who have vowed to build a better GitHub for agentic coding today can't seem to shake it off. Salesforce's lead/opportunity/pipeline object model from 1999 became the definition of what a CRM is. Every CRM company started since inherits it.

Read the whole story
emrox
2 days ago
reply
Hamburg, Germany
Share this story
Delete

Terminal Bench 4 Is Saturated | Hacker News

1 Share

You don't have to give it up entirely, you could grade on the interpretation they chose. Personally I'm a fan of models which seek clarification on critical ambiguities rather than barging ahead "autonomously" though.

But wow, they found a lot of issues with the benchmark.

Read the whole story
emrox
2 days ago
reply
Hamburg, Germany
Share this story
Delete
Next Page of Stories