It simplifies constructing your own "leaf" errors and streamlines the ability of collecting multiple types of errors while attaching more context to them (e.g. filenames, stack traces, user ids, etc.). It allows you to smoothly switch from "stringly-typed" errors to strongly-typed errors. You can create opaque errors to avoid leaking internal implementation details into your public API.
Applied to the code in the post:
use snafu::prelude::*;
use std::{fs::File, io::prelude::*};
#[derive(Debug, Snafu)]
enum Error {
#[snafu(display("Unable to open {filename}"))]
Opening {
source: std::io::Error,
filename: String,
},
#[snafu(display("Unable to read {filename}"))]
Reading {
source: std::io::Error,
filename: String,
},
#[snafu(display("Unable to parse {buffer} as a number"))]
Parsing {
source: std::num::ParseIntError,
buffer: String,
},
}
fn read_number_from_file(filename: &str) -> Result<u64, Error> {
let mut file = File::open(filename).context(OpeningSnafu { filename })?;
let mut buffer = String::new();
file.read_to_string(&mut buffer)
.context(ReadingSnafu { filename })?;
let buffer = buffer.trim();
let parsed: u64 = buffer.parse().context(ParsingSnafu { buffer })?;
Ok(parsed)
}
The key parts are the `derive(Snafu)` on the definition of the error enum and the usages of `.context` and `XxxSnafu` at the error sites.
Importantly, this example demonstrates a key feature of SNAFU, here shown as "not all `io::Error`s are the same". Opening the file and reading the file are two separate error conditions and should not be lumped together as one.
Obviously I realise which you'll say is best, but any comment on snafu vs thiserror/anyhow?
That's what I've used so far pretty much just because it seemed the popularly recommended way to solve the problem, but I wouldn't say it's been massively smooth.
Also, it seems unfortunate you won't get autocompletion (the first time anyway) for (the 'Snafu') part of XxxSnafu.
I'd like to provide a fair comparison [1] in the documentation, but I don't know thiserror / anyhow well enough to feel like I'd give them the credit they are due.
That said, to my knowledge, thiserror doesn't allow you to take an `io::Error` and sort it into two different enum variants (like the `Reading` and `Opening` variants in my grandparent example). To me, those are vastly different error states that just both happen to have the same error type. You can extend the metaphor with any larger error type from a crate (e.g. `reqwest::Error`).
Anyhow requires using a trait object (and potentially downcasting) and I prefer avoiding those when possible.
> seemed the popularly recommended way
Absolutely. The author of those crates is a giant in the Rust community [2] (they are also the author of serde, syn, and quote, for example!). If those crates suit your situations, then by all means — use them. I'd much rather the Rust community have better error types and messages by whatever means available. Even using `String` via `Box<dyn Error>` is better than nothing.
> you won't get autocompletion
You should, at least if you use rust-analyzer. I use it via emacs and have these settings enabled, but I think they were going to be the default:
> To my knowledge, thiserror doesn't allow you to take an `io::Error` and sort it into two different enum variants (like the `Reading` and `Opening` variants in my grandparent example)
Indeed that does look nice, I don't know for sure either, I'm only very basically using it. (Which is where it's been a bit of a pain at times - to be honest for where I've been using it so far I just wanted a dead simple 'I really don't care just return the basic error message in some way that compiles'.)
> The author of those crates is a giant in the Rust community [dtolnay]
For what it's worth, I haven't conducted any sort of objective metric-based comparison obviously, but I recognise both of your usernames equally as giants. ;)
> You should [get autocompletion on *Snafu], at least if you use rust-analyzer
Oh, clever. "[rust-analyzer] is a part of a larger rls-2.0 effort to create excellent IDE support for Rust." I'll have to check, but I'm pretty sure I'm using 'rls-1.0' (in vim).
With that example, since you use `Result::map_err` and specify the specific variant, you aren't really using anything from thiserror at the site of the `?`, correct?
Most usages I have seen, people use `#[from]`, which would end up having conflicting implementations. Is there a reason you didn't use `#[from]` for `ParseConfig`?
In short, I consider `From::from` implementations for errors to be an anti-pattern. It is super easy to become lax about adding context with these implementations in place. Especially as code is modified in the future.
I describe the approach that I use for errors in a detail in an article (https://kazlauskas.me/entries/errors.html) that has already been linked elsewhere in the thread. It is, as far as I can tell, pretty much equivalent to what `snafu` makes users to do.
Seems cool, but is a whole crate worth it for the `.context` function when you could just use e.g., `.map_err(|source| Error::Opening { source, filename })`? Seems like all `.context` provides is not not needing to name the originating error? (And obviously the `#[snafu(display(...))]` macros could just be moved into a `impl Debug for Error`.)
Yes. I'm not sure exactly what other response you'd expect from the author/maintainer of a library when they've already made a post encouraging other people to use it. ¯\_(ツ)_/¯
> when you could just use e.g., `.map_err(|source| Error::Opening { source, filename })`
That's not equivalent, as `filename` is a `&str` but becomes a `String` when stored in the error. SNAFU automatically calls `Into::into` for you, so the closest would be:
There's also the possibility of automatic construction of values (backtraces, location information, the current time, things captured from globals / thread locals, etc.)
Beyond `.context`, SNAFU also implements the `Error` trait (and associated methods like `Error::source`).
There's also convenience methods and macros to create leaf errors, those that originate in your code.
> obviously the `#[snafu(display(...))]` macros could just be moved into a `impl Debug for Error`
I'll assume you mean `Display`, not `Debug`. That also not quite true, as SNAFU offers a shorthand syntax that isn't yet in stable Rust:
"Unable to read {filename}"
would need to be one of
"Unable to read {}", filename
"Unable to read {filename}", filename = filename
> like all [...] provides [...] obviously [..] could just be moved
All code could have been written by your own hand or otherwise inlined. Your response feels (needlessly) highly dismissive of another person's work.
Thanks for your reply. I guess my question should really have been a statement: “based on this example, I don’t think it’s worth it”. But the info you provided does make it seem worthwhile. Cheers
As mentioned, these are attributes not comments. More specifically, the the `#[derive]` attribute on the enum lists which procedural macros to execute (Debug and Snafu here).
These procedural macros are handed the token stream for that enum, and can use that to generate new code. The Snafu macro here is also using attributes on each variant of the enum for information about how to implement certain things.
The documentation for Snafu has a page[1] describing what code is generated. There's nothing there that couldn't be done by hand, it's just tedious.
> It's still like putting comments above your functions.
If you're used to a certain language then sure that makes sense. Your comment seems like you are boxing yourself in, limiting yourself to just what makes sense in Javascript, very opposite of a programmer who looks to improve their craft.
Take a moment and think about that. Because it looks like Javascript comment...you only see it like that to a point of making this statement. There is a lot of ways different languages uses tokens/symbols to indicate something. Not everyone agrees what those symbols are used for. Some languages use it to define macros like C++ or preprocessor directives like C#. Some as comments JS/Python/. Some as like Java nothing.
Because the languages you use are use to it, and the language you use doesn't use something similar, it looks "wrong". That is in itself a narrow view.
I think you should question that kind of thinking, I believe it will be helpful.
It’s a little similar, except that these are attributes rather than comments. They’re part of the syntax of the language, and new macros can be written to add new attributes. Attributes are used for several different purposes within the language, so using them is familiar and common.
Rust also has two kinds of comments, one that shows up in the generated documentation and one that doesn’t.
The fact that failure conditions from both `File::open` and `read_to_string` become an `IoError` is a significant roadblock to making these errors useful. The mechanism as described in this blog post also fails to introduce contextual information about the reason a failure has occurred.
This means that errors, if implemented as described in this post, either formatted or handled don't give sufficient information to the caller/user on how to deal with the error.
EDIT:
I strongly recommend to use `op.map_err(|e| SomeError::Open(e, filename))?` and `op.map_err(SomeError::Read)?` as an alternative when propagating the errors. It is more typing at the location of propagation than just `?`, but the errors this approach produces are immediately actionable regardless of whether they are printed to the user or handled by a caller. Provided, of course, this pattern is applied consistently.
You can match on std::io::ErrorKind, which is available by calling kind() on your error. This effectively has the same result as error inheritance hierarchies in Java and C#.
I don't think you really would treat a failure in File::open and read_to_string() differently in most applications. If you have a function doing file IO you might retry reading a read_to_string() error, but you know that the file exists because you know which function you just called.
If you're at higher level of abstraction, where something is calling both open() and read_to_string() for you, are you really going to handle it differently that frequently whether it's a missing file or a file on an unavailable storage medium?
In my personal experience distinct handling of errors like these does indeed come up significantly more rarely. What ends up mattering to me much more often is how these errors are presented to the user, or what ends up in the logs in production. In those cases something like
error: failed to read contents of `foo`
caused by: a physical I/O error has occurred (...)
and
error: failed to open `foo`
caused by: permission denied (...)
are significantly more actionable than just the underlying I/O errors by themselves. To the point where a person reading these messages has a shot at addressing the problem without any prior knowledge of the code base.
In `anyhow` adding such context via its `context` family of methods seems fairly well accepted. I hope that my comment demonstrates there's no reason why `enum` based approach would need to be any worse than what `anyhow` can achieve.
> how these errors are presented to the user, or what ends up in the logs in production. In those cases something like [...] are significantly more actionable than just the underlying I/O errors by themselves. To the point **where a person reading these messages has a shot at addressing the problem** without any prior knowledge of the code base.
(emphasis mine)
This is a huge belief of mine as well. I call it a "semantic stacktrace".
I actually go as far as to say that _most of the time_ backtraces in errors are an antipattern, as it gives a false sense that an error is actionable ("oh, just look in the code!").
I even try to make it such that each leaf error is constructed in exactly one place so that grepping for the error identifies that one location.
For your logs you're going to print the error's default message anyway, so you should have the underlying message of the std::io::Error as long as it's listed in the source of your wrapped error.
Propagation without needing to use boxed trait objects can also be accomplished using `anyhow`[1]/`eyre`[2], which have nice downcasting API’s for recovering the original error type if you know what the possibilites could be. I only bring it up because they aren’t mentioned until the end of the article and only in passing but they offer really nice features for attaching context and downcasting that makes up for the pseudo-type-erasure.
> Propagation without needing to use boxed trait objects
You can also use enums, as shown in the post. You don't need trait objects.
EDIT 1
> without needing to use boxed trait objects [...] `anyhow`
It's my understanding that anyhow uses trait objects. The first sentence of its document says "This library provides anyhow::Error, a trait object based error type"
EDIT 1 END
> which have nice downcasting API
I'm not a fan of downcasting when not _absolutely_ necessary.
See my sibling comment about SNAFU for an alternate that allows keeping different errors separate while unifying them.
Sure but I’ve also looked at projects that have 15-20 enums (or more) for their error types and it makes things very cumbersome. `anyhow` makes it painless to have arbitrary errors. It’s trying to take the context and stuff it in to the type system. Which isn’t necessarily a bad idea but it can become unwieldy.
I’ve done both in production projects and they both have their merits but 9 out of 10 times I’ll start with `anyhow` if I’m writing library code and then refactor to enums later if I need it instead of the other way around.
EDIT: That said Snafu does look really cool for the cases where you do have a codified sum type of all possible errors.
DOUBLE EDIT: I already had Snafu starred on GH lol
> 15-20 enums (or more) for their error types and it makes things very cumbersome
Like everything, it can be a balancing act. I tend to be free about creating error types (one per module, usually, but it's not strange for me to create more). When you implement `std::error::Error` (to be able to make error trait objects) and have a tool like SNAFU (to make error enums), then having more error types isn't much of a hindrance in my experience.
> I’ll start with `anyhow` if I’m writing library code and then refactor to enums later
The newest (beta) version of SNAFU actually strives to make this case even smoother. There's a `whatever!` macro that you can use to create stringly-typed errors and then migrate case-by-case to an enum. Check out the docs for some example usage.
I used the wrong wording: They have 15-20 *enum variants* per enum definition. Or more. I worked with a project that had close to 30-something enum variants.
I find this kind of article that explains how to do something with vanilla Rust valuable because many articles will instead explain how to use a crate that offers similar functionality. This is the only way to know if a crate can actually pull its weight before you decide to adopt it.
For reducing the code to implement this pattern, `thiserror` is nice. `derive_more` also has similar functionality, and I tend to use that more personally because I use its other derives, and while thiserror has a slightly nicer API (derive_more will assume a unit struct's field is it's source, which is wrong more often than not in my codebase, since I avoid overdoing nested error enums), it's not worth adding another dep to do what the dep I already have can do: https://jeltef.github.io/derive_more/derive_more/error.html
The downside to derive_more from what I can tell is that it impacts compile times fairly noticeably compared to the equivalent handwritten code. YMMV, but I've seen it removed from some crates for that specific reason.
Error handling in Rust is one of the reasons I went back to Go. It's a lot of mental overhead having to constantly think about all the different types of errors from different crates that could be returned and how to handle them at the caller. I don't want to have to import somebody's hobby crate that claims to make it easier either. In Go it's just dead simple.
I've noticed a lot of Rust discussion is on how to write the code or make the compiler happy which is quite telling. I don't want to spend my time thinking about how to write the code and make the compiler happy - I want to get a problem solved and a ticket closed. Hence why I went back to Go (but I acknowledge these languages have distinct goals).
Your reluctance to use a crate which solves the issue you are facing is somewhat antithetical to Rust's crate-centric philosophy. After all, anyhow::Result does "get your problem solved and ticket closed" just the way you want.
That being said, in reality it's probably a pretty common sentiment among adopters, especially since the macro crates can slow down compilation. Thankfully, the ergonomics of error handling are indeed under consideration and work is underway to make it all better.
> It's a lot of mental overhead having to constantly think about all the different types of errors from different crates that could be returned and how to handle them at the caller.
If you care about handling errors robustly, you have to care about this stuff at some point, and if you don’t, just use something like Anyhow and never worry again.
When I write Rust stuff, I usually start with Anyhow::Result everywhere, then once everything’s built and working and/or we want more finesse over our error handling I progressively swap to ThisError/etc, I find it’s the best mix of utility and ease of writing.
> I've noticed a lot of Rust discussion is on how to write the code or make the compiler happy which is quite telling
I suspect this is because a lot of people come from other languages, and try to bring the prior languages quirks and idioms with them, quickly discover the stricter compiler flat out doesn’t permit some of them (unlike other languages), then they figure out the idiomatic/closer way and write up their thoughts on the process.
Error handling is one area I way prefer in Rust. It's trivial but having a single line "expect" to throw on some additional text to an error if it doesn't need anything else is really nice. In Go even something really simple might have a bunch of three line error checks everywhere and it just doesn't feel nice to me.
I've found that Go has significantly more mental overhead when it comes to error handling than rust does.
In Go, I don't have type information to know what kind of error it is. In rust, I can typically just "match err", and let the compiler tell me what error types it could be, and then handle each of them.
In go, I always have to read the entire function that returns the error, and any functions it calls, and so on, and then I have to think about how to handle it.
Is it an error from the os package, like "os.Open"? Well, maybe I need:
if os.IsNotExist(err) { /* handle */ }
Is it an error from the net package? I might need:
var addrErr *net.AddrError
var invalidAddrErr net.InvalidAddrError
var otherErr net.Error
if errors.As(err, &addrErr) {
/* handle addrErr */
} else if errors.As(err, &invalidAddrErr) {
/* handle invalidAddrErr */
} else if errors.As(err, &otherErr) {
/* some other net error */
} else if errors.Is(err, net.ErrClosed) {
/* handle this network error which is _not_ a net.Error */
}
Is it an error that used one of the third party error libraries, like "github.com/hashicorp/errwrap"? I may need even more special handling.
The fact that "errors.Is" and "errors.As" requires me to know if an error is a "sentinel" one like 'net.ErrClosed', or a struct error, like 'os.PathError', and I have to know if the struct is implementing error on a pointer receiver or not is also really annoying since it means I have to constantly go check error definitions, even if I vaguely remember what errors a function returns.
And, what's worse, the type system doesn't help me! It won't tell me if I get any of these things wrong. I can use "errors.Is" and "errors.As" backwards, and the type system won't help. I have no way to determine what errors a function can return without reading hundreds of lines of code.
Frankly, Go's error handling has more overhead than almost any other language, and I find it surprising you find it simple.
If all I'm doing is bubbling errors up, then rust's "?" operator works fine, and is easier than Go's "if err != nil" boilerplate. If I'm handling errors, rust's features are helpful, and Go is so much mental overhead it usually makes me forget the problem I was trying to solve by the time I've drawn out the full diagram of possible errors something can return up the stack. Usually rust's type-system has enough info that I don't need to read any code to know what errors I need to handle. That's never the case in Go, because idiomatically errors are "type-erased" into the error interface.
Oh, and this is all not to mention that go facilitates making unhandle-able errors. If someone returns "errors.New" or "fmt.Errorf", you're stuck with string matching, while rust encourages making typed and handle-able errors. The number of times I've had my go code break because someone reworded an un-exported error string I had to match on is pretty high. Hasn't happened to me in rust yet.
Not a rust aficionado: If you’re using error.As, wouldn’t you need essentially the same logic in Rust to match the error type and get access to the right fields?
Seems like the big difference is the compiler knows the variants of the error. The trade off is that Rust modules are tightly coupled by the error type since you have wrap each one into the module’s own error type.
I really like the thiserror crate for cutting down boilerplate and you can add annotations to automatically implement From. However, if you want to handle errors with more granularity, e.g. IO errors as noted elsewhere in the discussion, then you'll need to implement From yourself.
And as others noted, .map_err(...) is slightly better than .or(...) for wrapping errors.
The only downside of this approach is that you can't pass a dynamic string giving details on the error, but that's because Rust's enums are not powerful enough to express this (as opposed to Kotlin or Java).
Something worth mentioning is that your error types do NOT have to implement std::error::Error. The type parameters on Result<T, E> have no constraints.
So, only your public API should implement std::error::Error.
Likewise, you don't HAVE TO implement From for all of your error types. In fact, it has some of the same concerns that implicit constructors in C++ have in that it's easy to just throw try-operators around without thinking about whether you actually want to bubble up an error in a specific scenario.
I think a lot of people who pick up Rust might not realize that and end up thinking the error defining process is even more tedious than it really is.
Beyond Borrow checker, move/copy, and error propagation(different error in fn), what else should someone learn in order to have basic understanding of Rust?
Errors in rust are one of the hardest things for me to wrap my head around as a beginner. Thanks for this post!
I also really like this blog post that also walks you through the history of some features like the try! Macro and shows how they are implemented https://blog.burntsushi.net/rust-error-handling/
More hacks upon hacks to make up for the glaring inability of the Rust people to acknowledge they made a mistake by leaving out exceptions. Just like the "type after identifier" thing, the use of error values instead of exceptions was a pointless piece of Go envy in fashion at the time that we're all going to have to live with for decades.
Plus, Rust doesn't even get to avoid paying for exceptions, because it still has panics and still has to support unwind semantics --- even though they can be disabled at build time.
Avoiding exceptions was a needless technical error and an illustration of why we should design systems around tried and true techniques instead of jumping on fashion trends and ossifying those fashion trends into immutable mediocrity.
Calling exceptions "Go envy" or a "mistake" seems like naive criticism. Ignoring the fact that the Rust is as old as Go; there is a lot of time spent in building language features with an, from my point of view, academic level of rigor.
There have been a lot of RFCs discussing exceptions in Rust[1][2][3], and the tradeoffs and design have been discussed thoroughly. If you have a design in mind that is safe, robust and performant you are welcome to submit one - as my understanding is the Rust team does not have an ideological slant against exceptions.
I've never understood the hard-on by some C++ fanboys to rush exceptions. You can build (IMO) more reasonable code without abusing ~~gotos~~ exceptions, and a bad exception design (like Java's) is strictly worse than even Go's error handling.
Sum and product types are easier to reason about in total than exceptions. For errors they just integrate into the return type and can be handled as monads. They don’t require additional syntax and can directly be passed into other functions.
> ossifying those fashion trends into immutable mediocrity.
Can you explain what (in your opinion) was lost here? Because I don’t see how this is some kind of critical issue, it just looks like some minor semantics that you just don’t like.
I don’t see what type of “cost” was payed here that you think implies mediocrity.
Maybe I’m biased because I come from firmware development, but I’d rather have error handling be explicit and as close as possible to the error source.
For the type after identifier, the converse is what you see in C family languages. For example:
// in C
int x = ...
// in Rust
let x: i32 = ...
There's a couple of things that Rust can do better than C here. First, is that Rust is easier to parse because you know that let begins a variable declaration. If we look at C, maybe it's a variable, maybe it's a function and you have to get past the identifier to figure that out. The second more user-facing change is that you can omit the type in Rust if you can derive it from somewhere else, e.g. the return type of a function call or a primitive.
I think that there's a pretty strong case to be made that conditions, implemented in Common Lisp, are the best implementation of "exceptions"...by not actually being exceptions.
Conditions (plus restarts) have two advantages over exceptions (as implemented in almost every other language, including Go and C#) and Rust's error system that neither can replicate: they allow you to selectively keep the stack wound (both avoiding expensive recomputation and keeping error context), and they separate out multiple error recovery strategies from each other, and allow you to pick which one you want at runtime.
The latter fixes a core problem of all other error-handling systems: that the high-level code usually has the context to determine why a low-level operation was happening, and therefore how the error should be recovered from - but it doesn't (or shouldn't) have knowledge of what low-level operations are happening, because that breaks encapsulation. Meanwhile, the low-level code obviously knows about itself - but it doesn't have the high-level context necessary to determine which of several error-recovery strategies to take.
A condition system exposes multiple named restarts, which are error-recovery strategies, at the low-level code (technically, you can place them everywhere), and then the higher-level code can choose among those restarts (with names like "abort", "retry", "continue", "ignore") when an error occurs, based on the high-level context and the condition type (because conditions, like exceptions, also have types).
Note that these restarts do not necessarily unwind the stack. The stack can remain "wound" to the point of the error, or it can be partially unwound if a restart a few frames up is selected. Exceptions allow none of this - the moment an exception (or a Rust error type) is thrown/returned, the stack is unwound up to the error-handling point, and all context not explicitly encoded is lost (as well as the possibility of retrying an operation (perhaps with different parameters)).
Short example: let's say you're writing parser for a log format that goes line-by-line. There's the high-level "read-log-file" function, that calls several other functions that eventually calls a "parse-log-file-line" function. From the local perspective of just that low-level function, if it's fed a line it can't parse, there are multiple valid error-recovery strategies: it could try to repair the line as best as it could, it could add the corrupted line to a list, it could skip the line, or it could explode and die. With a normal exception system, you have to either thread a dedicated argument specifying the strategy down to that function, hard-code it to pick a particular strategy, or re-write that function for your particular application. With conditions, the parse-log-file-line function could throw a condition INVALID-LINE and expose each of the previous options as restarts, and the high-level code could pick one of them, or even handle the condition itself!
I like the article, though... the problems it illustrates with effects are usually solved by DI/IoC containers. It even says so:
> Effect handlers let us decouple the program logic from its concrete effect implementations without too much ceremony or boilerplate code. For example, we could completely override the behavior in tests to use a fake filesystem [..]
Yes, I understand the fundamental difference between effects and DI (effects suspend the execution and search the call stack for the handler - like exceptions, but without immediate unwinding).
Maybe we could solve the problem with code being able to throw arbitrary exceptions by injecting an "Error" interface as well and then using some discipline? (I.e., instead of throwing stuff directly, we invoke methods on the Error interface.) Actually, I have to think more about this.
Thanks for the link, it gave me some food for thought.
Common Lisp, as another commentator described in detail below. Common Lisp is a state of grace from which we have shamefully fallen. It's old out here, east of Eden.
It simplifies constructing your own "leaf" errors and streamlines the ability of collecting multiple types of errors while attaching more context to them (e.g. filenames, stack traces, user ids, etc.). It allows you to smoothly switch from "stringly-typed" errors to strongly-typed errors. You can create opaque errors to avoid leaking internal implementation details into your public API.
Applied to the code in the post:
The key parts are the `derive(Snafu)` on the definition of the error enum and the usages of `.context` and `XxxSnafu` at the error sites.Importantly, this example demonstrates a key feature of SNAFU, here shown as "not all `io::Error`s are the same". Opening the file and reading the file are two separate error conditions and should not be lumped together as one.
[1]: https://docs.rs/snafu/0.7.0-beta.1/snafu/index.html