I was really hoping that there'd be movement on a comment without-boats made in https://without.boats/blog/why-async-rust/ to bring a pollster like API into the standard library.
Rust has very good reasons for not wanting to bless an executor by bringing it into the standard library. But most of those would be moot if pollster was brought in. It wouldn't stifle experimentation and refinement of other approaches because it's so limited in scope and useless to all but the simplest of use cases.
But it does in practice solve what many mislabel as the function coloring problem. Powerful rust libraries tend to be async because that's maximally useful. Many provide an alternate synchronous interface but they all do it differently and it forces selection of an executor even if the library wouldn't otherwise force such a selection. (Although to be clear such libraries do often depend on I/O in a manner that also forces a specific executor selection).
Pollster or similar in standard library would allow external crates to be async with essentially no impact on synchronous users.
Not quite yet. Crates like reqwest and hyper tend to use tokio's io types internally to set up the sockets correctly and send/receive data at the right time. Those might have different APIs than the thread-pausing sync APIs.
Sans-IO crates exist but are kind of annoying to schedule correctly on an IO runtime of choice. Maybe lending iterators could help idk
I feel async is in a very good place now (apart from async trait :[ )
As a regular user who isn't developing libraries async is super simple to use. Your function is async = it must be .await and must be in an async runtime. Probably as simple and straightforward as possible. There are no super annoying anti-patterns to deal with.
The ecosystem being tokio centric is a little strange though
I love Rust and async Rust, but it's not true that there aren't annoying things to deal with. Anyone who's written async Rust enough has run into cancel-safety issues, the lack of async Drop and the interaction of async and traits. It's still very good, but there are some issues that don't feel very rust-y.
There are several ~~problems~~ subtleties that make usage of Rust async hindered IMHO.
- BoxFuture. It's used almost everywhere. It means there are no chances for heap elision optimization.
- Verbosity. Look at this BoxFuture definition: `BoxFuture<'a, T> = Pin<Box<dyn Future<Output = T> + Send + 'a>>;`. It's awful. I do understand what's Pin trait, what is Future trait, what's Send, lifetimes and dynamic dispatching. I *have to* know all these not obvious things just to operate with coroutines in my (possibly single threaded!) program =(
- No async drop and async trait in stdlib (fixed not so long ago)
I am *not* a hater of Rust async system. It's a little simpler and less tunable than in C++, but more complex than in Golang. Just I cannot say Rust's async approach is a good enough trade-off while a plethora of the decisions made in the design of the language are closest to the silver bullet.
Because async and sync programming are two fundamentally different registers. There are things you can do in one that you can’t with the other, or which have dramatically different tradeoffs.
As an example: Call N functions to see which one finishes first. With async this is trivial and cheap, without it it’s extremely expensive and error-prone.
The actor model proves that that isn't really as fundamentally a difference as you make it out to be. Write synchronously, execute asynchronously, that's the best of both worlds. To have the asynchronous implementation details exhibit themselves at the language level is just a terribly leaky abstraction. And I feel that if it wasn't a fashionable thing or an attempt to be more like JavaScript that it would have never been implemented in the way it was in the first place.
Async makes everything so much harder to reason about and introduces so many warts in the languages that use it that I probably think it should be considered an anti-pattern. And I was writing asynchronous code in C in the 90's so it's not like I haven't done it but it is just plain ugly, no matter what syntactic sugar you add to make the pill easier to swallow.
Or that you can't do it in a systems programming language whose main intent is to replace 'C'?
I don't want to start off with a strawman but in the interest of efficiency:
Because C is far from the only systems programming language and I don't see any pre-requisites in the actor model itself that would stop you from using that in a systems programming language at all. On the contrary, I think it is eminently suitable for systems programming tasks. Message passing is just another core construct and once you have that you can build on top of it without restrictions in terms of what you might be able to achieve.
Even Erlang - not your typical first choice for low level work - is used for bare metal systems programming ('GRiSP').
Maybe this should start with a definition of what you consider to be a systems programming language? Something that can work entirely without a runtime?
Sure, you can absolutely build systems with the actor model, even some embedded or bare metal cases. But Erlang isn't written in Erlang. I'm talking about the languages that you implement Erlang in.
Yes, I think "entirely without a runtime" in the colloquial sense is what I mean. Or "replace C" if you want.
Ok, interesting because if 'should be written in itself' is a must then lots of languages that I would not consider systems languages would qualify. And I can see Erlang 'native' and with hardware access primitives definitely as a possibility.
'replace C' is a much narrower brief and effectively forces you to accept a lot of the warts that C exposes to the world. This results in friction between what you wanted to do and end up doing as well as being stuck with some decisions made in the 1970's. It revisits a subset of those decisions whilst keeping the remainder. And Rust's ambitions now seem to have grown beyond 'replace C', it is trying very hard to be everything to everybody and includes a package manager and language features that a systems language does not need. In that sense it is becoming more like C++ than like C. C is small. Rust is now large.
Async/Await is a mental model that makes code (much) harder to reason about than synchronous code, in spite of all of the claims to the contrary (and I'm not even sure if all of the people making those claims really believe them, it may be hard to admit that reasoning about code you wrote yourself can be difficult). It obfuscates the thread of execution as well as the state and that's an important support to hold on to while attempting to understand what a chunk of code does. It effectively turns all of your code into a soft equivalent of interrupt driven code, and that is probably the most difficult kind of code you could try to write.
The actor model recognizes this fact and creates an abstraction that - for once - is not leaky, the code is extremely easy to reason about whilst under the hood the complexity of the implementation is hidden from the application programmer. This means that relative novices (which probably describes the bulk of all programmers alive today) can safely and predictably implement complex systems with multiple moving parts because it does not require them to have a mental model akin to a scheduler with multiple processes in flight all of which are at different stages of their execution. Reasoning about the state of a program suddenly becomes a global exercise rather than a local one and locality of state is an important tool if you want to write code that is predictable, the smaller the scope the better you will understand what you are doing.
It is funny because this would suggest that the likes of Erlang and other languages that implement the actor model are beginners languages because most experienced programmers would balk at the barrier to entry. But that barrier is mostly about a lot of the superstructure built on top of Erlang, and probably about the fact that Erlang has its roots in Prolog which was already an odd duck.
But you've made me wonder: could you write Erlang in Erlang entirely without a runtime other than a language bootstrap (which even C needs) and if not to what degree would you have to extend Erlang to be able to do so. And I think here you mean 'the Erlang virtual machine that are not written in Erlang' because Erlang the language is written in Erlang as is the vast bulk of the runtime.
The fact that the BEAM is written in another language is because it is effectively a HAL, an idealized (or not so idealized, see https://www.erlang.org/blog/beam-compiler-history/) machine to run Erlang on, not because you could not write the BEAM itself entirely in Erlang. That's mostly an optimization issue, which to me is in principal evaluations like this a matter of degree rather than a qualitative difference, though if the inefficiency is large enough it could easily become one as early versions of Erlang proved.
Maybe it is the use of a VM that should disqualify a language from being a 'systems language' by your definition?
But personally I don't care about that enough to sacrifice code readability to the point that you add entirely new footguns to a language that aims for safety because for code with long term staying power readability and ability to reason about the code is a very important property. Just as I would rather have memory safety than not (but there are many ways to achieve that particular goal).
What is amusing is that the Async/Await anti-pattern is now prevalent and just about the only 'systems languages' (using your definition) that have not adopted it are C and Go.
Honestly, this is why I find "systems language" kind of an annoying term, because you're not wrong, but it's also true that we're talking about two different things. I just don't think we have good language terminology for the different sorts of languages here.
> could you write Erlang in Erlang entirely
I think this sort of question is where theory and practice diverge: sure, due to turing completeness. But theory in this sense doesn't care about things like runtime performance, or maintainability.
> But personally I don't care about that enough
Some people and some domains do need to care about implementing the low-level details of a system. The VMs and runtimes and operating systems. And that's what I meant by my original post.
So, as the author of not one but two operating systems (one of which I've recently published, another will likely never see daylight): I've never felt the need for 'async/await' at the OS kernel level. And above that it is essentially all applications, and there almost everything has a runtime, usually in the form of a standard library.
I agree with you that writing Erlang in Erlang today is not feasible for the runtime performance matter, less so for maintainability (which I've found to be excellent for anything I ever did in Erlang, probably better than any other language I've used).
And effectively it is maintainability that we are talking about here because that is where this particular pattern makes life considerably harder. It is hard enough to reason about async code 20 minutes after you wrote it, much harder still if you have to get into a code base that you did not write or if you have to dig in six months (or a decade) later to solve some problem.
I get your gripe about the term systems language, but we can just delineate it in a descriptive way so we are not constrained by terminology that ill fits the various uses cases. Low level language or runtime-free language would be fine as well (the 'no true Scotsmen of systems languages ;) ).
But in the end this is about the actor model, not about Erlang per se, that is just one particular example and I don't see any reason why the actor model could not be a first class citizen in a systems oriented language, you could choose to use it or not and if you did that would have certain consequences just like using async/await have all kinds of consequences, and most likely when writing low level OS code you would not be using that anyway.
I mean, I'm also not saying async/await is critical for kernels. I'm only saying that "everything is an actor" isn't really possible at the language level.
Async/await is used for a lot of RTOS like things in Rust. At Oxide, we deliberately did not do that, and did something much closer to actors, actually. Both patterns are absolutely viable, for sure. But as patterns, and not as language primitives, at least on the actor side.
If not, your async code is a deterministic state machine. They're going to complete in the same order. Async is just a way of manually scheduling task switches.
The system Rust has is a lot better than that of Python or JavaScript. Cleanly separating construction from running/polling makes it a lot more predictable and easier to understand what's happening, and to conveniently compose things together using it.
I agree, with the possible exception of perplexing async stuff.