posts/rust-is-hard-or-the-misery-of-mainstream-programming

[utterances-bot]

Rust Is Hard, Or: The Misery of Mainstream Programming

https://hirrolot.github.io/posts/rust-is-hard-or-the-misery-of-mainstream-programming.html

[hengestone]

Question from the peanut gallery: what are your thoughts on Pony? It obviously doesn't have the momentum of Rust but it was designed with concurrency as a central objective, and compiles to native code with llvm.

[sggrigoriev]

Rust is good candidate to replace C++ I think. The only problem is that Rust doesn't provide the C++'s freedom which is bad for experienced programmers, and pretty hard for nubs.

[snowfrogdev]

I've been playing with rust on and off for about two years. I've found myself having the same thought as you. I'm amazed at some of the language features provided by rust and can't help but think that it would be an amazing language to build anything in if only you didn't have to deal with memory.

When I compare it with C# or Java, I find my self envying it's structural type system, error handling philosophy and constructs, often very useful and descriptive compiler messages, the absence of null, the best implementation of the iterator pattern I've seen in any language, traits, tuples, enums, amazing pattern matching and the list goes on.

If only there was a language out there that had made the same design decisions in those areas but was slightly higher level, it would be such a joy to work with. Unfortunately, for me, rust is sometimes a bit frustrating to try to build stuff in.

[QUINTIX]

I’m new to rust, and rusty on C++, so while I barely understand what’s being discussed here, it sounds an awful lot like what Mike Acton talked about in his CppCon 2014 talk: both templating in Cpp and generics in Rust represent a pre-compilation mixture of concerns that are best solved by text-preprocessing entirely separate from the compilation itself. Many things handled by generics within language could be handled just as well if not better by implementing for a concrete type and making a python/JavaScript/Perl template string out said code with fields for the type you wish to make more generic, then have that script generate specific code only for the concrete types actually used in your codebase. The only thing such dynamically generated code would lack is checking against traits/interfaces during code generation, or integration of such checks into your IDE of choice but I imagine that would not be a massive lift to fix. Javascript’s babel may already provide a model for such dynamic code generation.

[rosenk]

Excuse my ignorance but my small experience in Rust convinced me that the language actually promotes functional programming (you get bizarre errors when trying golang or c++ style of programming). Registering handlers doesn’t sound functional at all. Quick search about reactive functional streams leads me to interesting solutions. For example tokio::sync::broadcast. I would love to hear your opinion on this.

[sggrigoriev]

I see the point, thank you. I agree, it doesn't make sense to implement simple server w/o high memory or load requirements on Rust or C++ if one could use Go or Python for the task.

…

On Fri, Jun 3, 2022, 19:06 QUINTIX ***@***.***> wrote: I’m new to rust, and rusty on C++, so while I barely understand what’s being discussed here, it sounds an awful lot like what Mike Acton talked about in his CppCon 2014 talk: both templating in Cpp and generics in Rust represent a pre-compilation mixture of concerns that are best solved by text-preprocessing entirely separate from the compilation itself. Many things handled by generics within language could be handled just as well if not better by implementing for a concrete type and making a python/JavaScript/Perl template string out said code with fields for the type you wish to make more generic, then have that script generate specific code only for the concrete types actually used in your codebase. The only thing such dynamically generated code would lack is checking against traits/interfaces during code generation, or integration of such checks into your IDE of choice but I imagine that would not be a massive lift to fix. Javascript’s babel may already provide a model for such dynamic code generation. — Reply to this email directly, view it on GitHub <https://github.com/Hirrolot/hirrolot.github.io/issues/6#issuecomment-1146136162>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEDIB6IJJAEK7I5N3NMA7GTVNIURZANCNFSM5XXQUWHA> . You are receiving this because you commented.Message ID: ***@***.***>

[antmelnyk]

Excuse my ignorance but my small experience in Rust convinced me that the language actually promotes functional programming

I think it's a newbie trap and illusion that comes from the fact that, indeed, Rust took many good ideas from functional languages - idiomatic immutability, Option and Result functors, iterators, pattern matching. Indeed, Rust encourages you to write my_array.iter().map(myFn).collect() instead of doing for loop.

Rust does amazing job using some of higher abstractions on such a low level system programming language. With experience it's possible to use it for casual backend development and benefit from its speed and safety.

But in its core Rust in no way is a language for functional programming because it is too focused on imperative implementation details. Working with functions as first class citizens in Rust is hard - and the article greatly shows that. Author just wants a List of Tasks - how hard can it be, right? As you can see, turns out it's quite hard. Lifetimes and smart pointers are required to solve many things that are a bit abstract.

You cannot easily even write simple variadic compose, do currying, write abstractions like Functor or Monad because there is no higher kinded types, use optics...

Not saying it's not possible - most of things are, and there are many experimental crates that achieve them using macros. But, in my opinion, it's neither ergonomic nor idiomatic way to use Rust.

[snowfrogdev]

But in its core Rust in no way is a language for functional programming because it is too focused on imperative implementation details.

That's a good point. rust wants to give you performance and safety. It's hard to do the bare metal kind of performance rust is going for on current hardware without imperative code because the hardware itself uses "mutation".

That being said, I find it amazing that rust's implementation of iterators is a no cost abstraction. Though I guess iterators are technically not purely functional as they have state mutation... but they feel functional.

[tjhorner]

I tweeted something late last year that expressed this exact frustration: https://twitter.com/tjhorner/status/1475730841699500036

A lot of the pain came from an immature ecosystem of libraries that didn't really work together — either because they were written before async/await syntax existed, or they rely on non-standard things that were later added to std, or a handful of other reasons — but even in the short time since that tweet, things seem much improved. Async/await syntax, futures, etc. are finally stable and have been gaining traction.

Now with more mature libraries and better documentation, I find myself pulling my hair out a lot less these days.

[hirrolot]

I tweeted something late last year that expressed this exact frustration: https://twitter.com/tjhorner/status/1475730841699500036

"I guess you guys aren't ready for that yet, but your kids are gonna love it."

[ChristophePichaud]

C++ turns around 40. It has been updated from basic OOP template virtual Algo op redéfinition and C++ 98 milestone to now Modern C++ and a real big /huge strong STL. The problem in your Rust galaxy is that it requires full adoption of C++ like features like Move Semantics, Share_ptr and complete std::move std::forward and future and async/await understanding and REQUIRED AND FULL ADOPTION AND USAGE! With regular Modern C++ like C++20, in the C++ community, we do not constraint our developers to write apps only using those latest features.
With Rust, developers need and have no choice to think and write using those f**** move semantics everywhere... So you call it Rust is Hard.
My opinion: you are so asking for requirements but in the opposite your argue the low time compilation features makes no bug / your app is safe.
Ok, let me judge that in my way to design and write my apps please.
In C++ if I just want shared_ptr and vectors with simple usage of algorithm and some STL basic stuff like containers and some standard things, I can. No fear to be blamed because I don't want usage of move semantics. If STL allowing PTR ref instead of copying containers in a return vector from a function, ok great for me as a user of the STL but the beauty of C++ is that even if C++ Impl is a Champion Game, I am now REQUIRED to belong to the Elite Team.
I can just be a C++ STL user, a simple C++ developers and, It works so fast, si good with so power performance and optimisation inside.
You with Rust is just Tyranic Elite Required small community arguing you are so smart...
Go to hell.
C++ rules the world. And it will continue because you don't want to enter C++ world, you want to be the winner by replacing and asking for the King Castle.
It won't happen.
You have written all the fact that how you kill you inside your little universe.
C++ Expertise with appropriate hard stuff coded with latest features is so easy to read, write, maintain rather than your so called "better system programming language".
Arrogance is not a good sign when you are a less 10 years boy compared to us, C++ 40 years old.
Remember: we are the invisible foundation of Everything.
Ask Bjarne and listen to latest 66 minutes episode talking about that.
Even a small minor fix in Linux kernel, a firefox browser and some marketing post will make your community a winner.
Remember the fiasco of latest Rust internally communication disagreements ???? So cute.
Christophe Pichaud | Books author (3 C++, 2 C#)

[laughingraven]

Rust is an unnecessary language. If you are dealing with a safety critical, realtime application than you should be using Ada. Ads has been trusted for decades in the most safety critical of applications. Planes, trains, freightliners and more are all programed with Ada. Ada can be mathematically provable, has the best in industry tooling, and is trusted amongst such safety critical standards as DO 178.

[ivnsch]

General applications programmer here 👋 I'm nowadays writing everything in Rust (backend, frontend (WASM), even scripts sometimes) and don't see that much of an issue with it. Currently working on a fairly extensive, complex app, where everything is async. I'm not even fluent in "advanced Rust" - e.g. lifetimes, and tend to use alternatives like Rc / Arc, which don't have any drawbacks for these kind of apps.

In my experience it's possible to learn incrementally, where the baseline is a bit higher than languages like Java (or say, Kotlin, Swift..). And it's good to know that with some extra effort I can have C performance if I ever need to.

[hirrolot]

@ivanschuetz,

And it's good to know that with some extra effort I can have C performance if I ever need to.

With some extra effort you can write some C code and use it in a high-level language like Python. There is no need to use Rust if you can achieve the same with more high-level languages. And I don't understand people who write scripts in Rust at all; I think Rust is definitely not well-suited for scripting.

[bocc]

@ivanschuetz

I once tried to run a helloworld in Kotlin, and it failed with 'cannot allocate 3.something GB object heap'. There is no way some high level programming convenience is worth that much. Maybe Julia has a sweet spot with giving you information about the runtime percentage of GC, so you can optimize if you wanted to.

[hirrolot]

Zig has maturity issues as well, but I like its preference for the pragmatic.

Zig-style "generics" are more like C++ templates to me. Not something totally wrong, but the errors are really huge. I think this is because for a generic Zig function to be considered valid, we need to scan its body, while for a (any) Rust function to be valid, we scan only its signature. Yes, Rust can produce enormous errors too, but this occurs only for two reasons: either you are a type-level monster, or it's just a compiler bug or something like that.

[taophp]

I've just discovered Lia, a PL that compiles to Rust and seems to address some issues you pointed. Would you share your thoughts about it ?

[gabriel-fallen]

We just don't have an adequate high-level language with decent performance.

Sure thing we have. Among higher-performance (compared to JS/Pyhton it's not even funny) lower-memory-footprint high-level (memory) safe languages we have Haskell and OCaml on the FP side, and D on the OO-ish, closer-to-C++ side. Among top-of-the-line performance but high-memory-footprint we have good 'ol Java and C# which nowadays are pretty high-level and pretty safe, and newish "contenders" Scala and Kotlin which are just marginally "slower". Again compared to JS/Python/PHP the performance is over the top, and we have mind-bending complex and still pretty performant applications in JS/Python/PHP.

Then on a dynamic typing side of things we have Julia and it's community that constantly brags about being 10x to 100x as fast as Python. In certain cases Julia indeed demonstrably outperforms state-of-the-art C libraries. Good 'ol Common Lisp doesn't get attention new cool kids like Julia attract but it's literally decades-matured compilers produce native code as tight as your Rust in great many cases.

The gloomy sober bottom-line is no language guarantees you high performance. But if you're willing to go an extra mile writing in lower-level style conscious of memory usage and access patterns as you have to in Rust (to accommodate borrow-checker at the very least) you can achieve very comparable results in any of the mentioned languages with exactly the same usual techniques (preallocated memory arenas, manual reuse, indexing into arrays, you know).

But that's so boring and so last century! Wrestling with Rust is wa-ay cooler!

[rlsdevine]

We just don't have an adequate high-level language with decent performance.

I'm building an app that allows users to build data analysis plug-ins to extend the app itself and spent a long time looking at what's available (Nim, Zig, Kotlin, D, Go etc.). Python had to be in there of course but in the end I opted for Nim as the primary compiled language. Like others, it's missing Python's range of libraries but is fairly easy for data scientists to pick up.

[vikram2784]

While closures that accept references are bounded by specific lifetimes,

It's interesting to see that if the type is expilcitly stated in the closure's argument, it too works for all input lifetimes.

[hirrolot]

@gabriel-fallen, thank you for your analysis. I think most of the languages you mentioned are somehow get "unnoticed" by the mainstream. E.g., people think Haskell, OCaml, and Common Lisp are too alien because they are FP. The D programming language hasn't found its massive use in production too, for some unknown reason to me. Scala may be cool, but honestly, sometimes when I look into code written in Scala, I cannot understand a single line written in it.

The industry uses C#/Java a lot, although they do also have problems that Rust doesn't have. Java is still a lot of boilerplate, C# seems to be going in the direction of PL/I.

Yes, a lot of people use Rust just because it's trendy. I think the hype will disappear eventually. On the other hand, I'd really enjoy a high-level language similar to Rust -- it'd be still comprehensible by the mass programmer (unlike Haskell's FP or Lisp) and will provide fine type safety and tools to avoid code duplication, as well as other neat characteristics of Rust.

That being said, if I am to develop a regular web service, I'd choose Golang, Java or something else. But there are people who'd better choose Rust because of sum types, explicit checks, or other language features inaccessible from the mainstream.

[hirrolot]

"We just don't have an adequate high-level language with decent performance."

I already feel guilty of my phrase stating "decent performance". I think JVM and .NET have decent performance, I don't mean bare-metal performance like in C or Rust.

The focus should rather be on "adequate".

[gabriel-fallen]

Hi @hirrolot nice to talk to you here too! Cool post by the way. 😉

What I wrote is not an analysis it's somewhere between a rant and a personal conclusions. For the most part I'm just stating obvious and (relatively) widely known.

But what are we discussing here exactly? You stated the need for "an adequate high-level language with decent performance" and of course we have such languages, yes with perfectly adequate performance and development effort otherwise pretty much nothing around us would work! 😄 For sure we have way too many free services to whine about costs too high and everything. 😃

I think what we really need or at least want are applications with adequate performance. At least Jonathan Blow and Casey Muratory rant about that a lot. And I agree a lot.

The catch is — as I already mentioned — no programming language guarantee adequate performance to your application! Do we have huge bloated and slow C++ applications? I bet you're using at least one right now. Do we have surprisingly fast and responsive JavaScript applications? I can find a bunch of 3D games.

What conclusion can we make? Application performance mostly depend on the architecture and design, which stem from developer knowledge and experience (also dedication and time), not the programming language. Thus it's mostly an educational issue, not a technical one. As long as majority of employed programmers graduate from 6-months "get into IT quick" programs no change in programming languages' popularity will change the applications' performance distribution.

Contrary to the popular belief adequate on average performance of majority of (Web-)applications is a great testament to the amount of ingenious engineering going into JVM, CLR and alike and the evidence of their stellar efficiency which allows developers to deliver reasonable solutions without any regard to "systems" levels.

Again to me Rust's good track of efficient applications and libraries (so far) is a social thing. Rust attracts people who care about performance (and correctness) more than about their comfort (and "productivity"). Those who don't care just don't use Rust. (Many of those who do care still don't use Rust, as I mentioned too one can write very performant code in pretty much any language.)

So what do we get as a bottom line? If you personally want a "high level yet performant" language (for personal projects) you have quite a number of options: D and Nim on imperative OOP side (as long as Rust, Zig, Go are a bit too low-level); Haskell and OCaml on the FP side (and more advanced type systems). And these are pretty "mainstream" (at least production-ready) ones, if you're willing to venture into more "experimental land" you can find many more interesting languages (say, Koka).

And nothing we discuss and decide here among ourselves can affect the "grand scheme of things" in the "industrial mainstream" in a slightest! 😂

[hirrolot]

I know people use Haskell and OCaml in production, as well as Common Lisp (e.g., engineers at Grammarly are using it). Yet, show these languages to an average programmer -- the reaction is expectable. Yes, this is a social thing: many programmers would rather write code in a C-like language, even if say Common Lisp technically provides more advantages for their problem domain.

But if there was a high-level language like Rust, these people would have an easy time switching from their everyday language to this imaginary Rust. And yes, I think it'd be much better from the technical point than most of (high-level) mainstream languages nowadays.

Maybe we should instead educate people to not stick only with C-like languages but also try functional programming? Maybe, but I think this is just not going to happen in the near future.

P.S. I like Koka too, very neat language. It's my favourite imperative language so far.

[gabriel-fallen]

@hirrolot what's your idea of an "average programmer"? How would they react to Boost-like "production-level" C++ code? Or Metalang-style C code? OK, even Linux-like macros-heavy domain-specific C code? Or Teloxide-like Rust code? Would it be that different from Haskell/OCaml code? 😃

You have to teach people how to use any language. So instead of teaching superfluous Haskell we can teach how to write efficient, modular and robust Python. Surprisingly it will require teaching about CPU, memory, stack, OS kernel, I/O, networking stack and so on. Everyone's favourite low-level C stuff.

It's never about a language, it's about the system. The best part about Rust it brings system-awareness back into fashion. And it's a gateway drug to Haskell as we all know. 😂

[hirrolot]

An average programmer uses only mainstream technology and doesn't (want to) dig into less used or more specific tools. Boost-level metaprogramming is definitely out of mainstream, as well Metalang99, Haskell, OCaml, and Common Lisp.

So instead of teaching superfluous Haskell we can teach how to write efficient, modular and robust Python.

I agree that superfluous Haskell is a terrible idea. However, if we only teach Python profoundly, people will still be unaware of functional programming stuff. This will be just out of their spectrum of ideas, which is bad, because Haskell often showcases how common problems can be approached differently, thus greatly enriching a programmer's toolbox.

It's never about a language, it's about the system.

I'd say a language can "open your eyes" on system design. A language and a system are closely interrelated ideas -- after all, we use languages to describe system's structure and behaviour, which is, in its turn, restricted by language's expressiveness.

By the way, what's your favourite book on system design? I read some literature like "Code Complete", but authors often speak only about OOP stuff and pretend that other paradigms don't exist. Would love to see a textbook that explores system design in a more fundamental setting.

[gabriel-fallen]

@hirrolot

An average programmer uses only mainstream technology and doesn't (want to) dig into less used or more specific tools.

Here programming languages are again a poor marker to use. Let's take Aleksey Shipilev as an example. He writes pretty much only Java and C++ — as mainstream as it gets — and most likely doesn't know and doesn't care about Haskell or even Rust. Is he an "average developer"? Most certainly not.

After you've removed production-level C and C++ from the realm of "average programmers" altogether what's left? General Web and Mobile programming in Java/C#/Swift/PHP/Python/JS? Again, that's not about languages, Swift shows that your "average" iOS developers can pretty quickly pick up a new rather advanced programming language. It's still about the problem domain, values, constraints and knowledge.

if we only teach Python profoundly, people will still be unaware of functional programming stuff

For one thing "Python is an acceptable Lisp" as Peter Norvig revealed to us. 😃 But then again what's so special about Functional Programming? What about Logic Programming? Constraint Satisfaction Programming? Actor Programming? Is "different approach to common problems" a programming language's prerogative or a programming system's? 😉

I'd say a language can "open your eyes" on system design.

And I'd say it's a deficiency of an educational systems. The fact that we (you and I in particular) ever encounter new approaches ("paradigms" as they say) and systems only through our inquiry into new programming languages is a reflection of a poor educational program. It shouldn't be this way, it should be totally intentional, systematic and planned.

after all, we use languages to describe system's structure and behaviour

But is it the only way? Or the best one? And should we use programming languages or a mathematical language as Leslie Lamport recommends? 😉

By the way, what's your favourite book on system design?

I never claimed I'm an expert on system design. 😁 In reality I read very little on system design directly and can't give an authoritative answer. I suspect the best practical (rather honest) source is The Architecture of Open Source Applications books. Another recent and fashionable option is Software Design for Flexibility. Also I personally very interested and have high expectations about John Ousterhout's A Philosophy of Software Design. But I still read none of them.

[hirrolot]

Here programming languages are again a poor marker to use.

Yeah, that's why I said "tools" in general, not only "programming languages". Yes, there are excellent software engineers that write in Java/C++. Are GCC engineers "average"? :) of course, no.

I agree with the rest of your statements to some extent, though I need to think more about it. Thanks for the literature on system design, I'll look into it too.

[gabriel-fallen]

to think more about it

That's everything I can ever hope for! :)

[mryndzionek]

Hi! Good discussion. Regarding books I would recommend Concepts, Techniques, and Models of Computer Programming.