Leaving the comfort of lisp, I have started to wonder: why are all the looping facilities in other languages so awful?
Standing on the shoulders of giants, I made this little abomination: https://rikspucko.koketteriet.se/bjoli/goof-loop
It handles 98% of all loops I write, meaning I don't have to manage state (even for things like treating things like circular data). I find it removes many of the stupid errors I make when iterating over data, especially when I need nested loops and conditional accumulation.
I find it so provoking that someone implemented foreach and went "yes. That should be enough for everyone!"
Looping in Ruby is the opposite of awful. You only have to define an `#each` method in your class and include the `Enumerable` module, and you get a plethora of composable methods to iterate and accumulate in every which way. I find it nothing short of brilliant.
Yet, iterating over several collections and accumulating values is awkward.
It is not that it is easy to implement the iteration protocol, it is that the construct for looping universally sucks. Even the awful loop macro from common lisp runs laps around most other facilities. Racket's is even less powerful, but is miles better than what python offers.
Ruby can do some destructuring, but not enough to actually be useful.
It may be somewhat awkward; zipping collections to iterate over them in unison makes the first one (the sender) seem special, while it not necessarily is. And accumulating more than one value isn't always straightforward; you usually have to reduce explicitly. But it's so much better in the general case!
Like the example you showcase your macro with. In Ruby it would be:
You can see what it does at a glance, you don't have to immerse yourself inside the directives to follow the logic. And when you can't do that, declaring iterators and accumulators outside a loop and then iterating and accumulating "by hand" yields code not so different than your more complex examples.Your example of the flatten and grep will build two intermediate arrays before summing. That's inefficient.
I would also reach for higher order functions for most things, but when you need things to be fast that is not how you would do things. You would express it with nested for loops and ifs, and you would have to juggle state all by yourself. That is my main critique of looping constructs: they only cover the absolute simplest case, and then you are by yourself.
The partition example is not a particularly nice implementation of partition. It is there to show that you can write non-tail recursive loops, meaning you can use it to do tree walking and such. Using mutation, like your example, would make it prettier, but would make the code unsafe with schemes call/cc. Even a traditional named let (tail recursion) would be prettier.
For the record, if you want to avoid the creation of intermediate arrays, you can:
Not as clear, because the standard library doesn't have a `#flatten` method for lazy enumerators.But the point is the interface, not the implementation. Efficiency doesn't mandate assembly-like DSLs. Your interface could be as clear and clean as Ruby's and produce fast, inlined code by the power of macros. Ruby doesn't have macros, so chaining lambdas is the best it can do.
Ruby also has call/cc. None of the iterating methods has any provision to make them "safe" from it. They aren't safe from modifications to the iterated collection either. I think it makes sense; being forced to accumulate using only linked lists and having to be constantly on guard to support a rarely used quirky feature is a bad tradeoff IMO.
I dont think interfaces is the point. As it is now in ruby you have to pick between comfort and speed when you can have both.
Continuation safety has never been a thing in ruby, so caring about it doesn't make much sense regardless of the presence of call/cc.
And lastly, neither my loops nor others I have mentioned only accumulate into lists. My loops a re in the most general sense a left fold, although right folds are possible to write as well. I provide accumulators for just about any data type available, and writing a new one yourself is pretty easy. You can express lazy loops with it as well.
All without sacrificing performance. Any performance in most cases. I haven't written much ruby since the 1.9 days, but back then I remember having to rewrite the nice .each{}.blah.blah into for loops. I could prototype in a nice way which was nice, but in the code I ended up shipping I had to juggle state using for loops because the thing I prototyped ended up being a bit slow.
I use map, filter and friends in scheme all the time, but when those are insufficient or too slow I don't have to resort to named lets. I can just use the loop macro which compiles to a named let.
How is that different from select map and reduce in ruby?
You can iterate over strings and lists and arrays at the same time. You can do destructuring. You can accumulate values in every part of the loop, so if you have subloops you can accumulate values in the outer and inner and every intermediate loop and most important: it builds no intermediate results meaning it will be much faster.
Yes, you can do all that in ruby, plus you can choose to do it lazily such that chaining operations uses generators or by copy at any point so maybe this is a classic case of lispbian hubris.
Listen, my gripe isn't with ruby in particular. It is with every language out there.
Doing it that way is still going to be slower than doing it eagerly with a manual loop. I am not saying ruby should do away with the other ways of doing it. I am saying that ruby - and just about every language out there - should have a powerful looping facility that lets you do it fast without losing comfort.
If I - a non-programmer - could implement it using macros in scheme there is really no excuse for other languages to have such sucky foreach-like constructs.
For the vast majority of situations my goof-loop has no performance penalty compared to rolling the loop yourself. That is what none of the ruby examples I have been given can do. Using zip is not an acceptable way of iterating over 2 collections at once, at least not if you want to pretend to be efficient.
I don't think what I did is particularly good compared to something like the iterate macro, and all I want is for other languages to do better because I rarely have the pleasure of using scheme when doing coding.
I see the implementation of lazy zip here. But maybe by ruby you mean some other language.
https://ruby-doc.org/core-2.7.0/Enumerator/Lazy.html
Point is simply: No language is special or exceptional. If a language has yield, you can invert to your hearth's content. Even with lambdas/blocks you can invert by calling it in your loop. Speed is left to the efficiency of your jit or typed compilation. For example SBCL is able to compile incrementally by specifying types.
Adding to this: using laziness compared to a while loop in cases where you will process the whole array is just pure overhead.
Heck, testing now on 3.4.4 a while loop is faster than using each. Summing a million numbers (to make the overhead of the different approaches as clear as possible) shows that a while loop is almost 2x faster.
This is what my complaint is about. Why is the fast path so painful to use in so many languages?
Laziness is not free. An eager loop will be faster than a lazy construct, especially languages where the laziness isn't really a first class construct. Like ruby.
Rewriting the code you wrote (or was it the other guy? I am making food so I cant really check) to avoid intermediate collections as a for loop will be faster, especially if you can avoid creating those array pairs.
I am not saying ruby stinks. I am saying it - and just about every other language - is making it unnecessarily hard to write fast code.