I must admit - I still don't understand macros. I get that they're code that's generated at compile time. But I don't understand how that's different than a function which evaluates other functions. I guess the latter would actually be evaluated at runtime? I think I get it conceptually but I'm not sure I have the muscle memory to reach for them. Anybody here have an "ah hah!" Moment with macros?
> But I don't understand how that's different than a function which evaluates other functions.
You want a function that takes in arguments, but does not evaluate the arguments when called. So:
Normally, foobar() will be called twice - at the time of the call. With macro expansion, you can ensure that it's not the result of calling foobar that goes into the func, but this expression.A canonical example is if you want to write an if/then/else function:
It's quite possible that the else_path is invalid and will terminate the program if the condition is true. But if you wrote a function this way, it will evaluate both then_path and else_path - not something you want to do in a regular if expression!(I think you meant “if condition is false”)
Technically you can accomplish the same thing in languages with first class functions if the caller wraps their code in a lambda.
No, I meant "if condition is true".
The point is that in most/all languages, if the condition is true, the else branch is not evaluated. And it's usually OK to put code in there that can crash when the condition is true, because we know it won't be evaluated.
But if you make an if function like I did above in, say, Python, both the then_path and the else_path are evaluated before the decision is made.
> Technically you can accomplish the same thing in languages with first class functions if the caller wraps their code in a lambda.
True. For languages that have lambdas (which I guess is most of them nowadays).
Well, as mentioned that's why you create two lambdas as the branches. This is pretty much what thunks are in some languages (like Haskell), so this example doesn't strictly require macros per se.
This is true, but thunks carry overhead that macros don't in this case.
And I know people will think I'm crazy, but thunks are harder to read.
(And I have trouble reading macros).
Well, if we have a compiler that sees a constant value, it can completely optimize out one branch. But it of course depends on the semantics of the language.
But yeah you are right in case of a naive compiler.
Ok yeah looking at it again I understand now. Still learning to read.
Sure, macros are functions that take functions as input, and produce new functions as output. But they take the function's symbols as input and produce a new set of symbols. So a macro can extend the syntax of the language without having to modify the core language system. Anyway, what's unique about Lisp macros vs say, Rust macros, or C style preprocessors, is "homoiconicity". The data structure that a Lisp macro takes as an input, the lisp code, is the same data structure that the language uses normally (S-expressions, lists...), so writing a macro requires few new language skills compared to writing normal lisp (again, compare writing Rust macros, a dark art in comparison).
At the same time, you get a mushy tree as input, instead of a properly typed AST "object" with accessible "function name", "method parameters" everything that you can just refer to.
It will be the third arguments' 2nd arg with s-exprs, following your arbitrary pattern you figured you want to use. So it's very arguable which is easier to use, sure for some "party tricks" lisps will win, for anything more useful my vote would go with rust/Scala macros for sure. So again, you win nothing by homoiconicity, the by-the-compiler implemented parse function is more complex in rust, which is 100% abstracted away.
If you haven't read it, I'd suggest taking a look at Paul Graham's book On Lisp [0]. He says better, and with more examples than I'd provide in a comment block, what I'd write on the subject. Jump to chapter 8 for his discussion on the topic, referring back to chapter 7 if you find the macro definitions difficult to read.
[0] https://www.paulgraham.com/onlisptext.html
I think the best explanation, how Lisp macros are different from most other macro system is, that a Lisp macro is a function executed during compilation. It gets fed the parameters passed to it in literal form and then computes the source to actually compile by the compiler.
The power of this is that this function is also written in plain Lisp and it also sees all definition already existing in your Lisp image. Also, it is completely unlimited in power. So while being run, or as it is called "during macro expansion" the function could google for one of the things passed and use the result to create the output. Of course, this is not a practical example, but shows how flexible the system is.
function can do what a macro does, but it must run every single time while the macro can run just one time during the final code generation.
Let's say you have some Java-style a + b. It needs to work on native strings (concatenation) along with various kinds of ints and floats. The issue is that each one of these works differently internally, so the system is going to look at the incoming symbols and metasymbols (inferred type data) and is going to dynamically change between different functions like ADD_STR, ADD_INT8, ADD_INT32, ADD_FLOAT32, etc. If it did this at runtime by introspecting the type with some `switch(incomingType)` statement which involves all kinds of extra data and branches that clog the cache and create branches (two of the worst things you can do for performance). Instead, the macro (though they probably don't call it that) looks at everything and hardwires the correct output.
This hardwiring and its associated performance is the difference between a function and a macro. A macro `foo!(a b)` can inline into the current function while the function alternative must save all the registers to the stack and create a new stack frame which is very expensive.
The beauty of lisp macros is how functions and macros look the same. You may think that `(+ a b)` is the same as your favorite languages' `a + b`, but most lisps also allow `(+ a b c)` too. `+` is a macro rather than a function and it knows that `(+ a b c)` needs to convert to something more like `(+ a (+ b c))` and a typed lisp may go further to something like `(+f64 a (+f64 b c))`.
There's a bit of a mental model flip to make maybe.
> they're code that's generated at compile time
They're code that generates code at compile time. Macros can actively walk the AST of the parameters they process and rewrite them completely into new shapes. That transformed AST is what then actually gets compiled.
Open notepad and solve your problem in some language you make up on the fly, with macros that's now valid lisp code. A macro is a compiler and nothing more.
You could use them to:
1. Come up with an algorithm to define an algorithm.
2. Code expansion. Instead of typing out 1000 classes that are best represented as a template of a single class, you can define a macro then use it.
3. C++ at least uses them to provide generics.
4. They let you peel back the layers of abstraction to use the language itself as an API. Useful if you want to write static analysis to do analysis on code quality, security, linting, etc.
5. Anything you can imagine, it's metaprogramming.
If it helps, https://taeric.github.io/CodeAsData.html was my attempt at exploring "code as data" and what makes lisp different here. My specific focus was more to point out that "eval" in lisp doesn't just take in a string. But I think the same general points remain.
Many problems can be solved with either macros or higher-order-functions. The advantage of macros is two-fold (macros also have disadvantages, and Lisp allows you to use either solution; deciding which to use is often a matter of taste):
1. Syntax can be more familiar
2. Performance (to the extent that there is overhead for functions-calling-functions).
For example, let's consider a hypothetical lispy language that doesn't have a short-circuiting "and" operator. Macros would let you implement something to used like this:
Higher order functions would require you to do something more like: More noise, and more work for the optimizer (or in the worst-case, more work at run-time). Languages that rely heavily on higher-order-functions will tend to have terser syntax for anonymous functions. For example in javascript you might do:one way to see it is that it's a function that runs at compile time. for instance instead of dumping magic numbers/tables in a codebase you could put the code and substitute to their value at compile time.
but also it can change your code, so you get to do all the java annotation magic stuff.
The arguments presented to a macro don't have to be valid code. Your "function evaluating functions" are all individually valid functions you've composed at runtime.
=> actual, compiled Clojure function built up by `foo` parsing the string, producing a Clojure list, and calling `(eval the-list)` and returning it.I passed a string to `foo` in the example, but it could actually be anything the Clojure reader can parse even if it is semantically invalid—arbitrary Clojure data.
Macros are conceptually similar to FEXPRs in that they act like functions that don't evaluate their operands and return code. The (only?) difference is that macro are all expanded before execution/runtime.
[dead]
You do not need macros for anything. They are not the tool to "extend the language", but instructions for the compiler. And if the system does not even have a compiler, macros are useless, often incredibly stupid.
If you have an interpreter only, no compiler, using macros for metaprogramming anwyay at least prepares you for the eventuality that one day there will be a compiler. The macros will Just Work as before, only the expanded code is now processed by compiling.
Suppose you reject the idea that there will ever be a compiler. Macros are still useful for doing "compiler-like things" in the context of interpretation, like transforming code into something that will interpret better.
We can regard the interpreter as a virtual machine acting on a representation of the code; the macro system lets you manipulate the representation in a pre-computed pass, which has no further cost at run time.
If you have a code expansion pass on top of an interpreter, for supporting macros, you can use that as an excuse to perform built-in code transformations that are not macros; those enable you to have more flexibility in how special forms are implemented.
In TXR Lisp case is a macro (family) which performs certain optimizations like recognizing values in a range and emitting a table switch. This works interpreted or compiled:
It is faster to do some checks and interpret the sys:switch special form to dispatch by a numeric index than to do a large number of exhaustive comparisons.Ultimately, the way we optimize interpretation is by compiling, but it can still be worth it to have better interpretation here and there.
You don't want to do this kind of optimization at run-time; you don't want the interpreter to be evaluating the condition "are these cases integers (or characters) in a tight range?". That's a property of the syntax in which the cases are constants; it wants to be pre-computed once.
TLDR, but you are (probably) describing somekind of builtin property.
I am just saying that the end-user do not need macros for much anything, unless she has some specific optimization in mind. Bloody annoying when they start making cryptic macros to "extend the language".