When it comes to extensibility, Emacs and Emacs Lisp have one of the most compelling stories of all time. (In fact, is there anything that beats them?) If Andy is right that Guile can do for GNU what Elisp did for Emacs, only better, it would be downright amazing.
I wonder, though, whether some things that made Emacs/Elisp so successful would get lost in the switchover. For example, while dynamic scoping is bad for writing robust complex software, it does make Emacs easier to extend.
Emacs is not really an application or a platform, it is a framework: a framework for a text editor, one that comes with a well-thought-through model and a flagship application (Emacs itself) that doubles as a development environment. I don't mean "model" as in MVC, but in the sense of a consistent set of concepts that can be composed to do things. In Emacs those concepts are things like buffers and windows and text properties. Emacs has one of the best models I've ever seen; it should be taught as an example of how to do this kind of design. Emacs on the surface may be clunky and unlovable, but dive underneath and its world is beautiful.
When people extend Emacs, they're building programs out of that model using tools designed for building programs out of the model (which themselves were built out of the model). If you shift from Emacs to GNU as a whole, you're moving from that single -- self-consistent and composable, not to mention interactive -- design context into a hugely diverse environment with no common design context at all. If Emacs is a planet, GNU is a universe. It's not clear how well Emacs' extensibility magic can be reproduced at that level. If the benefits come mostly from the language, Andy may be right. I hope he is. But if the benefits come mostly from Emacs' framework and interactive environment (both of which are inseparably bound up with the language), then the magic is likely to be left behind with them.
Still, it's hard to argue against his case for choosing the most powerful possible extension language so as to maximize the extending that people will be able to do with it. To argue against this in favor of familiarity is particularly to miss the point.
How does dynamic scoping make Emacs easier to extend?
It gives you easy access to everything that's going on at runtime. For example, many Emacs buffers are read-only or have read-only bits of text (e.g. the prompt in a Slime REPL). If your code tries to change that text you get an error. But there's a trivial idiom for getting around it:
Any code invoked during the execution of this form, such as the system code that enforces read-onlyness, will now behave differently.
Because Emacs is dynamically scoped, everything works this way. You don't depend on the author of some module having said in the past, "Someone might want to modify this variable at runtime someday, so I'll declare it to be dynamic." Everything is readable and writeable by default. This is a big deal for Emacs' extensibility.
Dynamic scoping isn't good in general, but Emacs has benefited handsomely by making this tradeoff in its own case. There are thousands of useful Elisp modules that interoperate with remarkably little fuss, all extending the same system. Yet the language is dynamically scoped and has zero encapsulation. Most people, I think, would deny that that was possible if it didn't exist. Many people still deny it is possible even though it does!
I don't think dynamic scoping makes it possible -- I think it is used simply because it is there. If it wasn't there, some other solution, just as good (if not better) would be used.
When writing e-lisp, I find dynamic scoping limiting, and using lexical-let, especially when I want curried/partial-application, is extremely annoying.
For me, extending emacs is harder because of dynamic scoping. It is entirely possible to have code be executed in some sort of "Reader" monad/env, and execute it in a different than default env -- which gives you "dynamic scoping" on whatever input you want, without contaminating every other name.
I think almost every negative feature you could possibly implement would find "good use" that defended its existence (Dynamic scoping, $_ in Perl, implementation inheritance, etc).
In a system with dynamic scoping, you have access to variables which otherwise you might not. If it were written in scheme, it might be more difficult to know how many buffers are currently open or some other text-editory value than it is in emacs lisp.
Interestingly, from what I've seen dynamic scoping hasn't hindered emacs lispers at all. They have some sort of variable naming system that is specific enough that confusion is fairly rare (I've never seen anybody talk about variable naming conflicts, at least). For example, the value of the (chosen randomly out of my .emacs) ``ido-auto-merge-work-directories-length'' variable seems unlikely to be changed accidentally.
That being said, Scheme/Guile have dynamic fluids which allow for the same sorts of things to be done. Personally, I feel like these would have to be used heavily in order to for a hypothetical lexically-scoped guile-based emacs to work, but maybe thats just because I feel like dynamic scope makes things easier to extend :p (at least, in this sort of program).
It seems dynamic scoping is not really an issue for Guile to host Emacs Lisp.
See the paragraph from the README[1] of the current implementation:
flet and flet*:
---------------
These constructs behave exactly like let and let*, except that they bind
the function slots rather than the value slots, and so make dynamic scoping
available for functions, too.
To me this sounds like the dynamic scoping issue has already been solved.
As far as I know (but I'm not the person to ask), their goal is to swap a new (Guile-based) Elisp implementation into Emacs that is 100% compatible with existing programs. That seems like a sine qua non if their plan is to have any chance.
I wonder, though, whether some things that made Emacs/Elisp so successful would get lost in the switchover. For example, while dynamic scoping is bad for writing robust complex software, it does make Emacs easier to extend.
Emacs is not really an application or a platform, it is a framework: a framework for a text editor, one that comes with a well-thought-through model and a flagship application (Emacs itself) that doubles as a development environment. I don't mean "model" as in MVC, but in the sense of a consistent set of concepts that can be composed to do things. In Emacs those concepts are things like buffers and windows and text properties. Emacs has one of the best models I've ever seen; it should be taught as an example of how to do this kind of design. Emacs on the surface may be clunky and unlovable, but dive underneath and its world is beautiful.
When people extend Emacs, they're building programs out of that model using tools designed for building programs out of the model (which themselves were built out of the model). If you shift from Emacs to GNU as a whole, you're moving from that single -- self-consistent and composable, not to mention interactive -- design context into a hugely diverse environment with no common design context at all. If Emacs is a planet, GNU is a universe. It's not clear how well Emacs' extensibility magic can be reproduced at that level. If the benefits come mostly from the language, Andy may be right. I hope he is. But if the benefits come mostly from Emacs' framework and interactive environment (both of which are inseparably bound up with the language), then the magic is likely to be left behind with them.
Still, it's hard to argue against his case for choosing the most powerful possible extension language so as to maximize the extending that people will be able to do with it. To argue against this in favor of familiarity is particularly to miss the point.