ProgBlog: 2004

Saturday, September 04, 2004

Upcoming Lisp Book

After wandering the WikiWikiWeb for quite some time, I found out about a new Lisp book to be published sometime this year:

Practical Common Lisp

(available as of 9/4/2004
for pre-order).

The author, Peter Seibel, has posted the book for review and comment on the Web.

Friday, September 03, 2004

The WikiWikiWeb: you have to love it...

Tonight, somehow (and I'm not sure how this happened), I ended up at Ward Cunningham's WikiWiki site. I won't belabor the history or background of WikiWiki, as these topics are covered in detail over there.

However, the site is a HUGE repository of stuff related to programming languages, patterns, processes, practices and many other topics about Software Development in general. It even has quite a bit of information about Lisp.

After a couple of hours wending through the myriad links that relate to Lisp, I now know what a Smug Lisp Weenie is (and I don't want to be one), some of the differences between Lisp and Scheme, why we love and why we hate it, I've picked up a number of new URLs for later study, I've found out about many Lisp imple menta tions, how to handle incremental development with Lisp and lots of other little tid-bits.

And I don't even remember what led me...

Oh, yeah. I just remembered how I ended up at Ward's WikiWiki site. I was going to check out the CLiki (the Common Lisp Wiki), so I searched Google for "common lisp wiki cliki." I found a link to the the CLiki site, was well as a link to the CommonLisp topic on Ward's site.

I followed the link to Ward's site and never made it over to the CLiki site. I got completely side tracked at WikiWiki. Well, it's certainly not the first time that's happened. ;-)

If you've not spent any time at Ward Cunningham's WikiWikiWeb, I highly recommend a thorough review. You'll find lots to see and much to learn.

Saturday, August 28, 2004

Whither .NET Lisp by Franz?

Here's a short email conversation about whether or not a .NET version of Franz Lisp exists. This started with a comment posted to this blog by Zach Beane (who apparently thinks I'm a "Smug Lisp Weenie." Is this a Good Thing? See the July 18 entry on his Live Journal blog).

Zach mentioned a posting on Google Groups discussing the existence of a .NET Lisp. I sent the following message to the author, Duane Rettig of Franz, Inc.:

I just read your discussion on the (lack of) virtues for targeting the .NET CLR for a Lisp implementation (http://groups.google.com/groups?selm=47kp4j52c.fsf%40beta.franz.com).

Based on the date of your post, and that the study you referenced was two years old then, it's been four years since that study.

Has the CLR moved any closer to being Lisp-ready? Does Franz have any current plans to implement a .NET Lisp?

Thanks for your time.

Here is Duane's response:

We are researching it. I am not a part of that research, and I am getting ready for vacation (you just caught me before going out the door), so you might want to broaden your query to either sales@franz.com or support@franz.com. If you have our product, you can also ask your account representative directly.

Duane

So, per Duane's suggestion, I asked the same question of Franz' support team. Here's their response:

Thanks for your question. We are looking at ways to interface .Net with Allegro CL, but have not made any recent formal news announcements about it at this time. If you have special needs or things you'd like to see happen in this area, please let us know.

So, they're looking into it, but have nothing to report.

Too bad.

Wednesday, August 11, 2004

The Best Laid Plans...

You know, a few weeks ago, I was hot-to-trot about learning Lisp. Sadly, two weeks of traveling for work has taken a bit of the fervor our my plans.

Recent events at work now leave me with a strong sense that I need to learn .NET: ASP.NET, .NET Web Forms, .NET Windows Forms, etc., etc., etc.... Lisp Learning may have to take a backseat.

I'm also pondering the relative merits of Visual Python (a .NET version of Python) versus C#.NET (I'm fluent in VB, and could easily transition to VB.Net, but I've always preferred the C/Java/C# style of languages over VB).

I know Python will be more productive than C#, but will it's loose-typing and late binding be up to the task of building a high-quality, commerical, end-user application that leverages MS' product automation (e.g., linking together a number of apps through automation)?

Naturally, I wondered whether there's a .NET implementation of Lisp. There is. Sort of. There's a project at Northwestern University that is an implementation of Scheme that targets .NET. It hasn't been updated since 2002, though. And it's current version is 0.5 -- in alpha status at SourceForge. We won't be building out mousetrap in Lisp.NET.

Plus, the tools for Lisp, like XUnit (look for Common Lisp), ant, etc., don't seem to exist for Lisp.

Of course, it could be that the Lisp Way isn't conducive to creating tools like in other languages. Using Macros and "reflection" by parsing Lisp code directly is probably the Lisp Way.

We shall see....

At any rate, it looks like there's no real .NET implementation of Lisp. Too bad, too. That would have been fun.

Tuesday, July 20, 2004

A sort-of recent interview with Paul Graham

I just found this while tracking down the URL for Mr. Graham's site.

OnLamp.com interviewed Paul Graham in May 2004. The interview transcript is publicly available.

Not much to add to this yet...

I've purchased Paul Graham's ANSI Common Lisp cover

.

I've haven't gotten very far into it, as I now need to find a Lisp interpreter. So, I'll start that search later today.

Lisp's syntax will take a little getting used to. In common languages like Python, C/C++, Java, etc., function calls look like this:


  doSomething(arg1, arg2)

In Lisp, function calls look like this:


  (doSomething arg1 arg2)

The parentheses surround everything and there's no punctuation, save spaces between tokens.

But I'm keenly interested. Functions are first-class objects, which means that you can pass them around like other language constructs/tokens. Everything is a list, essentially, so the basic syntax is easy to learn: everything is a nested list of nested lists. I think I'm finally starting to get a handle on this functional programming idea. More on all of this later.

Anyway, I'll write more when I have more to say... It's too soon yet.

ttfn!

Friday, July 16, 2004

Lisp in the news, again...

In the last post, I didn't say what article in SD Magazine it was that talked about Lisp.

It was in the April 2004 issue of SD Magazine, a current events article, we find this statement:

Wu [Sheng-Chuan Wu, Franz's Vice President of corporate development] says, "People say 'Java is the gateway to the Internet.' We say 'Java is the gateway to hell.' Lisp is just a better alternative to the intelligent Internet..."

The auther recounts various applications that use Lisp. The closes with this:

Clearly, it's time to take another look at Lisp. And, if anyone thinks that there are no new frontiers that can generate dynasties like Microsoft, just listen to what Bill Gates advised computer science majors at MIT early this year in a question-answer session: 'If you invent a breakthrough in artificial intelligence so machines can learn, that [will be] worth 10 Microsofts.'

Surprisingly, when I searched for this April article, another Lisp reference popped up in the May 2004 article "A Little Goes a Long Way." In this article the author discusses "tags" a new feature implimented in the Tiger version of Java (1.5). Also known as annotation, or attributes in C#, these apparently act something like Lisp's macro feature.

Go figure. More features from Lisp being hijacked by other language.

And people wonder why I want to learn this lanuage.

This will be a blog about programming, in large part a discussion and history of learning a new language.

Currently my work focuses on Python and Java, and Jython. Over the years, I've learned to program effectively in nine (10?) languages (or twelve, or more, if you count variants or embedded languages [e.g., JavaScript in ASP or JSP pages]), and have made my living from six or seven of them.

However, I've decided to learn Yet Another Language.

I find myself in the interesting position of wanting to learn one of the oldest high-level languages: Lisp. I've mentioned this to some friends, and discussed the language in passing with others. Uniformly, they've disparaged the language and my decision.

The general consensus is this: Lisp sucks.

However, in a recent SD Magazine article, Lisp is shown as making a come-back. Shortly after reading this article, I came across Paul Graham's blog where he has a couple of essays about Lisp. In Revenge of the Nerds he talks about the strengths of Lisp. I'll share some quotations, below.

I'm very much language agnostic. As with editors, UML disarmers, debuggers, build utilities and the other tools of our trade, languages are actually little more than tools. Some are better suited for certain tasks than others. Some languages are better suited to certain brains than others (my brain refuses to wrap itself around perl, for instance). Mr. Graham explains this in one of his essays.

If I had to build a large, "classic" client/server application, with fat clients, no middle tier and one or more database servers, I'd reach for PowerBuilder. Hands down. Why? 1) Because I know it, I've used it in this environment and I know its capabilities -- and it's really good in this niche. 2) Its OO aspects allow you to churn out new client-side windows, once you build a core set of base windows to inherit from, at a blistering pace.

Would I use PB for a web-app? Not on your life. JSP, J2EE, ASP, and .Net shine here.

So, why Lisp, you ask? Why a language that is predated only by venerable FORTRAN? For reasons that are highlighted in Mr. Graham's essay, some of which I will share here.

Let us start with this paragraph:

If you look at these languages in order, Java, Perl, Python, you notice an interesting pattern. At least, you notice this pattern if you are a Lisp hacker. Each one is progressively more like Lisp. Python copies even features that many Lisp hackers consider to be mistakes.

So, why are "modern" languages adopting the features of an "old" language like Lisp? Could it be that Lisp knows something the "new" languages don't?

What I mean is that Lisp was first discovered by John McCarthy in 1958, and popular programming languages are only now catching up with the ideas he developed then....

...The short explanation of why this 1950s language is not obsolete is that it was not technology but math, and math doesn't get stale. The right thing to compare Lisp to is not 1950s hardware, but, say, the Quicksort algorithm, which was discovered in 1960 and is still the fastest general-purpose sort....
...Lisp looks strange not so much because it has a strange syntax as because it has no syntax; you express programs directly in the parse trees that get built behind the scenes when other languages are parsed, and these trees are made of lists, which are Lisp data structures...
...But the biggest win for languages like Lisp is at the other end of the spectrum, where you need to write sophisticated programs to solve hard problems in the face of fierce competition. A good example is the airline fare search program that ITA Software licenses to Orbitz. These guys entered a market already dominated by two big, entrenched competitors, Travelocity and Expedia, and seem to have just humiliated them technologically.
The core of ITA's application is a 200,000 line Common Lisp program that searches many orders of magnitude more possibilities than their competitors, who apparently are still using mainframe-era programming techniques.

These excerpts speak to what makes Lisp qualitatively better than other languages. Lisp is different than most other languages: it's self-extending; you can extend the language istelf toward your problem space, making it a more effective tool for solving your problems than a generic language like C/C++, Java or Python.

Expressing the language in its own data structures turns out to be a very powerful feature. Ideas 8 and 9 together mean that you can write programs that write programs. That may sound like a bizarre idea, but it's an everyday thing in Lisp.

The most common way to do it is with something called a macro.The term "macro" does not mean in Lisp what it means in other languages. A Lisp macro can be anything from an abbreviation to a compiler for a new language.

If you know OO (or non-OO programming for that matter), you know about abstraction. Abstraction allows you to generalize the logic and features of an application into a hierarchy of layers, where each successively "higher" layer is less language-oriented and more problem/solution oriented. With each layer of abstraction, you move away from the nuts and bolts of writing code, toward defining a "language" of objects that represent your problem space.

Now, imagine that you have at your finger tips a language tool that will allow you to forge a new dialect of that language that is a direct representation of your problem space. This, for me, is the single-most alluring aspect of Lisp: I will be able to describe my problems, and their solutions, using language constructs of my devising that are built in the language I'm using. Sounds recursive, eh?

I don't yet know how to do this, and I only have the barest hint of the implications. However, as the language agnostic practioner that I am, it gives me a little thrill whenever I think about it. Describing problems and solutions at an even higher level than Objects. Cool. Very cool. Even if the language is older than I am.

In another essay, "Beating the Averages," Mr. Graham outlines how he and a partner used Lisp to gain a competitive advantage, trounced their competition and eventually got bought out by Yahoo! -- due in large part to their use of Lisp. Here are some excerpts:

In the summer of 1995, my friend Robert Morris and I started a startup called Viaweb. Our plan was to write software that would let end users build online stores. What was novel about this software, at the time, was that it ran on our server, using ordinary Web pages as the interface.

A lot of people could have been having this idea at the same time, of course, but as far as I know, Viaweb was the first Web-based application. It seemed such a novel idea to us that we named the company after it: Viaweb, because our software worked via the Web, instead of running on your desktop computer.

Another unusual thing about this software was that it was written primarily in a programming language called Lisp. It was one of the first big end-user applications to be written in Lisp, which up till then had been used mostly in universities and research labs. Lisp gave us a great advantage over competitors using less powerful languages.

He quotes Eric Raymond, on becoming a "true hacker," thus:

Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot.

Mr. Graham then observes that Mr. Raymond's "advice is good," but that it's contradictory. Even though Lisp may be a better language to use, most programmers won't use it.

However, in his Viaweb start-up, Mr. Graham and his partnet chose not to follow the conventional approach:

So you could say that using Lisp was an experiment. Our hypothesis was that if we wrote our software in Lisp, we'd be able to get features done faster than our competitors, and also to do things in our software that they couldn't do. And because Lisp was so high-level, we wouldn't need a big development team, so our costs would be lower. If this were so, we could offer a better product for less money, and still make a profit. We would end up getting all the users, and our competitors would get none, and eventually go out of business. That was what we hoped would happen, anyway.

What were the results of this experiment? Somewhat surprisingly, it worked. We eventually had many competitors, on the order of twenty to thirty of them, but none of their software could compete with ours. We had a wysiwyg online store builder that ran on the server and yet felt like a desktop application. Our competitors had cgi scripts. And we were always far ahead of them in features. Sometimes, in desperation, competitors would try to introduce features that we didn't have. But with Lisp our development cycle was so fast that we could sometimes duplicate a new feature within a day or two of a competitor announcing it in a press release. By the time journalists covering the press release got round to calling us, we would have the new feature too.

He then hits on the central reason why, even though any given language may be better than the one you're using today, you won't switch:

What's so great about Lisp? And if Lisp is so great, why doesn't everyone use it? These sound like rhetorical questions, but actually they have straightforward answers. Lisp is so great not because of some magic quality visible only to devotees, but because it is simply the most powerful language available. And the reason everyone doesn't use it is that programming languages are not merely technologies, but habits of mind as well, and nothing changes slower.

As further anecdotal evidence of the power of Lisp, Mr. Graham offers this example:

But I think I can give a kind of argument that might be convincing. The source code of the Viaweb editor was probably about 20-25% macros. Macros are harder to write than ordinary Lisp functions, and it's considered to be bad style to use them when they're not necessary. So every macro in that code is there because it has to be. What that means is that at least 20-25% of the code in this program is doing things that you can't easily do in any other language. However skeptical the Blub [a hypothetical language Mr. Graham is using to exemplify other languages - tf] programmer might be about my claims for the mysterious powers of Lisp, this ought to make him curious. We weren't writing this code for our own amusement. We were a tiny startup, programming as hard as we could in order to put technical barriers between us and our competitors.

And this is where I find myself: curious about Lisp. Very curious. Curious about macros. Curious about how to write programs formatted as compiler parse trees. Curious about how to write programs that write programs. Curious about that 20-25% of the Viaweb code that couldn't easily be written in any other language.

So, I'm looking forward to this journey. I'll keep notes here for any who might be interested. I'll talk about problems, issues, insights and other stuff.

Along the way, Python, Java and Jython may find their way onto these pages. Perhaps editors, utilities or other software we use to write software may find their way here, too.

I've wanted to do something like this for years. And a blog should allow me to do this quite nicely.

See you soon.