Adding complexity is the fact that most applications execute in multiple contexts. For example, a web app is partly executing in the user’s browser, and partly at the server. Similarly an Android program is executing in that environment.

Why does this happen? Is it good? In general, software containers exist to provide a constrained, stable, long-running context for application code. The constraint is often technology specific: apache exists to deal with HTTP requests, whereas Swing classes exist to deal with user gestures and screen painting. These containers provide an event loop and do some basic pre-processing on these events, making them more addressable.

I’m not sure if this is good, to be honest. Why should I have a different event loop for each technology? Why should the management of those events have different configurations? How do I adjust the rules of the event loop? You see, something container authors love to do is constrain behavior and then provide constrained access to that behavior via config files. This has become a widley accepted standard practice: you parameterize your static code, and let only small parts of it vary.

Learning to program is learning how to manipulate these configuration files, as much as it is learning the API or language of your chosen runtime environment.

Another, related aspect to this kind of functional language is that a edges, not nodes, are given precedence. Or rather, predicates, not nouns, are important. This is strange and perplexing: human language evolved to give nouns precedence for a good reason: nouns contextualize predicates, but only rarely does a predicate contextualize a noun. This is an important practical consideration when writing code: we lose an important contextualizing filter. In clojure, for example, you get reams of predicates in a very flat organization.

Predicates do follow one simple pattern: that they summarize their arguments. This allows code to be self-similar, which is an extremely important and good property. The summarizing and linking nature of functions is deeply important. The simplest function that links it’s arguments is ‘reduce’. The simplest function that does not link it’s arguments is ‘map’.

Writing functional code is interesting because you can write it in an almost Unix style: building more complex expressions from simpler ones. This often entails writing simple test data as a placeholder, then replacing the placeholder data with a function that returns ‘real data’ although itself uses placeholder data. Recursively you define the ‘real’ program. It’s interesting because you are, at every step, anticipating the pattern of collapse of the flower you are about to write. One interesting problem that arises is when your flower needs a parameter that you just don’t have convenient access to. You can either a) get the parameter somewhere out-of-band, b) repeat the parameter all the way down the function chain, or c) instead of flowering (executing the function), just return the flower (function), relying on something higher up to provide the missing parameter.

This brings up an interesting question of when to name a function. A name is a like a root and can be used to reduce the apparent depth of nesting. It can also be used when you notice patterns arising in your code. But names come at a cost, particularly in flat functional programs.

Data structures in FP are a little odd. They are explicitly graph-like. They are immutable. Finally, (and this is a biggie): FP data structures do NOT have operations associated with them. Functions might require parameters of a particular shape. How those parameters get in the right place is up to you. (Indeed, so is defining how “the right place” is expressed.) In any event, FP provides a lot of different ways to address data, more than usual in fact, but it’s not that crazy.

Consider a typical user-interactive program. The client software constructs messages that are handled by the server. The client interprets the server responses. If the client is written in Swing, the “steady state flowering” primarily consists of a hierarchy of Swing UI objects and various attached event handlers. The client consumes all manner and combination of user gestures and constructs a message which, if some other gesture is sent, it sends to the server. That message is composed of a relatively small hierarchy of relatively simple data. In some sense the message could be considered the result of the client function. (But really, that’s not quite right because the client persists after the message is sent. And besides, clients can send many kinds of messages.)

The “seed” of the client is the REPL. The REPL, after all, has all the power of Java and the ability to create arbitary hierarchies, send internet messages. Indeed, we might start by defining a bunch of functions we can call asynchronously. These will be our messages. Next we want to make sure we can send messages over the network. This could either wrap our messages or our messages could wrap it; it doesn’t really matter and some natural organization is sure to present itself. At some point we make a GUI. The GUI is a restricted form of the REPL, limited to composing and sending messages. The new thing is “persistent composition”. Asynchronous messages are already handled (the REPL accepts commands asynchronously) The representation of message state could simply be a list of modifications to the message. The final message state is obtained by executing all modifications (probably want to simplify for serialization).

Just as the state of the REPL is itself a persistent side-effect, so too is the GUI. The side effect is in the Java Heap (and also display memory). What you’ve done is linked the pieces of the GUI into your code’s entry points via event handlers. Each key stroke and gesture will now create a flowering and a collapse within the context of the event handler. The question arises, where does the message mutation, whatever its representation, get stored?

Basically, the message must become a persistent side-effect, with access shared across event handlers. However, we can at least limit the scope of the message quite a lot. One way to do this is to rely on Swing state (which is itself a side-effect) to represent the message. Another would be to use transactional memory. (The analogue in the REPL of this side-effect is the command line itself before the enter key is pressed.)

What do event handlers know? What do they need to know? For one thing the event handlers know about their corresponding GUI component and the event itself. Through the GUI component they can traverse the GUI tree if they need to (which is kind of nasty when you think about it.) They may also have access to the root node of the  message (which was provided at listener registration time). Of course, it also has access to the entire namespace of functions.

This kind of begs the question: what is the ideal functional GUI? Would the entire GUI, on every user gesture, destroy the message and display objects, then recreate them and replay every past user gesture adding the most recent one? That seems kind of ridiculous. What is the nature of a long-running function? How does it receive messages? Is it recursive?

What about jquery? Doesn’t that exemplify a solid pattern of embracing side-effects when they are necessary and using functional ideas when they are most useful? Yes. And actually I’m not sure if FP so much generalizes jquery as represents an isomorphism of sorts: instead of traversing DOM elts you traverse functions.

The “seed” of the server is the REPL. The REPL, after all, has all the power of Java and the ability to listen to ports.

Here is my quick-and-dirty overview on relative strength:

• .Net is very strong on Windows and for desktop apps.
• Java is strong on the enterprise server and now very strong on mobile devices via Android.
• LAMP runs most of the public web.

Overview of focus:

• .Net seems to be having some surprising success in non-windows platforms via Mono, but mainly on the client side.
• Java motion is focused on new VM hosted languages and frameworks like Groovy/Grails, Scala/Lift, etc.
• LAMP motion is focused newish languages and frameworks like Ruby/Rails, JavaScript/Node and newish approaches to storage called “NoSQL”.

Overview of hosting trends:

• Hosted services have become very popular. Java has Google AppEngine. Ruby has Heroku, etc.
• Cloud services have become very popular. LAMP and Java have Amazon EC2, .Net has Azure, etc.
• Virtualized servers are still popular. Linode, etc.
• Shared hosts are less popular, being displaced by specialized hosted services like Blogger.
• Large arrays of low-powered servers have become very popular. SeaMicro, etc.

Big Data has become an important topic in the IT world. You’ll hear a lot about MapReduce, an algorithm used to operate on large datasets, and Hadoop, an implementation of M/R in Java that is so popular it’s growing a software ecosystem of it’s own. Running Hadoop in a cloud is apparently a very popular thing to do, which gives people a batch-oriented way to analyze and link huge (many terabyte) datasets.

NoSQL alternatives to relational databases are growing in popularity for every platform, or at least in buzz. It does seem like for enterprise Java the traditional Oracle/Hibernate/SpringMVC stack still dominates. There are some interesting document, key/value and graph databases out there which are strikingly different from RDBMSs, and you’ll hear mostly about MonogDB, Cassandra (from Facebook) or perhaps FlockDB (from Twitter). Notably, Google AppEngine provides only a NoSQL datastore based on their own BigTable for application persistence.

There are some other trendy topics like cloud-hosted consumer data (Amazon Cloud Drive, Netflix Streaming, Ubuntu One), APIs and DevOps. I won’t deal with that here.

#

What about clients? In sheer numbers, clients are growing far faster than any other computer space, especially in the cell phone market.

1. Browsers. Webapps have been getting thicker thanks to great JavaScript libraries like jQuery and a new appreciation of JS as a “real” language. HTML5 will make webapps even more like desktop applications. (GWT is an interesting approach to write browser clients in Java.)
2. iOS. Apple’s surprise success with the iPhone and iPad has encouraged lots of developers to buy Macs and try their hand at Objective C for writing native apps. Under a bit of a legal cloud these days thanks to a company called Lodsys suing app developers for patent infringement.
3. Android. Google’s alternative to iOS is free and Java based and actually quite popular. Under a bit of legal cloud these days thanks to a suit from Oracle.
4. Flash. Something of a red-headed step child these days, especially since the Canvas element became popular. But thanks to great authoring tools and a legion of Flash jockeys and the prospect of Flash-to-HTML5 compilers, I don’t think Flash is going anywhere. Certainly not on iOS.
5. Desktop. Traditional native apps are generally on the decline with the exception of traditional office suites, games, and developer tools. In the enterprise there is renewed interest in VNC style remote control of hosted applications on low powered machines like netbooks. Notably, Apple has dropped official support for Java on OSX, probably because they figure there are enough Objective-C programmers now.

#

The upside is that software developers have more options than ever. The downside is that if one considers computing capacity as a general commoditiy, then this commodity is being needlessly fragmented into ever smaller shards that are largely incompatible with each other. Java continues to be the overall front-runner, as it has very good reach into every form-factor. The ability of the JVM to host other languages is extremely powerful, the trio of Groovy, Scala and Clojure being my current favorites. JavaScript is surprisingly extending it’s reach beyond the browser into every other context, including app serving and storage, two fascinating examples being Node.js and CouchDB and even is even spawning it’s own languages like CoffeeScript, ClojureScript and GWT (which is basically JavaScript hosted Java).