Sunday, January 18, 2009

Thinking in Erlang

First, I'd like to say that my previous post was somewhat incorrect: there are MapReduce solutions written in Erlang. The great oracle, Google, has told me so. However, the ones I found are not quite what I envisioned. They all appear to be single-machine solutions, not distributed solutions. I'd like to see a multi-machine MapReduce solution written in Erlang. It looks like it could do it real well.

Anyway, the real meat of the post I is how I think when I'm thinking about writing code using Erlang. When dealing with databases, I think about how to structure the data so I get the results I want. Tables, columns, relationships, etc. With programs, I think about functions and data types. In those cases where I'm working in an object-oriented environment, I think in terms of objects. For web sites/applications, I think in terms of pages or interfaces, when dealing with AJAX requests.

Erlang is different. Instead of primarily thinking in terms of functions, data, etc., I'm thinking of in terms of processes and machines. I guess it's the fact that you can easily create and communicate with processes that it just becomes natural to think in terms of them. I'm trying to think of how to do a MapReduce style application and my first thoughts deal with how I'm going to try to distribute it across multiple machines and have them communicate with each other in a sensible fashion.

WTF?

Yeah, it's new to me. It's...interesting. I've never thought about applications like this before and it's an interesting experience. Granted, I may be biting off more than I can chew, but I think it's a reasonable starting point. Since this is an application that I want to work across multiple machines, it seems that the first thing that needs to be worked out is how the different machines communicate with each other. I know I can communicate with processes across multiple machines, but how do I do it in an organized fashion.

Needless to say, I'm intrigued.

Labels: , ,

Monday, January 12, 2009

MapReduce with Erlang?

A funny thought hit me yesterday before I forgot it and re-thought of it today: is Erlang the ideal platform for MapReduce applications? Let's see...
  1. Highly concurrent, even across multiple machines.
  2. Is functional, therefore it does have map and "reduce-like" functions.
  3. Designed for high-reliability, including the ability to upgrade without shutting down.
Interesting, no? It's definitely something to think about and I think if I do get a chance, I'll have to see what I can do along those lines.

Perhaps I should suggest that the Computer Language Shootout should have a MapReduce benchmark...

Sunday, January 11, 2009

What can a broken toilet teach you?

A couple days ago, a toilet in my house cracked and leaked water on the floor of our bathroom. My wife and I got a replacement yesterday and I managed to get it installed pretty quickly, so it was not too much of a deal. However it got me thinking a bit. This makes a good computer analogy.

You see, we got lucky. We have been working on our secondary bathroom off and on, mainly off right now, since August, and we got the old toilet re-installed and working. This worked out well as I didn't have to resort to writing my name in the snow. How does this apply to the computer world? Having one of something is nice. Having a backup is a good idea when it's a critical system. :-)