Programming in E, one year on

E is a "secure distributed pure-object platform and p2p scripting language". I've been writing programs in E for a little over a year now. Here's a quick summary of the cool features I've found so far in this surprisingly overlooked little language.

Note that the version on the E web-site is rather out-of-date and has some race conditions that may even prevent it from starting. However, the current svn version is very stable, and I've made some releases of it through 0install, so you should be fine if you use that:

  $ 0alias rune http://repo.roscidus.com/e/e-swt

This is just a quick run-through of the features. See the E tutorial for more details.

Immutable-by-default

Lists, maps (dictionaries), sets and even "variables" are immutable by default. For example:

  def list := [1, 2, 3]
  process(list)

Here, we know that the "process" function won't modify the list, because it's immutable. That avoids some bugs and makes it safe to share a single copy of lists when you need them.

This is also useful for distributed computing. When you call a remote function with an immutable list, it can be sent literally over the network. If you call a remote function with a mutable list, E passes a (remote) reference to the list. So, local and remote functions get the same behaviour, unlike many other languages, where passing a mutable list makes a copy.

All the built in types have predictable, reproducable behaviour in all cases. For example, iterating over the keys of a map will process them in the order in which they were added to the map.

Constructors

In E, an object is essentially a function that takes a verb and a list of arguments, with a bit of syntax to keep things simple. For example, this defines an object representing the point (3,4):

  def point {
    to getX() { return 3 }
    to getY() { return 4 }
  }
 
  point.getX()

There are no explicit classes in E. Instead, you wrap the object definition in a constructor function:

  def makePoint(x, y) {
    return def point {
      to getX() { return x }
      to getY() { return y }
    }
  }
 
  def p1 := makePoint(3, 4)
  p1.getX()

Notice that the constructor arguments are automatically in scope for the object's methods. There's no need for special syntax for object fields, or for copying ("this.x = x", etc).

Private fields and methods are expressed simply as oridinay definitions in the constructor, e.g. here 'count' and 'logStatus' are effectively 'private':

  def makeCounter() {
    var count := 0
    def logStatus() {
      traceln(`count is now $count`)
    }
    return def counter {
      to increment() {
        count += 1
	logStatus()
	return count
      }
    }
  }

Escape expressions

An escape expression defines a function that can be used to finish execution of the body of the escape early. For example:

  def result := escape done {
    if (x == 1) {
      done("one item")
    }
    ...
    "many items"
  }

Kernel-E

There is a simple core language called Kernel-E. This is what is actually executed. Full E is defined in terms of expansion into Kernel-E. Some examples (a full E expression followed by its expansion):

  1 + 1
  1.add(1)
 
  println("Hello")
  println.run("Hello")
 
  def double(x) { return x }
  def double {
    to run(x) {
      escape __return {
        __return.run(x)
        null
      }
    }
  }
 
  a[4]
  a.get(4)
 
  <http://example.com>.getText()
  http__uriGetter.get("//example.com").getText()

Many important constructs, such as while-loops, for-loops, break, continue and return are just expansions into this simpler form.

This makes the interpreter simpler and thus easier to secure, while the Full-E parser can be sandboxed. It also allows different versions of E to define different expansions, but still run the code with a single interpreter (which is also useful if sharing code over the network).

Quasi-literals

Expressions in other languages can be embedded into E using the backtick notation. In its simplest form (`Hello $name`), it does simple string interpolation. However, you can define your own custom ones. e.g.

  sql`UPDATE users SET email = $email WHERE id = $id`

  return xml`<email>$email</email>`

In both cases, the interpolation follows the rules of the language. The SQL example does not fail if $email contains quote characters for example, and the XML example will quote < characters if $email is a string (but not if it's of type XML-fragment).

This effectively prevents SQL-injection attacks.

Quasi-literals are also implemented as an expansion to Kernel-E:

  xml`<email>$email</email>`
  xml__quasiParser.valueMaker("<email>${0}</email>").substitute([email])

So, to create your own just define an object named something__quasiParser.

Event-based processing

Each E thread runs an event loop, pulling messages (method calls) off a queue one at a time and processing them (processing one queued item = one "turn"). To queue a message instead of doing it right away, use the arrow notation:

  counter.increment()	# Do it now
  counter<-increment()	# Queue it up

If counter is an object in another thread or on another machine, these asynchronous calls are the only thing you're allowed to do. This means that one thread only affects another thread's state between turns. In other words, you don't have to worry that another thread will change an object's state while you're using it. This avoids a lot of threading problems.

Promises

The result of an asynchronous call is a 'promise' for the result. You can schedule something to happen when the result is known, or you can pass the promise around or invoke (asynchronous) messages on it. Assume 'factory' is a remote object. This program creates a remote job, starts it processing some inputs, and then displays the results when they arrive:

  def job := factory<-createJob()
  def result := job<-process(inputs)
  when (result) -> {
    println(`Result is $result`)
  }

The "when" block schedules a callback, but execution continues after it immediately. E programs never stop mid-turn. This makes it very easy to write asyncronous code.

Cleverly, E will even pipeline these messages over the network (sending "process" immediately after "createJob", without waiting for the reply to come back).

You can also create and resolve promises locally:

  def mysteryObject
  mysteryObject<-run("Hello, World")
  bind mysteryObject := println

Here, we create a promise, queue up a "run" method on it, and then resolve it to the "println" object.

Pattern matching

E has decent pattern matching support:

  def [a, b] := twoItemList
  def [a] + rest := someList
 
  def `Hello @name` := message
  println(`Extracted name = $name`)

Quasi-literals can also be used for custom pattern matching. For example, you can do regular-expression matching as well as the simple string matching shown above.

Note: in for-loops, the body is executed only where the pattern matches:

  for `ERROR: @msg` in log {
    logError(msg)
  }

There's also a match-bind operator which can be used to execute some code only if the pattern matches, and with the matching variables set:

  if (lookupCustomer(name) =~ customer :notNull) {
    println(`ID is ${customer.getID()}`)
  }

No authority by default

Library files in E execute in an environment with no access to the outside world. By default, a library can't do anything except consume CPU and memory, unless you actually pass it some objects. For example, we know that the makeCounter library function here can't write to any files or access the network, because we never gave it those things:

  def makeCounter := <this:makeCounter>
  def c1 := makeCounter()
  c1.increment()

Instead, you can give out just the access a library needs to do its job. Here, we can see that the 'grep' library only has access to the /var/data directory, and read-only access at that:

  def grep := <shutils:grep>
  def results := grep(pattern, <file:/var/data>.deepReadOnly())

The standard library behaves sensibly regarding security at all times. For example, giving out a directory object gives access to its children, but not to its parent (".."), making it easy to partition things up.

As well as the obvious security benefits, this also makes it a lot easier to understand how a program works, since all authority must be passed explicitly from the main script to the bits that need it. Unit-testing also becomes much easier.

Library support also integrates rather well with 0install, with <environment name="mylib"> bindings in the feed file turning into <mylib> bindings in the corresponding E module. For examples of this, see the EBox sandboxing tool.

Not only does the default environment have no authority, it doesn't even provide access to non-deterministic behaviour. By default, library code does not get access to a source of random numbers or to the time of day, for example.

Easy secure, encrypted communications

It's easy to generate a key-pair and start communicating between two processes. This program exposes an object on the network, prints its address, and waits for incoming requests:

  def myObject {
    to doSomething() { ... }
  }
  introducer.onTheAir()
  def sr := makeSturdyRef.temp(myObject)
  def uri := introducer.sturdyToURI(sr)
  println(uri)
  interp.blockAtTop()

The URI printed is a <captp> URL, which includes the server's public key and an unguessable number granting access to the object. This program invokes the object's method:

  introducer.onTheAir()
  def sr := introducer.sturdyFromURI("captp://...")
  def myObject := sr.getRcvr()
  myObject<-doSomething()