blog.houseofmoran.com

What’s the opposite of Nostalgia? Nowadays, I can think of an idea, code it up and have it on heroku in minutes. Working at a start-up in 2001—2004 was somewhat different.

Back then, the first task was to decide which of the many (over-priced) Cisco routers you should stick in your closet. The people at our service provider were all friendly and helpful, but the idea of personally visiting your boxes now seems mildly archaic.

Going with a cloud-based provider is still not a trivial choice. Little sysadmin-level expertise within-company means you are more beholden to AWS/Azure/Who-ever to keep your service running. Security and trust are issues to be settled, but it’s a welcome choice to be able to spend on your unique selling points.

In summary: you’ve never had it so good.

p.s. We also had to sweep the lake.

If I did have a time machine, I’d be spending a lot of my life getting slapped in the face by my future self.

Last week I spent far too long debugging why Heroku was refusing to accept my rails app when I did a “git push heroku master”. I had all sorts of theories and learned about the various reasons Heroku has for saying “Heroku push rejected, no Cedar-supported app detected”. The actual reason? I was on a local git branch and Heroku was being pushed to from my master branch, which was not a rails app.

SLAP!

Incidentally, future Mike, if you do want to push from a non-master branch (e.g. “stuff”), then do this:

git push heroku stuff:master

I’ve been playing with data from the Project365 group I am a member of. I would usually do this analysis using some combination of perl, ruby, and unix shell (sort, uniq etc). This time I thought I’d stretch myself a bit.

These are the two books I’ve been using to learn “R”. Why R? Apart from being ‘interesting’ to google for, “R” is distinguished by often popping up on the visualisation blogs I subscribe to. It also offers another way to slice and dice data. Although I can achieve some of these things in a combination of Excel and shell, it always feels very brittle and throwaway. R offers the hope of some re-usability, or at least, repeatability.

Taking the Project365 data as a driving example, I plunged into learning R.

From a first pass, “R” is definitely ‘new Lego’: lots of large pre-formed parts that look great, when they fit. However, it doesn’t really feel like a cohesive language. For example, what do you think as.vector() does? It sounds like it should always return a vector, yes? Nope, it tries its best, and if it doesn’t work, well, then you get back what you put in. There may be a perfectly valid reason for this, but it still feels ‘perly’. I like my languages to tell me when something hasn’t worked.

If you follow along with the documentation, then you can get a lot done. However, I found myself often falling flat whenever I tried to generalise; it did not follow the principal of least surprise.

There is power there but it is often frustrating getting to it. I’m going to leave it for a bit before I dive back in.

This is the first time I’ve been on holiday in Germany and not launched Eclipse. Also: not a single line of Java. It’s refreshing.

I now know I can write incredibly slow Ruby, albeit for a good cause (“The Diagonal Method vs. The Rule of Thirds”). I rediscovered a love for Unix. I finally admit: maybe this Rails thing is quite good, and: git is better than svn.

Oh ye, and I got loads of presents ;-)

More specifically, how can I find this out using freebase? Turns out the steps you should take are:

1. Find the film you want to use as a constraint.

2. Slap the film id and type as constraints into the query editor, using “*”: null to get all properties of that type.

3. Expand your query until you find the actor somewhere within the tree (keep on using "*": null at each level down to get more detail)

4. Select the actor leaf in the query and use “Turn inside out” to arrange the query so that it will return actors constrained by film.

5. Repeat the above for the tv series

6. Combined the film and tv series constraints into one query.

7. Voila, you have all actors who appeared in Tron and Babylon 5.

You can easily extend this to do things like finding all Science Fiction films that anyone in Babylon 5 has starred in.

What you definitely shouldn’t do is:

1. Find the MQL documentation

2. Read it

3. Wonder why you need to know so much about type theory

Now, that was a wasted hour. Honestly, just go straight to the Query Editor. You know it makes sense.

It’s LSystem Sunday! I was curious about investigating an assertion that I’d made the other day: that the dragon curve loops back and traces over itself. Turns out, I was talking nonsense:

The above shows the dragon curve at it’s 7th expansion, with an animation of how this is drawn on the right. As you can see, it has overlapping points, but not overlapping line segments. I was wrong.

Btw, yes, I know, but today is still logical Sunday for me.

Note: animated gif rendering supported by http://elliot.kroo.net/software/java/GifSequenceWriter/

Btw, doing conditional GET in Jersey using ETags is piss-easy. Combine this with apache server-side caching and you have a very nice persistent caching solution. I can now do some pretty inefficient LSystem drawing without worrying about my server melting ;-) For example:

LSystems have once-again distracted me, this time from German homework:

---

---

---

More arsing around with Scala. This time, trying to reimplement lesscss. This is about two hours worth of hacking:

object LessParser extends RegexParsers {
  def parse(spec: String) : Sequence = {
    parseAll(sequence, spec) match {
      case Success(l, _) => l
      case Failure(m, _) => throw new IllegalArgumentException(m) 
      case Error(m, _)   => throw new IllegalArgumentException(m) 
    }
  }
  def sequence = rep(namedNested | atom) ^^ {
    Sequence(_)
  }
  def namedNested = nestedName~"{"~rep(atom)~"}" ^^ {
    case name~"{"~contents~"}" => NamedNested(name, contents)
  }
  def nestedName = """[\.\#][a-zA-Z]+""".r
  def atom = variableDef | variableUse | text
  def text = "[^@{}]+".r ^^ { Text(_) }
  def variableDef = "@"~variableName~":"~variableValue~";" ^^ {
    case "@"~name~":"~value~";" => VariableDef(name, value)
  }
  def variableUse = "@"~variableName ^^ {
    case "@"~name => VariableUse(name)
  }
  def variableName = "[a-z-]+".r
  def variableValue = "[^;]+".r
}

abstract class Tree {
  def eval(defs: Map[String, String], out: StringBuffer) : Map[String, String]
}
case class Sequence(parts: List[Tree]) extends Tree {
  def eval(defs: Map[String, String], out: StringBuffer) : Map[String, String] = {
    parts.foldLeft (defs) { (accumDefs : Map[String, String], part : Tree) =>
      part.eval(accumDefs, out)
    }
  }
}
case class NamedNested(name: String, contents: List[Atom]) extends Tree {
  def eval(defs: Map[String, String], out: StringBuffer) : Map[String, String] = {
    out.append(name)
    out.append(" {")
    contents.foldLeft (defs) { (accumDefs : Map[String, String], part : Tree) =>
      part.eval(accumDefs, out)
    }
    out.append("}")
    defs
  }
}
abstract case class Atom() extends Tree
case class VariableDef(name: String, value: String) extends Atom {
  def eval(defs: Map[String, String], out: StringBuffer) : Map[String, String] = {
    defs + (name -> value)
  }
}
case class VariableUse(name: String) extends Atom {
  def eval(defs: Map[String, String], out: StringBuffer) : Map[String, String] = {
    out.append(defs(name))
    defs
  }
}
case class Text(value: String) extends Atom {
  def eval(defs: Map[String, String], out: StringBuffer) : Map[String, String] = {
    out.append(value)
    defs
  }
}

object Less {
  def main(args : Array[String]) : Unit = {
    println(LessParser.parse("@nice-blue: #5B83AD;"))
    println(LessParser.parse("@nice-blue"))
    println(LessParser.parse("@nice-blue: #5B83AD; @nice-blue"))
    println(LessParser.parse("@nice-blue: #5B83AD; #header { background-color: @nice-blue; }"))

    var fullExample = """
    @nice-blue: #5B83AD; 
    #header { background-color: @nice-blue; }
    """

    var buf = new StringBuffer
    LessParser.parse(fullExample).eval(Map(), buf)
    println(buf)
  }
}

Output:

Sequence(List(VariableDef(nice-blue,#5B83AD)))
Sequence(List(VariableUse(nice-blue)))
Sequence(List(VariableDef(nice-blue,#5B83AD), VariableUse(nice-blue)))
Sequence(List(VariableDef(nice-blue,#5B83AD), NamedNested(#header,List(Text(background-color: ), 
VariableUse(nice-blue), Text(; )))))
#header {background-color: #5B83AD; }

It does variables and the beginnings of mixins. It’s actually quite lame e.g. it throws away whitespace. Hohum, time to learn more about parsers in Scala…

So, another summer holiday is at an end, and another programming interval. This time I was using my LSystem Explorations as a way to get more practice with Scala and Android.

Random aside: I find it hard to get excited about non-JVM languages nowadays. I develop on a Mac and deploy to Linux, Windows (sometimes) and now Android. I really don’t want to worry about whether the language or platform I’m using has native components hidden somewhere inside. I’ve spent enough time doing “./configure” in the past to be seriously bored with non-portability. That was even going between two Unixes (Solaris and Linux)! I know Java is really “write once, debug everywhere” (e.g. Graphics2D problems on the Mac), but at least these are problems in particular library features, not a total inability to run.

Anyway, what have I learned? Scala in Eclipse has become usable enough for me with the latest Eclipse plugin for me to use Scala as my main Programming Language. It’s still the case that the Scala builder gets confused too often and you have to do a Project->Clean. This even happens with pure Scala builds, and not just with mixed Java/Scala projects. The Editor integration is also flaky e.g. copy and paste between files can cause errors, and you can forget about consistently completing a Rename Refactor. However, even given all that, the ability to mix Scala and Java in one project is what really does it for me. Overall, it’s still basic compared to the JDT. I don’t yet want to push it for my day job, but for personal projects, where you can absorb the extra time fixing things if you break them, it’s fine.

Scala as a language is really growing on me. As previously mentioned, I was working through the “Programming in Scala” book. Even though Scala fundamentally has a small core, it has the appearance of a much larger language. I really needed to read most of this book to get past quite a few roadblocks, but I did manage it.

For example, I’d looked at parser combinators before but found the examples posted at the time and the scaladoc impenetrable. This is not too surprising as operators in Scala appear as methods on the class, so class-level docs can’t really tell you much about how to use them in context. The JSON example in the book was large enough to give me sufficient transferrable knowledge to write a simple parser for LSystem descriptions:

import scala.util.parsing.combinator._



object LSystemLanguage extends RegexParsers {  

  // This parses "B; F -> FF, B -> F[+B]F[-B]+B; FB" into an LSystem

  def parse(spec: String) : LSystem = {

    parseAll(lSystem, spec) match {

      case Success(l, _) => l

      case Failure(m, _) => throw new IllegalArgumentException(m) 

      case Error(m, _)   => throw new IllegalArgumentException(m) 

    }

  }

  def lSystem = axioms~";"~rules~";"~visible ^^

    { case axioms~";"~rules~";"~visible => LSystem(axioms, rules, Set() ++ visible) }

  def rules = repsep ( rule, "," )

  def rule = variable~"->"~rep( variable | constant ) ^^

    { case lhs~"->"~rhs => new Rule(lhs, rhs) }

  def axioms = rep(element)

  def visible = rep(element)

  def element = variable | constant

  def variable = 

    "[A-Z]".r ^^ ( (s : String) => { Var(s.charAt(0)) } )

  def constant = 

    ("[a-z]".r | literal("+") | literal("-") | literal("[") | literal("]") ) ^^ 

    ( (s : String) => Constant(s.charAt(0)) )

}

Apologies for the probably non-idiomatic Scala (and lack of syntax-highlighting on this site, I’ll upgrade one day, honest), but it has the important property that it works, as opposed to most of my previous attempts. From further reading elsewhere, I get the impression that they are papering over some details (the !RegexParser combines both tokenisation and parsing, which they don’t cover separately from what I read of the book). However, who cares, it works!

I’m now seriously considering supporting the full Fractint LSystem syntax, which I wouldn’t have considered doing in Java. Obviously, you can do all this with JLex etc, but that means going back and forward between languages and probably, gack, writing an ant script. Seriously, I spent more time during my holiday getting a simple ant task to work than I did writing this parser.

Modulo one instance where Eclipse crashed and I had to rebuild my workspace, programming in Scala using Eclipse has been fun. The combination of an object-functional language, a half-decent IDE and rich platform libraries is very powerful. It’s important to emphasize what this can give you. For me it means that if I have a spare hour or so in which to add a new feature or tweak an existing one (like extending the parser) I’ll consider doing it and expect to complete it, and enjoy it.

Technically, I’m capable of getting the same feature implemented in Java, in combination with other tools. However, I’d start out with a big “harumph” as I picture the fields of verbosity laid out before me. It would definitely not be fun.

This brings me to Android. The Scala compiler spits out JVM bytecodes. The Android platform, though not strictly a Java VM, does claim to support Java class files. Theoretically, I should be able to write a library in Scala and use it on Android.

I am explictly not trying to write an Android app in Scala. Instead, I created an LSystem library, written in Scala, which exposes itself solely through these interfaces:

public interface LSystemViewFactory {

  public LSystemView evaluate(String spec, double angle, int rounds);

}



public interface LSystemView {

  public void renderTo(Renderer renderer);

}



public interface Renderer {

  interface Point {

    double x();

    double y();

  }



  public void addPath(Iterable<Point> path, int length);

}

In this way, I was trying to avoid relying on any special help that could come from using one Scala project with another. It does work, in a way.

Warning: what follows is a history of my stubborn depth-first method of getting something showing on my phone. There may a better method.

In Eclipse 3.4, you need only make an Android project depend on a Scala project and everything will compile. However, when you run it on the phone, it will be missing the required Scala class definitions. You can download the scala-library.jar and add it as a required library. The Dex compiler then starts churning away on this file, heating up your lap as a side-effect, and then fails with a compiler error. Schäde.

There is some joojoo in “scala-library.jar” that the Dex compiler just doesn’t like. I think it is an unsupported annotation of some kind, probably somewhere in scala.xml.transform, but since it takes a minute or so for dex to do its thang to this library, I decided to adopt a slash and burn approach, by quickly removing anything I knew I didn’t need on the phone. So, goodbye to any of scala.actors, scala.reflect, scala.testing or scala.xml.transform.

Random Aside: it was at this point I had the afore-mentioned elbow injury^H^H^H^H…problem with ant. It is frustratingly hard to get ant to behave in any sort of sensible and consistent manner. In this case, I had the temerity to require more than one exclude pattern (for the multiple packages). The docs claim that you can put them in one string, comma-separated or space-separated; obviously multiple elements are for wimps and no-one will ever need a pattern that contains both a literal space and a comma. I tried this, and it didn’t seem to work. There is, of course, no “verbose” option on the “jar” task so I couldn’t confirm whether my patterns were getting picked up. I ended up using a separate “excludesfile”. A muted “hurrah” and I’m back to work. It’s at times like this that I briefly consider moving back to Make. But only briefly.

This did the trick and dex now compiled the scala-library.jar. My Android Activities and Views (which depend, indirectly, on Scala classes) successfully resolved all references and I could run my app.

All is not satisfactory, however. As previously-mentioned, the dex compilation phase now takes a minute. This compilation happens in the background whenever you make a change to any file, whatsoever. That’s right: tappitty-tap, muscle-memory says “auto-save”, and bzzt you’ve got a minute penalty. On top of this, the dex loader on the phone takes about 10 to 15 seconds to parse the app description. I haven’t yet packaged the app and used it outside of the debugger, so I can’t tell if this 15 second penalty is incurred ever time you launch it.

Both the IDE and run-time problems should be solved by running proguard over it. If I choose the set of interfaces above as the ones to preserve, then I should be left with a much smaller working set of classes. I didn’t try this, partly because I ran out of time, but mostly because it would probably involve writing an ant build file. Can you tell I don’t like ant?

Phew, so after all that I have a (slightly buggy) drawing of an LSystem on my phone:

For reference, here’s the same LSystem rendered using the same library, but with a Java2D backend: