Tuesday, December 27, 2011

Code Google

UPDATE: Added structure and a few more ideas...


Have you ever seen someone include a giant open source library in a project, just to access some small part of its functionality? It's very important to have someone else write, debug, and maintain a large part of your code, but when you do this over and over, it can add quite a lot of bloat to an app for comparatively little gain in functionality.

This is a huge problem, as I see it, but maybe there is a solution?

What about allowing people to integrate only small subsets of modules into their code. I'm calling this idea "code google," for lack of a better term, although it's a LOT more than just search. It involves search, analysis, IDE integration, and social networking, so that library developers can track what parts of their code are actually being used. It's far from trivial to implement, but I think it could be very useful

Before our company started, a friend and I wrote a fair-sized game in LISP (a couple tens of thousands of lines) and later versions used OO languages (both class-based and prototype-based).  My friend and I seem both to be coming to the conclusion that what is happening in the Java world is not a good thing.  A lot of modern programming seems actually to be just gluing together libraries and frameworks.  Many times, frameworks are chosen because of a few of their features and glued onto other frameworks similarly chosen.

Module systems concentrate on versioning in the large -- each framework is versioned as a whole, despite the fact that so many people (apparently) pick a small set of features and use that.  Programmers achieve modularity by marking off a segment of code and declaring it to be a module.  APIs are defined explicitly by developers, but I, on the other hand, often need to use implicit APIs that the developers didn't consider to be "first class," when they wrote their code.

Code Search

What I think is needed, in addition to explicit module declaration, is a "code Google" that searches the vast sea of code and finds a small subset that does what I need at the time.  Curiously enough, this already exists, in part, for Haskell programmers, with Hoogle, that lets you search for functions based on their type signatures.  It might be hard to imagine how this could possibly work in Java, because you won't necessarily know the name of the actual type you are looking for, but it becomes more practical as Java gets closer to LISP (or Haskell), because it becomes easier to express generic behavior without knowing as many explicit names.  A "code Google" for Java could allow you to inline a method signature without knowing the name of the SAM type that the code needs.  This could be done before Java actually has a lambda syntax.

Closures will go a long way toward making Java simpler to use, since so many design patterns are trivial with them -- I made a command-driven framework in Java 1.0.4, before there were even anonymous inner classes and had to make a separate class file for each of several hundred commands.  You can look at how much much easier AWT became after the introduction of anonymous inner classes for an example of this detangling; you don't have to subclass Button, anymore, when you make a GUI, you just make an anonymous event handler inner class.  In Java 8 (or later?), each of these 1.0.4 files would just be a lambda expression -- same anonymous inner class mechanism as in Java 6, but it's a LOT easier on the eyes.

Code Snippet Retrieval

In addition to search, Code Google should be able to retrieve the dependencies of the code snippet you need and pack it all up for you in a library jar with source included.  Hoogle does not do anything like this -- it just gets you to the module version.  Again, as Java gets closer to LISP, dependencies won't need to be as tangled up because as closures become easier to use and more prevalent in code, it becomes easier to write code that stands on its own (see the AWT evolution for examples).  It's already possible to do this in Java, by using anonymous inner classes, but these are verbose and a lot of people don't like to use them (except when they write event handlers :) ).

Social Networking and IDE integration

Once you have the snippet, all jarred up, there should be a way to track the search query and the version of the module it came from, so you can automatically update your code.  When someone downloads a snippet, there should be a social networking tie-in with tools that the developers use so that they can see an annotation in their code to let them know that people are using that particular API, so that the the developer doesn't unintentionally change or remove that implicit API. If the search does fail because the code is no longer in a module, someone can still spawn another open source module to support it.  The social networking support should also indicate how many people are using a piece of functionality and allow people to attach comments to the code so the developers and other users can see them.

As an alternative to IDE integration, there should also be command line and web-based tools for your project.  Mods to Gitweb and Fossil, maybe?


  1. Code Search indexes vast amounts of code and lets people search for what they need by specifying type signatures in addition to just unstructured text
  2. Code Snippet Retrieval incorporates a snippet and its dependencies into your project's codebase and includes metadata that describes where the snippet came from (the module, version number, site, etc.)
  3. IDE Integration puts this information at your fingertips so users can be informed when new versions are available and developers can be informed which pieces of their code that people are finding important and how many people are using the code
  4. Social Networking lets people comment and collaborate with the developers to keep them connected to the users of their code

The Eclipse Code Recommenders plugin looks like it's trying to do the search portion, at least.

Tuesday, September 27, 2011

Podcast interview about Death of the Vele

I already blabbed this in a bunch of emails to people, but just in case... Scott Dunphy of New Style interviewed me about Death of the Vele, here. This was the first time I've been interviewed for anything; I was surprised I sounded coherent -- I thought it might come out a lot less cohesive than it did when I listened to it.

Thursday, August 18, 2011

Quick update on The Philosopher's Stone

I have changed TPS many times in the past year and a half, but the core idea is still to be a collaborative story telling RPG that feels like a traditional RPG. The latest version of it is called Death of the Vele and the (lame) website is here It has a link to the latest rules.

Wow, taking a look at the old rules, I see that I have streamlined the mechanics a LOT since last year. They are simpler, lighter weight, and more intuitive. There are also tools to promote more integrated and structured stories (spotlights, hitting aspects, etc.). Death of the Vele also adds a back story (which you can remove if you don't like, of course).

I'm hoping to publish Death of the Vele within the next 12 months. In the mean time, the rules are out there for free, so anyone who wants to can use them.

Here is the feature list from the latest copy of the rules:
  • Collaborative, but feels like a traditional RPG
  • Your characters are yours
  • Supports campaign-length stories
  • Game play stays at the table
  • Characters don’t always do their best -- sometimes they have an off-day, even if their stats say they’re fine, but this is your decision
  • The Right of Challenge -- if the other players don’t buy it, you can’t do it
  • Show, don’t tell -- a lot of communication happens through scenes and events
  • Answer questions with scenes -- if players want to know, scenes reveal the answer
  • Characters are naturally “woven together”
  • Your characters will get the spotlight
  • Supports visiting gamers

Friday, July 8, 2011

Applied Lambda Calculus

I made a slide presentation introducing Lambda Calculus and some functional concepts (lazy evaluation, how to represent data, etc.). It also talks about a Lambda Calculator I wrote, which does LC in three ways:
  • compiles into JavaScript
  • interprets through substitution (alpha, beta, and eta) and shows the steps
  • compiles into virtual machine code (and can execute it)
The VM still fails on some complex cases, but when I have that working, I want to make it generate native code, using LLVM.

I open sourced all this, under the ZLIB license.

Oh, I also started on a Lambda Calculus version of space invaders, which is in there, too :)

LC is a really nice language. I wasn't able to find a modern, untyped, lazy language. It seems like functional languages have all gone the way of static typing, but I'm hoping it doesn't have to end that way. Looking at what Anders Møller and co. are doing with TAJS at Aarhus University and BRICS gives me hope that some day, the computer will tell the type information to the programmer and not the other way around like it is, today.

So I wrote my own "Lambda Calculator." Really, it was so I could teach a friend about computers, but I am relatively new to LC and I thought I really ought to write something in it, which is why I chose Space Invaders. So far, I have 55 lines of space invader code written. It doesn't do all that much, but I think it's still informative. In the mean time, I had to make that slide presentation and fix bugs in the calculator.

I had a lot of fun writing space invaders and I think I'm starting to understand the power of having lazy evaluation, currying, etc. built into a language (shame on me for not using Haskell, yet). Check out the slides for more info :)