Code for America: Philly 2012

This past weekend I attended the Code for America event held at Azavea’s offices around 12th and Callowhill. Their office was spacious and well accommodated, but the wireless was slow during the day.

I met a coworker, Hector, there and had some coffee and breakfast. We didn’t really get started until at least quarter of ten, a bit late. Organizers passed out Post-It notes for us to write down ideas. Various people involved in city government and other civic organizations took turns presenting their ideas of projects to work on. I took notes as best I could and when they were done, we discussed which seemed most interesting to work on. People were encouraged to stick the ideas written down onto a whiteboard at the front of the room. Half of the participants helped to group the ideas by topic.

Hector and I talked again about what to work on. Organizers meanwhile passed out colored dot stickers to use in voting for ideas. Once the herd thinned out, we voted as well. Once the voting was complete, 8 main topics/ideas were selected and participants self-selected into those different groups. We considered working on a project to make lobbyist information accessible, but decided to work on a project to publicize statistics about the plight of the more downtrodden areas of the city.

Originally, the concept was to take statistics about the city such as crime and poverty rates, and relate them to a person through their neighborhood. Someone would enter either their zipcode or various other statistics about themselves (perhaps highschool class size), and the site would use that information to give them a statistic like:

If you lived in Philadelphia, only 58 of the 100 students in your class would’ve graduated.

or

If you lived in Philadelphia, 3 of your 200 neighbors would’ve been the victim of violent crime.

We discussed this concept with other people, and someone suggested this type of site might make someone glad they didn’t live in Philadelphia rather than promote civic engagement. This helped to lead to the concept that we eventually ran with. We decided that we should use people’s friends to illustrate the statistics.

So, we went to work implementing Facebook authentication, querying the Graph API, designing the site views, and writing code. I mostly paired with Hector and focused with him on the Graph API. We needed to retrieve friend names and profile pictures. We worked on various other pieces such as sampling friends for the statistics, wiring up the views, and pushing to Heroku.

We had some last minute difficulties shortly before our demo, including:

  • Heroku having limited functionality for the free-app tier
  • Problems connecting the site on Heroku with the Facebook app
  • Wireless network issues
I am proud of the project we completed, especially since I think we all had fun working on it. There are some things that could be improved, but it was a great experience overall. It’s up now if you’d like to check it out at http://theotherphiladelphia.org. We wound up winning 3rd place. Here’s the repo location: https://github.com/CfABrigadePhiladelphia/the-other-philadelphia.

In case you’re wondering, the project that won first price was http://lobbying.ph/, the site that provides information about Philadelphia’s registered lobbyists. Technically Philly has a writeup about the event. You can find our presentation on Viddler.

OSX Lion Rubygame Install

I tried setting up Rubygame on Snow Leopard this time last year. I had many difficulties setting up SDL. I had some time to kill so I tried to set it up again, this time on my new MacBook Air running Lion.

This old blog post from 2005 helped me this time around: http://inquirylabs.com/blog2005/?p=21 to nail down the requirements. Instead of following the installation instructions here, I used Homebrew to set things up.

One of the key requirements is rubysdl, ruby bindings for the SDL multimedia library, and the rsdl gem. The following Gist is a rubysdl Homebrew formula that will help install the required packages: https://gist.github.com/1394440. The SGE formula is not part of the included formulas. I forked the following Gist to provide SGE: https://gist.github.com/1394414/.


# install the sge graphics lib
wget https://gist.github.com/raw/1394414/4a6bfa4a8bab179ebe263b1d044a4322c7628f60/sge.rb
brew install sge

# install rubysdl ruby sdl bindings
wget https://raw.github.com/gist/1394440/8a71160306ad277f08fbad3eaf028b2e8700f2f1/rubysdl.rb
brew install rubysdl

# install the required gem
gem install rsdl
gem install rubygame

Afterwards, most of the rubygame samples ran fine, but the ones requiring opengl had issues. I installed ruby 1.9.2 using rbenv, compiled a new dynamic library libruby.dylib, based on the 1.9.2 source. Then, I cloned this repo providing an upgraded ruby-opengl gem: https://github.com/theymaybecoders/ruby-opengl.

# clone the repo
git clone https://github.com/theymaybecoders/ruby-opengl
cd ruby-opengl

# make sure that the libruby.1.9.1.dylib compiled from the
# ruby 1.9.2 source is present in the gcc class path. For
# me, it was ~/.rbenv/versions/1.9.2-p290/lib/libruby.dylib
# use build option --enable-shared

rake

# build the gem
rake gem

# install the gem
gem install pkg/ruby-opengl-0.60.1.gem

Afterwards, the rubygame opengl samples compiled fine.

Philly Give Camp 2010: Part 5

The Builder

To keep the teams working as separately as possible with the least interference, we decided that the PAAL Builder, the tool used to build student schedules and lessons, would be completely HTML and Javascript. That means that there would be no server-side code, no ERB or HAML template files, and no Rails special sauce. This allowed the designers to work without developer support, and allowed developers who did not know Ruby or Rails to work solely on Javascript code. The Builder would interact with the API via ajax calls with JSON responses. A simple format for the response envelope was designed:


{
version: 0.1,
error: '',

data: {
  task: {
    id: 1,
    name: 'Brush Teeth'
  }
}
}

This is an example response upon the creation of a task with the name “Brush Teeth.” On each request, the error property would be checked to make sure that it was blank. The version property would also allow changing the API in the future without breaking the application.

In the previous post I discussed the problems of having a simple API. One common scenario is to create a task and link it with a parent task. This would happen if you wanted to create a task like “Brush Teeth” and steps underneath it like “get toothpaste,” “get toothbrush”, “brush front teeth.” Here is the code necessary to do that (I am just posting the necessary jQuery calls rather than the wrapper functions I wound up writing):

// create the task via a POST
$.post(
  '/tasks',
  taskData,
  function(envelope, textStatus, request) {
    if ( envelope.error != '' ) {
      // error
    }
    else {
      // on success, tie the newly created task
      // to a parent task
      function callbackSuccess(newTask, envelope) {
        $.post(
          '/task_parents',
          {
            task_parent: {
              task_id: newTask.id,
              task_parent_id: some_parent_id
            }
          },
          function(innerEnvelope, textStatus, request) {
            if ( innerEnvelope.error != '' ) {
              // error
            }
            else {
              // success!
            }
          },
          'json'
        );
      }

      callbackSuccess(envelope.data.task, envelope);

    }
  },
  'json'
);

Wrapper functions make this simpler to understand, but the fact that it requires two posts troubled me. Still, I avoided adding a new method to the API to do this all in one step.

When we were working on the project, the designers and another developer worked mostly on basic UI. I then worked on “installing the plumbing” as I put it, making all the API calls and using the live data. A lot of this plumbing went into the product after Give Camp. One aspect of the project I found hard after Give Camp was figuring out how a feature or UI element would work. Often, I’d be hooking up the plumbing, but be unsure of whether they wanted a stand-up shower or a bathtub. Both would get the job done, but I wasn’t sure what they had in mind, and it wasn’t clear from the design.

Part of this was due to the fact that I didn’t have the most recent design. We switched versioning systems two times during the course of Give Camp, and repository locations 3-4 times. Part of this was due to lack of notes or obvious visual stimuli. In general, after Give Camp was over, it was hard to coordinate with each other.

Philly Give Camp 2010: Part 4

Thinking about the problem domain, Trevor, Sebastian, and I decided to abstract away the complexity. The problem (as I understood it then) can be explained thusly:

Students need a way to follow a list of scheduled tasks throughout the day, to assist them in performing daily activities. These tasks can be broken into more detailed tasks with steps or choices. A choice, such as what to have for lunch, is non-linear. A choice helps a student develop his sense of independence. Each choice and task may have accompanying text and audio.

Day to day within a week, the schedule of tasks may be different. Weeks can also differ from each other.

Given these simple business requirements, we decided that a step, a task, and a choice were all the same in that they had a name, audio content, text content, and that they had a parent-child relationship with each other. We decided that all of these would be “tasks” and that there would be a “task type” field to differentiate between these. The “task” paradigm yields the following definitions:

  • A task is a task with linear/sequential sub-tasks
  • A step is a task with no sub-tasks
  • A choice is a task with non-linear, non-sequential sub-tasks

We also decided, after further discussion to make certain other things tasks. We thought that days and weeks were similar enough to a task in the parent-child relationship. We decided that

  • A day is a task with linear/sequential sub-tasks whose name is a weekday
  • A week is a task with day sub-tasks (days as defined above)

This allowed us to have a single Task model, and a Task Parent model, which greatly simplified the API. Instead of making REST calls to create a step, create a week, or create a choice, there would be just one create call, with a task type defining the behavior of the task. This would also make copying tasks and task trees easier, since we would only have to work with one model when doing a deep copy of a task tree.

Although this made the API really quick to build, when I started writing the builder code, interacting with the API, I found the API to be a bit low level. For example, I would have to make two calls to bind a new task to a user. First, I would have to create the task via the API, and then create the user-task relationship via the API. As I was writing nested callbacks with the jQuery ajax functions, I questioned our decision to make the API so simple. Still, I think the API is a thing of beauty, and I am still proud of what we did that weekend.

Philly Give Camp 2010: Part 3

As the MECA staff were describing the application, I didn’t have a clear sense of how beneficial our work would be. To illustrate how our application would help people, they showed us a video of a student going to a gym. In the first clip, the student has trouble following directions and signage about the use of gym equipment. In the second clip, the student is following auditory prompts and exercising at the gym with little trouble.  Another student was able to prepare a meal with the help of visual prompts from a PDA. It was illuminating to see the difference that prompts could in someone’s daily life and spurred us to work even harder.

We decided to split the application into 3 components:

  • A REST API written using Rails
  • A HTML/Ajax builder web application
  • An iPhone user application

We had a team of about eight people, and we knew we had to separate tasks somehow. There were some individuals with iPhone experience (mostly novice stuff like simple view demos), and some with Ruby experience, and then there were two designers with CSS and some jQuery experience.

We figured Trevor, who has extensive Rails experience should lead the API effort, and that Brian, who has done the most, albeit little, iPhone development using MonoTouch should lead the student application effort, while the designers should focus on designing and implementing some of the builder code. (The interface for the user application had already been worked out by group consensus.)

I decided I could best help with the API, so I worked with Trevor on that. Something that I think amazed all of us was that once we had assigned ourselves to teams and broken the project up, we functioned seamlessly, all of us entering the zone in our respective areas. It’s the kind of synergy and flow you get when the majority of your team knows their shit. It was an incredible experience to be a part of that.

We had a good framework and sample response API for the student interface and builder to start building plumbing by the time we left on Friday. Meanwhile, the iPhone app team had gotten some samples working using Monotouch. The builder interface was looking really sharp. We were off to a good start when we decided to break for the night around midnight.

In the next post, I will describe some of the insights behind the API.

Mediawiki-Usage/Changes Storage and Visualizations

A coworker and I have been doing some work on visualizations of wiki updates. The server-side code, written in Ruby, stores changes from a media-wiki feed.

Media-wiki Changes Feed

Media-wiki Changes Feed

I got a chance to use the Google Visualization API, which is really simple and useful for a wide variety of domains.

He also turned me on to HAML, which offers a concise, structured way of specifying HTML.

Project Page: http://github.com/hectcastro/mediawiki-usage/
Demo: http://www.angelforge.org/mediawiki-usage/

Media-wiki Changes Visualization

Media-wiki Changes Visualization

Stikker

Some coworkers and I have been working on Stikker, a cli interface for the Stikked FOSS php pastebin. I haven’t used it for awhile, but we recently set it up at work to facilitate code sharing. Unfortunately, it hasn’t been updated recently and there are some things which are broken which I’ll need to address. One of the great things about OSS is that fixing things yourself is possible!

Ruby Hiccups

I decided to try out some Ruby stuff this weekend. I wanted to do a quick little game with the rubygame gem, but I got some weird error attempting to install it on my 64-bit Snow Leopard Intel Mac Powerbook. I abandoned the effort, since I hate to expend time in system administration and installation.

Next, I decided to try running through a Rails demo. I wanted to use MySQL instead of the default SQLite, but got an error, whose text is below, when I tried installing the mysql gem. I resolved it by using the command string that follows, which I found on a MySQL website forum post.  I think the main issue was that I had a non-standard installation of mysql on my machine via macports, so I had to specify the locations of the various MySQL directories in the gem install command.

Error:
cdERROR:  Error installing mysql:
ERROR: Failed to build gem native extension.

/usr/local/bin/ruby extconf.rb --with-mysql-dir=/opt/local/bin
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lm... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lz... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lsocket... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lnsl... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lmygcc... no
checking for mysql_query() in -lmysqlclient... no
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers.  Check the mkmf.log file for more
details.  You may need configuration options.

Command:

sudo gem install --include-dependencies mysql -- --with-mysql-dir=/opt/local/bin --with-mysql-config=/opt/local/bin/mysql_config5