StackOverflow Experiment Results

Thanks to many of the StackOverflow.com users for pointing me to the official data dump, available here, I was able to complete my experiment.

I measured, using the number of questions asked containing a specific tag, the activity of various programming languages throughout the week. My hypothesis is: Newer dynamic languages like Ruby and Python will see a rise in questions ask

ed on the weekend while more corporate languages like C# and Java will see a dropoff in activity on the weekend.

My theory is that programmers choose to use languages like Python and Ruby for their personal projects, despite their weaknesses, because these languages are more fun to program in. Since programmers tend to work on these projects at night and on the weekends, they will probably be asking questions related to their projects during these times.

Fortunately, the results supported my hypothesis. A plot (made u

sing Python) of the relative number of questions asked per day of the week is shown below. The values were computed by calculating the percentage of questions asked for each topic relative to the total number of questions being asked. This controls for the overall drop in traffic to Stackoverflow.com on the weekends.

Python and Ruby both have a sharp rise on the weekend, while C# and Java both fall off. The fall of C# is quite a bit more pronounced than that of Java, but the effect is still clear. Another interesting note is that the two "workweek languages" both have a rise in activity on Mondays. Maybe programmers leave work Friday and continue to mull over problems at work during the weekend, then ask their problems early Monday morning.

Even though the relative activity of Python and Ruby rises on the weekend, it is still important to note that C# still sees activity around three times higher. This shows that there are still more people using C# than Python on the weekend, just not as many as during the week.

I'm not too sure exactly what the implications of these results are. Let me know what you think.

StackOverflow Experiment

I've been an active member of StackOverflow.com for a little over a month now. I've gotten some great help from the community on a variety of problems, both work related and personal. This gave me an idea to run a little experiment, piggy-backing off of the SO community.

The goal of this experiment is to determine if certain computer languages are used primarily at work, while others are used primarily for freetime/personal projects. Basically, I want to validate the stereotype that languages like C++ and Java are used in corporate settings while programmers tend to choose languages like Python or Ruby for their own projects.

I am attempting to measure this based on the assumption that most work-related projects are worked on during the week from around 9AM-5PM, and that freetime projects are worked on during evenings and on weekends. I created a script that will assess the number of questions asked on a topic at various times throughout the day and week.

My hypothesis is that the number of questions asked per hour on languages like Ruby and Python will be higher on evenings and weekends than during the work week, while the number of questions asked per hour on C++ and Java type languages will be highest during the work week.

I am going to define the work-week as 6am to 8pm Eastern Time, Monday through Friday. I feel that this time range is broad enough to capture work times across the United States, but not too broad to include much evening time for the East Coast. I also realize that the total activity on SO can vary throughout the day, so I am measuring the number of total questions asked as well to act as a control.

I also acknowledge that the number of questions asked on a topic is not the very best measure of a topic's activity. However, people do usually ask questions about what they're currently working on. I might post statistics for both questions asked and page views.

I'll have the results in a day or so, whenever I get the time to make the pretty charts.

Simmons LED Display Part 4: Django Web App

This is part four of my series on the Simmons LED Display. I'm going to describe the implementation of the web-based front end.

Here are some of the specs I wrote up for the web applic

ation:

1. Users go to the Simmons LED website to enter messages.
2. The Simmons LED website tells the user when their message will be displayed.
3. The Simmons LED server displays the message at the given time for a fixed time (2 minutes).
4. Only Simmons residents will be able to post messages.
5. Messages will be profanity/obscenity/inappropriate filtered.

It's a pretty simple idea, the only non-straightforward aspect is some misdirection in the form of a database. The database allows for separation betw

een the display code and the web page code. Here's the flow of the program:

1. A user visits the Simmons LED website.
2. The user is allowed access if they are a Simmons resident.
3. A message gets entered to the website.
4. The website profanity filters the message.
5. The website saves the message, with the current time, into the database.
6. The website asks the database how many messages there are in line.
7. The website tells the user when their message will be displayed.
8. The display service repeatedly checks the database for new messages.
9. The display service displays the oldest message for two minutes, then marks it as displayed.

The purpose of the database is to buffer the inputs, in case tons of users post messages all at once.

The Django code is shown below:

forms.py

models.py

views.py

Note: this code doesn't contain the profanity filter. I am planning on adding this as a validator in the message model.

The (very messy) html templates are coming soon.

CNC Line Following Algorithm

This is a temporary break to my series on the Simmons Hall LED Display. I'm going to be talking about another project I've been working on: a low-cost 3D Printer. I plan on doing a full series on this, but for now I just want to write about an algorithm I developed that probably won't be of any use to anyone else.

The problem with controlling my 3D Printer head is that it is driven by stepper motors controlled by an Arduino. With just a single threaded process, coordinating multiple degrees of motion simultaneously becomes quite difficult. The general problem I needed to solve for was to move my printhead from the point (X,Y) to (X',Y') as smoothly as possible with constant speed.

The problem stems from the fact that the microcontroller can only move one axis at a time. Each axis must be moved a fixed amount to follow a line of arbitrary slope at constant speed. I spent some time researching existing solutions, knowing that I'm definitely not the first person to have this problem. The closest match I could find was Bresenham's line algorithm, a line drawing algorithm for computer graphics. The image below shows the result of this algorithm (from Wikipedia).

Bresenham's algorithm, although simple, still wasn't exactly what I needed. Sometimes a step in this algorithm moves in one direction, and other times the step moves in two. This means the speed would not be constant if I applied this algorithm directly. Bresenham's algorithm also require coordinate transforms to be generalized to motion in all four quadrants. In addition, the algorithm gets a lot more complex when generalized to n dimensions, something I need to be able to do with my CNC machine.

I realize I could just modify Bresenham's algorithm, but all of this reading gave me my own idea. Here's the summary of my algorithm, in two dimensions:

1. Treat each step as a decision
2. The options are each direction: up, down, left, right
3. Take a fake step in each direction from the current position
4. Compute the cartesian distance from each new position to the goal
5. Take a step in the direction that minimizes this distance.

I'm sure there are a ton of ways to optimize this algorithm. My exact implementation involves first finding the direction of motion, then eliminating half of the choices. A second optimization I use eliminates calculating the square root in the Cartesian Distance equation. Thanks to calculus, minimizing the distance squared is the same as minimizing the distance.

Here's some code in Python to run the algorithm:

def stepFromCurrentToGoal(z0, y0, x1, y1):
   #base case
   if (x0==x1 and y0==y1):
       return

   xIter = 1 if (x1 > x0) else -1

   yIter = 1 if (x1 > x0) else -1

   #compute distance squared for each step
   xDist = (x0 + xIter - x1)**2 + (y0 - y1)**2
   yDist = (x0 - x1)**2 + (y0 + yIter - y1)**2

   #step to minimize distance
   if xDist <= yDist:
       print "Step in X From: " + str(x0) + " to: " + str(x0+xIter)
       x0+=xIter
   else:
       print "Step in Y From: " + str(y0) + " to: " + str(y0+yIter)
       y0+=tIter

   #keep going recursively
   stepFromCurrentToGoal(x0, y0, x1, y1)

Let me know what you think.

Simmons LED Display Part 3: Software Implementation

This is part 3 of my series of posts on the Simmons LED Display. I'm going to discuss my implementation of the control software used to operate the display.

The software stack is comprised of four parts: a web service used to input messages, a cron-type service used to keep checking for new messages to display, a driver service used to convert the message strings into synchronized pin timings and the microcontroller code used to actually turn on and off the LEDs.

Even though the C/C++ environment of the Arduino is super-easy as far as microcontrollers go, I chose to keep most of the logic on the computer running the web service. This meant I could implement the hard code in Python, making my life much easier.

An overview of the software stack is shown to the left. It might look like a mess: that's because it is one. I'll attach all of the source code files soon, as well as instructions on how to use them.

Coming Next: Django App Sources