Now, almost three years later, I finally made it out to Tahoe with Tim McKague, Sally Lee, and Sean Dy. Stanley Chan, whom I worked with at TellApart last Summer, also managed to join us for about half the trip. We all went to Squaw Valley to ski and snowboard for the entire weekend.

My track in Winter sports isn’t really the best. I have only went skiing twice before then and performed so poorly that strangers were yelling at me telling me that I “completely suck”. Nevertheless, I decided the “bite the bullet” and see how things go.

On Saturday, the first day of skiing, I think I fell ~50 times. Everyone helped in instructing me, especially Stan. I think Stan was bored enough that he decided to document my struggle. Like this one:

And this one:

 

On the second day though, I felt a lot of improvement and I didn’t fall nearly as often. I managed to join Tim and Sean on their skiing/snowboarding adventures. Sally decided to snowboard for her first time that day and was left to struggle for the rest of the day on the green hills. I guess Winter sports aren’t that bad after all.

The CAPTCHA in the screen above immediately caught my attention. The distortions in it seemed very simple, the text was not warped in any form and no overlap between characters.

The following is a URL for one of the CAPTCHAs:

http://www.aljazeera.net/Sitevote/SiteServices/
Contrlos/SecureCAPTCHA/
GenerateImage.aspx?Code=EANmyyXghpajFhOX6rCRKQ==&Length=4

Opening the URL above and refreshing the page a few times gives the following CAPTCHAs:

Aljazeera’s CAPTCHA can easily be broken by doing the following:

1. Removing the dashed grey lines
2. Finding the characters in the image
3. Separating the characters in the image
4. Classifying each character

I’ll be using Octave/Matlab for the above tasks and will be explaining my algorithm using the following CAPTCHA as an example.

I’ll first begin by loading the CAPTCHA in Octave/Matlab using the line of code below. I have already saved the CAPTCHA image in a file called captcha.jpeg.

N = imread('captcha.jpeg');

N is a matrix where each element in the matrix corresponds to the color of the pixel at the corresponding location in the image. For example, the matrix below is a submatrix of N corresponding to the letter ‘J’ in the image. If you look closely, you can almost see the letter ‘J’ in the matrix.

255  249  249  255  255  255  255  255  255  255  255  255
189  255  246    7  251  255  249  255  255  246  249  255
170   76   11    0  253  255  255  250  245  255  255  245
254   24   59   60  255  233  255  255  255  255  230  255
246    1    9   96  170  255  238  255  247  255  246  255
255    2    0    9   60  251  240  255  255  255  255  250
238  255    2    0    0  206  154  251  241  245  255  254
255  253    5    0    9   13  193  128  246  255  253  255
255  243    0    4    0    0    9  241  255  247  255  255
245  251  255    0    0    0   10    0  255  255  246  255
255  255  233   21    0    7    0    2    0  255  245  255
255  246  255  231   11    6    0    5   11    1  245  243
255  253  252  255  248    0   15    0    0    0  255  255
255  255  240  243  255  254  250   11    8    0    0  248
238  251  255  255  255  250  255  241    0    0   10  255
255  252  255  238  244  255  255  240  255   18    0  255
255  246  255  253  255  242  251  255  244    0   12    0
255  251  255  249  254  241  255  253  255    0   17    0
255  255  244  255  255  246  255  250  246    6    0    9
255  243  255  249  255  253  255  243    8   11   12  254
251  250  255  249  245  233  255   10    1    0    0  255
246  255  249  251    8   15    0    0    0    0  255  243
246  255    0   11    0    5    1    0  255  255  237  255
254    0    5    4   13    0  244  255  250  255  255  248
255  255  255  247  249  255  255  244  255  247  255  255

A value of 255 corresponds to a white pixel while 0 corresponds to a black pixel. All values between 255 and 0 exclusively correspond to shades of grey.

1. Removing the dashed grey lines

The grey lines serve as a distortion to make the image harder for a computer to read. We can remove all shades of grey using the following line of code:

N = N > 100;

The line of code above changes all the elements that are light-colored to one. The matrix becomes a representation for a binary image where a value of one corresponds to white and a value of zero corresponds to black. The following image is what we see after executing the line of code above. Notice how all shades of grey have been removed and we’re left only with the black letters.

2. Finding the characters in the image

I can take advantage of the fact that the characters are well-separated with whitespace in order to identify each character in the image.

First, the columns containing black pixels are identified using the code below.

  % find location of dark colours (rows and columns)
  [r, c] = find(N==0);
 
  % find out the columns that contain black pixels
  c = unique(c);

c is a vector that now contains the column numbers, in sorted order, that contain black pixels.

Second, cluster the columns that are close together. Each cluster of columns is then considered a character. Here, I am declaring two columns to be “close together” if they are no more than three pixels apart. The line of code below does exactly that.

  clusters = [0; find(diff(c) > 3); length(c)];

find(diff(c) > 3) finds the columns that are not close together, while 0 and length(c) are for the boundaries.

3. Extracting the characters from the image

The submatrix corresponding to each cluster in the image corresponds to a character. Running the following loop extracts each of the four characters from the image.

  for i = 1:4
    char = N(:, c(edges(i) + 1):c(edges(i + 1)));
    % Classify the character here...
  end

The following are the extracted characters:

4. Classifying each character

Now that each character’s image is separate, all that remains is to identify what character is in the image. As I mentioned earlier, each character of the 26 characters has two forms, a straight form and a rotated form. This brings the total number of characters to identify to 52.

Straight form of a character:

Rotated form of a character:

Classifying a character is slightly less straightforward than the previous sections since there are some minor differences in the shape. Take the two characters below as an example; they are the same character, but there are some slight differences in each image.

So how can they be identified as the same? A very naive approach to solving this problem that seems to work very well is the following:

• For each of the 52 characters, compute the number of white pixels in each column.

The computations from the task above forms our dataset. This can be calculated by executing the following command for each character:

sum(char_image);

Where char_image above is the matrix corresponding to the character that we want to compute the number of white pixels in its columns. The command above works since each white pixel in the matrix has the value one while the other elements of the matrix (i.e. black pixels) are zero.

• Given an image of a character, compute the number of white pixels in its columns and compare it to that of the 52 characters in the dataset. The character in the dataset with the least difference in white pixels is the most likely match.

  % The 52 characters in our dataset
  chars = ['L' 'I' ... characters of the dataset ... ];
 
  % Sum of white pixels in each column for each of 
  % the characters above.
  m{1} = [40 28 24 21 28 33 38 39 40 40 40 39 38 39 40];
  m{2} = [...]
  ... rest of the data set ...
 
  % Compute the number of white pixels in each columns for the 
  % image we're trying to classify.
  char_image = sum(char_image);
 
  % Assign a score of how different the character we're trying
  % to classify from a character in the dataset. The lower the
  % score, the more likely these characters match.
  score = 100;
  char = '-';
 
  % Compare each character in our dataset to the character
  % we're trying to classify
  for i = 1:length(m)
    candidate_char = m{i};
 
    % Only compare characters that are the same width
    if length(candidate_char) == length(char_image)
      candidate_score = max(abs(char_image-candidate_char));
      if candidate_score < score  % Did we find a better match?
        score = candidate_score;
        char = chars(i);
      end
    end
  end
 
  % char has the solution at this point

I tested the algorithm on roughly 100 CAPTCHAs and the success rate has been 100%.

For completeness, I used the bash script below to fetch CAPTCHAs directly from Aljazeera’s website.

VOTING_URL='http://www.aljazeera.net/Portal/KServices/supportPages/vote/SecureVote.aspx'
CAPTCHA_CODE=$(wget -qO- "$VOTING_URL" | grep ?Code= | sed "s/.*?Code=\(.*\)&.*/\1/")
 
echo "Captcha code: $CAPTCHA_CODE"
 
CAPTCHA_URL="http://www.aljazeera.net/Portal/KServices/Controles/SecureCAPTCHA/GenerateImage.aspx?Code=$CAPTCHA_CODE&Length=4"
CAPTCHA_FILE=$(echo $CAPTCHA_CODE | sed "s,/,_,g")
 
wget "$CAPTCHA_URL" -O $CAPTCHA_FILE.jpeg

If you’re interested, you can download the entire dataset and source code of the algorithm here. You will need either Octave or Matlab to run the scripts.

It’s really a shame that a well-known news network like Aljazeera would use such a silly measure for security. Anyways, happy new year!

TellApart is the fourth company I join for an internship and, over the course of sixteen weeks, my internship at TellApart grew to become the most rewarding and memorable internship I have experienced so far.

One of my main goals in joining TellApart was to experience the startup culture in Silicon Valley. TellApart has a very open and transparent culture; every member of the team knows about everything that goes on in the company. The team, aside from being super-talented, is always willing to listen to ideas, comments and suggestions (even from the intern!) A big portion of my internship was spent working on ideas that I proposed myself, and I started working on them within a week of proposing them!

Being a small company, it was very insightful to see the company evolve over a period of 16 weeks. To highlight just a few of the events that happened during my time there:

• TellApart closed its series B funding round of $13 million. In the process, and in occasions thereafter, I had the chance to meet and get to know top tier venture capitalists.
• Amazon Web Services blogged about TellApart’s technology and how it leverages Amazon’s infrastructure. A picture of the entire team (including me in my boxing gloves) was included!
• TellApart was selected in VentureWire’s FASTech 50 most innovative startups for “paving a unique path technologically”.
• TellApart hosted a “TellAparty”, where it invited techies from Silicon Valley over to the office to get to know more about the company. John Lilly, the former CEO of Mozilla, was also invited to come and speak about his experiences.

Company culture aside, TellApart works on really interesting technical challenges in machine learning, data analytics, and in scalability. Because we were such a small team, everyone got a big slice of the pie when it came to dealing with all these technicalities. As an intern, my work was no less relevant than anyone else’s work and I had a major impact on the company’s performance and revenue. Some of the technical challenges we faced include:

• Responding to thousands of bid requests per second while keeping the response time for each request under 120ms.
• Predicting how likely users are to make purchases given the data that we have about them.
• Serving ads customized for each of TellApart’s millions of users.
• Running experiments against millions of users to measure the performance of certain features.

Following up on my previous post on the El-Tetris algorithm, a Tetris player that clears 16 million rows on average per Tetris game, I thought I would provide an implementation, rather than just a description of the algorithm.

This algorithm is implemented fully in Javascript and the rendering is done in HTML5 canvas. The rendering is purely for cosmetic reasons (so you can actually see how the game is progressing). If you’re only interested in the final score, you can choose to speed up the game by enabling “Hardcore Mode”. In that mode, rendering the board will be disabled and the algorithm will run continuously in the background. You can also change the size of the board; the smaller the board, the shorter the game.

Full source code can be found here.

Note: For faster execution, use Google Chrome.

We started a project dubbed “inFormed”. The aim of the project is to raise awareness on global issues around the world. Currently, it’s a Firefox plugin. As you browse the web, it will analyze the content of the page you are browsing and, based on that content, will show a fact, or a statistic, that is both relevant to the content of the page and related to a global issue. Along with that, it will provide a link to a charity where you can donate and/or get involved.

For example, if you are buying a book online or browsing an educational site, you would see, at the bottom right-hand corner, something like this:

Have a look at the screenshots below for some more examples. Take a close look at the fact displayed at the bottom right-hand corner and notice how it’s related to the content of the page.

[Show as slideshow]

To summarize, the goals behind inFormed are the following:

• Help you stay informed on global issues around the world.
• Facilitate how you can be involved by providing links to related charities.
• Provide a seamless and an uninstrusive user experience.

Behind the scenes, inFormed sends the URL of your current page to the server where we fetch the content of that page, extract the text, and run it through a Naive Bayes classifier to select what is likely to be the most relevant fact or statistic on that page, and feed that back to the browser.

This event is the first hackathon we ever participate in, and we are well proud to have made it to the final round! We didn’t win the event, but were extremely impressed at the quality of the projects that people presented.

We had some votes on twitter as well:

inFormed will need a little more work to be ready to publish. Should we invest the time in doing so? Would you use it? Let us know!

Within a span of eight days, I surfed through six cities in five flights over four continents. Here’s the timeline:

• May 7, 2011: Singapore, Singapore ✈ Dubai, UAE ✈ Cairo, Egypt
• May 14, 2011: Cairo, Egypt ✈ Paris, France ✈ New York, USA
• May 15, 2011: New York, USA ✈ San Fransisco, USA

My one week visit to Egypt marks my first visit after the January 25th revolution. Traces of the revolution are certainly visible everywhere you go – stickers on street carts saying “January 25″, spray painting on walls and bridges denouncing Mubarak and his regime and barbed wires around the Maspiro Television building. There was also a protest to express solidarity with Palestine in Tahrir square on Friday, May 13th. Hopefully political and economical improvement would be as visible in the near future.

For the first ten days in California I stayed at USA Hostels near the Civic Center in San Francisco. The commute time to work was around 75 minutes. One of the weekends the hostel was fully booked and didn’t have room to accommodate me. Thankfully, Stan Chen, my workmate, let me stay over at his place then. I have now settled in a house a few blocks away from work.

I am really enjoying my work at TellApart so far. In many ways it was exactly what I was aiming for this work term. It’s a great mixture of really smart people to work with, a startup environment, and serious scalability issues to deal with.

Aside from that, I really don’t know what else to say. In fact, I have no idea why I am writing this post right now. Off to bed.

This algorithm is part of a project I was working on last term as part of my Artificial Intelligence class. This algorithm, which I will be referring to as “El-Tetris”, is an algorithm for playing Tetris by inspecting only one piece at a time (as opposed to two or three pieces in some variations of the game). It is based on Pierre Delacherie’s Tetris algorithm, which is known as one of the best one-piece Tetris playing algorithms.

Before discussing the details of the algorithm, let’s briefly look into how you, a human player, would play tetris.

When you play tetris, you are faced with two decisions every time you are given a tetris piece:

1. Where to position the piece
2. Which orientation of the piece to play

Naturally, you want to eliminate as many rows as possible and maximize your score. To accomplish that, you would (subconsciously) be doing the following:

while the game is not over:
  examine piece given
  find the best move possible (an orientation and a position)
  play piece
repeat

What would be the best possible move then? You would usually try to eyeball certain features to help you determine that. You might, for example, ask yourself these questions:

• If I were to play the move, would that create holes in the game board?
• If I were to play the move, how many rows would I clear?
• If I were to play the move, what would be the height of the highest column?
• … and so on

That’s exactly what El-Tetris does. For every given piece, it evaluates every possible orientation and position against a set of features. The move with the best evaluation is the one that is played.

There are six features in total, formally outlined as follows:

1. Landing Height: The height where the piece is put (= the height of the column + (the height of the piece / 2))
2. Rows eliminated: The number of rows eliminated.
3. Row Transitions: The total number of row transitions. A row transition occurs when an empty cell is adjacent to a filled cell on the same row and vice versa.
4. Column Transitions: The total number of column transitions. A column transition occurs when an empty cell is adjacent to a filled cell on the same column and vice versa.
5. Number of Holes: A hole is an empty cell that has at least one filled cell above it in the same column.
6. Well Sums: A well is a succession of empty cells such that their left cells and right cells are both filled.

The evaluation function is a linear sum of all the above features. The weights of each feature were set and determined using particle swarm optimization.

The following table shows the feature number (as outlined above) and its weight respectively:

El-Tetris’ performance has been compared to Dellacherie’s using a framework written in C called mdptetris. The following are the results after running 35 games on a standard sized board (10 columns x 20 rows).

We were trying to see as much of Indonesia as possible within these five days so our schedule was really tight. Knowing how much I love bullet points, these are some of the places that we visited (in chronological order):

• Mount Bromo, an active volcano in east Java
• Surabaya, Indonesia’s second largest city
• Sampoerna’s Tobacco Factory
• Borobudur, the largest Buddhist temple in the world
• Yogyakarta City
• Kraton, the palace of the Yogyakarta Sultanate
• Jakarta, Indonesia’s capital and largest city
• University of Indonesia
• Masjid Istiqlal, the world’s third largest mosque
• National Museum

Other “side accomplishments”:

• Took four flights in five days with four different budget airlines (Jetstar, Batavia, Air Asia and Tiger Airways)
• Hiked up a mountain peak across from Mount Bromo to see the volcano at sunrise
• Horseback rode up to the crater of Mount Bromo (lots of ash)
• Rented motor bikes and drove around the villages in Borobudur
• Attended a lecture on sequential circuits at the University of Indonesia (taught in Indonesian)
• BONUS 1: Did not get pickpocketed!
• BONUS 2: Stayed within budget!

During the trip, I only managed to learn the Indonesian word “terima kasih” meaning thank you. For all the great adventures that you offered me, terima kasih Indonesia.

Learning Chinese is an eye opener for me in many ways. Besides the obvious intent of learning Chinese for the purpose of communicating with Chinese people, there are some insights that I can see. Let me give you some examples:

1. Tones

Being both an Arabic and an English speaker, the concept of tones was very new, and very strange, when it was first introduced to me. It turns out that the pitch in which you pronounce a syllable determines the word that you intend to say. To see what I mean, listen to the audio clips in the table below. The four rows in this table refer to the four tones/pitches that are found in Mandarin Chinese. They are all pronouncing the same syllable, but the pitch in which it’s pronounced determines which of the four words below you mean.

Weird, huh?

2. Culture and Society

In case you didn’t know, Chinese doesn’t have an alphabet. It is, more or less, a set of characters, the majority of which are based off of pictures. Consider the Chinese character for the word “home” (pronounced “Jiā”) and how it evolved over time:

Current Chinese Character In 259 B.C. In 1046 B.C.

Looking at the origin of the character, it is a picture of a house with a pig inside. This gives an insight in what the Chinese consider to be a home (shelter, roof and live stock) at least at the time the character began to be used.

Another example is 外婆 (pronounced “wàipó”), which means grandmother on mother’s side. This word consists of two characters. The first character (“wài”) means outside, indicating that the grandmother is considered an “outsider” in the Chinese family.

In Arabic we have the saying “من تعلم لغة قوم أمن مكرهم”, which roughly translates to “He who has learned the language of people is safe from their mischief”. Looking back, I couldn’t agree more with this statement. When you learn a language you are not just learning how people communicate with each other, but also how they think.

Sources:

• http://www.foreigners-in-china.com/chinese-symbol-history.html
• http://mandarin.about.com/od/pronunciation/a/tones.htm

While you’re at it, checkout the full blog and the making of below (just so you know what we’ve been through)