Recap

It's a bit late to recap the year of 2010, but since I mentally make milestones from birthday-to-birthday instead of arbitrary Dec 31-to-Dec 31, I suppose I am actually in the middle of the year I am going to recap.  The next one will in fact be in May 2012 :)

So, what did I do in 2010?  I studied a handful of books:

• XMPP: The definitive guide
• C# in Depth
• F# For Scientists
• Expert F# and
• Beginning Blackberry Development

I experimented with these in various ways.  I made a small Blackberry app, which I had grand visions for; but I abandoned this when I realized that RIM was going to dump the O/S in favor of a new one.  At the time, I also wanted to get into Android development as well, so it meant that although I was still going to work in Java, I was working on a better platform (that can actually use generics, among other things).

I worked through quite a few Project Euler problems, and then grew a bit weary of F# in general.  I just didn't have any interesting projects that would make use of it.  I would still be very open to anything that came up, but overall, I think that I won't be pursuing functional programming right now.  I suppose this will change next year, after I have experimented with other things that I will write about towards the end of this post.

C# in Depth was quite a good book to read.  I was often confused about the different versions, and the book not only points out what changed, it actually takes you through different techniques and examples, showing you what it looked like in the different versions.  Version 4 came out after I read this, and it is actually quite powerful.  I suppose you don't really need F# if you have C#, but there are some really good things about F# that make programming in it much more pleasant.  There are a lot of things to like about F#: computational expressions, active patterns, even the syntax itself looks very pleasant (if you don't mind whitespace instead of braces).

XMPP was the most interesting thing I delved into last year.  I had dabbled in some Jabber implementation very early in my career, but didn't pursue it at all.  When I read through the book and went through the docs online, I saw that it was actually being used everywhere I looked.  Later on in the year, I found out even Facebook chat was XMPP-based (as is Google talk and Wave [RIP]).  I was quite excited by it and ran an ejabberd server for a bit while I tried to figure out some server components to add in.  It was difficult to start programming for XMPP though, with a lot of big players in the market.  Since I was trying to use F# in those days, I wanted to use C# libraries, which are all closed-source and needed licensing.

Setting up XMPP is not easy.  The servers are fine, but adding in TLS and trying to program for it seems to be very difficult to get into.  I was able to get TLS working on ejabberd, and was able to chat using Trillian with Google talk accounts.  However, I didn't enjoy wrestling with all the different libraries to try and extend either servers or clients.  I later discovered that there are better implementations, but they haven't been around as long and decided to wait it out.  I am talking about telehash (which is gaining a lot of traction these days), and also STOMP.  Anyway, I am now looking into node.js for similar functionality.

With respect to work, I mainly spent the time working with Sharepoint, and have reached a stage where I think I am content with moving on.  There isn't much left to work on, and I think I will not move on to Sharepoint 2010 as I had planned.  I successfully managed to implement a "minimal master page" and managed an HTML5 boilerplate-style reset, which is actually a lot harder than it would seem.  I had to get rid of Windows on my server though, to start experimenting with node.js.  The reset was actually quite stimulating, and if Silverlight wasn't all over 2010, I think Microsoft would have had a lot more success with it.  If I had continued with my experimentation, I would have sought to completely replace Silverlight with HTML and javascript, making it a completely HTML5 (now just HTML) solution with a minimal master page (FTW).

I was approached to work on a Drupal site for a large company here, but after a bit of soul searching, I think I've come to the conclusion that I am not content with CMS development.  I think I've done this kind of thing for 5 years now, and I am discovering a lot more new stuff now that excites me.  I'm going to let it plateau for a bit as I explore the following technologies:

• Neo4j - a graph database, which allows you to query large amounts of relationship data.  You can think of it a bit like LinkedIn, which is their showcase example.  Think of the query which lets you display the connections withing 5 hops of you in some kind of field like "programmer".  Doing this in a relational database wouldn't perform very well with a large data set.  I am quite excited to be working on a test neo4j database, trying to work out how to solve these kinds of problems, because these kinds of problems are actually quite common-place.
• CouchDB - this is a document-based noSQL solution, which is great for storing and retrieving data that doesn't have set form.  What I mean by this is that whereas you would hard-wire relationships between entities in a relational database, in Couchdb, you generally "shard" them into "databases" and then code the relationships in your code itself.  This way, you are able to store a piece of information, along with all of its related info in one place, and slice it up as you want using code (in this case, map-reduce in javascript).  The data is all stored as JSON, and you can attach arbitrary binary data as well (though retrieving these is slow).  I see it as being great to store things like blog posts, comments, etc., but it has quite a few applications.  I will be using it a fair bit this year, I think,
• Node.js - This basically allows you to process a lot of simultaneous client connections on the server, where you run javascript.  It basically can accomplish most things that XMPP would be able to.  Chat, games, presence info, etc. can all be programmed in, and it can connect to couchDB for added javascript goodness.
• GWT - I am quite happy with how GWT is performing.  I have a unique need for it, as I am working towards an app for educational institutions, which has a dependence on Google Apps for Educational institutions, Google APIs, and Neo4j; and needs a web front-end.  This fit the bill nicely.
• HTML(5), javascript, CSS3 - The cornerstones of what I do will not be neglected this year.  I think out of the three, javascript is going to be in everything I work on, but I will still keep my eye out as IE9 pops out and the browser wars continue.  Let's hope that IE6 dies a slow quick death this year.  I am also going to be taking a closer look at user interface this year.  There are a lot of problems that I'm working on that actually translate into how well you can get the user to enter in what they want.  The interface solves a lot of challenges that you can't possibly account for in the back-end, and instead of second-guessing, just make it so the user can tell you what they want, using the broadest toolset, which needs to be as broad as it is abstract, without alienating the user and having the shallowest learning curve possible.  It's actually a huge problem, but I'm hoping to come up with some solutions that can be used as a base in future work.

I suppose that covers the gist of what I am interested in these days.  I am always on the lookout for interesting projects, but I think I have my hands full with my own ones right now.  The best part of that is that the things I am working on are translatable to things others are also looking for, so I'm hoping this pays off down the road.

Tools of the trade

This is a continuation of the series on what happens behind-the-scenes during the development process when I work on a site or project. Today, I will outline some productivity shortcuts that I use constantly that makes an hour of my time these days worth a few hours of my time when I first started out. I'm going to start out with smaller things and work up to the major ones.

• First up is a small program called SlickRun. It is a launcher for Windows. It's highly customizable and very handy - it also comes with something called QuickJot, which is a tiny program that lets you put in arbitrary text and stores it for you (it is completely persistable, in that it will re-appear even if your comp crashes). You call up SlickRun with Win+Q and QuickJot with Win+J. These two shortcuts are now almost like reflex reactions.
  Why are they important? Well, pressing Win+Q then "ps" pops up Photoshop, "wiki" and then a query opens up a new tab with search results in Wikipedia, "ie" pops up IE; the list goes on. It's very handy.
  Win+J is also a time saver. It lets me hold something in memory while copy-pasting, like license keys, unformatted HTML, etc. and is also my place to store ideas that I have that I can think about later.
• In the same vein as the above is a small utility called AutoHotKey. It allows you to program "macros" for Windows. It's hard to describe, but you can make it do just about anything. I've currently got it only doing basic text-replaces for things I type so often that I would get RSI doing it as many times (URLs, my email addresses, the basic HTML template page markup, etc.) It's very powerful though, and I could technically replace SlickRun with it if it weren't for QuickJot.
• In order to keep track of projects and to estimate my time, I use a program called ToDoList. It is very easy to use, and in all my searching, I haven't found a better tool.  I am consistently getting very good results for my estimations these days, as I can look back on previous projects and use the estimated/actual times from them to help me.
• Coming in to actual coding, I've got my IDE of choice - NotePad++. It has some excellent features and plugins by default, which save me a lot of time: ZenCoding plugin, escaping " to \" and back, upper/lower case, HTML Tidy, block and line duplication, multi-caret (allows you to type the same thing in multiple places in the file simultaneously), search-replace in multiple files, conversion of tabs to spaces and back, split line by clipboard character, and so many other little things that can all be assigned keyboard shortcuts. Some very common scenarios are below:
  
  • Inline CSS to block
    
    Say I had this CSS:
    
    #elem { background:transparent url(/someurl) no-repeat 0 0; color:#FFF; text-decoration:none; display:block; padding:3px 0 5px; margin:0 5px 0; }
    
    Now I want to re-write it block-format. All I have to do is copy ";" to the clipboard, highlight the line, hit my shortcut (Ctrl+Alt+1) and it splits it all into lines that I can re-indent with block-level indenting with the tab key.  You might think to do this with a search-and-replace, but this is a lot quicker.  For search-and-replace scenarios, I use The RegEx Coach, which I'll discuss later on.
  • ZenCoding
    This is possibly the the best plug-in for an IDE that was ever made.  It is completely unbelievable that you can type the following:
    

    table#someID>(thead>tr>th*4)+(tbody>tr*2>td*4)

    and get back this:
    

    <table id="someID">
        <thead>
            <tr>
                <th></th>
                <th></th>
                <th></th>
                <th></th>
            </tr>
        </thead>
        <tbody>
            <tr>
                <td></td>
                <td></td>
                <td></td>
                <td></td>
            </tr>
            <tr>
                <td></td>
                <td></td>
                <td></td>
                <td></td>
            </tr>
        </tbody>
    </table>  

  • HTML Tidy
    
    This plugin allows you to take some formatted text, say from a Word document, and convert it to HTML using configurable settings, saving you hours of typing and formatting. Of course, you need to do some work configuring it to be useful (it's not by default), but once you have it down, you can enjoy hours of saved effort.
    
• The Regex Coach is a program that I use almost constantly. You can say that I might be a little like the character in XKCD who proclaims, "Stand back! I know Regular Expressions!"
  
  I don't think I can name a single project where I have been given copy to put on a page, and I did not use this program to reformat the content into HTML for me (usually to shove into HTML Tidy).  It's blazing fast too.  For instance, let's take the example of my previous blog post, where I needed to list out the RSS feeds that I use.  The program only exported the feeds as "OPML", which was not useful to me.  I used the Regex Coach to pull out the important bits (URL and feed name) and used the replace function to generate some nice HTML anchor links as shown in the image below (the four text fields are my search expression, the original string that I needed to format, my replacement expression, and finally the HTML markup I could use in my blog post).
  
  
  
  
  There is the added advantage that using this program has taught me a lot about regular expressions in general, and I am confident about a lot of concepts like look-aheads, greedy * and +, grouping, escaping, and which abilities are available in the languages I use. Also, I now know a lot more of the inner-workings of the engines, and can plan for optimization. I would not say I am an advanced user of regular expressions, but I am a lot further up than when I first started out.
  
• Firebug/IE Developer toolbar/ Webkit Inspector
  These tools are miles apart in terms of their capabilities, but together, they let you debug just about any problem you have with front-end development.  My general rule-of-thumb is: use Firebug first; if there is a problem with it or to debug $(document).ready, use IE.  Use Webkit inspector last.
  
  I don't particularly like the Webkit inspector as an alternative to firebug.  I find it doesn't scroll into view the correct element, and you can't edit the properties like you can in Firebug.  In fact, with Firebug, you can edit all the HTML/CSS on the page and see your changes live, while using the console to make changes to the scripts or to do further modifications via jquery.
  
  What Firebug lacks, the Web development toolbar add-on can fill in.  You can peek at all the javascript on a page, all the CSS, manipulate forms and elements, and outline specific groups of elements in  a page (broken images or table cells, for instance).
  
  I find myself using IE's dev toolbar a lot to debug javascript, as it throws a bigger tantrum for errors.  Some errors just don't show up in Firebug, especially when they are in the $(document).ready function.  Of course, as with all other things in javascript, the error messages themselves are completely useless, but at least you know that an error occurred!
• Javascript beautifier / Closure compiler
  
  These two tools go hand-in-hand.  The Closure compiler minifies javascript for you, and the js beautifier allows you to glimpse minified javascript and trace out what others have done.  This is useful in certain instances like when using third-party tools in inherited projects, when you don't really have access to the code but need to debug a problem.  It has helped me before to look at problems with Sharepoint's weird functions for dynamic menus (especially the nefarious left-hand nav), Omniture code (don't get me started), and to catch a glimpse into how certain sites accomplished some javascript that I thought was interesting (20 things comes to mind).

The combination of these and other tools let me confidently declare that the present me is orders of magnitude more efficient than the me from 5 years ago who was just starting out professionally coding websites. This doesn't even take into account the efficiencies in actual code - just in the tools that I have learned to use.

Keeping up

This is the first of a series of articles geared towards explaining how I work, and what you don't get to see about me during the hiring process. These are productivity shortcuts and techniques that I've adopted over the years that allow me to justify my claim that my time is highly efficient and productive.

The list below is a current view of all the RSS feeds that I currently have in my reader of choice (the desktop app Klipfolio, which is powerful, lightweight, and incredibly reliable). I've been using Klipfolio and a lot of these feeds for over 3 years now, and they have served me well.

• Slashdot
  I use it mainly for reading the comments, as the news takes a little while to get posted on the site by the curators. The users of the site are often knowledgeable and humorous.
  
• TEDTalks (video)
  This should require no introduction. I discovered it a few years ago and combed through and watched all the videos that interested me, and have since combed through the videos as they get added to the site.
  
• Hacker News (Ycombinator)
  This is my main source of information related to my programming and entrepreneurial activities. The stories and great and get to me before any other source. The comments are generally good, but I don't really read them as I read the articles long before the comments start to appear on the site. Also, the links go directly to the article and not to the site itself, so I manually go back to the site to read comments if I think that the article needed further elaboration.
  
• The Economist
  I don't read it as often as I used to, but the Economist does have some of the best written articles out there.
  
• GlobalPost
  The GlobalPost also has some very well written articles, but again, I don't have the time these days to go through them unless they are really of interest to me.
  
• optics.org research channel
  They specialize in optics, naturally, but they do feature a unique slant on the news stories. For instance, you might read an article about the full-body scans and the public outcry and media fanfare in other feeds or through other media, but from optics.org you get the perspective of the technology itself: what imaging techniques and apparatus are used, and other technical details you would not find elsewhere.
  
• Core77
  It is a good source of web development news.
  
• Smashing Magazine
  This one is mostly targeted at web "designers," but does feature occasional articles geared towards web developers. It is a little skewed towards introductory-level techniques, but if you find yourself in that boat (as I often do), it offers a good introduction and features some good out-bound links.
  
• Yanko Design
  This one is mostly to look at design inspiration, as most of their features don't exist (and probably can't exist). It offers a good perspective on UI though, as a lot of the designs feature unique views for interacting with objects.
  
• Six Revisions
  You find more in-depth articles here than Smashing Magazine, and it is a bit more skewed towards web development than web design.
  
• QuirksBlog
  It is always good to keep up with his research and tinkering.
  
• ReadWriteWeb
  This one is a fairly recent addition to my feeds. It features a lot of very current work in real-time uses of the web. You would find information about XMPP, node, and other technologies geared towards providing interactivity and presence in real-time across the web using existing protocols and technology.
  
• WebWorkerDaily
  GigaOm - I purchased the annual subscription, mainly for their work with inspecting the current state of mobile technology, but their articles are generally good.
  
• GigaOM Pro
  See above
  
• myNoSQL
  This is the latest addition to my feeds. I just couldn't keep up with the noSQL trend as there are so many different solutions out there. Each has its little niche and this site tracks the various goings-on in this rapidly emerging field.
  
• xkcd.com
  You need to read this every Monday, Wednesday, and Friday.
  
• Dictionary.com Word of the Day
  I sometimes try to use the WotD, but I mostly find myself dismissing them as too obscure.
  
• Mandership
  If you have an interest in design or interactivity, you need to read this blog. I've read every single article and I find them all extremely well-written and thought-provoking.
  
• PhysOrg.com - latest science and technology news
  This last one is usually the one with the most articles every day, and some of them are not as well-written as others. However, from a research perspective, it usually has the broadest breadth of information, and you would easily be able to find the journal article in question and read through it if you so wished. I use it generally to catch a glimpse of the state of various emerging technology from bio-tech and engineering.
  

So you see, I comb through a tonne of information on a daily basis. I find that I am well-informed, usually within a day of some emerging event or trend. I go through all of these feeds religiously, every day, and have developed the ability to scan through a lot to identify trends on my own as well. You can watch a story progress from one site to another and be able to judge the general level of activity around it. For instance, if twitter or gmail went down for a little while, you would notice it come up in Hacker News with something like "Is this service down for anyone else?" followed through with a bunch of blogs taking up the story and proclaiming them as being down. You would then see it happen again when the service came back up, and people would try and figure out what happened and what to do in the meantime. After all of it is done, within a day, you would see post-mortem analyses by various blogs, and perhaps an official story making its way around as well.

Keeping up and watching trends like noSQL emerge from nowhere and take over also helps to keep me interested in my field of choice. I am actively trying to integrate these technologies into my work, but until the opportunity presents itself, I would not jump in without a reason to do so. In the meantime, I spent about a year reading up on map/reduce and learning about the differences in the approaches that BigTable, Hadoop, MongoDB and CouchDB all take (to choose a very small sample). It is of great interest to me as a long-time user of SQL and having generated many crazy and complicated queries over the years. Some things are just not meant for SQL, though I suspect I will always be using both one SQL solution combined with a noSQL solution.

I feel that my efforts in staying current with this rapidly changing landscape makes me invaluable in helping with evaluations, planning, and coding using cutting-edge techniques. I am not afraid to delve into anything, and always make a point of checking out a site's source code (putting it through jsbeautifier if necessary). I like to incorporate techniques that I see being used in the wild, and I always make sure to recreate, not steal. It is for the purpose of learning, after all.