AI in networking: Looking beyond the hype

"The question of whether machines can think is about as relevant as the question of whether submarines can swim." – Edsger Dijkstra

Artificial intelligence was the term coined by the famous computer scientist John McCarthy, primarily as a replacement for the term “Cybernetics”, which was romanticized by culture of the day as the utopian end all to all problem solving. McCarthy tried to distinguish the field by having a solid foundation in mathematics and not the “what if” scenarios extrapolated from minor progress in certain aspects of the science. Unfortunately, the term “Artificial Intelligence” has suffered the same fate decades later.

Defining a true AI is hard, because there is no true definition of Intelligence itself. Is a pig more intelligent than a dog? Is a dog more intelligent than a 1-year old human? How do we quantify these differences? The pursuit of an artificial general intelligence is typically limited to academia and research labs of bigger multinational corporations.

The majority of the industry is using the term AI for a more specialized, domain specific, algorithmic enhancement to knowledge compression and expression. We’d refer specifically this concept as AI in this article.

The tech industry is rife with confusing and interchangeable use of the terms AI, ML, Advance analytics. The goal of an ethical company should not be to create a technology that caters to the particular hyped up term but create the simplest products that solves a real customer problem. (Occam’s Razor!)

With that disclaimer, we need to find some logical separation between the technologies out there and assign some term to it. Let’s look at some points of separation and overlap between these technologies:

1. Feedback loop: Most of the experts agree that ANY intelligent system should have a feedback loop to learn from mistakes and learn new things. So is an Artificially intelligent system, just a ML system with a feedback loop? How long is the feedback loop?  This depends on the problem at hand.
   Let’s take the problem of predicting network bandwidth for capacity planning. We could start with a clean slate and have the AI agent learn the network behavior over the course of an year, or if we already have the data, train it in one shot and deploy the model directly. To avoid model drifts (where the network behavior changes over time), we can retrain every few months to avoid loss of accuracy. Depending on the amount of data we could throw at problem and how the problem is described, an end user might not see a difference between the predictions done via different methodologies.
2. Feature Extraction: Certain problems don’t lend themselves well to feature extraction and have fuzzy rules. These include problems natural language processing, image labelling etc. Deep neural networks can just be fed raw bytes and trained to internally create cluster of neurons (that are ‘imagined’ as features). The higher layer neurons then combine these features to predict the necessary outcome.
   This approach is not necessary for problems that have fixed feature sets. Coming back to the network capacity planning problem, the features of a network are well quantified (jitter, latency, loss, bandwidth etc) and do not really need to be deduced by a neural net.

Again, from an end user perspective, the gain in accuracy obtained by using simple linear regression vs a state of art GAN (general adversarial networks) might be non-existent or too little to justify investment of resources. Does the end user care if he’s being helped by AI or ML or advanced data analytics? What if we could quantify the gains for the end user instead.
We could borrow the idea of product maturity from the Autonomous Car research and classify an analytics product in levels:

• Level Zero - No analytics: Think of these networking devices as just providing basic snmp/rest api support and you have to build all the tooling and logic around it yourself.
• Level One - Basic Analytics support: With these networking devices, you get analytics support. You can drill down on traffic patterns, analyze typical issues and take corrective actions. Most of the network analytics dashboard in existence fall into this category.
• Level Two – Partial Automation: These networking devices allow setting up of thresholds for admin notification, automation of service requests via precanned scripts and service platforms. E.g. a rule saying “Increase the bandwidth by 10% if the bandwidth threshold alarm is active for 10 minutes”. This level also allows for simpler forms of machine learning and could allow for rules like “if daily moving average of traffic rate exceeds the monthly moving average, then notify the admin”.
• Level Three – Conditional Automation: This level takes a massive leap over level two in the sense that the system itself determines the parameters and thresholds for the networking device or the whole network. In this level, a networking device will decide on its own threshold and decide the extent of its own corrective actions. The template of action would still need human action. Using the same example as above, the system could “Increase the bandwidth by X% if a bandwidth threshold of Y mbps is breached for Z minutes”. The values of X, Y & Z will be learnt automatically by the system. 
• Level Four – High Automation: This is the next automatic evolutionary step for stabilized level three systems. This allows for complete automation in controlled use cases and would be built over the abstraction of level three. E.g. if the latency between offices in Mumbai and San Jose increased, it would invoke Level 3 automatons to figure out the cause of slowness in traffic. It would even try to remedy some of them with human defined actions/notifications.
• Level Five – Self Healing Networks: This would be the holy grail of networking products. The automation and machine learning would be sufficiently be advanced enough to not require any human intervention for most of network maintenance tasks. The whole network would behave as one distributed self-healing entity that not only heals itself but also optimizes itself for the best performance.

Each level requires a lot of investment in infrastructure and research. Each level is a layer of onion that you can only get to after peeling the outer layer. Level One systems are industry standards. Lot of new analytics products are Level two (including Versa Analytics). And we are incredibly close to building a Level three analytics products.

Huge interest in machine learning, increase in the amount of generated data, decrease in the cost of compute & storage and the rise of cloud computing are fueling incredible leaps in domain specific usage of machine learning. We are on our way to produce something truly transformative. 

Encoding the zietgeist for predicting outcome of a cricket match

TLDR: Here's the code.

This happened a few months ago...

World cup cricket mania was gripping the nation, and I was feeling left out because of my own ignorance towards the sport. I decided to fight the battle in my own battlefield and tried to make the game a bit more interesting for myself. I wanted to take part in company pool without putting in a lot of effort to follow the sport, but also not be the person to not have anything to talk about when all that people talk about is cricket.
Given all the buzz surrounding AI/ML and work that we are doing at Versa, I decided to just build a minimum viable solution to help me winning the pool.

The constraints I had for myself were simple: 

• Don't spend a lot of time. Maximum of 1 day to implement an end to end solution. This ruled out any massive model building using play by play stats, players stats, team dynamics etc. 
• Encode the zeitgeist and perception of the sports fans and enthusiasts rather than focus on micro indicators. 

Here's how I went about winning the pool (shared it with two other folks).

Home team advantage:

I don't claim to be an expert in any sports. However, I do claim to be an expert listener of sports blabber, and one thing that everyone seem to agree on in home team advantage. To factor this into the model I needed to figure out if the teams are playing in their home country. Our ML algorithm can make the connection between a victory and location of the game easily, so I scraped a random website from google to get the data:

	Date	Ground	Team_A	Team_B	Winner	City	Host
0	2000-01-02	Eden Park	New Zealand	West Indies	New Zealand	Auckland	New Zealand
37	2000-01-04	Owen Delany Park	New Zealand	West Indies	New Zealand	Taupo	New Zealand
39	2000-01-06	McLean Park	New Zealand	West Indies	New Zealand	Napier	New Zealand
67	2000-01-08	Westpac Stadium	New Zealand	West Indies	New Zealand	Wellington	New Zealand
98	2000-01-09	Brisbane Cricket Ground	Australia	Pakistan	Pakistan	Brisbane	Australia

Encoding gaming form advantage:

Every sports fan I have spoken to, talks about a team or a player under performing or  outperforming because they are in bad/good form.

How do we objectify a "good" form? If team A is a vastly better team than team B, but team A is in its worst form and team B is in their best form, then who has a better chance? Surely there is an upper bound from the benefit of a good form!!

Based on these questions, I formulated few axioms. These are not true "facts", but universally accepted ideas.

Sridhar's AXIOMS of team sports (SATS):

Axiom 1: A previous win against any specific team can improve the winning probability by 𝛼
Axiom 2: A previous loss against reduces the winning probability by some 𝛽, where 𝛼 < 𝛽
This is based on the observation that sports fans always say that a team is in form because they have won quite a few matches in the recent past but are quick to retort the form is broken if the team loses a single game. This points to the fact that a loss is a heavier blow to the psyche than upliftment provided by a win.

Axiom 3: Specific form against the competitor 𝛼' and 𝛽' also is a factor
If team A has always won against team B, then it'll have a higher probability of winning against team B, even if team B has been winning recent string of matches. This effect can be seen in world cup cricket matches between India and Pakistan.

Axiom 4: The contribution of form to the winning probability is capped to some number

Axiom 5: The form is function of winning streak. i.e. a team will have "good form" if it has been winning more matches in the recent past.

From Axiom 5, we know that 𝛼=f(streak). So, let's just define streak and define its growth function. I'm arbitrarily choosing a decay of 0.8 for winning streak

Let's take a look at the function growth for streak:

Here the max streak is 5 that can be achieved by around 30 continuous wins. However I'm going to linearly decrease the streak when a game is lost. Note the arbitrary nature of the decay function and streak increase/decrease. I don't need to get the function exactly right.

The machine learning algorithm will figure out the importance to give to the streak.

After running a gradient boosted tree classifier on the test set, I get an accuracy of 67%. I don't expect a high accuracy on this model because the inputs are highly subjective. It's not great, but acceptable for a few hours of work.

Once all the i's were dotted and t's crossed, was able to run the prediction on the actual matches. Here are few test predictions:

   
print_update_prediction('Australia','Afghanistan', host = 'England')
Australia, 0.9750151038169861

print_update_prediction('India','Australia', host='India') 
India, 0.6103534698486328


print_update_prediction('India','Australia', host='Australia') 
India, 0.5572055578231812
  

Thanks to the untimely rains in the world cup, 4-5 games were rained out. Few of those games might have been mispredicted by my model. All in all, I was able to predict 87% of the games correctly.

Check out the code on github. I suspect that this would work for any team sport where the teams don't change. It would work for soccer, field hockey etc, but not for IPL cricket or NFL/NBA where the teams change every year.

Prophesying APPL stock movement

Disclaimer: Lot of statistics and technical jargon ahead. Jump directly to the summary section for a quick walk-through. I'm neither a statistician or claim to be knowledgeable about the stock market in general. Use the information presented here at your own risk.

At Versa, we have been doing research into crunching network related data and providing actionable insight into network issues and network behavior in general. My wife works with financial models and a discussion on various tools used in econometrics led me into the rabbit hole of statistics and modelling (I had some experience with it back in school, but tech has come a long way in the past decade).

Although, I am fluent in Python, nothing came close rapid prototyping power of R, so had to pick it up along the way. After few datacamp courses, I had been trying to come up with a generalized statistical model that could work across a wide variety of similar data sets. There is a lot of literature that deals with model fitting and there are lot of libraries for number crunching (both in Python & R), however most of them were never designed for the deluge of data that we have now, nor do they deal with multiple seasonalities well (how the data looks per hour, per day, per week, per month, per year). After trying out stationary model fits and regressions, I moved on to time series models; These include a barrage of models, including Holt-Winters, Arima, ets, tbats, etc to fit a time series model. I have yet to try a LSTM to fit the data.

While browsing the interwebs, I stumbled across a generalized time series model fitter by Facebook's data science team: Prophet. It is fairly simple for use by an amateur like me and yields surprisingly good results. I decided to run this on Apple's daily opening price and tried to predict the motion of the stock for the next one year. There was no assumption on my part that this would yield any information on stock price (else statisticians would have been billionaires). What I wanted to figure out was the general direction of the stock's motion.

The prophet library allows the accounting for events affecting the stock price, which are mainly the holidays and the release announcements (and the days leading up to it). The following R code did the heavy lifting:



Here are some of the graphs generated by this model:

This is the graph of log(stock price) per day (This is done to normalize the variance of the graph). As you can see, even though we have a decent fit, the exact forecast is very klugy. We can't rely on the point forecast, but it does predict that after a small dip in early 2018, the stock will continue to grow up. Now let's look at each of the components and their effects:

The first graph shows that the stock will be on rise in 2018, but the curve shows sign of plateauing. The second component shows that the holidays and events have a fairly small effect on the stock price in general. The last graph shows that stock prices have been at their lowest near the end of January/early Feb (over the past 1 decade).

Another analysis we could do is to predict the growth of the stock, and by deduction, it's volatility. We repeat the same exercise for date vs gain in opening price:

Even though, there are few small spikes (which represent the upward movement of the stock), there are no downward spikes. Barring the temporal volatility of the market, AAPL is a solid stock to bank on. Doing the component analysis on the gain, we get:

Now the first graph here doesn't mean that the stock is going down, it points to the decrease in volatility. There seems to be a more pronounced effect of holidays and announcements on the gain observed, and stock dip is seen typically at the end of Jan/early Feb (as earlier graph predicted) on a Monday. September end, and June end/Early July seem like a good time to sell the stocks (on a Thursday).

What does it all mean?

• It is not a good time to buy APPL stocks right now. They are at an all time high.
• Jan end (on a Monday) might be a good time to pick up APPL stocks.
• June end (on a Thursday) might be a good time to sell these stocks.
• Since the stock is getting stabler, do not expect meteoric returns on investment.
• The author of this blog is not a statistician or a stock market guru, so ignore the first four points.

Language, Personality and Google

"Dude, pass me the book" or "Could you please give me that book?". Which of the two phrases do you prefer, and which one would make you awkward? To me, both of them would be awkward if uttered by the wrong person, which would convey the wrong intent. If my co-worker (who I'm not friendly with) uses the first phrase, that would signal a lack of respect. If a friend uses the second one, that would signal aloofness.

However, this analysis of intent is often wrong. Both, my friend & the co-worker might be preoccupied with something else. In all honesty, I don't care about which phrase is used, they might not care either, but there are people who do. Your friend circle is a small group of people who don't misplace your intent, and then later start harboring good feelings towards you. As mentioned in my earlier blog post, difference in societal stratification and economic status plays a big role in perception of malcontent in the language. If a billionaire friend (Bill Gates is not my friend... yet) were to suggest that we get a chartered plane to some remote island, I'd have to decline and be sad about it. If someone from my circle suggested it, we'd start talking numbers and try to come up with a plan to either make it happen, or laugh at the idea as a joke.

Language and the words we utter, the grammar, plays a big role in how we are perceived by other human beings. The rule of the game is to use the language according to the setting and the person/people you want to impress or at the very least, not alienate. There is a reason Steve Jobs was the face of Apple Inc., even though Steve Woz designed and developed the computers. Enthusiasm is contagious, it is a positive life force that gets everyone excited around you. Conveying enthusiasm with words is an art form. Engineers(like me) are typically succinct and to the point, not a great quality to have in a social setup. However, being Spock like logical and terse, is an occupational hazard. This leads to the perception of mal-intent by others who do not communicate in a similar way (which is why most friend circles have people with one dominant occupation)

Enter, Google Mail's Smart Reply. Smart Reply in it's current form and the planned future upgrades (http://www.kurzweilai.net/ray-kurzweil-reveals-plans-for-linguistically-fluent-google-software) is going to change the way we (or rather I) communicate.

This is how most of my gmail group threads look like:
Alice> Hey Guys, We were thinking of doing a potluck this Saturday. Let me know if you guys are in.
Bob> Sure.
Carol> +2
Dave> +2, might be a bit late

With Smart Reply, here's what it'd look like:
Alice> Hey Guys, We were thinking of doing a potluck this Saturday. Let me know if you guys are in.
Bob> That sounds awesome. I'm in.
Carol> Sure thing. Do you want me to pick something up on the way.
Dave> Sounds good. We'll be there. Might be a bit late.

If you were the fifth person (Eve), which meetup would you go to? The people are the same, the event is the same, the intent is the same. Google's Smart Reply just made Bob, Carol and Dave sound a lot more affable and enthusiastic. In the absence of body language, written words are more likely to be misconstrued. Google's smart reply takes the intent of Alice/Bob/Dave, which is just "yes", and dresses it up in a way that is palatable to a majority of people. It changes a conversation with neutral tone to a positive one. Pretty soon one would have to work to convey apathy and antipathy in their conversation, and not have it introduced due to lack of time for the device in use (replying to a mail on phone).

One could also argue that as conversations get more positive, the net morale of the human race might improve (or at least amongst the people using Gmail). I don't think I should extrapolate this any further and guess the emergent properties or this cultural change. Small apps and features have changed human interactions in unpredictable ways (Twitter/Facebook have very well toppled/made governments, Uber/Lyft have caused a dip in Car ownerships). Let's hope for the best.

A case for and against God

A recent death in my family has caused me to re-examine my relationship with God. Being an atheist (will get back to this), every condolence worded in clichés like "She's with God", or "She's in a better place", trigger a very emotional and existential whirlwind of a crisis within me. What is the purpose of life if being with God is the end goal? What good are your degrees, patents, accolades, your unnoticed sacrifices, your selfless acts, if you cease to exist? What is the point of writing a book that no body will read in its entirety and then burning the book? Should there be point?

I've been on both side of the isle. I've been a devout Hindu while growing up and been a complete Dawkinian atheist for most of my adult years. However, I find myself trying to go to the temple and think about God in times of joy and sorrow. And when someone asks about my allegiance, I give them a wide range of answers, ranging from the obtuse:"What is God?", to vague:"mostly atheist", to random:"Yes, I'm a pastafarian".

I can't call myself pure atheist, because, well... I like going to temple & praying. I can't call myself an agnostic, because I'm not ambivalent, I do not have any doubts about the existence of God, and I'm not religious as I don't think that reciting certain texts can give me an edge over others. So, what am I? I am all three, and I'm neither of those at the same time. All the three groups approach the concept of God differently.

Credit: Andrea Baldwin (http://6iee.com/378783.html)

Life is an uphill journey that starts with birth and ends with death. That is the undisputed, cold fact. Imagine a life, of just climbing up a steep set of endless stairs, with no guard railing on the side. The sole purpose of life is to thrive and propagate, but how does an animal, who, by the freak of nature, questions the futility of the exercise & the purpose of existence, survive? The first thing, most of us, who are afraid of heights, would do is imagine a guard railing. Is the railing really there? No. Does it help me climb the endless stairs towards death? Yes, of course. That guard railing we imagine is God.

This is where the factions come in. A general atheist with a mouth piece would yell that there is no rail. He doesn't care if the person relying on this mental construct is scared of heights. He doesn't care if he could help pull someone up the stairs. If he is not scared of heights, he rallies that no one else should be either. A garden variety religious person, on the other hand, tries to convince a scared person that the railing is real. He rallies hard to convince others that not only the guard is real, but his stainless steel railing is much better than the plastic railings others have. He convinces others that it is perfectly alright to lean on this railing. He convinces others to fight for his cause. Of course, we need stronger railing to stop us from falling into the bottomless abyss, don't we.

Not everyone is bad though. An enlightened atheist might distract you from the deep abyss and show you the wonders that lie ahead of you. You can always create other mental constructs to alleviate your fears. Even most people who identify themselves as religious, fall into this category. Why else would they believe in the power of small sheets of paper with numbers printed on it? (Money.. in case you didn't get that). Similarly, there are a lot of religious people I know, who would just point to the railing to comfort the scared climber. Nothing wrong with that.

An agnostic, would just give an ambivalent answer. He would look straight ahead and tell that he's not sure if the railing exists. He is not sure, either because he is not sure whether the person is referring to an actual railing, or, he just doesn't care about the answer.

So what am I? I'm whatever the person who needs help climbing the stairs of life needs me to be. My own relationship with my God, or any other mental construct, is my own, and varies with the ebbs and flows of life. Defining any relationship with a single word, will take the depth out of it. Love without devotion is nothing. Devotion without conviction is nothing. Conviction without awareness is nothing. We rise from nothingness and we go back into it. Nothing encapsulates everything.

Humans have had the need to box off their uncertainties in their life and attribute it to a higher power, or a foreign country, or a sect, or anyone but themselves. This disassociation helps them achieve extraordinary feats in life, and has definitely helped the human race progress at an accelerating pace. As long as one knows what God is, and use it to imbue positivity in their own life and in the life of others, God is and will remain, the most redeeming feature of the human race.

Interoperability of AI assistants..

I don't think that it'd be presumptuous of me to assume that most of you would have used an AI assistant of some sort; be it SIRI on iPhone/MacBooks, Google Now on Android, Cortana on Microsoft Windows or Alexa on Amazon Echo. Even if you used the "speak to type" option on your phone, you have used it.

Most of the AI assistants have limited functionality and do certain things quite well. I don't care if Alexa is as smart as Google Assistant. It makes up for that with an extensive skill/rule set. Google Assistant would shine in situations where the queries are more free form. SIRI/Cortana/Google Assistant/Alexa all depend on the information you are sharing with them. This inherently limits the efficacy of each. Google Express is not going to replace Amazon for my shopping needs, Microsoft Live mail is not going to replace Gmail and I'm not going to exchange my Macbook or Windows desktop for a Chromebook. So what is the solution?!

The browser wars in the 90s and early 2000s have shown us how this is going to go down. Couldn't the industry leaders pool in their resources to come up with something like a W3C standard and provide a standard browser like shim layer (Reactor/Proactor design patterns) to dispatch requests to individual subsystems (AI assistants)? This could be a simple rule engine in the first phase, that punts my email queries to Google Assistant, my shopping inquiries to Alexa, and my file related queries to Siri/Cortana etc.

Taking the browser parlance a bit far, I would want to theme my AI assistants too. I don't want to call them Google/Cortana/Alexa, I want to call them Mr Chekov or Mr Sulu and I want them to refer to me as Captain and always with a sense of impending doom (Star Trek joke for the uninitiated).

A unified AI assistant might just be a pipe dream. Hopefully, it doesn't take the AI assistants a decade to mature (like it did for the browsers).

Takeaways from RedisConf

I have been using Redis as a swiss army knife database, both at work, and for hobby projects. So I decided to attend RedisConf this year. Low cost of entry, more tech talks than marketing spiels, and a chance to meet Salvatore Sanfilippo, the creator of Redis, were the big draw to the event.
The event was a great introduction to upcoming and lesser used redis features (used by me that is). It was a fun and eye opening window into a lot of different domains and use cases. When all you sell is cheese, the concept of sour cream is interesting..
Some that blew my mind were:

1. IoT: Prior to this conference, I had a limited understanding of it's use-cases. IoT, to me, was just using a web-controlled device. A light-bulb, or a smart fridge, that'd ping me if the milk ran out etc. I.e. a very Silicon Valley centric first world problem solver.
   
   The use-cases that Laura Merling provided in her talk, just blew me away:
• Disney MagicBands: When you navigate your way through disney parks, your hotel automatically knows that you are coming, your door automatically unlocks & most importantly, if you get dangerously close to a moving ride, the ride adjusts itself (can stop) to get you out of harm's way!
• The oil & gas industry: The oil rigs generate a lot of real time information on the equipment’s performance and health — pressure, temperature, flow rates and multitudes of other variables. IoT based solutions provide real time analytics for predictive and preventative measures. This link explains this further.
• Factory safety: Remember the incident, when an automotive factory robot arm crushed a factory operator into a metal plate. This happened because the robot arm mistook the person as another part of the machinery. There have been around 26 deaths in the past 30 years in US alone. This number, although low, can be further brought down by judicial use of IoT tech (just like Disney's magicBand).
• Home Automation: Home automation industry is fairly nascent right now. There is an app to control the lights, another one to control your locks, another one for your TV and so on. There needs to be push from the consumers for tighter integration of products for arbitrary control. E.g. to turn down the volume of the music system if someone is sleeping in the other room, or turn on the lights when the doors are unlocked, and so on.


2. Distributed systems & the rise of DevOps: There were couple of talks by companies like Netflix and Scopely. The amount of data served and generated by companies these days is monumental.
   
   Avram Lyon from Scopely gave a behind the scenes look at the data usage. He talked about a simple mobile game where the goal is to hit oncoming zombies with some objects. His game has a modest 100,000 users active at any given time. To generate insights into the game (E.g. People in California like to throw rocks at zombies, whereas people in New York like to hurl trash-cans at them, or that no-one seems to like throwing hammer at zombies), the amount of data generated and recorded is huge. This is used to provide the development and marketing teams with actionable intelligence. Few TBs of data is not huge by today's standards, and conventional databases are ill-suited for this endeavor. Redis, along with it's other supporting actors lends well to this use-case.
   
   The conference was filled with 2-4 member teams who had set up massively scaled systems within 5-6 months using FOSS(kafka, spark, redis etc) on el-cheapo AWS cluster. Horizontal automatic scaling being the key takeaway. One doesn't need to be a star programmer to build these systems. The architecture has matured enough that systems can be deployed with minimal effort (relatively).
   
   Another emergent database concept was Eventual consistency. Unlike traditional database systems, that dictate that the view of the databases need to be the same for everyone, eventually consistent databases are like.. meh, it's not the end of the world if are able to access the new season of "House of Cards" few minutes after someone else. Heuristics are further tied to extrapolate behavior of the system to get massive scalability. Amazon, Netflix etc have this kind of systems. Take a look at this to get more detailed view.
   


3. Hardware advancement: Traditionally, Redis has been used as a memcached replacement. Something that sits in-front of traditional sql databases. Apps like Twitter et al would store the hot keys in a massively distributed redis cluster (in order of TBs) and would later save these to some backend stores. This adds another layer of complexity. With the advent of flash memory, the backend stores typically now sit on flash memory. However, the newer NVDIMMs & NVMs perform orders of magnitude better than flash memory at just 1/10th the price of RAM. With the closing gap between RAM & Flash, the functionality and scalability of Redis is increasing by leaps and bounds.
   


4. Redis Modules: Usual databases are just dumb datastores. Their main utility is to store the data efficiently in the memory and provide an easy way to retrieve it. All the intelligence to consume that data goes into the client code. Salvatore announced that starting 4.0, Redis would support loadable modules that can be used to offload client functionality to Redis. You could use this to keep track of counter rates or a running average or use simple ML algos to automatically classify the images being saved on the DB or apply some sort of filter on them. The possibilities are endless.
   

Smaller advancements in hardware typically bring in much bigger change in software, that exponentially and radically changes the application development paradigm.

About a decade ago, a simple classifier on an MPI Beowulf cluster took few weeks to implement. These days, I could use a python ML classifier to do the same thing in just few hours.

As complicated technology becomes more pervasive, the entry points to the tech itself gets easier, which further paves way for more complicated technology. Redis has already enabled a lot of industries, which are further revolutionizing everything from healthcare to robotics.
We live in interesting times.

Programming and the art of shoe polishing

One of the most most undocumented occupational hazards of being a software engineer is that the rest of the world is not privy to our thought process, nor do they have a deeper understanding of what we do. Yes, coding is a part of it, and a very big part of it, but, it is understanding of the system that separates good engineers from the bad ones. Coding is easy, coding a system is hard. There is a whole lot of difference between learning english and writing a novel. Just as an exercise, let's imagine that the task at hand is polishing shoes. Our protagonist is the engineer(SE). The product manager(PM), the quality assurance(QA) engineer & shoe polisher(SP) have the supporting roles in this play.

Bad Engineer's Life
PM: I want our company to polish shoes. We will provide supplies to our shoe polisher team who will polish the shoes.
SE: I'll get on it.

[CODE]
- Distribute shoes to each SP(shoe polisher).
- Distribute polish to each SP.
- Ask each SP to use the given polish on each shoe.
- Take shoes from them.


QA: Hey! Your team used brown polish on black shoes. I'm filing a bug.
SE: Oops. I didn't know that. Let me fix it.

[CODE]
- Distribute shoes to each SP(shoe polisher).
- Each shoe has a COLOR.
- Distribute polish to each SP. Each polish has a COLOR.
- Ask each SP to use the given polish(COLOR) on each shoe(COLOR).
- Take shoes from them.


QA: Hey! Your team put black leather polish on my black canvas shoes. I'm filing a bug.
SE: Oops. Hey, why would the customer even want to polish canvas shoes?! Please test my system  properly.
QA: I don't care. You should not ruin your customer's shoes.
SE: Let me talk to PM.

PM: You should not ruin your customer's shoes.
SE: Ok.

[CODE]
- Distribute shoes to each SP(shoe polisher).
- If shoe is CANVAS then return to the customer.
- Each shoe has a COLOR.
- Distribute polish to each SP.
- Each polish has a COLOR.
- Ask each SP to use the given polish(COLOR) on each shoe(COLOR).
- Take shoes from them.


QA: Hey! SP still polishes my hip hemp shoes from SF. And why should the customer wait for shoe distribution to get his canvas shoe back? I'm filing a bug. Hey PM, the code is unstable!!
PM: Hmmm :(
SE: Let me fix it.

[CODE]
- Get shoes from Customer.
- Return shoe if NOT leather shoes.
- Distribute shoes to each SP(shoe polisher).
- Each shoe has a COLOR.
- Distribute polish to each SP.
- Each polish has a COLOR.
- Ask each SP to use the given polish(COLOR) on each shoe(COLOR).
- Take shoes from them.


QA: After 1 work day, the shoes don't look polished. Something is not working.
SE: I don't know, it works for me. Can you try it again and let me know?
-- After one day --
QA: See.. I told you so. I'm filing a bug.
SE: Oops.. the polish got over after a day. Let me add more polish each day.

[CODE]
- Get shoes from Customer.
- Return shoe if NOT leather shoes.
- Distribute shoes to each SP(shoe polisher).
- Each shoe has a COLOR.
- Distribute polish to each SP.
- Each polish has a COLOR.
- Ask each SP to use the given polish(COLOR) on each shoe(COLOR).
- If SP is working more than a day, then distribute more supplies to each SP.
- Take shoes from them.


QA: The product passes the test cases.
PM: We can sell this code to SUPER-MEGA-STORE then. Let us do some scale testing. They order around 100 shoe polishes a day! This could be a big order for us. I feel like Steve Jobs.
QA: Sure, whatever, I'll test out the cases.

QA: PM, we polish 100 shoes in a week! We need to handle that many in a day!
PM: SE.. make it so!
SE: I'll try. We can handle around 15 shoes in a day, we need to get 7 times as many SPs, maybe 6 times if we get stronger and faster SPs.
PM: Hmmm... We'll have to pass the cost to SUPER-MEGA-STORE

SUPER-MEGA-STORE: We like the fonts on sign board outside your establishment, take our money                                           and give us the product please.
--- After Deployment ---

SUPER-MEGA-STORE: Hey, nobody wears black leather shoes in our Sunnyvale store. We have too many unused black polish cans in our stores right now.
SE: I'll fix it.
...
SUPER-MEGA-STORE: Hey, one of the SP died while working. The shoes are now piling on!
SE: I'll fix it.

[CODE]
- Get shoes from Customer.
- Return shoe if NOT leather shoes.
- Distribute shoes to each SP(shoe polisher).
- Each shoe has a COLOR.
- Distribute polish to each SP.
- Each polish has a COLOR.
- Ask each SP to use the given polish(COLOR) on each shoe(COLOR).
- If SP is working more than a day.
- If polish(COLOR) is over, replace it.
- If SP is dead, get new SP.
- Take shoes from them.


SUPER-MEGA-STORE: Hey, we want canvas shoes to be washed too.
SE quits.

This happens over the period of a year. SE's own unscalable and spaghetti code have tormented him. The fluidity of requirements from the customer adds fuel to the fire. When one SE quits, other SE takes his place, mutating this code even more into an unrecognizable mess.

Now let's see what happens with a more experienced engineer:

Good Engineer's Life
PM: I want our company to polish shoes. We will provide supplies to our shoe polisher team who will polish the shoes.
SE: I'll get on it. What kind of shoes though?
PM: Leather shoes.
SE: Any specific set of colors? Google lists some common colors as BLACK, BROWN, NEUTRAL, WHITE, CHERRY RED & CORDOVAN.
PM: Hmmm..
SE: Also what about speciality leather shoes like those make with crocodile leather?
PM: Market research shows that people mostly wear BLACK and BROWN shoes, so let's stick with that.
SE: Any particular brand of polish you want me to use.
PM: Use the CODEYMAN brand. We can get bulk discount from them.
-- A week later --
SE: Hey I found out that one box of polish can polish about 250 shoes. I also spoke to my SP friend. He can shine 10 shoes in a day. Do you know about how many shoes we'll be shining a day?
PM: You ask too many questions, I somewhat hate you. SUPER-MEGA-STORE might be interested, but that is just a rumor I heard. But you haven't yet started writing code.
SE: So how many shoe shines a day?
PM: About 100 maybe?
SE: So we need about 10 SPs working each day. We can have 20 SPs who work on two rotating shifts. This would reduce the downtime if something were to happen to them.
PM: Stop doing a PhD in this! I want the code.
SE: Also 100 shoes a day is 700 shoes a week, which means 3 shoe cans would be used in a week. We need to keep more inventory of cans, to keep SPs from waiting on a polish can if someone else is using it. You can use this to set a price point for the product.
PM: I hate you! Give me an ETA.
SE: I'll sketch some informal design plans and revert in 2-3 days.

.. Time goes on ..
SE: PM, I'm going to be code complete in some time. I want the QA to add test cases.
PM: So the product is done?
SE: No, I've just written the code and did some unit testing. System testing etc will take at least a  month.
PM: So the product is done? I'll tell the Sales team to sell the product.
QA: I'll start testing it as soon as I'm done testing some other thing the SE doesn't care about.

SE: Here's the code

[CODE]
- Don't allow customers without BLACK or BROWN leather shoes.
- Let there be X BLACK polish cans and Y BROWN polish cans.
- Every 12 hours:
    a. New set of 10 SPs come in.
       - If less than 10 SPs come in. Pay the exiting employees overtime to pick up the slack.
b. SPs distribute the shoes amongst themselves, trying to maximize the amount of shoes of a particular COLOR.
c. Each SP takes the cans of polish corresponding to the shoe colors he has. He decrements the number of shoe polish cans available by 1.
- If the number of shoe polish hits the threshold T, he puts an order into the system to get more polish. This way they don't have to wait for polish if it runs out.
d. If SP runs out of shoes to polish:
- He'll put the polished shoes on the rack.
- Hel'll repeat steps from b. again.
e. When SPs are ready to leave:
- If less than 10 SPs leave then:
  - Find if SP is dead and put a replacement SP for the next rotated shift.
  - If SP is hiding, then ask him to leave.
- Take shoes from them.


PM: (During review): Please use spaces instead of tabs and use the function names according to  coding standards.

QA: Hey, edge condition issues. If I'm getting 100 shoes/day and one SP dies, the backlog never goes away.
SE: Oh yeah. Let's add a fudge factor of 11 SPs. Will take care in release.

PM: How is the quality?
QA: Seems to mostly work. Not many open issues I think.
PM: Great, let's deploy it.

SUPER-MEGA-STORE: Hey, your system doesn't handle bursts of more than 110 shoe-shines. We need a burst of 200 during Christmas holidays.
PM: Sure. We can include redundancy and burstiness of shoe-shine traffic in your tarrif. This would translate to hiring seasonal workers.

SUPER-MEGA-STORE: I need to get canvas shoes washed too.
PM: That feature will be supported in the next release.

SE: Shouldn't be a big deal. I'll design similar product for canvas-shoes and encapsulate both the products behind an api that is multiplexed by shoe type.
PM: Whatever.. just do it.

SE goes home.
SE: Honey.. I'm home.
Wife: Whatever! you spend all your time at work. I hate you.

.. SE is sad.

Mosh: The ssh client replacement you probably didn't think you needed.

I had to take a breather after writing the heading to avoid giving an over the top introduction to Mosh (mobile shell), but it deserves nothing less. It is a relatively new software (2012) from the well known MIT AI labs. Mosh is based on a new protocol SSP (State Sync Protocol), that runs over UDP instead of TCP. It does all the session management at the higher levels, which makes the connection incredibly robust. Few places where Mosh really shines on my setup (Mosh+Tmux):

1. When moving between networks (Wireless to Wired to VPN at home), the connection automatically migrates to the new IP. This wouldn't have been possible if TCP had been used. Consequently, my shell connectivity to servers stay up... No more banging the keyboard when ssh session abruptly freezes, or me trying to clear up a ssh terminal after the computer resumes from sleep. The terminal comes back up at the same place no matter what the network situation is. Its Magic!! I'm running tmux within mosh and the difference between local and remote machines has completely blurred.

2. Mosh has predictive typing! When typing on a ssh client, your keystrokes are relayed to the server, and displayed to you when it appears on the pseudo terminal at the remote end. This works great when you are working on a network with low latency, but try working from a slower connection from home, or god forbid, the caltrain, a coffee shop or from an airplane using the gogo inflight connection, there is a very significant delay between when you type and when the letters appear on your screen, which makes any sort of remote work next to impossible. Mosh gets around this issue by showing you what you type, bunching the updates to your screen in a low latency setup and later syncing the updates to your screen in background (e.g. if you type an 'i' in vi command mode, it should not show up, but it will show up when the network is very very laggy, however it will be erased when the state is synced from the remote server).

3. Mosh uses SSH protocol to communicate, so it automatically uses all the SSH configurations and writes to ssh logs. All one needs to do is call "mosh ", just like ssh. One doesn't need to muck with _any_ configuration files.

One important distinction between ssh and mosh is that ssh is a set of protocols whose utility goes well beyond providing a secure shell. It allows tunnelling, file transfer, X forwarding etc. None of those things are supported in mosh. Mosh was only meant to be a more robust shell.

That being said, Mosh hasn't been battle tested the way ssh has (specifically OpenSSH), but the design looks sound and simple with a relatively small attack surface, which gives me a lot of confidence in using it.

Go try it out.. and let me know what you guys think.

You are not special..

Are you from a country that has some sort of currency? 

Now that you are intrigued by my idiotic question, please allow me the liberty of asking another one. If you are filthy rich in one country, lets say that you are a millionaire in US, living in a house that was featured on "Cribs", and you move to country that doesn't recognize your wealth. Do you instantly become poor? The answer is a resounding yes. Then all this proves is that you were rich only because you and other people around you, attached some value to the pieces of paper with numbers scribbled on them. The whole economy functions only because everyone sees some value in those pieces of paper. If I'm able to convince people that my torn sock has some value, then other people will think I'm rich. I'll be actually rich, when I believe that my torn sock is valuable, or if I exchange it for some belief system that I actually believe in (e.g. hard cash) before the reality catches up.

Let's look at social circles. There isn't much disparity between people within a circle, I.e. you wouldn't find people who are a lot poorer or a lot richer than you in your active circle of friends. Money act as differentiator here because it forces a change in priority, e.g. you wouldn't take a trip to Hawaii with friends if you are short on cash, but would definitely go on local hikes because it hardly costs anything.  Gradually friends turn into acquaintances and are sometimes forgotten. Slowly but steadily, you lose touch with people (Note that there are other reasons why people lose touch, but I'm just trying to point out this particular one). If you have lived a relatively fuller life, you'd accept the fact that people move in and out of our lives, just as we move through their's. Without our knowledge, forces of economics drive people closer to each other and rips us apart. All this, because we believe in a common currency!!

Forget money for a second, and think about culture. Think about it as money. What happens when a white catholic from America meets an upper caste Hindu brahmin from India? How does that interaction go? Let's assume for one second that the American has never heard or know anything about India, and the Indian doesn't know anything about America.. what then? Their interaction would completely depend on the place and environment in which they meet. If they meet at the Indian's house, the Indian would probably find the American less modestly dressed, lacking any knowledge of his religion, his traditions, bordering on blasphemy. The American would see the Indian in the same light if the meeting happened in America, he would find the clothes, food, traditions, deities with multiple hands completely weird. Both of them would find each other less cultured. Assuming both of them to be decent humans, the Indian would encourage the American to go to temples and introduce him to scriptures, on the other hand, the American would do the same if he is religious, if not he'd encourage the Indian to watch some opera or Broadway to get some "culture". If either of them disagree, there would be hostilities thrown around. 

Sadly that is how most of the wars start, because of both parties fail to understand the different point of view because it is so radically different. We got around the money problem by having currency exchanges which arbitrarily pits currencies against each other based on supply and demand and not the purchasing power it facilitates in the region where it is prevalent. I.e. if a 1 USD = 60 INR, and a couch costs 1 USD in USA but 30INR in India, who is richer, the guy with 30INR or the guy with 1USD? Still, these currency exchanges allow retention of economic status all over the globe... because everyone believes in some currency and they are more or less inter changeable.

Should we setup a culture exchanges like currency exchanges then ? If you can distinguish between Chardonnay and Sauvignon Blanc, should you be considered a high priest in Navajo? Would you consider Dr BalmuraliKrishna and Puccini similar? Would you see Baptism and Upnayanam in the same light regardless of your religious inclinations?

Of course that would be absurd. Cultural, economical, sociological inequalities are all absurd because they depend on a belief on an arbitrarily defined concept perpetuated by people. These concepts might be bit alien to people who have not interacted a lot with other cultures and have spent their whole life in a cocoon. 

Attacks on our rigidly held belief system tend to anger us. If the belief system is big enough, it might even start a war. A lack of a belief system, however, evokes nihilism, depression and general sociopathic tendencies. This seems to be a catch 22 situation. A belief system has given us distinct evolutionary advantage, even if has been something which defies logic. However, evolution's only concern is the survival of the species, not it's well being. (A recent study shows that chicken taste good to humans because chickens wouldn't have survived in the wild without human help, so humans started rearing tastier breeds of chickens while ignoring the other breeds). It may very well be the case that a fundamentalist faction exterminates all other races (as has been tried multiple times in the past) on earth.. We might end up becoming a paragraph in a grade school book. We might all be doomed to the same fate as Neanderthals when homo sapiens roamed the earth. Of course, the process won't end there. There'll be fractures in the victor's factions, a power struggle between the sub-factions, and eventual destruction of culture and advancement. 

Humanity is stuck in a feedback loop and is shaped by our own intellect, the limits of which, we cannot comprehend. We need a better approach, and we need to shape our own destiny as a race. What matters to you, doesn't matter to anyone else and vice versa. Why shouldn't we accept the disparity in the belief systems without being combative?

Stop yearning for yester years..

Let me articulate a problem that people don't realize they have, and two words that cause this: Happiness & Sadness. We people like to be happy (whatever be the cost), and do not like to be sad. However, almost all of us, mistakenly assume happiness to be the exact opposite of sadness. This assumption is ingrained in our psyche, thrust onto us by our education system, our family, the society, the media, almost universally. But why is this wrong?

Take something banal from your own life, from say 10 years ago, say sharing a cup of road side chai with one of your friends/roommate, or taking a bus ride home on a bus that had barely any place to stand, or maybe just looking outside the window of your room (10 years ago). How do they make you feel now? Nostalgic, yes.. but also happy. How did you feel when you were actually living those moments? Just meh!

Now, take some actual defining moments in your life, when you were actually happy in that particular instant: these includes graduation, marriage, winning some contest that only 50 people in the world care about. How do they make you feel now? Those were just a blips in your existence when you were almost ecstatic. You were definitely happy during those few hours or few days of festivities, but your journey to those defining events, your college years, your courtship period is what made you more happy than those events itself.

So are you happy now? Most of us too jaded by the monotony would respond to this ambiguously, and even if someone replied that they were happy, they wouldn't be able to articulate why. Let me articulate it now for every one of you reading this post; You were happy in the past.. you will always be happy in the past.. 10 years from now, you will think you were happy at this very moment.

Lets switch to sadness. What makes you sad? Traffic Jam, chores at home, office work on weekend, existential crisis, when things don't work according to you.. in short pretty much everything.

What about some sad defining moments in your life? Loss of a loved one, an accident, getting fired from a job? This generally tend to generate two different kind of reaction, either you will look at them fondly (yes you heard me right) because it made you what you are today (the hardship you faced gave to skills to progress in life), or it will make you shed a tear whenever you think about it. The first kind generally tend to invoke happiness indirectly when you look back on your struggle, the second one might put you in a downward spiral or sadness, but you do eventually recover from it.

On an average, when are/were you sad? Now.. and when are/were you happy? In the past. This justifies the predisposition of people to yearn for the past, sometimes elevating it up to the utopian standards. This is a sad universal truth; in America, people want to live the way their parents lived, when the jobs were plentiful and everyone had a car and a house, but they discount and turn a blind eye to slavery. In India, the movies and soaps puts every one on a trip down memory lane, with all the bahus (daugher-in-laws) taking care of the house and guys bringing the moolah, but discount sati and dowry practices.

Everyone is chasing the proverbial happiness carrot, without realizing that it is something that cannot be achieved, only looked at. If you are not sad now, then you'll be happy in the future thinking about this very instant, so don't be sad because you are not happy now, because that'll rob you of your happiness in the future.

Learn from your experiences, and make new ones, but don't try to yearn for old ones.

PS: Inspired by the Republican rhetoric and Jon Stewart :)

Stop the card madness...

Let's say that you have 5 five friends and you want to find what they are up to. How do you interact with them?

1. You meet/call/mail them each individually and talk to them. 
2. You have a conference call, facebook/watsapp group message, group email thread, where you just ask what is going on. People respond is random order, but you process the response serially. You lose some of that personal one on one conversation that you might have had, but you still easily touch base with them. All of them talk to each other, crack some common joke, and after a while the group messaging stops. You have effectively reduced the amount of work needed to communicate, and you have checked up on them so they know that you exist, but you still managed to alienate them... but you might not have spent the energy on an individual mail had it not been for the group chat, so this is still a win win situation.
3. Now imagine that you ask your friends to continuously tell you what they are doing, but instead of communicating serially, they start shouting simultaneously. This is the craziest and the most inefficient way to communicate and that is being thrust down our throats by all the card interface craziness going on.

Let me explain; Pinterest, google currents, flipboard, and as of today, google plus, use card interface. What that means is they'll have each nugget of information on cards, and depending on the application, you can swipe, close, add, append etc these cards. Each of these cards takes a hell lot of screen space and has all the gooey gradient goodness with shadows and what not. Looks great and works great too.. if all you are reading is few occasional tidbits. If you are one of them, just imagine facebook stream arranged into huge squares and thrown about all over your screen.. or just imagine if each email in your inbox was delivered as these boxes. This interface does makes sense if you are discovering content, but not if you are going through a curated set of content, such as RSS feeds or social networking stream.

There have been tons of UI studies where the eyeball of the users were tracked and found to gravitate towards left hand edge of the screen/web page, by forcing these eyeballs all over the screen, these companies are causing a disservice to their customers. 

Now, admittedly, I'm still pretty hung up on Google Reader, but that is how I get to the gobs and gobs of information, I read about 20 articles daily, but for that, I go through 200+ feeds daily. The ability to go through everything and selectively read what I want is what makes stream readers such a good interface. Same thing holds for facebook. You scroll endlessly until something catches your eye, then you click that and explore. Imagine if every object in your news feed, the ones that you don't care about, took as much time; you wouldn't be able to process all that information in time.

With that said, card interface is not all bad. They work on mobile devices, but the powers that be should realize that it doesn't make sense on a desktop, and should show a device specific interface. It might even make sense for something like stumbleUpon.. but please.. not my email, not my feeds and not my social stream.

Sh*t will always fly

Are you living in the perfect world of your own? Do you want something to change? Chances are that you do. If you don't then you probably have achieved nirvana and are reading this blog from a higher state of existence.

Dial your life back by a few years, dial it as far as you can go, access your oldest memory. What were your problems back then? You were probably not old enough to understand any financial problem that your parents might have, you were probably fed by someone, someone did your laundry, somebody chauffeured you around, you had friends who could play with you, and even a discarded carton was a spaceship to you... but what were you worried about? Grades!! thats right. Even though, your life was perfect, so much so that songs have been written about fantasies to get back to childhood, you worried about that 1/2 a grade that you didn't get in history. You thought that life would be perfect if you could just get that 1/2 a grade. Of course, now you know that, your life was perfect, and it was immature of you to want that 1/2 a grade.. or maybe you think that the want of that 1/2 a grade shaped who you turned out now.

Fast forward a few years now, just before you go to college. You think that you could ace all the classes you took few years back, if you could do it now.. and you'd be right. Now you are hot headed enough to ignore any problems that your parents might have, and think of yourself as a new generation of whatever. Still everything is paid by family so life is good. But you are still worried about grades and the college you get into. You obsess over it for a year or two. And you think that life would be good if you could just get into a good college. Of course, now you know that, your life was perfect, and it was immature of you to base your whole life around the want of the greatest university.. or maybe you think that the want shaped who you turned out now.

Few more years into your life, you are now in your prime, just before you graduate. You have the closest friends you'll ever make in your life close to you, you have a clear goal of what you want to do in life. Your bills are still paid by your family and you are too busy to be worried by any problems back home. And what do you obsess about? Your job.. that's right. You obsess and obsess over finding the right job. Maybe you find it, maybe you don't. You still think the life is not perfect.

A good university, a good job, a new gadget... everything gets old and transparent. We humans are evolved only to notice change, not something that is stagnant.

Few years into the job, your life is still not perfect, you have a good job, but now you obsess over your personal life. You think that your life would be perfect if you just got married. Even with all the accolades you won, all the friends you have, you don't have a perfect life... And of course, we want to make it perfect, don't we.

So fast forward a few years, you'll get married. This is where my experience ends.. of course there are still things in my already perfect life that I need to change to make my life "perfect"... But we as a species should realize that our lives are perfect the way they are. Life changes, and since a non-believer in destiny, you have the reigns of your life in your own hands. Life adds more code to your monolithic operating system, and you should know that more code mean more bugs, but remember that more code also means more functionality. If life didn't have imperfections then you'd not move the train of your life to a better destination.

Don't think about that half a grade, your college search, your job search, your spouse search, or any other imperfections that you handled. Think about your schooling, your college, your friends, your job, your spouse. You'll see that you have a rich life.. enjoy it and be thankful for the imperfections that makes it richer.. cause even if you are billionaire space cowboy with pet unicorns, sh*t will alway fly.

Macbook pro reviewed with vile and hate...

I've made it very clear in my previous blog posts/status messages and random conversations, my disdain for closed source systems, primarily Windows and MacOSX. I was a happy camper in my last company because I got to use Ubuntu with wmii as my primary work environment.. and I was churning code like there was no tomorrow.

That all changed when I joined Cisco. Even though you can install whatever you want on your machine, I decided to go with the IT supported Windows 7 so I don't have to muck around with any of the internal proprietary softwares that is generally vogue in any company of Cisco's size. At the end of the day, I had to vnc over to a linux/solaris build server so I didn't care much. Windows was just a thin client, and for most part it didn't get in the way. Operating systems don't really matter if all you need them for is IM, browsing and email (the same software is available on all the oses anyway).

Fast forward 3 years, I needed to refresh my laptop. I decided to go with a mac book pro this time, only because it had much higher hardware specs than it's counterpart. I wondered if I had made a wise choice. In the first one week of badly missing the dock (not the OSX dock, but the hardware dock you put the thinkpad on) and the pin mouse(most people think that it is useless, but it is quite useful if you know how to type and don't want to move your hand from the keyboard), my experience with mac seemed like a self fulfilling prophecy.

I was ready to hate this mac book pro, and write rants for the next 3 years.. but then I really started using the damned machine. With few tweaks, I got my .vimrc, .bashrc and .screenrc to work on mac (some of these files, I've been building for over 10 years!). Adium, which to my surprise is implemented on libpurple, filled the void pidgin had left. And chrome.. what can I say about this fantastic piece of software, once you log in to the browser, you won't know the difference if you are browsing on windows, osx or linux. So OSX just got out of the way as Windows 7 did and let me do my work as I used to do earlier.  All the apps like evernote, rescueTime that I depend on pretty heavily just worked as expected. 

But then the fun started, when it started seducing me to its vertically integrated ecosystem. To install some apps, I had to get AppleID, so I got one. To get the nifty notifications I configured my facebook, twitter, gmail and flickr accounts, which automatically imported the calendar and contacts, so with barely any effort, I was using OSX's calendar, notification system and contacts. Coming from the windows/linux world, I was used to either open a software and search for relevant entries or just grep for configs/data in the respective directories, but spotlight changed all that. Searching for anything became much easier. To my surprise, some software ran faster on a windows virtual machine on mac than on windows running on my old laptop (with similar memory allocation). The douchey way to scroll became "natural". It just looked like a really really well designed ubuntu system (I know people will find it funny because Ubuntu has been taking GUI cues from OSX.. but I come from that world).

The hardware itself is a joy to use. The 15'' retina display is great and it makes the HD IPS display of my photo processing workstation look like the screen of windows with 1024x768 resolution with 256 colors. Keyboard lights that turn on when lights are dim, extremely thin dimensions,a great touchpad, the completely silent 256G SSD and the monstrous 16G RAM makes this machine a joy to use. 

Am I a fanboy? No.. and probably will never be. Why? Well the cost of the machine with all the connectors etc have run to about $3000.. Unless someone is paying for it, there is no way in hell I'll be shelling that amount(in this case my company pays for it). I hate non-standard keyboards. I don't like the command key, I don't like the omission of page-up/down, end/home keys, and I'll never get used to the "MAC" way of doing things because I'll always be working in an environment where I'll be using Linux and working with people who own heterogeneous set of machines and operating systems, so I'd want something that doesn't stump a non-mac user. Even if the keys were eliminated from the laptop itself, they shouldn't be disabled when a usb keyboard is connected. Also, don't know what'll take Apple to put a hardware dock support to the mac books, but connecting a keyboard, a mouse, two displays and a power chord is a chore. The available third party solutions are nowhere close to Lenovo docks. The battery life of my 3 year old Lenovo laptop is still much much better than the new Mac Book Pro. Also MS office softwares work much better on Windows machine.

So what's the final verdict; undoubtedly I'm in love with the machine and is much better than my old Lenovo laptop, but since the hardware configs were different, it is not an apple to apples comparison. Do I still feel the same way about closed source software.. absolutely. Would that stop me from using a decked up mac book pro.. absolutely not :).

PS: Incidentally I wrote this entry, on MBP, sitting in a street corner starbuck, sipping a skinny latte, in San Francisco. I don't know the hipster/douchebaggy smugness is my own or one induced by this vile machine :)