Mosh: The ssh client replacement you probably didn't think you needed.

I had to take a breather after writing the heading to avoid giving an over the top introduction to Mosh (mobile shell), but it deserves nothing less. It is a relatively new software (2012) from the well known MIT AI labs. Mosh is based on a new protocol SSP (State Sync Protocol), that runs over UDP instead of TCP. It does all the session management at the higher levels, which makes the connection incredibly robust. Few places where Mosh really shines on my setup (Mosh+Tmux):

1. When moving between networks (Wireless to Wired to VPN at home), the connection automatically migrates to the new IP. This wouldn't have been possible if TCP had been used. Consequently, my shell connectivity to servers stay up... No more banging the keyboard when ssh session abruptly freezes, or me trying to clear up a ssh terminal after the computer resumes from sleep. The terminal comes back up at the same place no matter what the network situation is. Its Magic!! I'm running tmux within mosh and the difference between local and remote machines has completely blurred.

2. Mosh has predictive typing! When typing on a ssh client, your keystrokes are relayed to the server, and displayed to you when it appears on the pseudo terminal at the remote end. This works great when you are working on a network with low latency, but try working from a slower connection from home, or god forbid, the caltrain, a coffee shop or from an airplane using the gogo inflight connection, there is a very significant delay between when you type and when the letters appear on your screen, which makes any sort of remote work next to impossible. Mosh gets around this issue by showing you what you type, bunching the updates to your screen in a low latency setup and later syncing the updates to your screen in background (e.g. if you type an 'i' in vi command mode, it should not show up, but it will show up when the network is very very laggy, however it will be erased when the state is synced from the remote server).

3. Mosh uses SSH protocol to communicate, so it automatically uses all the SSH configurations and writes to ssh logs. All one needs to do is call "mosh ", just like ssh. One doesn't need to muck with _any_ configuration files.

One important distinction between ssh and mosh is that ssh is a set of protocols whose utility goes well beyond providing a secure shell. It allows tunnelling, file transfer, X forwarding etc. None of those things are supported in mosh. Mosh was only meant to be a more robust shell.

That being said, Mosh hasn't been battle tested the way ssh has (specifically OpenSSH), but the design looks sound and simple with a relatively small attack surface, which gives me a lot of confidence in using it.

Go try it out.. and let me know what you guys think.

The Buildbot Experience..

I've had been pushing myself to submit a patch to buildbot and then write a post about it, but days,weeks and months have passed by and I haven't even read the source code (let alone writing some of my own).. well what can I say, I've been incredibly lazy (the word "busy" can be substituted here.. but that would be a lie). Let me spread some buildbot love rather than just be yet another leach.

Now what is buildbot? According to the website:

The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. The guilty developer can be identified and harassed without human intervention.

It's a build/test automation system that can run on a variety of platforms. Well there are a variety of free and commercial apps out there which do the same. I chose it because it was relatively light weight and was written in python (an impressive list of clientèle was a factor too).

As the relatively small company (note "small" here just means employee strength) where I work in grew; software maintenance, integration and the task of porting the code to all the platforms and then testing them just became an incredibly arduous task. We already had a semi-agile system in place and were using version control, but it was just not enough. In a small firm, even a single dev day spent on anything other than coding/design is a day wasted (and we devs are known to be lazy and don't really like doing the same things again and again).

Our product is supported on many different platforms, ranging from the ancient(linux 2.2, mac 10.2, Sun Sparc 5.8 etc) to relatively new ones(freebsd7, macosx-universal, windows 2008 etc) totaling to about 13 different platform. Commercial systems were out of the question because none of them supported all the platforms that we had, so were any .NET based ones (not possible for *nix systems) and we preferred python based system over java because our test bed was completely in python and we just didn't think that adding another language to the mix would be a good idea in the longer run. Setting up buildbot on the newer systems was a breeze (just apt-get/yum/portage etc was good enough in most of the cases, a rare recompile on some others). The older systems and Windows were a bit messy (big surprise there!). Some of the systems didn't even have python (or had a really old version), let alone twistd (a buildbot dependency). After 3 days of hushed cursing, head banging and a lots of ugly hacks, I got buildbot to run on all of the *nix platforms. Windows kind of made me feel completely handicapped because once the install failed, I had absolutely no idea how to get around the situation. I ended up setting up a proxy Linux system for windows (credit for the idea goes to one of my colleagues), which identified itself as a windows machine and just did the needed compilation over ssh on Windows (using VS2008).

Once the crude system was setup, we started adding bells and whistles. Every svn update now triggers a build on all the 12 different platforms, sends emails to a group of people who want to be notified if something is broken and to the person who broke the code. Another process triggers a nightly builds which updates the code,does a clean build, run a set of core regression tests, creates a package(tagged with date and revision number) and posts the status on a pretty page. The status page (html and css) was also hacked to list the tests that failed on particular machines. It also uploads the package to a different repository which can then be used by the QA team. Later some more hacks were done to maintain just a small history of packages (nightly builds, not the production builds) and then some (idiosyncratic to the product)... Check out Google Chrome's buidbot page.. now imagine that for 12 different platforms!!.

We did get the obvious benefits:

• no integration downtime.
• no downtime to port the product to different OSes.
• continuous integration made the development process more robust/agile.

There were some not-so-obvious benefits too:

• Since creating a package on all the platforms just involved clicking a button on the web interface, debugging->packaging->testing cycle was made a lot faster as the QA didn't have to wait for developer to create the package (who would normally try to include all the fixes in a package before creating a package), which means faster feedback on the remaining issues/builds, which means a happy PM :)
• Automating build across different platforms meant that all the platform specific hacks would have to be cleaned up, which meant a more elegant build process, which lead to faster build time and did point to some issues that were being overlooked.
• The green color signals a "pass" on the buildbot status page. Surprisingly we(all the devs) find it rewarding to see a green on the status page with our names beneath it, I think I can safely say that our productivity has gone up and our favorite color is "buildbot green" :) .
  

The whole system works so well and has relieved me of so much of repetitive/boring work, that I was guilty of using it for free. And since I am a poor developer, all I could do was offer some CPU cycles on my home machine (did that for about 4-5 months, until the summer heat forced me to switch off my PC) and offer my help to write some code (which I'll get around to doing one of these days).

On a totally unrelated note, I'll be taking a trip to Europe (Paris, Zurich, Munich, Prague, Brussels, Brugge, Amsterdam), so drop me a line if you've been to any of these places and would like to recommend me something that I should not miss.

vim sorcery: read/write pdf files

I hate switching windows to refer some manual while coding.. completely breaks the flow. I was browsing through some vim tips, which had an entry for reading MS Word docs in vim and wanted to do the same with pdf files... and then decided to take it a notch higher. By adding the following entries to your .vimrc, you can read/write pdf files transparently, as if they were just regular files. Of course, you can also open these pdf files in regular pdf viewers.

Here's how to do it:

1. Get xpdf and cups-pdf. On Ubuntu this should do the trick:
   sudo apt-get install xpdf cups-pdf
2. cups-pdf prints the pdf files and saves them in ~/PDF/ by default. I am assuming that these settings wont be changed.
3. Add the following lines to ~/.vimrc
   autocmd BufReadPre *.pdf set ro nowrap
autocmd BufReadPost *.pdf silent %!pdftotext "%" -layout -q -eol unix -
autocmd BufWritePost *.pdf silent !rm -rf ~/PDF/%
autocmd BufWritePost *.pdf silent !lp -s -d "%"
autocmd BufWritePost *.pdf silent !until [ -e ~/PDF/% ]; do sleep 1; done
autocmd BufWritePost *.pdf silent !mv ~/PDF/% %:p:h

BTW, I have also added this tip to vim Tips. Once setup properly, you can open, save, write pdf files in vim as if they were regular text files.(I guess I already said that earlier.. but what the heck)

Tiddly oooo...

First of all let me introduce you to my foray into the world of javascript : http://sridharv.net .

I was never too much of a web enthusiast (from a developer's perspective).. the reasons being that presentation is a big part of a web application and I hate spending hours on a wysiwyg editor to get the right look. Also there does not seem to be any consistency in the way the pages are rendered on various browsers.

This website began as an exploratory project into the tiddly world when Paul Reiber introduced me to it at an SVLUG installfest. The prospect of an all javascript website sounded alot more interesting than spending hours on dreamweaver/quanta/bluefish/frontpage etc etc. Tweaking a few if and for statements was all I needed to get the right look (and some occasional div tweaks).

There were a lot of different technologies that went into this website that I had no or little experience with:

1. TiddlyWiki: This is a microcontent wiki created by Jeremy Ruston. Here's an extract from tiddlywiki.com:

   TiddlyWiki is fundamentally different from a conventional Wiki because it is not based on separate, entire pages of content, but rather items of MicroContent referred to as Tiddlers that live together on the same page.

Tiddlers can not only be used to edit and publish new content but also to edit and modify the source of the webpage itself. This enables a highly modular plugin architecture. All you have to do is write a small plugin to do your bidding or better yet.. import one from someone who has already done it. Check this tiddler out. This small plugin was written by me to fix some rss rendering inconsistencies on IE... simple enough. Websites like tiddlytools provide tons of cool plugins that make adding any snazzy formatting/functionality a breeze on tiddlywiki.

2. DNS Management: Although this is pretty easy and does not take more than 20 minutes; setting up sub domains, CNAME records, forwarding etc is interesting in many respects (not fun though). It was sort of an educational experience for me. Apparently if you use meta tag information on a page for redirection, Google considers the page as a duplicate and penalizes the page which will reduce its page rank and thats as bad as it can get... who knew.
3. RSS management: Ok.. I cheated here. I used yahoo pipes to join couple of feeds, imposed limits on the number or elements, made a few cosmetic changes via feedburner and plugged it into the website. Yahoo pipes is one great tool that you have to check out.

One thing that was quite evident while testing the websites on different browsers was that they still have to go a long way towards standard compliance. I developed this website on firefox, IE put quite a fight, Opera was the official loser and Safari turned out to be a ringer. There was a big difference in the way iframes were rendered.

Anyhoo.. do check out my website and let me know if you stumble upon some bug or irritating feature.

Gadget bonanza...

A week ago on Black Friday, I purchased a Canon Rebel XT DSLR camera. It's not the coolest camera on earth, but it is the perfect entry level camera for a hobbyist and amateur photographer like me and fits smugly into my budget too. I had been busy for the past few days getting acquainted with all its features, terminology and the sheer number of settings to be made to take a sellable pic. Since I hardly get any time these days, I was taking a long time to get "OMG" pictures..

Just when I thought I had my share of toys.. I got another shiny one.. Nokia N800. At Mailshell, we have an annual game event when we play some RPG and the whole company plays against the bots or someone online as a team..but the trouble was that the developers almost never had the "right" operating system and it took ages to get the games working on all the machines.. The CEO, Tonny, decided to give everyone a common platform, a Nokia N800.

Its a neat little piece of hardware running a modified version of Debian. Unfortunately Tonny underestimated the geekdom in the development team :P. The CTO flashed the device and upgraded the OS to a newer kernel. I installed ssh, opened an xterm on it and was apt-getting away to glory, started pairing it with all the bluetooth devices I had access to (keyboard, cell phone)... i.e we did everything, but played games on it. The damn thing wouldn't connect (mostly because we had just hacked the stuff to run on the beta OS)... nevertheless it met its purpose.. we hacked away as a team :D.

I can hardly wait for the weekend to really get dirty with it.. for starters I will install a sniffer on it with crackers and keyloggers to see how much data I can gather during one casual stroll to the neighboring Starbucks :P .. just kidding.

Wrestling

Note :Do not try this at home without adult supervision

I have been using linux for the past 5 years and use it as my primary OS.. but once a while I have the "OOPS" moment. (No.. this does not mean that there is a problem with linux, just means that if it was any other OS I would have just done a boring clean install because there was nothing else I would be able to do).

I have got so used to larger disk space that when I installed Debian Etch on my "new" 900Mhz machine with 10 GB hard disk, I never put any forethought while partitioning the harddisk. I just went for the default 2G root partition and ~8 G /home partition. As expected, with my rampant abuse of the apt-get command, I pretty much filled the root partition. Now my 500G external disk was mounted on home folder and I did not need any space in there, so it made real sense to redistribute the space. Complication 1: The /root was on a primary hard disk partition and /home was an extended one so I could not use gparted and do any easy cut paste operation.

The right thing a sane person would have done was to cpio the whole disks, repartition and then cpio the stuff back and fix the tiny inconsistencies along the way..but no. I was crazed beyond imagination. I created two disk images with dd and tried to restore them after repartitioning. This wouldn't be a problem for the root partition as the size of the formatted drive was bigger than the disk image..not so with the /home partition. /home didn't have anything anyway, so I thought that even if the image got partially copied, I would be able to fix the number of blocks in the superblock. (Now you know the meaning of the phrase: Little knowledge is a dangerous thing). I had never used dd earlier and had no idea whether this would work.. but what the heck, its all about keeping the spirit of adventure alive.

So after copying the larger disk image onto the smaller /home partition. I ran mkefs2 -S to fix the superblocks and ran e2fsck -y to check the filesystem and fix the errors automatically. It cleaned out my /home directory.. by that I mean it deleted everything.(btw I also tried dls and foremost before performing the step.. these are industrial strength forensic tools.. courtesy of Debian repositories).

Finally I did a clean install but fell asleep half way while configuring it. After waking up I discovered that I had deleted the font files, X, gdm, and few other gnome packages while asleep. I couldn't figure out the packages I had deleted while asleep (may be I wanted to get rid of them at a sub-conscious level :) and no amount of apt-get seemed to startup gdm.. I gave up and did another clean install. Hopefully this should be the last install.. today. All the software mishaps predicted by Murphy's law have already happened.

This exercise made me realize how rusty my sysadmin skills have become since linux became all "user-friendly".. ah the good old rh6 days.

Why do you need PGP?

Lets answer a more basic question first: Why do you need any security constructs like AntiVirus, firewall, IDS etc? Maybe because:

• You have secret/vital information to protect.
• The exploit/attack is so easy that any casual user can do it.. for fun.
  

On my laptop running Linux I don't have an antivirus, firewall or IDS.. I don't have any critical information on my laptop (the critical information is encrypted using my DNA sequence, scrambled using my iris pattern as seed, divided into blocks and stored on NSA servers :P), nor does it run continuously to give the attacker any time to conduct the attack... It's a different story altogether for workstations running 24x7. That said, why do I need PGP to encrypt/authenticate my mails then? It's not that I need to protect my mails from the public eye or even a passive listener.. come on, what can a person get by reading the emails I write to my friends and family!!.. The problem arises when he can write emails to my friends and family using my email ID.. and be undetected. The point of concern is how much easier it is to do so.. let me show you how:

PLEASE NOTE: Doing this is illegal in US and EU unless you own the email addresses you are using. Posing as a third party is considered Identity theft and is a criminal offense. I am using the email address that belongs to me and this demonstration is only for the educational purposes. I am not responsible for anything you might do with the information provided here.

• Cover your tracks: What better way to do that than tor+privoxy!! Since I will be using telnet for this exploit, I need a way to torrify telnet. torrify command does that but I was too lazy to figure out the syntax. I grabbed tor_aliases to do that. Its a small script that you add to your .bashrc , which automatically torrifies various net tools like telnet, scp etc.(Thanks t3rmin4t0r for pointing it out to me on #linux-india).
  
• Find the smtp server of the victim: I am going to send a mail to testingusage@gmail.com(again the ID belongs to me and I use it for testing purposes). Lets figure out which smtp server gmail uses..


sridhar@pico:~$ dig mx gmail.com
; <<>> DiG 9.3.4 <<>> mx gmail.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8033
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 7

;; QUESTION SECTION:
;gmail.com. IN MX

;; ANSWER SECTION:
gmail.com. 3534 IN MX 50 gsmtp183.google.com.
gmail.com. 3534 IN MX 5 gmail-smtp-in.l.google.com.
gmail.com. 3534 IN MX 10 alt1.gmail-smtp-in.l.google.com.
gmail.com. 3534 IN MX 10 alt2.gmail-smtp-in.l.google.com.
gmail.com. 3534 IN MX 50 gsmtp163.google.com.

;; ADDITIONAL SECTION:
gmail-smtp-in.l.google.com. 214 IN A 209.85.199.114
gmail-smtp-in.l.google.com. 214 IN A 209.85.199.27
alt1.gmail-smtp-in.l.google.com. 237 IN A 64.233.167.114
alt1.gmail-smtp-in.l.google.com. 237 IN A 64.233.167.27
alt2.gmail-smtp-in.l.google.com. 285 IN A 66.249.91.27
gsmtp163.google.com. 5883 IN A 64.233.163.27
gsmtp183.google.com. 4017 IN A 64.233.183.27

;; Query time: 64 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Sun Jul 29 14:05:57 2007
;; MSG SIZE rcvd: 270



MX points to the servers that receives email. The numbers 50,5,10,10,50 you see are preference numbers. Lower the number, the higher the preference.. for us it means that we have higher probability of a successful attack. which implies that we will be using gmail-smtp-in.l.google.com.


• Speaking SMTP: The last step is to speak in SMTP to make the server to accept the mail. I am not going to teach you SMTP here. Check out RFC 821 for the details.
  
sridhar@pico:~/blog$ telnet gmail-smtp-in.l.google.com. 25
Resolving gmail-smtp-in.l.google.com. through tor... 209.85.129.27
Trying 209.85.129.27...
Connected to 209.85.129.27.
Escape character is '^]'.
220 mx.google.com ESMTP o11si2850482fkf
502 5.5.1 Unrecognized command o11si2850482fkf
helo
250 mx.google.com at your service
mail from: <testingusage@gmail.com>
250 2.1.0 OK
rcpt to: <testingusage@gmail.com>
250 2.1.5 OK
data
354 Go ahead
From: Someone <someone@someone.com>
Subject: Testing

I see the light!!

.
250 2.0.0 OK 1187318464 o11si2850482fkf
quit


Please do note that testingusage@gmail.com is my own ID, so no harm done. What have I just done? Sent an email to testingusage@gmail.com as someone@someone.com... I could have used any email address there and nothing will be traceable back to me. The following screenshot shows you the result:

As you can see that the message was marked as spam, that is because I did not create the mail properly and someone.com may be in its blacklist. It's fairly easy to fool those spam filters.

Infact the method just demonstrated is used alot by spammers.. just a bit more refined and automated. Nothing is stopping your mortal enemy from impersonating you and wreak a havoc in your life. Yes there are ways to detect a fake mail.. IP analysis, route it takes etc etc, but it all requires the receiver to be a paranoid geek. How many mails do you receive daily and how many times have you stopped to check the authenticity of the mail?

So what is the solution?.. PGP or it's free, open source implementation GPG(GNU Privacy Guard). There are millions of tutorials on the web so am not going to write another one here.

In a nutshell, it attaches a keyed hash to your mail. Since it uses DSA, only you posses the key to create the hash. All others users can only verify your hash using your public key (which has to be published on keyring servers e.g. http://pgp.mit.edu ). The email client takes care of the whole process so you don't have to do anything except the initial setup.

Is it fool proof? NO.. This will only stop casual attacks. The attacker may spoof your identity, create a public key with your credentials and ask the receiver to download his public key, so the receiver might consider your emails fake and the attacker's emails real, but of course this is not something your friend might do for fun.

Bottom line; if you are breaking up with someone on email, proposing someone or making a million dollar deal, use GPG to tell the receiver that it's really you :)
BTW my pgp key is this. That said I can always deny sending a mail which doesn't have my signature... which I might use to satisfy my evil ends ..Mwahahahaha.

PS: There are chances that the attack shown above might not work. This would happen if your IP is blacklisted.

Whistling at your computer

Those of you following Slashdot would have noticed the IBM's article to provide commands to your computer by whistling at it. I decided to give it a shot on Ubuntu7.04. Follow these steps to get it working flawlessly on Ubuntu:

• $sudo apt-get install libsndfile1 libsndfile1-dev libglut libglut-dev
• Follow the instructions in the linked article and download the file cmdWhistle.pl
  
• Replace the functions getEpochSeconds and getEpochMicroseconds in cmdWhistle.pl with this code:
  
use Time::HiRes qw(gettimeofday); #Add at the beginning of the file

sub getEpochMicroSeconds {
my $seconds=0;
my $microseconds=0;
my $timeVal;
($seconds, $microseconds) = gettimeofday;
$timeVal = $seconds.$microseconds;
$timeVal = substr( $timeVal, 6);

my $padLen = 10 - length($timeVal);
$timeVal = $timeVal . "0" x $padLen;

return($timeVal);
}#getEpochMicroSeconds

sub getEpochSeconds {
my $seconds=0;
my $microseconds=0;
($seconds, $microseconds) = gettimeofday;
return $seconds
}#getEpochSeconds



• After this you can pretty much follow the same instructions that the article explains.

I tried running the few commands(did not try the windowing commands using xwit though) after which my mouth started to hurt and I could not whistle a consistent tune afterwards :( It's a good piece of software, and the sndpeek program that it uses is really awesome and feature rich but I think using whistle as an input is totally infeasible for me.
On the other hand, if you know how to play any kind of wind instrument(like flute, clarinet etc) you can easily write a superkewl password login utility that takes a particular tune as an input... sounds right out of a hollywood movie, doesn't it :)

Nostalgia

Remember the good old days of RH7 et al when almost all the software had to be recompiled and adding a new hardware meant atleast a day's work to get everything working properly.. and when just an installation would make you entitled to use the word "Geek". Yes...those were the days.. It really was a great feeling back then.. you felt completely in control of your PC.

Now evolution is indispensable .. so Linux evolved.. from an experimental system to a commercially sellable system.. We had a barrage of distros.. touting compatibility, snazzy graphics and point and click solution to most problems. The public applauded and accepted Linux but deep down inside I hated them for liking linux just because of its looks..

Anyway.. my taste in distros underwent a drastic change from RH to Fedora to Debian and finally to Ubuntu. I was not a fan of the pointing device but the apt-get and the highly up to date repos have kept me hooked up. There were hardly any instances when I had to mess with something. Scripts like Automatix took care of most of the painful jobs.. Even if I had to recompile the sources, ./configure make "make install", hardly counts as hacking.

Anyhoo.. coming to the meat of the subject a friend of mine got some HP laptop with AMD 64 Turion, 2 GB RAM and a Broadcom 4310UART wireless card and was bugging me that he could not get wireless to work under Ubuntu. Normally I dont like support calls but this changed it all. There was a bug in Broadcom drivers(Specifically the way NDISwrapper used them) for AMD64 having more than 1GB data.. An opportunity too good to miss.. I went to my friends place to fix his card(yeah am mad and was totally bored).

After two hours of hair pulling, RTFMing and shouting out F&^* loud.. I got the damn thing to work. It was the most refreshing thing I had done in a while :).

PS: If you are having a similar problem..read this, this and this.

The sound of music..

Am not a music enthusiast, but when I got to listen to it, I need the song. Although I do have an MP3 player, I am fond of internet radio.. live365, pandora, raaga.com, dishant.com et al. Most of them work flawlessly on Linux.. but some Indian Bollywood radios seem to fail pathetically. I always thought that it would be some compatibility issue since they seem to work fine on Windows. Incidentally, few days back I started firefox from the terminal(well.. i generally click on the icon) and wistfully tried playing a song from raaga.com. (Firefox on wine is really frustrating). Of course.. nothing happened, but I found a SEGFAULT reported to the terminal... this was the beginning of a big adventure :)

I googled out the issue and found that someone had found the issue and interestingly enough posted the solution on bugzilla long ago. I downloaded the latest tar ball and saw that the errors were not yet fixed! Big deal I thought.. lemme compile the code myself. There were no INSTALL or READMEs in the directory.. but there was a Makefile... I didn't care to look into the makefile and just called make. That didn't work.. I found that Makefile called individual makefiles in hundreds of subdirectories and those makefiles were not there in the subdirectories!! This is where the "real" fun began.

I have to give you a brief idea of the helix project on Linux(Real's backend) before I can proceed further.

• Its a mixture if highly convoluted C and C++ code. The building process is handled by a python script called Umake (which is used for cross platform compilation).
• The same blob can be compiled into helix server, helix client, real player.. etc etc. There are 32 different project into which you can compile the source code into.
  
• There are 634 build targets!!
• This whole setup reads another configuration parameter to know the destination build system.. including most almost all versions of Windows, Unix and linux.
• I'd leave the maths up to you.
  

It makes sense now that a static Makefile was not supplied with the release tarball. But I was pretty adamant on getting RealPlayer to work with my firefox installation. This manual will tell you that how freaking hard it is to get the damned thing compile.

I had to become a Helix community member, set up CVS access to the repository, accept all the BEULA craphola(I am not allowed to fix bugs and let people have it even though its free), spend countless hours debugging the substandard code and build it. The whole code base has tell tale signs of cut paste code, unnecessary optimization and non standard C++. I was messing up with the latest stable release when one of the Real developers informed me that the branch was dead!! I got one of those nightly builds that had fixed few, but not all, of the issues. I then quickly made the changes and got the thing to build. The most stupid errors were the ones where the variables were used without definition inside a #ifdef clause, casting a boolean into integer etc.

To cut the long story short, Raaga.com now works with firefox on Linux!!! and no unfortunately I cant give you the binary, you have to build it yourself.. RealPlayer's license needs another entry. It is clearly an example of __evil__ open source (The good one being GPL of course). I cant check in the code cuz I was not authorized to modify them at the first place.

Anyhoo.. the songs sound so much more melodious now :)...

PS: A developer asked me to submit a patch. If all goes well, then the whole Linux desi community would soon be thanking me :P

PPS: My patch is accepted. You can download the nightly build by tomorrow.. or wait for the next release. It will be a part of the next release... Yay!!

PPPS: Download the latest binaries here.

Tech update..

So it's time to bore you with stuff that keeps my boredom at bay. Few of the technologies that I have embraced and felt great about. Here are few of them:

• Google web history: Few days back, google's personalized search history metamorphosed into Google web history. There were a lot of murmurs that google now has all the knowledge about you.. Guys give it a break!!. Even if google has it.. so what?? Believe me, there are a million ways to track you down, sniff on your IMs, see your browsing history and even get your password. Google's worth is around 100 billion and would certainly not be interested in identity theft scams, so give it a rest. It's really helpful if you have forgotten to bookmark something (delicious ofcourse) at work and want to look at it later.
• Google recommendation(GR): This was bound to happen. Clearly, Google let its snazzy neural networked monster lose on my web history to recommend me web pages. It looks a lot like StumbleUpon, but it's not. StumbleUpon is to GR what yahoo directory search was to google search. StumbleUpon gives you some random sites, some funny ones, you hardly spent 30 seconds on them. GR on the other hand gives you well defined searches, some you can spend the whole day on without regret. It helped me find the right drivers for my card, a tutorial on a topic I had totally given up on.. its simply great. It still needs a little bit polishing though. It quite clearly also takes the site that it recommends as an input for the next iteration. This might be desirable at times but sometimes the results just get too convoluted. E.g I was searching for some actress' web page. GR recommended me another one the next day.. fine, I thought.. but now since GR thought that I have visited two sites (one on my own and one by recommendation), I would really be interested in them and started bombarding me with web pages of different actresses. I had to go and delete the web history to take care of that. I guess the next step for them is to monitor the time you spend on a website to compute its weightage... they surely would have some brains working on the issue.
• Ubuntu Feisty Fawn: Finally upgraded to feisty fawn. It's without doubt the best desktop distro I have ever used. It has rock solid stability and flexibility. Now Fedora is definitely more secure than Ubuntu but I find Ubuntu much more usable.. and I dont care much about security on my laptop.
• AIGLX + Beryl: I had posted earlier about XGL+compiz on my Dapper. It made the system so unstable that I never even thought of installing it on Edgy. People all over user groups were crying over lack of Linux driver support for ATI cards, so I had given up all hopes. GR popped me a link about installing AIGLX+Beryl on ATI drivers (somehow I never though about aiglx, was too fixated on getting fglrx to work on my system with mediocre results). Did the installation and am totally in love with my desktop. I have totally shifted from metacity to beryl. I removed all the obtrusive features like wobbly windows and magic lamp effect to get me simple snazzy desktop. For once my terminal windows are truly transparent/translucent. The beam out effect on closing windows is fast, fashionable and unobtrusive. Of course I retained the rotate cube functionality when I switch desktops by mouse(which i hardly do) to show off when needed :P.
• Evolution+Gaim(Now Pidgin): Well.. I was using them for ages but never customized them. Gaim 2.0.0.6 kicks any other IM clients a** (civility enforced :P). It gels amazingly with the desktop environment. Help me remove redundant entries for user.. i.e. it shows only one entry per user irrespective of the number of accounts or aliases he/she has logged in as. The psychic mode is another awesome feature that notifies me when someone has just started typing me a message before they complete it.. works really well to spook people out. Evolution did a great job importing contacts from gaim and I find its junk filtering mechanism more usable than thunderbird.

I was planning on a new laptop (this one is almost 2 years old), possibly a MAC.. but I think I've changed my mind. Ubuntu can be, whatever I want it to be.. it supports non-free restricted drivers.. and has my 2nd favorite OS underneath (minix is my 1st fav but has still a long way to go).

I am High!!

begin
const n=10;
integer array A[n];
integer x,i;

proc Sort
begin
integer k,x;
i:=1;
k:=n;
do ~(k<1)>
 i:=1;
   do i
     if A[i]>A[i+1] ->
           x:=A[i];
           A[i]:=A[i+1];
           A[i+1]:=x;
           []
         1=1 -> x:=1;
     fi;
     i:=i+1;
   od;
 k:=k-1;
od;
end;

$Input Table:
i:=1;
do ~(i>n)->read A[i];i:=i+1; od;

$ Test Sort:
i:=1;
call Sort;
do ~(i>n)->write A[i];i:=i+1; od;

end.

Hmm.. before you bombard me with any questions, let me answer them all..
1) What Language is that?
Ans: PL

2)What does it do?
Ans: Its a really bad inefficient implementation of bubble sort.

3)Does it actually compiles and runs?
Ans: Yes, Of Course.

4)How do I know it?

Ans: Cause I WROTE THE COMPILER!!! I can not express in words, how elated I am. Well, of course, its just another project. But.. its a friggin compiler!! I wanted to make one, when I first came to know about them(about 10 years ago).. It's totally surreal. It gives me error messages when I make an error(Type error, Syntax error, out of Scope etc etc) and it took almost 3 months to finish it (well.. only 2 weeks for coding though). It totally amazes me that how a figment of my imagination can suddenly become so tangible, so real.. so alive.

Sincere thanks to Dr Per Brinch Hansen, for helping me realize a dream. Its been a pleasure being his student.

There is also sadness deep inside.. I realize that I may not get to work on a compiler ever again in my life..but there is hope..
If I have a geeky kid or a grandkid, you know what stories I will be telling them :P

Its alive!!

Remember the good old school/undergrad days.. when you thought you knew everything. No? Well then I guess it was just me. I was as enthused by open source and Linux then, as I am now, but was apparently not a very good programmer(good by the standards of ppl of my age though)... but of course I didn't think so then. I made a Linux Task Manager on the lines of Windows task manager which had a similar interface and used to parse the data from the /proc directory. It was one of the best term projects and I so was excited by it that I went and uploaded it on Sourceforge. Of course I did not maintain it... I was young and brash then :P.

Fast forward four years.. I g0t a mail regarding missing files in that project!! I went and checked the sourceforge page and was shocked to find a 93.1% activity and about 96 downloads!! I downloaded the old tarball and it was missing a few source files... who would want to download such a project?? I found an old CD containing the source codes of my projects.. used qt3to4 to convert the project format and re-uploaded it. Ofcourse, I marked it "ceased development".

I just looked into the source files to see if I could quickly fix some of the stuffs there... There was a bigger surprise waiting for me. I had done implementation in the .h files... used namespace-less header.h files, used fscanfs in C++ code... programmers doing any of the above stuffs should be immediately stopped from using the computer and should be sent to concentration camps and tortured until they start following the guidelines.

There was a time when I would bring the project into any discussion I was involved in... Fortunately there is still time and there is still no such product. I swear to totally change and refactor the code as soon as I get time...

PS: If you are a Qt expert and are free, you can contribute to the project.. wont take long to get it up and running.

Compiler Design....

Before I dive into the meat of the topic, I would like to mention that I am a HUGE fan of Per Brinch Hansen... so whatever viewpoints I express in this post, will totally be partial to him.Per Brinch Hansen is teaching me Compiler Design this year and I need to implement 6 phases of a compiler, for a minimalistic language(PL), incrementally for the course completion. I am already done with the scanner and parser.Programming should be a thoughtless process. It should be as obvious as Maths.

Most of you guys(that is, if you are in Computer Science) would have taken design courses(OO etc ). How many times did you have trouble implementing a properly designed code? If your design is good, you should have none, but the idiosyncrasies of the programming language sometime do corner you at some nasty implementation issue. Everyone has seen a nasty memory leak or a typo ruining the whole logic of your implementation. According to Prof Hansen:

Programming should be a thoughtless process. It should be as obvious as Maths. If you are asked the value of 5+2, you would say 7. Have you ever stopped and tried to derive that. Programming should be like that.

Yesterday I was writing the parser, which mainly involved sifting through the PL code and producing a JAVA implementation for the PL's BNF. PL's grammar had some 50 rules which needed to be implemented in the form of around 50 functions. Following the design rules and restrictions imposed by him, I was able to code the parser in just four straight hours!! The code compiled with no syntax errors!! I spent around a day testing it rigorously..and nothing failed!! In fact at one point of time, programming seemed more like documentation. I even wrote vi macros to construct the function skeleton from the grammar rule in the comment (I hate typing:P ). Cant wait to finish rest of phases...

PS: BTW the EFS project ,I was talking about in the earlier post, is implemented and all the issues are resolved. Too bad its proprietary :(

PPS: A colleague forwarded me this link. Turn out that Per Brinch Hansen was the one who came up with the idea of an OS kernel!!

Beginner's guide to Stack buffer overflow...

If you are a C/C++ geek with an ounce of interest in system programming, you would have definitely tried stack buffer overflow. Most of the websites out there are either too detailed or too abstract. Some of the popular websites for buffer overflow claim success on age old machines.. of course the techniques listed on these sites don't work and are terribly hard to replicate.

So all you linux newbs, here is a simplistic buffer overflow exploit written in C... Well I wont call it an exploit, more of a way to modify the return address. A little bit of assembly knowledge would help but is not necessary. I did it on Ubuntu Dapper... fasten your seat belts now.

void function(int a, int b, int c)
{
char ret5[1];
}

int main()
{
int x;

x = 0;
function(1,100,3);
x = 1;
printf(``%d\n'',x);
}
*Adapted example from http://www.cs.wright.edu/people/faculty/tkprasad/courses/cs781/alephOne.html

The above program just prints 1 on the console... what did you think? BTW do keep the debugging option on while compiling the code, i.e your command line should be:

$gcc -o program program.c -g

Now its time to wear that black hat and fire gdb.

(gdb) break 1
Breakpoint 1 at 0x8048360: file temp.c, line 1.
(gdb) r
Starting program: /home/sridhar/bufov/program

Breakpoint 1, function (a=1, b=-1082010236, c=-1082010228) at temp.c:1
1       void function(int a, int b, int c) {
(gdb) s
3       }
(gdb) info registers
eax            0x10     16
ecx            0xbf81d58c       -1082010228
edx            0x1      1
ebx            0xb7ef2adc       -1209062692
esp            0xbf81d4a8       0xbf81d4a8
ebp            0xbf81d4b8       0xbf81d4b8
esi            0xbf81d584       -1082010236
edi            0xbf81d510       -1082010352
eip            0x8048366        0x8048366 
eflags         0x200282 2097794
cs             0x73     115
ss             0x7b     123
ds             0x7b     123
es             0x7b     123
fs             0x0      0
gs             0x33     51
(gdb) print &ret
Hmm so what is the address of ret?
$1 = (char (*)[1]) 0xbf81d4b7
(gdb) disassemble main //Lets see the return address in main
Dump of assembler code for function main:
0x08048368 :    push   %ebp
0x08048369 :    mov    %esp,%ebp
0x0804836b :    sub    $0x28,%esp
0x0804836e :    and    $0xfffffff0,%esp
0x08048371 :    mov    $0x0,%eax
0x08048376 :   add    $0xf,%eax
0x08048379 :   add    $0xf,%eax
0x0804837c :   shr    $0x4,%eax
0x0804837f :   shl    $0x4,%eax
0x08048382 :   sub    %eax,%esp
0x08048384 :   movl   $0x0,0xfffffffc(%ebp)
0x0804838b :   movl   $0x3,0x8(%esp)
0x08048393 :   movl   $0x64,0x4(%esp)
0x0804839b :   movl   $0x1,(%esp)
0x080483a2 :   call   0x8048360

|----this is the return address. How did I know that? well its the statement
          after the function call
0x080483a7 :   movl   $0x1,0xfffffffc(%ebp)
0x080483ae :   mov    0xfffffffc(%ebp),%eax
0x080483b1 :   mov    %eax,0x4(%esp)
0x080483b5 :   movl   $0x80484b4,(%esp)
0x080483bc :   call   0x80482b0 
0x080483c1 :   leave
0x080483c2 :   ret
End of assembler dump.
(gdb) x 0xbf81d4b8 //Go back up and see the value of ebp...
what's it pointing to? Notice its just below ret
0xbf81d4b8:     0xbf81d4f8
(gdb) x 0xbf81d4b9 //hmm the return address should be some where nearby
0xbf81d4b9:     0xa7bf81d4
(gdb) x 0xbf81d4ba // nah.. this is not the one
0xbf81d4ba:     0x83a7bf81
(gdb) x 0xbf81d4bb //still not there
0xbf81d4bb:     0x0483a7bf
(gdb) x 0xbf81d4bc //BINGO!!
0xbf81d4bc:     0x080483a7
(gdb) print &ret[4] //Now lets find out how far is ret away from the return address
$2 = 0xbf81d4bb "��\203\004\b\001"
(gdb) print &ret[5] //GOT IT
$3 = 0xbf81d4bc "�\203\004\b\001"

Now that we know that ret[5] contains the return address, lets go for the kill. A brute force way would have been to just fill ret with long strings so that the buffer overflows. If we know the the position of a code in the memory we can overwrite the return address to branch to that address instead of back to main. For the sake of simplicity, I'll just skip a statement in main(), so that the output is 0 instead of 1 (i.e.the statement x=1 is skipped).

From the disassembly of main() we know that the return address should be 0x080483ae instead of 0x080483a7. Which means i need to increment the return address by 0x080483ae-0x080483a7=7.

Lets take a look at the code now..

void function(int a, int b, int c)
{
char ret[1];
*(long *) &ret[5] +=7 ;
}
int main()
{
int x;
x = 0;
function(1,100,3);
x = 1;
printf("%d\n",x);
}

*Adapted example from http://www.cs.wright.edu/people/faculty/tkprasad/courses/cs781/alephOne.html

Ok WTF is *(long *) &ret[5] +=7 ??
Well it turns out that data is stored on word boundaries for efficiency, and we know that word is of the size of long. Hence the above statement dereferences the long data buffer pointed to by a char pointer. Wait for some time till that concept sinks in...
Feeling better now??good..
Now compile it $gcc -o p2 p2.c
and run it

$./p2
0

If you followed everything till this point, you are no longer a newb...

Update: I reffered this article.

Hack your audio collection

DISCLAIMER: The code shown here is purely for educational purposes. The author does not suggest anyone to use this code....he himself hasn't used it. Everything that follows in the next paragraphs is a lie. This is just to demonstrate the versatility of the Perl language. Music piracy is a heinous crime and I urge everyone to pay for all the songs you download. All stunts performed here are done by professionals, please don't try them at home.

Well since this is the first time I am using red fonts on my blog, you would have guessed that I am up to something nasty(keep the disclaimer in mind). Coming right down to business, I wrote a Perl script( with a little BASH littered around) that downloads all the songs of any bollywood movie, fix their weird names and put them nicely in a directory. Also since the id3 tags are intact, they are perfectly well suited to be played from an mp3 player as well as from your computer.

Here's the script. Currently it runs only Linux and needs mp3info and LWP module for Perl installed (which.. if you use Ubuntu/Debian is just an apt-get away). Since they both are there on windoze too, readers are encouraged to go ahead and try it on Windows. Basically what it does is, it just goes to a free mp3 site and downloads all the songs. Later it reads the songs' title on the id3 tag and renames them.
E.g $./mugBolly.pl fanaa
this would create a directory called fanaa and download all the songs from the movie fanaa in that folder. I downloaded 1 GB of bollywood songs yesterday...

I would specifically like to draw your attention to three lines of code that can create wonders



$realname=`mp3info -p\"\%t\" \"$filename\"`;
$realname=~ tr/" "/_/;
$realname=$realname.".mp3";
rename($filename,$realname);

Now simply putting it, this tiny perl script reads the mp3 title from the ID3 tag of the song. The second line replaces spaces with underscores and the last line renames the file. Imagine having gigabytes of songs with stupid names that your friend though were funny.. now you can name them automatically. You can be as creative as you like.. I had a friend who dumped all his music collection in one folder and soon had an unmanageable mammoth collection of songs, I just modified this script a bit which auto-organized the songs into folders named after the album(again from ID3 tags). Dont you love Perl and Linux after this??

Tags:Technology