13 December 2017

An open letter to hospitals that make patients relatives run around to get blood

Note: In Karnataka, you can get blood from these places: https://blood.kar.nic.in/Home1.aspx

A relative of mine was being treated at a hospital, and I heard the hospital was making his friends and relatives run around, to arrange for replacement blood donors. Since he was undergoing plasmapheresis and dialysis, there was a huge demand for blood. Moreover, even though they brought blood donors and paid the donors, they still had to pay extra for blood units that the hospital gave. That was a shame. Especially for a hospital that claimed to treat its patients well.

I wrote a letter to them. The links in the letter are important. They will take you to a whole new level of knowledge.


I know there's no point in taking you on a guilt-trip of making relatives of patients running around trying to find blood donors, so let's get down to some facts:

I understand hospitals have their own logistics to worry about, but when you have a marketing term that claims to treat people well, the hospital's word-of-mouth advertising will definitely improve when people know that this is one hospital that does not inconvenience people. Apart from the blood donation aspect, I really appreciate other procedures that are followed well in the hospital.

Kindly forward this to the person who can take decisions on organizing blood donation drives. Meanwhile, I hope your hospital could immediately stop inconveniencing people and instead organize for processed blood units to be brought to the hospital from other blood banks (which follow strict guidelines). You could of course charge a premium on it.


For the reader: If you can organize for donors, then please do. There is indeed a shortage of blood. But be aware that you are not obligated to.
Bangalore itself needs an average of 800 units of blood everyday!!! Do encourage and participate in blood donation drives. It's the best way to ensure there is safe blood available for everyone in need. Encourage hospitals to organize frequent blood donation camps instead of just complacently indulging them with replacement donors.


Some points from the links above:
  • With no official complaints raised by the public, corporate blood banks have no incentive to walk away from replacement blood. “Grievances can be filed at the PM’s portal,” Centralised Public Grievance Redress and Monitoring System (CPGRAMS), “which collates grievances from multiple sources and forwards them to the state government, nodal officer, or blood bank concerned,” says Dr Shobini.
  • Shobini Rajan, assistant director general, National Aids Control Organisation (NACO), said the blood policy doesn't say it is illegal for hospitals to ask for donors but it is undesirable.

More on blood donation: http://nrecursions.blogspot.in/2015/02/blood-donation-what-went-right.html
And volunteering: http://nrecursions.blogspot.in/2015/07/how-to-start-volunteering-or-how-to.html

09 December 2017

Forgot BIOS password and removing CMOS battery does not work?

In the "good old days", if you forgot your BIOS password, all you had to do was open up the computer, remove the CMOS battery or remove the jumper near the battery and your CMOS password would get reset to no password.

That's changed now. I recently reset my laptop to CMOS factory default settings, saved and exited, and it asked for a password the next time I wanted to enter the CMOS settings. I had not even set a password!!!

Opening up the laptop and finding the CMOS battery was an interesting proposition, so I did just that. Removed the battery, replaced it after 30 seconds, put back all components, started the laptop and it still asked for a password. Darn! Going through the laptop manual showed that the battery had to be removed and kept aside for at least an hour.
Ok good. So I did that. Waited for more than an hour, put back all components, started the laptop and bang. There was the password prompt again!
Turns out that there was a person who even kept the battery aside for three months and yet the password didn't reset. It's likely to be stored in a more permanent location in memory in today's computers.

The solution:
After a lot of anxious searching, I found this website. The ultimate guide to resetting the BIOS password.
Turns out that when you take your computer to the official service people, they don't need to know your BIOS password or hack into it with brute force techniques.
They simply use a default password that the manufacturer programs into the motherboard. These are called backdoor codes.

So all you have to do is try out some of the openly available backdoor codes on the website, and if that doesn't work, then try entering some random password three times and your BIOS password prompt will show you a number. Enter that number into this website https://bios-pw.org/ and it'll show you the backdoor password for your BIOS.

That's it! I was able to access my CMOS settings screen again and this time I set my own password.
Do try this before you take your laptop to some service person who'd use the same simple technique and then charge you a bomb because you'd think they used some super secret hacking technique to elegantly solve your problem for you.

30 October 2017

You can check your voter details online

The draft electoral rolls for Bangalore are out already, and voters can now check/edit their details online.

There is a regional website where the backend didn't function properly, so I'd recommend going to the other regional website which takes you to the national website. Sadly, none of the websites have https, so anything you enter there could be viewed by someone else. Your browser will inform you about this the moment the webpage opens.

Once on the national website, you can search for your existing voter information. I typed in my details and it didn't find anything, but on clicking the Search by EPIC number tab, I entered my voter ID and was shown my details which were exactly what I had entered earlier, and the backend didn't even return an exact name search match. Since the websites were powered by C-DAC, I wrote to them and hope they'd introduce HTTPS.

Still, as long as you are ok with not using https, you can use the website to submit forms online and track your application status online.
Forms are available for:
  • Inclusion of names for residents electors    
  • Inclusion of names for overseas electors    
  • Any objection on inclusion of names       
  • Correction of entries in the Electoral Rolls     
  • Transposition within Assembly

Some thoughtful person has also created a nice caricature document explaining general voter information. Do go through it and show it to kids.

22 October 2017

Do a 'Google Trends' on Google Books

This may actually be quite a good incentives for scholars and literature enthusiasts, for uploading more books online.
Similar to how Google Trends gives you graphs to compare various Google search terms used over the years, Google Ngram viewer allows you to search and compare the usage of words and phrases used in books over the years. You can even add wildcard characters like the asterisk, and even specify if what you are searching for is a noun or an adjective etc. Two researchers, Sparavinga and R.Marazzato even wrote a paper on it.

For example, searching for "how are you" vs "how do you do" shows that "how do you do" was used more in the 1700's, but slowly the usage declined. On a side note: the reply to "how do you do" is "how do you do", and not "I'm fine".

More on how Ngram works: https://books.google.com/ngrams/info#

If your work is primarily on Google Scholar, there are good search tools (including advanced search) and metrics information available to gauge the visibility and influence of articles published. Useful tools.

21 October 2017

Mostly, time spent online is time well spent

Which is better? Finding a bunch of intellectuals online, who love discussing your favourite topics on artificial intelligence, general knowledge, TED talks, research efforts worldwide or switching off your phone and computer and spending time with relatives who gossip about their neighbours, discuss cricket runs scored and discuss religious rituals? *yawn*

Have a look at this toon from Wumo.

Interacting online has the advantage of filtering out a HUGE amount of irrelevant data, awkwardness and annoyances. If you like it, there's nothing wrong with liking it. Time is precious to all of us. Let's spend it doing what we like, and with the people we like. Whether the people are online or physically present with you is irrelevant.
It's the genuineness of your connection with them that matters.

That being said, physical meetings have a flavour of their own, and should be encouraged at the comfort level that people would like it to be encouraged.

As for Simon Sinek's narrow observation...

...it's easy to use the pointing finger to highlight "what's wrong with this generation", but one has to know that this is what every generation has been saying about the next generation for the past millenia.

The truth is, there's nothing ever wrong with a generation. There's nothing ever wrong with how people do things. Some people build good relationships, some don't. Some people adopt and use technology in specific ways, some don't. Some people will always be less confident and unhappy, some won't. That's how we are meant to be as a population. There's nothing wrong in that.

Every single species on this planet was not meant to live and behave in a certain way forever. We evolve. We are meant to evolve. When you see someone doing something different, it's because that's what they are meant to do. We as a generation, have better access to information. We know and understand relationships better. We know whom to connect with and when to run away from toxic people. We've even researched and spread awareness about addiction to social media. We learn from people all around the world.

Accept change for the good it brings. For as someone once said: There is only one constant: "Change". Rather than point fingers at a generation of people, see how these well informed, intelligent people can help you and you'll be surprised!

12 September 2017

A tutorial on Differential Evolution and Otsu's method for image thresholding

The first thing you need to know before your foray into computational intelligence (CI), is that the creators of these algorithms themselves don't know how it works. CI algorithms are inherently analytical techniques that depend on randomness to give solutions that are close to optimal, but not necessarily the best solution. It's like finding a lovely house in a city and choosing it as your home, but you have not actually found out if it is the best house that you could find in the city, because you just did not have the time to explore. The fact that people come up with naming these algorithms after animals (bat algorithm), insects (ant colony, bee colony, glowworm algorithms) and bacteria (bacterial foraging algorithm) is an amusing practice because the lack of theoretical backing makes the creation and optimization of the algorithm more of an art than a science.

Still, the way these algorithms improve over the generations is very impressive. This video shows you how:

This tutorial covers the topic of finding thresholds for a grayscale image using a CI algorithm named Differential Evolution (DE). We check if the thresholds are good enough by using the Otsu criterion. The best references for DE and Otsu criterion are the original papers presented here and here.

What is image thresholding?
Consider Lena's image. When you convert the red, blue and green pixels to gray colours, you end up with colours ranging from 0 to 255. Zero is black. 255 is white and the numbers in between are shades of gray (I'm not talking of Fifty shades of gray). Now if you decide that all pixels below the shade 118 are going to be converted to black and all pixels above or equal to 118 are going to be white, you end up with an image like this:

This is called "image thresholding" or "image segmentation". There are more complex ways of doing image segmentation though. The basic idea is to try and separate the image into distinct segments based on our interest. Sometimes we want to distinguish between foreground and background. Sometimes, to distinguish between parts of objects or multiple objects. If the threshold were closer to 50 or 10, the image would appear too dark. So we have to figure out which threshold will give us an optimum image. An image where the pixels are shaded in such a way that the objects in the image are well distinguished from other objects. An "optimal threshold".

So how did we find this threshold at 118?
We used the "Otsu" technique. A person named Otsu figured out that a simple and nice way to segment images was to first create a histogram of the pixels and then statistically analyze the histogram.
This is what Lena's histogram looks like:

The vertical columns are the count of the pixels of a particular shade of gray. Notice the bottom of the graph showing the shades of gray. At shade 100, the vertical column just crosses 2000. It means that the Lena image has a little more than 2000 pixels which have the gray shade of 100.  These vertical columns are called "bins".

In statistics, there is a concept of "class", which is a group of objects with a similar property. So in the above histogram, we want to find a threshold which would separate the pixels into two classes. Black pixels and white pixels.
Otsu figured out that he could statistically find the best combination of classes (and hence the location of the best threshold), by examining the mean and variances of the classes. He created a formula and called it "finding the between class variance". This between class variance will be used as a value that gives us the "fitness" of the image. Higher the value, better the threshold.
If there are two thresholds, there would be three classes. As shown in the image below, the image would be eventually shown with all pixels below grayscale 32, as white. Anything equal to or above 32 and below 106, as gray colour 32 and any pixel equal to or above 106 as white.

The calculation goes like this:

The gray histogram is first normalized. Each of the 0 to 255 bins of the histogram are iterated as i.

probability of i = number of pixels in bin i / total number of pixels in image
total mean = sum of probability of i of all bins

If there are t thresholds, then there will be t+1 number of classes c the image can be segmented into. So with a threshold of t=1, there would be classes c1 and c2. Class mean values would also be computed as mean1 and mean2. Or, as shown in Figure 2.1, if there are two thresholds t1 and t2, there would be three classes and therefore three mean values calculated.

pc = probability of class occurrence c = sum of probability of i of all bins in c
class mean = sum of (i * probability of i of all bins in c) / class occurrence c

Class variances of each class c are calculated as:

class variance = sum of ((i - class mean)2 * probability of i / probability of class occurrence c)

The variance between the classes gives the fitness of the threshold. 

between class variance = sum for all bins(pc * (class mean - total mean)2

Note that the between class variance (BCV) is one single value. You don't calculate between class variances for each threshold and sum them up. You use the one formula given above to calculate it.

Ok, so now we know how to calculate the fitness of a threshold, we can write a simple for loop that places the threshold at the zero'th position in the histogram and calculates the BCV. Then move the threshold to position 1 of the histogram and find the BCV and keep going on for values 2, 3, 4 ... till 255. The threshold with the highest BCV will be the optimal threshold. For a single threshold of the Lena image, it is at 118.

Now you'd say. Hey, that's it? We just have a search space of 0 to 255? Then why do we need a computational intelligence algorithm? Couldn't we just use normal for loops?

Well, for one threshold you can do an exhaustive search from 0 to 255. For two thresholds, you'd have to run two loops nested. So that's 2552. For four thresholds, the number of iterations are 4294967296. It would take 95 days to compute on Matlab.

Enter the saviour: Computational Intelligence algorithm: Differential Evolution:

The basic idea of DE is to place thresholds randomly within 0 and 255, and keep evaluating the BCV. With some luck, you will get good thresholds which won't be the best, but will be good enough.

The secret weapon of DE is "mutation" and "crossover". Similar to how we have a population of humans and our genes undergo mutations and crossover of traits, the DE algorithm initializes a 'population' of vectors. If we are solving a problem of finding two optimal thresholds, then each vector will contain two random thresholds. During the DE algorithm, these vectors will be combined randomly with other vectors in the population to provide some variety to the result, and somehow, we end up finding near-optimal thresholds. Nobody knows how, but it works. Magic.

If there are 4 thresholds and a population of 3 vectors, the vectors would look like this:
X1 = [t1; t2; t3; t4];
X2 = [t1; t2; t3; t4];
X3 = [t1; t2; t3; t4];
Where, the t1 of X1 does not have to be the same as the t1 in X2 and so on.

This is the pseudocode for my DE algorithm:

  • Load image and convert to 8 bit grayscale
  • Generate histogram from image. Bins ranging from 0 to 255
  • Initialize DE parameters: PopulationSize, thresh, crossover probability (cr), beta.
  • Set thresh number of thresholds each, randomly within bin space for each population entity X.
  • Evaluate initial fitness of X
  • forEach generation for a defined number of iterations
  • forEach population p for PopulationSize
  • Select population entity p from X.
  • Select 3 distinct populations from X. The thresholds should not be equal to p.
  • Generate mutant Up = X1 + beta * (X2 – X3)
  • If Up == Xp, regenerate threshold at random position for Up and continue;
  • Crossover at least one threshold from X to U and other thresholds if generated rnd <= cr
  • If mutant thresholds are outside histogram range, regenerate threshold at random position
  • end for population
  • Evaluate fitness of Up
  • forEach population p for PopulationSize
  • If fittest mutants in Up are fitter than fittest Xp, replace Xp with fittest Up
  • end for population
  • decrease beta by 40-1
  • end for generation

Beta value: Ranges from 0 to 2. A high beta value ensures that the mixing of vector values have a much larger variation, so the mutated vector will end up far from the existing vectors. This is called "exploration", and helps visiting many other threshold positions within the 0 to 255 values. Smaller values of beta keep the variations of threshold values closer to the existing thresholds. This is called "exploitation", as you have found a good threshold already and are trying to exploit that area to find better thresholds in the area. Beta is also called the constant of differentiation.
The crossover probability: Ranges from 0 to 1. Having a low crossover probability like 0.2 was found to help reach an optimal threshold faster, but it does not necessarily reach the most optimal threshold. A higher value like 0.8 helps reach a more optimal threshold, but it can take three times more number of generations to do so. Setting cr=0.3 appeared to help reach an optimal threshold with a slightly higher number of generations than cr=0.2. Allows exploitation of symmetry and separability of a function.

That's all there is to it. You can have multiple thresholds and the time of execution is just around a minute or two in Matlab even for 6 or eight thresholds. This is a major advantage of CI algorithms. You could achieve the same results with genetic algorithms, particle swarm optimization, bee colony algorithm or bacterial foraging algorithms.

The code for the DE algorithm and Otsu method, I've listed here: https://github.com/nav9/differentialEvolution

31 July 2017

Creating a dual boot with Windows and Ubuntu when you have UEFI

People on many blogs say that you have to disable secure boot when you have UEFI and want to install Ubuntu alongside Windows 8 or 10.
Not necessary. You don't even have to switch to Legacy boot.

The basic concept to understand is, that if you have UEFI, you'll have to tell the boot manager where to find your boot file. That's all.

After a lot of searching and ending up at all the wrong blogs and posts, I finally found this post. Worked perfectly!

The steps are:
  • Install Windows, leaving a separate partition for Ubuntu
  • Install Ubuntu on that partition
  • Reboot and enter BIOS screen
  • Follow the steps here about selecting grubx64.efi
  • Save and exit BIOS

That's it! You've got a dual boot system. Grub will show you the options for booting into either Windows or Ubuntu.
If anything goes wrong, you can always press F12 when starting your computer to see the boot options.

Oh btw, don't disable secure boot. It's a very important feature that prevents your system from getting infected at boot time, by malware that might be in an external drive that's connected to your PC at boot time.

27 July 2017

Our Income Tax department has a strange maintenance message

I logged into my income tax e-filing account yesterday, and saw this message: "You are not authorized to view this page because either your bank details of your PAN and profile details are incorrect or these details have not been passed onto e-Filing. Please contact the owner of the information for details".

I was completely perplexed. I had filed my taxes correctly, so why this message? I couldn't even access my dashboard or file a grievance.

A bit of Googling revealed that few other people had faced the same issue since 2014. One of the answers on a forum said that this message shows up when the website is undergoing some routine maintenance, and we should check back after six to twelve hours.

I logged back in the next day and sure enough, everything was back to normal.


My guess is that the message appears when there are too many connection requests to the server. I do hope our e-filing department changes that message to a less-scary one like "Routine website maintenance ongoing. Please check back after few hours".

This is for all of you who were perplexed by this message and didn't know whom to ask. Hope it helped.

18 June 2017

How to download files or an entire folder from Google Drive?

I've found it very strange that a product of Google didn't have a straightforward way to download files or an entire folder in one shot.

There is a way though:
  • As shown in the image below, click the folder or file you want to download
  • Then click the menu button to open the menu.
  • Then if you don't see a download option in the menu, just click the empty area at the bottom of the menu.

You'll get your folder in zipped format.

I assume it's just a bug that the download icon isn't visible.
Another way to download is to double click the file to open it and on the top right corner of the screen you'll see the download icon.

UPDATE: I wrote to Google support about this issue, and as of August,  the CSS issue with the hidden download button is fixed. There also was another issue of people using Chrome being unable to download files. Google told me they are working on fixing that too. Until then, you could just use incognito mode to download.

14 June 2017

Wear that helmet. It's actually worth it

Few years ago when taking my friend back home on my bike, he noticed me taking my helmet and asked "You wear a helmet for short distances too?".

I couldn't really believe what I was hearing. Still, there is a large number of people who hate helmets because it's claustrophobic, because they think it causes hair loss (it does not), because they actually feel they'd never get into an accident and sometimes simply because they can't afford one.

The safety lectures are not the only reason to wear them:

Stones, insects, tree branches: Even when riding a bike at 40kmph, these objects can hit your face at high speeds. Have a full face helmet with a visor. I've encountered these projectiles at least once a month. The stones that get half under a vehicle tyre and shoot out at you are particularly dangerous. A colleague's car windshield looks like as though a bullet hit against bulletproof glass and left a dent there. It was hit by one such stone.

Wind, dust, sunlight and rain: Very useful to be protected with a helmet.
And of course...safety: The visor protects your nose and the front of your face from getting damaged too much. A full face helmet protects the sides of your face (which are most probable for damage during an accident).

Look at all the pics here.

Get a helmet. Wear it.

21 May 2017

Writing GATE exam of another branch

I've written about how I was forced into doing Mechanical engineering even though what really clicked with me was computer science. Ever since then there was a desire to pursue an education in CS.

So what can a Mechanical Engineer do to get a degree in Computer Science? Either write GATE or PGCET. The PGCET people were obstinate about not allowing a Mech engineer to write a CS paper, but the GATE organizers (bless them) agreed.
With five months of preparation time and a work schedule which many-a-time required me to work on weekends, I started preparing. Even though a week before the exam I couldn't study due to work deadlines, I wrote GATE and waited...remembering the amusing moment when the examiners at the centre, double-checked my photo ID with my face, wondering if I was a genuine candidate (others who wrote the exam were much younger).

Results were declared and I was among the top 11% of candidates across India. To me it initially didn't seem a big deal that I qualified. It was only later that a friend from IISc told me that qualifying in GATE itself is a big deal. On top of that, I cleared GATE from a different branch than what I studied in my bachelors. This was accentuated by my meeting some CS graduates who told me that they couldn't clear GATE when they wrote it.

Even more surprising:  Only 16% of the students had qualified. Articles from HindustanTimes and IndiaToday.

I also wrote back to the GATE organizers thanking them for the opportunity and the organizing chairman Prof.G.J.Chakrapani replied back with his best wishes.

What helps with preparing for GATE?

The wonderful people who created GateOverflow. The single best resource on the internet. There's a PDF you can download which contains questions from previous years, segregated subject-wise.
They also have a stackoverflow-style forum where you can ask your doubts.
This is a very useful piece of knowledge, because it lets you know how you have to study the concept, in order to be prepared for the exam.

Now I'm finally doing my masters in Computer Science. A bunch of mind boggling mathematics as part of the syllabus, but I'm glad I got here.

And to the socially conscious parents...

I still meet people who have newly become parents, who are already planning to send their child to college to become a doctor or engineer. I ask them if they'd be willing to consider what their child's interests are, and they don't really care.
I feel sad for their kids.
A bunch of lovely dreams squashed by the perception of social status and bullying.

As long as the education system and the industry allows people to switch to what they like doing, I feel our people and our nation will have a bright future. Better still, might be the concept shown in Man of Steel, where the genetic makeup of a child is determined at birth and they are trained from childhood to become the best at whatever career they are genetically and mentally most inclined to like and perform.

"Knowledge is most precious to the one who truly seeks and respects it"
- Navin

28 April 2017

What exactly is the "mothers love" part of home cooked food?

One of my seniors in college always loved saying (whenever we were at a restaurant) that mother's food is always the best food because it is sprinkled with a necessary ingredient: Love.

I mentioned the same thing to a pragmatic uncle of mine, and his immediate response was "Bull***t!" :-)
At least he wasn't of the opinion that the taste of food comes from an emotion.

So what exactly is the "mothers love in home-cooked food" or to ask it differently, what makes home cooked food more healthy? What makes outside food bad?

Outside food answers you are familiar with:
  • Artificial ingredients
  • Too many spices
  • Reused oil

But that's not all. There are some things you've never noticed because your stomach was either resilient enough to handle it or you just didn't know:
  • The food was not cooked fully (many cooks aren't really cooks. They were just looking for a job and someone taught them the basics and they have too many customers to serve. Who has the time to cook food fully?)
  • The food has burnt particles in it (I know there are a lot of silly people who have told you that the black portions of rotis and parathas are ok. They are not)
  • The ingredients used were of low quality
  • The food contains H Pylori. Causes horribly painful stomach aches.

So the "Love" in the food is...

The love is the care with which the food is cooked. Not every mother can do that though. Many of them hate being in the kitchen and many aren't very knowledgeable about cooking. But the more access to knowledge the mother or father or any family member has, the better their cooking will be and the more healthy it will be for you.
There's one other thing: Certain foods get associated with memories of one's mother or family. For example, there are friends of mine who love Ragi Mudde (plenty of childhood memories associated with it). Others can't even stand the taste of it.

If you don't have access to healthy home cooked food, buy a stove (even if it's an electric induction stove) and learn to cook your own food. Trust me, it'll be entirely worth it. If you live in a place that does not allow you to cook, then leave that place and find a place where you can cook.
Invest in your health and you will reap benefits. Ignore the food part of health and every other part of your body will suffer.

ps: On a side note, if you've heard of people saying that eating with the fingers is tastier than eating with a spoon but were never able to explain why; I have the answer for you. I asked this question on skeptics.stackexchange.com and here's the answer:

There is scientific research which says that touching (tactile sensations) itself, not just the changes in the diffusion of taste and odor compounds with different textures, can adjust the perception of taste and smell. The studies don't specifically say that the taste is always improved, but tactile sensations affect the perceived taste. 

18 March 2017

Getting a quick summary of table sizes and rows in MySQL

Just a quick post to show you a technique my colleague showed me on how to get the sizes of tables and an approximate number of rows it has, without having to wait long for a select count(*) from table.

Use this command:

             table_schema as `Database`,
             table_name AS `Table`,
             table_rows AS "Quant of Rows",
             round(((data_length + index_length) / 1024 / 1024/ 1024), 2) `Size in GB`
        FROM information_schema.TABLES
        WHERE table_schema = 'ballast'
        ORDER BY (data_length + index_length) DESC;

This will show you a result like this:

| Database     Table                           Quant of Rows  Size in GB
| ball            TableHosp2015               110654400             11.05
| ball            TableHosp2016               115890383             10.30
| ball            Table2016_temp                87027666               6.49
| ball            Table2014_temp                73765370               5.78
| ball            Table2015                          62064795               4.86
| ball            Distances                               158372               0.02

13 March 2017

The Indian Education System inadvertently trains you to deal with customers

The first part of this blog post is anecdotal.

As Indians, we tend to complain a lot about our education system. About how it focuses only on securing marks and does not spark interest in students.

A neighbour recently asked me for my book on C++, saying she wanted a few theory concepts to write as part of an assignment her teacher gave.

As the book was an old one, I told her the best place to get her info was on the internet.
Her answer surprised me. She said she wanted to refer the theoretical concepts in the book and write those for her assignments because that's what the teacher would expect. Scoring the marks was of a higher priority than learning the theory.

Two ways of looking at it:

Way One: Here's where you start cursing the Indian education system which designed assignments only for the purpose of scoring marks. Pull your hair out and ask for the system to be changed so that kids would have better practical programming knowledge.

Way Two: Our Indian education system had just given birth to another little girl who saw the problem from the customer's perspective. This is exactly what every CEO wants their sales people to understand.

Find out what the customer wants and tailor your response and product to that. You make the sale!!!

My little neighbour learnt how to anticipate what the teacher wanted and gave the teacher exactly that. Marks scored!!!
You see students all over India finding such shortcuts which get the job done. Finding innovative ways of impressing the teacher, pretending to be extra humble for some leniency or marks, pulling strings to get hold of the studious guy/girl's notes...and so many more...

Rings a bell? Somebody's famous quote of "I will always choose a lazy person to do a difficult job because a lazy person will find an easy way to do it".

We are expected to do this as engineers too. If there's a solution already available, just use it instead of spending time doing things from scratch and allowing the competition to gain an advantage over us.

The customer focus:

As a person who designs the Architecture of software, my primary goal is to ensure that the customer's requirements are fulfilled, instead of choosing an architecture and technology that would look good on my resume/CV.
If the customer's work can be done with a simple technology, it is better to use that, than to choose a different, more complex, more time consuming technology that would add a lot of punch to your portfolio, but would be of little or no value to the customer.

When you do find people who care about what the customer wants, treat them like a precious gem. You have a valuable find.

ps: This is not to misunderstand those genuine people who spend time understanding concepts and improving on the theoretical understanding. This is also expected of good engineers. To be well informed, educated and at the top of the game. These engineers are a league apart. Inventors. Scientists. People who create value for the world.

27 February 2017

12 February 2017

How to speed up MySQL inserts

One would assume that if an application had two threads, and while one thread inserted data into one table and the other thread inserted data into another table, the inserts would be faster than if both threads inserted into the same table? No. It isn't faster.
The various threads that MySQL has, are to manage the other different operations.

There however is a technique that can improve insert speed further.

Smart programmers already know that using a batch insert is more efficient and faster than calling INSERT multiple times from their program.

What you perhaps do not know, is that after you send the batch of insert statements to MySQL, they are executed internally as separate inserts.

INSERT INTO Table1 VALUES (1,2,3) 

What if you could combine it into a single insert statement like this:

INSERT INTO X VALUES (1,2,3),(4,5,6)

It'll actually execute faster! Some people report a 2x increase in write speed. Others, report a 40x increase.

How to do it? 

Just set the rewriteBatchedStatements=true value in the connection string.

So if you normally connect to MySQL like this...

Connection conn = DriverManager.getConnection("jdbc:mysql://"+ domainName +"/"+configRef.getBallastDatabaseNameOfSQL()+"?" + "user=" + username + "&password=" + password);

...you just have to change it to:

Connection conn = DriverManager.getConnection("jdbc:mysql://"+ domainName +"/"+configRef.getBallastDatabaseNameOfSQL()+"?" + "user=" + username + "&password=" + password + "&rewriteBatchedStatements=true");

In my opinion, MySQL should have kept this as default behaviour and the user should have had to disable it only if they were doing very large batch inserts.

Do try this out even if you don't execute too many inserts. In applications dealing with a lot of data though, this option can come as a life saver! You save a humongous amount of time with these super fast inserts.

Avoiding memory problems when working with huge amounts of data

There would be times when you run applications with libraries like Apache Storm or similar programs, and you'd be surprised when these applications crash and restart with no apparent reason.

The reason in many cases (as you'll infer from the error logs) is that the application ran out of memory. You just didn't have enough RAM.

Why this happens:
If you specify (using -Xmx2g) that your application can use a maximum of 2GB memory, you have to remember that there are other applications running that might also be free to use that much memory.

Let's say you have 4GB RAM.
App1 takes up 2GB.
App2 takes up 2GB.

App1 deletes some large datastructures that it's using, before creating a new datastructure.

The request for creation of a new datastructure makes the Garbage Collector try to allocate more memory. Sometimes, operations happen so quickly that the GC has no time to remove the unused memory before allocating new memory.

Since you've run out of RAM space, the OS would typically allow your application to use virtual memory. To do that, you need a swap space. If you don't have one, your application might crash with the message that there was insufficient memory. Your application might not even catch this exception and yet crash.

In many managed servers like AWS, the nodes don't have a default swap space.  You have to create one. It's very simple and completely worth it.

How much swap space to create?
Go for double the size of your existing RAM. I know that sounds like a lot, but believe me; I've seen how a system with 4GB RAM has used more than 4GB of swap space when I had allocated it 10GB of swap space. Hardware is cheap, and we now have disks with plenty of space, so create large swap partitions/files.

How to create the swap space?
First check if it's already there with

free -m

and check for how much disk space is available with

df -h

On an Ubuntu 14.04 system, just run these commands:

sudo fallocate -l 6G /swapfile (the "6G" means 6GB of disk space)

sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

To make the swap file permanent:

sudo vi /etc/fstab

and at the bottom of the fstab file, add this line (press tab instead of a space wherever you see a space in the line below):
/swapfile   none    swap    sw    0   0

If you like tweaking things more, you can also tweak the swappiness setting which determines how often the OS will transfer data from the RAM to swap space.

30 January 2017

22 January 2017

USB tethering. Use internet on your Desktop PC via mobile phone internet

Whenever I've used internet via a mobile phone's WiFi Hotspot, I've always wondered if it was transmitting extra data just for the sake of being in sync with the laptop. Or perhaps adding extra bits to the packets when transferring data.
Turns out it is true. Using mobile hotspot actually takes up more data.

While some people have suggested tethering your mobile with the laptop using Bluetooth tethering, which is said to (but I haven't verified it) use lesser data, I feel a wired connection would be much better.

That alternative I found in USB tethering.

All this while I was actually considering buying a WiFi network card for my desktop PC, and I came across USB tethering which solved my problem elegantly and with no additional cost.

How to do it?
  1. Just take the cable you usually connect your mobile phone and desktop PC or laptop with, for transferring data via USB.
  2. Go to your mobile phone settings and activate USB tethering.
  3. Switch on your mobile phone internet.

That's it. Your phone is now a modem. You can use internet on your desktop PC or laptop. Best tethering option because your phone is connected to the PC, and it even gets charged in the process.

Kudos to those who came up with these brilliant solutions!

20 January 2017

Viruses in Indian cybercafe's

Happens all the time. Take a pen drive with a word file to get a printout at a cyber cafe or a shop where they do printouts, and you come back with a virus infected pen drive.
At least this used to be ok as long as the virus didn't do anything to the file. You could still take the printout. Now-a-days, there's a bigger problem: The virus copies all the files into a hidden, unnamed folder and all you see is an empty pen drive. How are we to take a printout now?

Sadly, the popularity of Microsoft Windows and the availability of pirated copies is one factor that contributes to the spread of these viruses. What's worse, the cyber cafe owners refuse to believe that their computer has a virus!
I wonder when people will realize that pirated operating systems are very likely to have backdoors and more malware? Something you have to be very aware of when you type your passwords in cyber cafe's.

Having an antivirus is not enough. I've seen numerous cases where antiviruses are unable to detect viruses and worms. An antivirus detects virus signatures (a certain pattern of characters that a virus file contains). When the creators realized this, they started creating viruses that would alter their signature whenever they copied themselves to another disk, using an encryptor and decryptor. Then came polymorphic viruses which could change the signature of the encryptor and decryptor too. Antiviruses also use heuristics (virus behaviour like replication, overwriting files etc) to find viruses.

The alternative

The better way to stay safe, is to use an operating system which is heavily monitored by a community of voluntary software programmers all across the world. People who are dedicated to maintaining security and privacy.

One such operating system is Linux. Although some of the other variants of Linux were good earlier, I now dislike the interfaces of Fedora and CentOS.
One good Linux operating system you could use is Ubuntu. It's as good as Windows, it is safe and it is free of cost. Ten years ago, you could email the Ubuntu community and they would send you an Ubuntu CD for free by post. They've stopped that now of course, but you can still download Ubuntu and either burn it onto a CD or create a bootable pen drive from which you can either try Ubuntu or install it into your computer.

The best part is, that you don't even have to install Ubuntu to try it out. Just create a liveCD or do the same in a USB stick and you can run it to see if you like it.

Do note that if you have an old computer that cannot boot from a pen drive, then you will have to burn Ubuntu onto a CD or a DVD to create a bootable disk which can be used to try or install Ubuntu.

A message for Cyber Cafe owners in India

Enough of viruses and malware. Please install Linux on all computers you have.
If your users still want to use Windows, then you can install Linux and have a free Virtual Box in Linux inside which you can install Windows.

Some users would want to use Microsoft Word and Excel on the computer. Linux has an alternative named Libre office which comes pre installed with Linux.
If you don't like Libre office, you can always use the awesome Google Docs, Google Sheets etc.

One thing I would advise you should check though, is the compatibility of your printers and scanners with Linux. Do this before you install Linux, by trying out Linux with the Live CD option. Take a few sample printouts and scans and see if everything works fine. If any old printers need drivers to be installed, you could either contact the vendor and ask about linux drivers or search the internet using the device model number and I'm very sure you would find free drivers which you can download and install for Linux. The modern printers I think would work without any extra drivers. Do try it out.

Why I'm writing this

I went to a cyber cafe to take a printout and ended up with a virus that messed up the pen drive's partition table (and also hid the files, so I couldn't take the printout). Came home and had a hard time restoring the partition table.
For anyone who encounters this error:

"The driver descriptor says the physical block size is 2048 bytes, but Linux says it is 512 bytes."

Use this command to fix the blocks on your pen drive:

sudo dd if=/dev/zero of=/dev/id_of_your_device bs=2048; sync

Most likely, the id_of_your_device would be "sdb". Make sure you confirm that, because if you choose "sda" and it happens to be your hard disk, all the data on it will get permanently erased. 

The dd command is used to: Convert and copy a file, write disk headers, boot records, create a boot floppy. dd can make an exact clone of an (unmounted) disk, this will include all blank space so the output destination must be at least as large as the input. In this specific case, these posts will help you understand the meaning of the bs and 2048.

Did that, and the disk is as good as new!

15 January 2017

Visits to some North Indian tourist spots [Part 2]

Continued from Part 1

Disclaimer: Views expressed here are based on a single visit, so would be lacking a complete perspective. Advice from locals are also included.

When in North India, have a plan for food. Especially if it's tough for you to handle poorly cooked food or food with burnt particles in it.
Most people in the north are accustomed to eating chapathis, rotis, naans and parathas which have black burnt spots on them. They feel it is normal because it is so prevalent there. Burnt spots are actually not a part of the recipe.
See this.
Eating burnt food causes cancer, and other health complications.
Try searching for South Indian restaurants where they take care to cook food properly (Saravana bhavan was one example). Idli's and dosa's early in the morning are a safe bet. Try not to eat from restaurants where the chutney is too watery. You can actually check that by walking up to the kitchen and asking them to show a sample, before you sit at the table.

Your body produces heat via the muscles (which is why you shiver), and having a good amount of nutrition is essential for your muscles. In cold weather, make sure you have full meals which are well cooked and do not have burnt particles in them. Even if you have to eat at an expensive restaurant, it is worth it. Buy fruits too, but only after verifying that they are sufficiently ripe.
You'll notice that it's very tough to find bakeries there. It's always handy to have some bread with you, and if your stomach suffers from having eaten restaurant food, drinking some curd will give relief. You could also buy some steamed rice, buy the curd separately, mix them and have curd rice.
Also, drink water constantly whenever you feel a bit thirsty. Hydration is important during travel.
You'll get 1L mineral water bottles of reputed brands for Rs.20 even from chemist shops. The 2L bottles cost Rs.30, so it's better to buy those. Refuse to pay more than the MRP. Even for soft drinks kept in cold storage. It's illegal and there are shops where you can find it at MRP too. Both in Delhi and Agra. Even at the bus stands.

Shimla and Kufri

Note: I haven't traveled much in Shimla, so these views may be very biased. There would be a lot of other people who would appreciate these places for their beauty, heritage and resources. For a rational mind though, it's the experience that counts, and I didn't like it. These opinions are geared toward the precautions and realities.
These are also two overrated places. Perhaps they were good in the past, but now both places are worth avoiding because:
  • Tourist operators have taken tourism for granted. They will actually insult you and speak to you rudely if you do not accept their services or are not willing to pay the exorbitant amount of money they demand. Lies are also part of the package. You'll be charged Rs.2000+ for a little taxi drive of 12km to around 4 tourist spots. We traveled double the distance for a lesser price at Manali.
  • A friend tells me that people throw stones at people who bring their own vehicles for touring the place. I don't know how far this is true, but given what I saw about the nature of the tourist operators, I would be inclined to believe it.
  • The buildings and alleys are narrow and steep. One hotel area we went to was infested with monkeys.
  • A friend says it isn't worth visiting Shimla for the snow because the crowd is so great that all you see it boot snow. ie: Snow that has been walked upon so much that it's black.
  • Snow also means power cuts and no water supply. See the article from Times of India below.
Click image to see clearly

Perhaps some reasons to visit Shimla would be for the apples, the view, the snow at Kufri and the toy train ride. But given that you can get all of these at other places in India, it might actually be worth skipping Shimla and Kufri altogether. These places are overpriced and the tourist operators just bring a bad name to the place.

Why you could skip the tour operators at Kufri:
If you can get a bus or app based cab or a local friend or a bike which can take you to Kufri, then no problem. Go ahead. Enjoy the view and the snow during winter.
But. There are these few things you have to be aware of:
  • At Kufri, the tourist operators get a commission when you spend a lot of money at certain places, and they take you there for exactly that. Our guide made sure we went out of our route to take us to an ATM where we had the opportunity to withdraw enough of money. While we initially thought of him as an angel, we later realized it was so that he'd get his commission.
  • Hip Hip Hurray: The tourist guide had so highly recommended this place that we thought it would be a large area full of adventure activities, like Wonderla. It turned out to be an extremely extremely extremely tiny place with a few cramped up amusement activities. You can take part in each of those only once, unlike Wonderla where it is unlimited. And yet, they charge Rs.700 per head. What's worse, if you choose to skip it, they will actually start insulting you. Try avoiding the place altogether.
  • Skip the horse ride at the Kufri hill top: This is another place where the tourist guide gets a commission from the horse ride organizers. They'll tell you you have to climb up for 4km and 11k feet height to reach the top, and you won't be able to do so unless you take the horse ride for Rs.500. That seemed dumb to me. If a horse could climb, then so could I. For 10 minutes, we walked up a 45 degree incline huffing and puffing and after that the route was easily walkable at a 20 degree incline. The people walking down congratulated us on not taking the horse ride, as they said the hill top was just 1.5km of a walkable distance away and the organizers had cheated them by offering the ride for Rs.600 per head. They said even Rs.500 was not worth it because the amount you pay is only for the ride up. To ride down, you have to pay more. People who took the horse ride were even complaining they didn't enjoy it. One woman got off the horse out of fear because the hooves of the horse were slipping on the smooth rocks along the path. It didn't seem safe to ride a horse on a sharp incline.

Once on top of the hill, you'll have to pay an additional Rs.10 to enter an area where you pay Rs.50 if you want to get photographed with a Yak. Rs.50 for 5 shots with an air gun with which you shoot plastic bottles 20 feet away. You get to dress up in traditional attire and get photographed for a fee. You get some less than impressive mountain views and some makeshift stalls where you get cooked Maggi for Rs.60 (price for a packet is actually Rs.10).

Since we decided to skip the expensive Hip Hip Hurray and horse rides because we could see we were being cheated, our tour guide actually had the cheek to ask us "If you didn't have money to spend, why did you come here".

This is the reality of Shimla and Kufri.
You'll find snow and adventure for less than half the price and with zero insults and plenty of hospitality at other places in India. Go there instead.


The experience of being to Manali was better than Shimla, Kufri and Mussourie. Although at this time of the year many of the trees had shed their leaves, the people of Manali had not shed their hospitality. Everyone we met were warm, hospitable and very helpful.

Freshening up: If you haven't booked a hotel, you can freshen up at one of the government latrines. It's maintained reasonably clean, and you can get hot water from a geyser. It's Rs.10 for number1, Rs.20 for number2 and Rs.20 for bath. There's a separate wash basin with freezing water during winters. For women, the costs are lesser, and the caretaker there is polite and helpful.

Food at Manali: Unfortunately this sucks even in Manali. Restaurants; even the posh one I went to (hotel Kunzum) had food that wasn't cooked fully. The waiter there was however kind enough to seat me near a room heater which was a huge relief from the freezing cold.

Entertainment: New year sees a stage being setup at the main chowk at Manali, near the bus stand where drama, magic shows, songs and street plays are hosted. They also provide a public fire source near the stage where you can warm your hands.

Market: The goods here are reasonably priced (woolen clothing though is priced a bit more than Mussourie, but you can bargain), and you can get fruits, shoes, jackets etc. of good quality for a reasonable price. But make sure you bargain and are ready to walk off if you feel the price isn't right.

Tours: Taxi operators are willing to take you around for Rs.1500 with no receipt (which was far cheaper than Shimla, because in Manali they take you much farther distances). The operators were very polite and gave us a generous helping of whatever other information we needed to know about the place. You can visit places like Solang valley etc. A visit to the friendship peak though, needs more preparation and more than one day.


Food at Solang: At the adventure spot where paragliding is held, try to eat only the Maggi served there. I ordered rice and rajma and had to return it because it was full of large burnt pieces. The cook was cheeky enough to tell me it was tadka. They don't really care about the food they cook because they know you'll eat bad food anyway.

Paragliding: Rs.700 for the short and safer version. Rs.3000 for the one which launches you from much higher up. You use the cable car ride to reach the top of a hill where your paragliding guide and his friends hurriedly lay the parachute on the ground and within a few minutes, without any apparent safety checks, they ask you to run and the guide and you get launched into the air. For some people the ride is smooth. For others they oscillate in the air at such angles that we thought they would fall to the ground. We heard of a case where the parachute got caught onto something during launch and both the guide and the tourist got very badly injured. A colleague told me (of his experience, where) the string of the parachute got cut before they started, and the guide simply tied the cut strings into a knot and proceeded with the paragliding. Dangerous and a complete disregard for safety. We skipped it.

Cable car ride: At Rs.650 per head this is a tad expensive, but is a world class product and is a nice experience. You get to go up and come back down with that one ticket (unlike Kufri where the horse ride of Rs.600 was just for the way up). You might find the seats a bit dusty. We inferred it was from the paragliders who land on their bottom on the mud and then use the cable car to go up.

Skiing: After it snows, there is also skiing available at Manali, but I think you have to get trained for a few days before you actually attempt it.

Spring water: There was a place we were taken to where people said they got fresh spring water which they drank directly. Never drink water from such sources. The places where this water flows from can be infested with rodents and insects which do all their stuff in the water.

From Wulffmorgenthaler (now Wumo)

I'd recommend Manali. Nice adventure activities, good people. Reasonable rates. The food can improve though. Try to go during the season when you can get apples for very cheap prices.


Triveni ghat (Ganga, Yammuna and Saraswati rivers meet here)

Honestly, this is a boring place for people who aren't into religion. There's a small tempo style bus that'll take you to Rishikesh in 1.5 hours from Dehradun ISBT (Inter State Bus Terminal). Once you reach the bus stand, you could approach an auto rickshaw terminal where they give you a bill for a tourist ride to spots in Rishikesh. We were charged Rs.520 which we felt was too much.
Anyway, the auto driver loyally took us to spots like the Lakshman jhula etc. and my co travellers got to dip their feet in the Ganga river and bring back some Ganga jal which they had been taught was holy water. You also get to see the temple which was the founding center for Rishikesh as a religious pilgrimage spot. There's also a Banyan tree there which is actually 3 trees joined together.
The auto driver takes you to a place, and promises to come back in some time, once you are done visiting the spot. In one case he took so long to come back, we thought he cheated us. He came soon enough though. Turns out he didn't expect us to return so early and had gone for lunch. Many of the other temples looked like apartments that were converted into a temple.

Rishikesh is apparently not a good spot for shopping. The auto driver told us everything is priced 3 or 4 times more than usual because of being a tourist spot.

Everyone we met, recommended doing water rafting in Rishikesh, and I even overheard a girl saying "Bahut maza aaya water rafting karke" (she had a lot of fun rafting). Might have been nice to do it, though later a colleague mentioned how his raft turned upside down, all of them ended up in the water and were luckily saved. Apparently you have to sign a declaration stating that you know what danger you are getting into. All it would take is for you to slam your head onto a rock or get impaled by an underwater piece of wood wedged between rocks. There's also bungee jumping at Rishikesh.

Rishikesh as a place was crowded and narrow. A bit suffocating. Some places have a lot of monkeys, but the monkeys didn't harm us. Plenty of foreigners. Plenty of police around. Still, it didn't feel like a safe place to be in.


This was one beautiful city. Neat, wide roads, plenty of trees all around and plenty of open space. You get excellent quality curd there. The food wasn't very impressive though, and a tad expensive for what it was.

Auto rickshaws don't have a meter, and charge a reasonable fixed rate. The locals are aware of the rate and the rickshaw driver didn't charge us extra even though he knew we were not locals. He was willing to stop at multiple places when we wanted to buy things in between and gave us a cartload of information about whatever we wanted to know about the city. Very hospitable.

Make sure you visit the market at Sector 19. You'll get branded clothing for 1/3rd of the MRP, because they are manufactured just a little distance away in Ludhiana. A classmate of mine with his own clothing manufacturing business says that you can bargain even at the fixed price shops in this market, and you are actually expected to bargain. When you start walking away they'll come to you with a lower price. Buy as much as you can from Chandigarh. Woolens, jeans, pyjamas, shoes...whatever clothing items you need. Also note that there are different quality of woolen clothing available. Learn to identify it.

It's worth visiting Chandigarh to buy clothing before you go to the colder places like Manali and Shimla. It's also a good place to do some shopping to buy things for taking back home.

The International airport: You need to make sure you aren't taken to the old airport at Chandigarh. Go to the Chandigarh International Airport, which the elderly Ola taxi driver told us is not actually in Chandigarh, but in Mohali. Throughout the ride he told us about the Sikh religion, the generosity of Gurudwaras, the culture in Punjab where if there's an accident on the road, no vehicle would go forward until the accident victim was helped. He also recommended we try the Makkey di roti, Sarson da saag, lassi and the parathas of Punjab. To which I mentioned that I developed stomach problems after eating burnt parathas and roti's for the past few days. To this he said that parathas will be well cooked and not burnt if they are cooked properly.
This is what inspired my question on Seasoned Advice where I got it confirmed.

Be aware though, that Chandigarh airport sees a few flight cancellations during winter because of the fog. As of 2017, they haven't yet upgraded to CAT III. Might be better to book a flight from Delhi.

Accommodation: When booking a hotel, make sure you get a place in one of the sectors, which are the well planned, neat areas. If you get a room in an industrial phase instead, you'll end up in what our taxi driver referred to as a 'third class area' with potholes and filth. This is one of the problems of booking a budget hotel with Oyo. I've written to Oyo about also mentioning the locality type along with pictures of the room.


You could also try a trip to Dhanaulti, Chopta or Auli. Nice places with a lesser crowd and plenty of clean snow during winter. If springtime, visit the valley of flowers.

Wishing you a good trip to North India. Hospitable people, nice places (but recently, horrible food). Hope my bunch of tips and info would be of help. Do let me know in the comments.