Programs to install after a fresh Ubuntu installation (specifically for 16.04)

Install vim:

sudo apt-get -y install vim

Create a bash aliases file.

gedit .bash_aliases; source .bash_aliases

vim .bashrc, and add this to the end of your .bashrc: (not necessary Ubuntu 20 onward)

if [ -f $HOME/.bash_aliases ]
then
  . $HOME/.bash_aliases
fi

 

Download Google Chrome and install it:

sudo dpkg -i google-chrome-stable_current_amd64.deb 

If there's an error with appstreamcli:

sudo apt install --reinstall libappstream3

Remove Amazon:

sudo rm /usr/share/applications/ubuntu-amazon-default.desktop
sudo rm /usr/share/unity-webapps/userscripts/unity-webapps-amazon/Amazon.user.js
sudo rm /usr/share/unity-webapps/userscripts/unity-webapps-amazon/manifest.json

Install various programs:

sudo apt-get update; install gimp; install vlc; install git; install audacity; uninstall rhythmbox; install audacious; install kdenlive; install kazam; install inkscape; install gnome-tweaks; install meld; install kdiff3; install gparted; install shutter; install htop; install iotop; install tcpdump; install atop; install nethogs; install nmon; install glances; install dmsetup; install swapspace; install snapd; install redshift; install git-cola; install bleachbit; sudo apt autoremove -y;

Remove the boot curtain and the ugly pink grub background:

https://nrecursions.blogspot.com/2021/07/what-to-do-when-ubuntus-boot-time-is.html

Change grub timeout:

gksudo gedit /etc/default/grub; sudo update-grub

Install Snap if necessary and change the update time:

sudo snap set system refresh.timer=fri,3:00-3:01

Install virtual environment for Python:

PyEnv

Get rid of Ubuntu 16.04's annoying drumbeat sound:

sudo mv /usr/share/sounds/ubuntu/stereo/desktop-login{,-disabled}.ogg; sudo mv /usr/share/sounds/ubuntu/stereo/system-ready{,-disabled}.ogg

Other tasks

• Install Firefox extensions.
• Change colors of terminal.
• Change settings of Gedit.
• Change the wallpaper. 
• Create another user.
• Disable the guest account.
  

Creating a dual boot with Windows and Ubuntu when you have UEFI

People on many blogs say that you have to disable secure boot when you have UEFI and want to install Ubuntu alongside Windows 8 or 10.
Not necessary. You don't even have to switch to Legacy boot.

The basic concept to understand is, that if you have UEFI, you'll have to tell the boot manager where to find your boot file. That's all.

After a lot of searching and ending up at all the wrong blogs and posts, I finally found this post. Worked perfectly!

The steps are:

• Install Windows, leaving a separate partition for Ubuntu
• Install Ubuntu on that partition
• Reboot and enter BIOS screen
• Follow the steps here about selecting grubx64.efi
• Save and exit BIOS

That's it! You've got a dual boot system. Grub will show you the options for booting into either Windows or Ubuntu.
If anything goes wrong, you can always press F12 when starting your computer to see the boot options.

Oh btw, don't disable secure boot. It's a very important feature that prevents your system from getting infected at boot time, by malware that might be in an external drive that's connected to your PC at boot time.

Our Income tax website has a perplexing error message

I logged into my income tax e-filing account yesterday, and saw this message: "You are not authorized to view this page because either your bank details of your PAN and profile details are incorrect or these details have not been passed onto e-Filing. Please contact the owner of the information for details".

I was completely perplexed. I had filed my taxes correctly, so why this message? I couldn't even access my dashboard and didn't know what to do.

A bit of Googling revealed that few other people had faced the same issue since 2014. One of the answers on a forum said that this message shows up when the website is undergoing some routine maintenance, and we should check back after six to twelve hours.

I logged back in the next day and sure enough, everything was back to normal.

Phew!

Whether it's a routine website maintenance or too many connection requests to the server or because there's an actual issue with the details not matching, I hope the message could be changed to display something that specifies what the actual issue is, so that we could be informed of it and do what is necessary to fix it. For example:

• If it's routine maintenance or too many connections to the server: "Routine website maintenance ongoing. Please check back after few hours".
• If it's a problem with the bank details: "Your bank details for <bankName> are incorrect. Please ensure that <problem> is fixed by <solution>".

This is for all of you who were perplexed by this message and didn't know whom to ask. Hope it helped. In most cases, you just have to wait a while before you try logging in again. Otherwise, you can always contact the income tax helpline to find out more.

 

Update 2020: The issue still seems prevalent, as shown in the comments below. I hope contacting the relevant authority or submitting feedback or submitting a grievance or contacting your bank to mention this problem may help resolve the problem faster.

How to download files or an entire folder from Google Drive?

I've found it very strange that a product of Google didn't have a straightforward way to download files or an entire folder in one shot.

There is a way though:

• As shown in the image below, click the folder or file you want to download
• Then click the menu button to open the menu.
• Then if you don't see a download option in the menu, just click the empty area at the bottom of the menu.

You'll get your folder in zipped format.

I assume it's just a bug that the download icon isn't visible.
Another way to download is to double click the file to open it and on the top right corner of the screen you'll see the download icon.


UPDATE: I wrote to Google support about this issue, and as of August,  the CSS issue with the hidden download button is fixed. There also was another issue of people using Chrome being unable to download files. Google told me they are working on fixing that too. Until then, you could just use incognito mode to download.

Wear that helmet. It's actually worth it

Few years ago when taking my friend back home on my bike, he noticed me taking my helmet and asked "You wear a helmet for short distances too?".

I couldn't really believe what I was hearing. Still, there is a large number of people who hate helmets because it's claustrophobic, because they think it causes hair loss (it does not), because they actually feel they'd never get into an accident and sometimes simply because they can't afford one.

The safety lectures are not the only reason to wear them:

Stones, insects, tree branches: Even when riding a bike at 40kmph, these objects can hit your face at high speeds. Have a full face helmet with a visor. I've encountered these projectiles at least once a month. The stones that get half under a vehicle tyre and shoot out at you are particularly dangerous. A colleague's car windshield looks like as though a bullet hit against bulletproof glass and left a dent there. It was hit by one such stone.

Wind, dust, sunlight and rain: Very useful to be protected with a helmet.
And of course...safety: The visor protects your nose and the front of your face from getting damaged too much. A full face helmet protects the sides of your face (which are most probable for damage during an accident).


Look at all the pics here.

Get a helmet. Wear it.

Writing GATE exam of another branch

I was forced into a different branch of engineering, even though what really clicked with me was computer science. Ever since then there was a desire to pursue an education in CS.

So what can engineers of other branches do to get a degree in Computer Science? Either write GATE or PGCET. The PGCET people were obstinate about not allowing me to write a CS paper, but the GATE organizers (bless them) agreed.
With five months of preparation time and a work schedule which many-a-time required me to work on weekends, I started preparing. Even though a week before the exam I couldn't study due to work deadlines, I wrote GATE and waited...remembering the amusing moment when the examiners at the centre, double-checked my photo ID with my face, wondering if I was a genuine candidate (others who wrote the exam were much younger).

Results were declared and I was among the top 11% of candidates across India. To me it initially didn't seem a big deal that I qualified. It was only later that a friend from IISc told me that qualifying in GATE itself is a big deal. On top of that, I cleared GATE from a different branch than what I studied in my bachelors. This was accentuated by my meeting some CS graduates who told me that they couldn't clear GATE when they wrote it.

Even more surprising:  Only 16% of the students had qualified. Articles from HindustanTimes and IndiaToday.

I also wrote back to the GATE organizers thanking them for the opportunity and the organizing chairman Prof.G.J.Chakrapani replied back with his best wishes.


What helps with preparing for GATE?

The wonderful people who created GateOverflow. The single best resource on the internet. There's a PDF you can download which contains questions from previous years, segregated subject-wise.
They also have a stackoverflow-style forum where you can ask your doubts.
This is a very useful piece of knowledge, because it lets you know how you have to study the concept, in order to be prepared for the exam.

Now I'm finally doing my masters in Computer Science. A bunch of mind boggling mathematics as part of the syllabus, but I'm glad I got here.


And to the socially conscious parents...

I still meet people who have newly become parents, who are already planning to send their child to college to become a doctor or engineer. I ask them if they'd be willing to consider what their child's interests are, and they don't really care.
I feel sad for their kids.
A bunch of lovely dreams squashed by the perception of social status and bullying.

As long as the education system and the industry allows people to switch to what they like doing, I feel our people and our nation will have a bright future. Better still, might be the concept shown in Man of Steel, where the genetic makeup of a child is determined at birth and they are trained from childhood to become the best at whatever career they are genetically and mentally most inclined to like and perform.

"Knowledge is most precious to the one who truly seeks and respects it"

- Nav

What exactly is the "mothers love" part of home cooked food?

One of my seniors in college always loved saying (whenever we were at a restaurant) that mother's food is always the best food because it is sprinkled with a necessary ingredient: Love.

I mentioned the same thing to a pragmatic uncle of mine, and his immediate response was "Bull***t!" :-)
At least he wasn't of the opinion that the taste of food comes from an emotion.

So what exactly is the "mothers love in home-cooked food" or to ask it differently, what makes home cooked food more healthy? What makes restaurant food bad?

Answers you are familiar with (about restaurant food):

• Artificial ingredients
• Too many spices
• Reused oil

But that's not all. There are some things you've never noticed because your stomach was either resilient enough to handle it or you just didn't know:

• The food was not cooked fully (many cooks aren't really cooks. They were just looking for a job and someone taught them the basics and they have too many customers to serve)
• The food has burnt particles in it (I know there are a lot of silly people who have told you that the black portions of rotis and parathas are ok. They are not)
• The ingredients used were of low quality.
• The food can contain H Pylori. Causes horribly painful stomach aches.

So the "Love" in the food is...

The love is the care with which the food is cooked. Not every mother can do that though. Many of them hate being in the kitchen and many aren't very knowledgeable about cooking. But the more access to knowledge the mother or father or any family member has, the better their cooking will be and the more healthy it will be for you.
There's one other thing: Certain foods get associated with memories of one's mother or family. For example, there are friends of mine who love Ragi Mudde (plenty of childhood memories associated with it). Others can't even stand the taste of it.

If you don't have access to healthy home cooked food, buy a stove (even if it's an electric induction stove) and learn to cook your own food. Trust me, it'll be entirely worth it. If you live in a place that does not allow you to cook, then leave that place and find a place where you can cook.
Invest in your health. If you ignore the food part of health, every other part of your body will suffer.


ps: On a side note, if you've heard of people saying that eating with the fingers is tastier than eating with a spoon but were never able to explain why; I have the answer for you. I asked this question on skeptics.stackexchange.com and here's the answer:

There is scientific research which says that touching (tactile sensations) itself, not just the changes in the diffusion of taste and odor compounds with different textures, can adjust the perception of taste and smell. The studies don't specifically say that the taste is always improved, but tactile sensations affect the perceived taste. 

Getting a quick summary of table sizes and rows in MySQL

Just a quick post to show you a technique my colleague showed me on how to get the sizes of tables and an approximate number of rows it has, without having to wait long for a select count(*) from table.

Use this command:

        SELECT
             table_schema as `Database`,
             table_name AS `Table`,
             table_rows AS "Quant of Rows",
             round(((data_length + index_length) / 1024 / 1024/ 1024), 2) `Size in GB`
        FROM information_schema.TABLES
        WHERE table_schema = 'ballast'
        ORDER BY (data_length + index_length) DESC;

This will show you a result like this:

-------------------------------------------------------------------------------------------
| Database     Table                           Quant of Rows  Size in GB
-------------------------------------------------------------------------------------------
| ball            TableHosp2015               110654400             11.05
| ball            TableHosp2016               115890383             10.30
| ball            Table2016_temp                87027666               6.49
| ball            Table2014_temp                73765370               5.78
| ball            Table2015                          62064795               4.86
| ball            Distances                               158372               0.02
-------------------------------------------------------------------------------------------

The Indian Education System inadvertently trains you to deal with customers

The first part of this blog post is anecdotal.

As Indians, we tend to complain a lot about our education system. About how it focuses only on securing marks and does not spark interest in students.

A neighbour recently asked me for my book on C++, saying she wanted a few theory concepts to write as part of an assignment her teacher gave.

As the book was an old one, I told her the best place to get her info was on the internet.
Her answer surprised me. She said she wanted to refer the theoretical concepts in the book and write those for her assignments because that's what the teacher would expect. Scoring the marks was of a higher priority than learning the theory.


Two ways of looking at it:

Way One: Here's where you start cursing the Indian education system which designed assignments only for the purpose of scoring marks. Pull your hair out and ask for the system to be changed so that kids would have better practical programming knowledge.

Way Two: Our Indian education system had just given birth to another little girl who saw the problem from the customer's perspective. This is exactly what every CEO wants their sales people to understand.

Find out what the customer wants and tailor your response and product to that. You make the sale!!!

My little neighbour learnt how to anticipate what the teacher wanted and gave the teacher exactly that. Marks scored!!!
You see students all over India finding such shortcuts which get the job done. Finding innovative ways of impressing the teacher, pretending to be extra humble for some leniency or marks, pulling strings to get hold of the studious guy/girl's notes...and so many more...

Rings a bell? Somebody's famous quote of "I will always choose a lazy person to do a difficult job because a lazy person will find an easy way to do it".

We are expected to do this as engineers too. If there's a solution already available, just use it instead of spending time doing things from scratch and allowing the competition to gain an advantage over us.


The customer focus:

As a person who designs the Architecture of software, my primary goal is to ensure that the customer's requirements are fulfilled, instead of choosing an architecture and technology that would look good on my resume/CV.
If the customer's work can be done with a simple technology, it is better to use that, than to choose a different, more complex, more time consuming technology that would add a lot of punch to your portfolio, but would be of little or no value to the customer.

When you do find people who care about what the customer wants, treat them like a precious gem. You have a valuable find.

ps: This is not to misunderstand those genuine people who spend time understanding concepts and improving on the theoretical understanding. This is also expected of good engineers. To be well informed, educated and at the top of the game. These engineers are a league apart. Inventors. Scientists. People who create value for the world.

Aha!

Continued from the previous Aha!

Vendors

Share with this link

 

Continued in the next Aha.

 

 

How to speed up MySQL inserts

One would assume that if an application had two threads, and while one thread inserted data into one table and the other thread inserted data into another table, the inserts would be faster than if both threads inserted into the same table? No. It isn't faster.
The various threads that MySQL has, are to manage the other different operations.

There however is a technique that can improve insert speed further.

Smart programmers already know that using a batch insert is more efficient and faster than calling INSERT multiple times from their program.

What you perhaps do not know, is that after you send the batch of insert statements to MySQL, they are executed internally as separate inserts.

INSERT INTO Table1 VALUES (1,2,3) 
INSERT INTO Table1 VALUES (4,5,6)

What if you could combine it into a single insert statement like this:

INSERT INTO X VALUES (1,2,3),(4,5,6)

It'll actually execute faster! Some people report a 2x increase in write speed. Others, report a 40x increase.


How to do it? 

Just set the rewriteBatchedStatements=true value in the connection string.

So if you normally connect to MySQL like this...

Connection conn = DriverManager.getConnection("jdbc:mysql://"+ domainName +"/"+configRef.getBallastDatabaseNameOfSQL()+"?" + "user=" + username + "&password=" + password);

...you just have to change it to:

Connection conn = DriverManager.getConnection("jdbc:mysql://"+ domainName +"/"+configRef.getBallastDatabaseNameOfSQL()+"?" + "user=" + username + "&password=" + password + "&rewriteBatchedStatements=true");


In my opinion, MySQL should have kept this as default behaviour and the user should have had to disable it only if they were doing very large batch inserts.

Do try this out even if you don't execute too many inserts. In applications dealing with a lot of data though, this option can come as a life saver! You save a humongous amount of time with these super fast inserts.

Avoiding memory problems when working with huge amounts of data

There would be times when you run applications with libraries like Apache Storm or similar programs, and you'd be surprised when these applications crash and restart with no apparent reason.

The reason in many cases (as you'll infer from the error logs) is that the application ran out of memory. You just didn't have enough RAM.

Why this happens:
If you specify (using -Xmx2g) that your application can use a maximum of 2GB memory, you have to remember that there are other applications running that might also be free to use that much memory.

Let's say you have 4GB RAM.
App1 takes up 2GB.
App2 takes up 2GB.

App1 deletes some large datastructures that it's using, before creating a new datastructure.

The request for creation of a new datastructure makes the Garbage Collector try to allocate more memory. Sometimes, operations happen so quickly that the GC has no time to remove the unused memory before allocating new memory.

Since you've run out of RAM space, the OS would typically allow your application to use virtual memory. To do that, you need a swap space. If you don't have one, your application might crash with the message that there was insufficient memory. Your application might not even catch this exception and yet crash.

In many managed servers like AWS, the nodes don't have a default swap space.  You have to create one. It's very simple and completely worth it.

How much swap space to create?
Go for double the size of your existing RAM. I know that sounds like a lot, but believe me; I've seen how a system with 4GB RAM has used more than 4GB of swap space when I had allocated it 10GB of swap space. Hardware is cheap, and we now have disks with plenty of space, so create large swap partitions/files.

How to create the swap space?
First check if it's already there with

free -m

and check for how much disk space is available with

df -h

On an Ubuntu 14.04 system, just run these commands:

sudo fallocate -l 6G /swapfile (the "6G" means 6GB of disk space)

sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

To make the swap file permanent:

sudo vi /etc/fstab

and at the bottom of the fstab file, add this line (press tab instead of a space wherever you see a space in the line below):
/swapfile   none    swap    sw    0   0

If you like tweaking things more, you can also tweak the swappiness setting which determines how often the OS will transfer data from the RAM to swap space.

Aha!

Continued from the previous Aha!

Packaging

Share with this link

 

Continued in the next Aha...

 

 

USB tethering. Use internet on your Desktop PC via mobile phone internet

Whenever I've used internet via a mobile phone's WiFi Hotspot, I've always wondered if it was transmitting extra data just for the sake of being in sync with the laptop. Or perhaps adding extra bits to the packets when transferring data.
Turns out it is true. Using mobile hotspot actually takes up more data.

While some people have suggested tethering your mobile with the laptop using Bluetooth tethering, which is said to (but I haven't verified it) use lesser data, I feel a wired connection would be much better.

That alternative I found in USB tethering.

All this while I was actually considering buying a WiFi network card for my desktop PC, and I came across USB tethering which solved my problem elegantly and with no additional cost.


How to do it?

1. Just take the cable you usually connect your mobile phone and desktop PC or laptop with, for transferring data via USB.
2. Go to your mobile phone settings and activate USB tethering.
3. Switch on your mobile phone internet.

That's it. Your phone is now a modem. You can use internet on your desktop PC or laptop. Best tethering option because your phone is connected to the PC, and it even gets charged in the process.

Kudos to those who came up with these brilliant solutions!

Viruses in Indian cybercafe's

Happens all the time. Take a pen drive with a word file to get a printout at a cyber cafe or a shop where they do printouts, and you come back with a virus infected pen drive.
At least this used to be ok as long as the virus didn't do anything to the file. You could still take the printout. Now-a-days, there's a bigger problem: The virus copies all the files into a hidden, unnamed folder and all you see is an empty pen drive. How are we to take a printout now?

Sadly, the popularity of Microsoft Windows and the availability of pirated copies is one factor that contributes to the spread of these viruses. What's worse, the cyber cafe owners refuse to believe that their computer has a virus!
I wonder when people will realize that pirated operating systems are very likely to have backdoors and more malware? Something you have to be very aware of when you type your passwords in cyber cafe's.

Having an antivirus is not enough. I've seen numerous cases where antiviruses are unable to detect viruses and worms. An antivirus detects virus signatures (a certain pattern of characters that a virus file contains). When the creators realized this, they started creating viruses that would alter their signature whenever they copied themselves to another disk, using an encryptor and decryptor. Then came polymorphic viruses which could change the signature of the encryptor and decryptor too. Antiviruses also use heuristics (virus behaviour like replication, overwriting files etc) to find viruses.


The alternative

The better way to stay safe, is to use an operating system which is heavily monitored by a community of voluntary software programmers all across the world. People who are dedicated to maintaining security and privacy.

One such operating system is Linux. Although some of the other variants of Linux were good earlier, I now dislike the interfaces of Fedora and CentOS.
One good Linux operating system you could use is Ubuntu. It's as good as Windows, it is safe and it is free of cost. Ten years ago, you could email the Ubuntu community and they would send you an Ubuntu CD for free by post. They've stopped that now of course, but you can still download Ubuntu and either burn it onto a CD or create a bootable pen drive from which you can either try Ubuntu or install it into your computer.

The best part is, that you don't even have to install Ubuntu to try it out. Just create a liveCD or do the same in a USB stick and you can run it to see if you like it.

Do note that if you have an old computer that cannot boot from a pen drive, then you will have to burn Ubuntu onto a CD or a DVD to create a bootable disk which can be used to try or install Ubuntu.


A message for Cyber Cafe owners in India

Enough of viruses and malware. Please install Linux on all computers you have.
If your users still want to use Windows, then you can install Linux and have a free Virtual Box in Linux inside which you can install Windows.

Some users would want to use Microsoft Word and Excel on the computer. Linux has an alternative named Libre office which comes pre installed with Linux.
If you don't like Libre office, you can always use the awesome Google Docs, Google Sheets etc.

One thing I would advise you should check though, is the compatibility of your printers and scanners with Linux. Do this before you install Linux, by trying out Linux with the Live CD option. Take a few sample printouts and scans and see if everything works fine. If any old printers need drivers to be installed, you could either contact the vendor and ask about linux drivers or search the internet using the device model number and I'm very sure you would find free drivers which you can download and install for Linux. The modern printers I think would work without any extra drivers. Do try it out.


Why I'm writing this

I went to a cyber cafe to take a printout and ended up with a virus that messed up the pen drive's partition table (and also hid the files, so I couldn't take the printout). Came home and had a hard time restoring the partition table.
For anyone who encounters this error:

"The driver descriptor says the physical block size is 2048 bytes, but Linux says it is 512 bytes."

Use this command to fix the blocks on your pen drive:

sudo dd if=/dev/zero of=/dev/id_of_your_device bs=2048; sync

Most likely, the id_of_your_device would be "sdb". Make sure you confirm that, because if you choose "sda" and it happens to be your hard disk, all the data on it will get permanently erased. 

The dd command is used to: Convert and copy a file, write disk headers, boot records, create a boot floppy. dd can make an exact clone of an (unmounted) disk, this will include all blank space so the output destination must be at least as large as the input. In this specific case, these posts will help you understand the meaning of the bs and 2048.

Did that, and the disk is as good as new!