VMware to Virtualbox Migration

After updating my Fedora Kernel , VMware Workstation 12 stopped working. This moment I thought to switch back to Virtualbox as it has support for latest Kernels. There are pretty some good documentation on how to migrate from VMware to Virutalbox format

You can find a good document on Migration Here . In Summary the steps are

  • Power off your guest which you are migrating
  • Open command promt as an Adminstrator and Navigate to   C:\Program Files (x86)\VMware\VMware Player\OVFTool
  • Run  ovftool <source.vmx> <output.ovf>
ovftool "C:\Users\diaryfolio\Virtual Machines\Windows 7 x64\Windows 7 x64.vmx" C:\Users\diaryfolio\Win7Export.ovf
  • This would take about 15mins . So please be patient
  • Once complete Import into your Virtualbox using "Import Appliance"

Errors that might occur

  • Error: "Failed to open disk” - This normally happens when the Guest VM is not properly shutdown or stopped
  • Error while importing to Virtualbox 
Failed to import applianceHost resource of type "Other Storage Device (20)" is supported with SATA AHCI controllers only,
line 47. Result Code: VBOX_E_FILE_ERROR (0x80BB0004)
Component: Appliance Interface: Appliance {xxxxxxx-4add-4474-5bc3-xxxxxxxx}
The quickest solution is to open it with any text/xml editor, and
1. replace word “ElementName” with word “Caption” in the whole file
2. replace “vmware.sata.ahci” with “AHCI”
3. Then  change SHA accordingly
[root@diaryfolio Win10 x64]# sha1sum "Win10 x64.ovf"
0baac9938935a10b254e8cc18fc47fa3242168bb  Win10 x64.ovf

Chromecast vs AppleTV vs Raspberry Pi vs Roku vs Others (2015)

Never waste your money on buying SMART TV's :)  But buy one of these to make your LED/LCD tv smart and control with your phone/home automation

Let's compare between  Chromecast vs AppleTV vs Raspberry Pi vs Roku vs Others in 2015. Previous version comparison can be found here

Chromecast 2

  • Google came with a new design for Chromecast 2, with a much better antenna (802.11ac) and perfect price ($35)
  • Built in apps like HBO, Spotify, Youtube, Netflix (but all its competitors have these apps)
  • Though aesthetic is much better, In my opinion it doesn't matter as it sits behind the Television
  • Much improved casting from your phone, Laptop.. almost any devices with a cast option
  • Also I liked that it doesn't have a remote, coz you can control from your phone
  • Lacking power to play games

Apple TV4

  • Coming with Top Specs and a cool Remote Control (Though I feel your iPhone is more than sufficient), Apple TV4 released in 2015-September boasts of (Price around $150)
  • 32/64GB storage, 802.11ac WiFi, A8 Processor
  • USB C (3C) Connectivity but only for service/support
  • Bluetooth 4.0, IR sensor, HDMI 1.4, Ethernet, Power cable
  • Remote Control with Siri voice recognition, Bluetooth & IR, & Lightning port for charging

Raspberry Pi 2 (model B) 

  • Priced around $35 and best suitable if you are a developer. Can be very well used as Full HD entertainment mechanism. Details here
  • Released in 2015 Feb, 900 Mhz A7 Cortex CPU, 1GB RAM, 4 USB ports
  • Full HD supported. Can run full Linux or Windows 10 (who really want to run Windows?)
  • HDMI, MicroSD slot, 800 MA power, Approx 45gms 
  • Sits Quiety and very easy to put an OS and do whatever you want !!  Check here to find various media center's/ OS available for RPi2

Roku 4

  • All the hype around Roku4 is the 4K compatible/ready playback  and priced around $130. 
  • Optical audio out (phew its old tech !!), 60 frames/sec, Roku OS7 
  • The design of console is cool, but the remote seems out of date ( I would have done other way around as you use remote more than console)


  • Amazon FireTV - I have attached a sample compare sheet with Amazon FireTV (I don't really care about FireTV as of now)

Encrypted Filesystem - Virtual filesystem with folder, file and content encryption

Linux is wonderful when it comes to security aspects and data protection. My challenge was

  • Laptop having personal details in files and folders
  • Have to encrypt these data so that if laptop is lost, the data shouldn't be replicated
  • Have to use "git" to backup the data normally and version it

Solution for Linux

Design your folder/directory structure

  • Ensure that all your personal files are well structured
  • Ensure all files are put into a single directory/folder hierarchy  (eg "/home/myuser/personal/")

Setting up encfs

In Ubuntu, run 

sudo apt-get install encfs

Encrypted Directory and Mount location

encDir="/home/diaryfolio/Docs/encr" # Where encrypted files are stored
mntDir="/home/diaryfolio/Docs/mydocs" # Mount created

# Create encrypted directory and mount directory
encfs $encDir $mntDir

# Press Enter which uses default encryption
# Provide a strong password and REMEMBER it in future

# Copy content/files/directories to $mntDir once you mount it

# Unmount $mntDir afterwards
fusermount -u $mntDir

# You can now only see $encDir  and encrypted files

# To remount and view files in plain format. It will ask you for your password
encfs $encDir $mntDir

OpenVPN configuration for CentOS

This article is specifically for PrivateInternetAccess on CentOS. As you might see there are no written packages/support for CentOS. I've followed a very good article from this link , but that was not enough as I had to tweak a lot afterwards.
Just to write down the key points here

  • Install OpenVPN using yum (enable EPEL beforehand)
 sudo yum install openvpn

Please follow all steps as per this article. 
Hope you have copied all VPN files in /etc/openvpn and the credentials to /root/.pia

When you start, VPN it throws an error

[root@localhost openvpn]# systemctl start openvpn@server.service
Job for openvpn@server.service failed. See 'systemctl status openvpn@server.service' and 'journalctl -xn' for details.

This is because you need to ensure the files/configurations you put in /etc/openvpn needs to be "Set files with the openvpn_etc_t type"

How to set files with a specific type

Please follow below instructions
sudo su - 
cd /etc/openvpn/
semanage fcontext -a -t openvpn_etc_t  <each_file>
/sbin/restorecon -v  <each_file>

later you check status or start/stop vpn
systemctl status openvpn@server.service
systemctl start openvpn@server.service
systemctl stop openvpn@server.service

NUC5i5RYH - Barebone PC as Linux Server

After thinking of  installing VMWare ESXi on NUC5i5RYH barebone , I decided not to go ahead. This is because you need a separate system to manage (vSphere Client) to manage your lab which in my case was not helpful. I decided to go ahead and install CentOS 7 as my main OS(host) and install VMware workstation to support Windows Server (for ActiveDirectory) as guest.

I faced few issues while installing CentOS into the i5 NUC. Mainly because the CentOS Kernel & drivers are bit outdated compared to NUC. Just to document what I've done to fix them !!


  • Download Minimal CentOS7 from main site
  • Use "UNetbootin" to put the ISO into thumbdrive. 
  • FAT32 formatting of your USB is best suited


  • Attach bootable USB to your NUC and enter ensure your USB as primary boot.
  • Ensure you create a "root" user and password
  • Reboot and CentOS installation is straight forward. 

Network Issues

Your Wi-Fi won't be listed in your detected devices (ifconfig, iwlist etc are not installed)

NetworkManager to your rescue

NetworkManager Command Line (nmcli) is very powerful. Full details here
sudo su - # All activities in root
nmcli device show   # Shows all devices available. Only your ethernet and loop back would show
nmcli con show # Shows connections available. You won't be able to see anything initially

Installing Relevant Driver. 

(You need an extra laptop for this)
I had CentOS 7 with Kernel 3.10.x . NUC5i5RYH uses Wireless 7265 ,but the downloads are meant for Kernel 3.13+ onwards. So I had to download multiple drivers thus facing lot of hit & miss. I can confirm, the driver that works is  "iwlwifi-7265-ucode-" though it's for a higher Kernel. Extract just the "iwlwifi-7265-9.ucode" file from the archive and copy it into usb drive and copy it to /lib/firmware/ of your CentOS installation.  (USB mounting tips at this link). Now REBOOT your NUC

Verify if your Network is detected. 

sudo su - # All activities in root
nmcli device show   # Shows all devices available. Now it show a wireless device (eg wls2p0)
nmcli con show # Connections won't be still available
if no devices are shown, something went wrong. Need to debug by running "dmesg | tail -100" to see if there are any errors when OS rebooted.

Add your interface

Now try adding a new connection to your interface using networkmanager cli (wls2p0 is my ifname shown when I ran  "nmcli device show") . More Details in this link

nmcli con add con-name wls2p0 ifname wls2p0 type wifi ssid MyInternetConnectionName ip4 gw4
service NetworkManager restart# This should restart network
You should be now able to see ifcfg-wls2p0 file in /etc/sysconfig/network-scripts/  location with above details and will get a MAC address automatically.

Adding wifi plugin to NetworkManager

Unfortunately "NetworkManager" itself doesn't contain wifi setup plugins. You need to download "NetworkManager-wifi" rpm offline package separately into your laptop. Copy this again using usb into your NUC.   Install it using  rpm -Uvh <rpmfile> 
Verify if the package is installed
rpm -qa | grep NetworkManager
service NetworkManager restart
service NetworkManager status

Connect your Wifi

nmcli device wifi connect <MyInternetConnectionName> password <myPassword>

Verify by running 
curl -k https://www.google.com 
Atlast, it should all work now !!

Now you could install GUI by
sudo yum groups install "GNOME Desktop"

Big Data - Jobs, tools and how to ace it

Big Data : Overview of Structure and Jobs 

The demand for big data resources have increased dramatically in past few years. The requirements to create and get most out of "Big Data" environment is classified into 3 tiers

  • Base Layer - DevOps and Infrastructure
  • Mid  Layer - Understanding & manipulating data
  • Front Layer - Analytics, data science
I feel the jobs surrounding "Big Data" would also ultimately reflect this. Learning Big Data should be also based on these tiers.

Software Suite/Tools

Base Layer - Summary

This layer forms the core infrastructure of "big data" platform and should be horizontally scalable.
  • OS - Linux is the way forward for big data technologies. RedHat, SuSe, Ubuntu, CentOS
  • Distributed Computing tools/software - Hadoop, Splunk
  • Data Storage - Splunk, MongoDB, Apache Cassandra
  • Configuration management - Ansible, Puppet, Chef
  • Others - Networking knowledge, Version Control (Git)

Mid Layer - Summary

This layer forms the engineering role of the project with focus on data mining, normalisation etc.
  • Regex (Regular Expression) Knowledge and machine learning
  • Various Application system knowledge to understand data
  • Data normalisation techniques, data enrichment, integration to external data stores
  • Technologies - Java, Scala, Splunk Search language, python, NoSQL,JSON
  • Tools - Apache Spark, Splunk, Mahout
  • Web Development - jQuery, AngularJS, D3.js

Front Layer - Summary

This layer forms the end-user experience, business interaction experience
  • Business Analysis - Requirement gathering, Data availability etc.
  • Web Development - Analytics, Visualisation techniques, D3.js, mathematical modelling etc.
  • Data Science - Data co-relation, regular expression, statistics and data modelling
  • Tools - Apache Ambari, Splunk, D3.js, XML/JSON formats

Big Data Jobs

The key thing here is to concentrate on one the above layers/tiers. Depending on the flavour of your previous experience it would be quite easy to fit your profile into one of the above Tiers.

Software Suite/Tools

  • Apache Products (Hadoop suite, Spark, Ambari) are so varied and flexible. But return on Investment (ROI) is poor in initial years and takes time and effort to create a fully fledged system. The integration of Apache products need to be designed and upgrade plan should be well in place as products change quite often.
  • Splunk on the other hand encompasses all into a single suite. Splunk is not a free product, but licensing is based on data volume you index. ROI is very quick and great and can build Proof of Concept systems (POCs) immediately.

DevOps : Search and replace files with JSON dataset and template Engine

Today's code build & continuous deployment models are highly diverse thus leading to handwritten and complicated perl/awk/sed scripts. DevOps should come out of age old hand-crafted find and replace scripts with much modern template engines.
              Of course template engines are available in wide variety. All enterprise configuration management (chef, puppet, ansible ) software are equipped with their own flavour of template engines and playbooks.  This article however concentrate on "Mustache" template  which is logicless template system and work on any text based data (Web pages, scripts, dataset, config files etc..)
              The example below focusses on replacing dynamic text using a JSON dataset. Let's define terminologies
  • Source :   Template Parent Directory with all files/directories +  dynamic variables in it
  • Dataset:    JSON based self defining dataset to replace the above source(s)
  • Params:    Extra parameters that are supplied (eg: Environment PROD, DEV etc..)
The key idea here is to replace all templates (recursively) within the "Parent Source Directory" and find and replace with dataset. In summary this is what I'm trying to achieve
  • All template files in "Source" to have an extension of  ".mustache"
  • Recursively scan whole of "Source" directory and children and identify  .mustache files
  • Feed these files into mustache renderer
  • Rending information supplied using "Dataset"
  • Filter out required elements of "Dataset" using input "Params"
  • Copy directory structure and files into a temporary location with filled values