Skip to main content

Posts

Big Data - Jobs, tools and how to ace it

Big Data : Overview of Structure and Jobs  The demand for big data resources have increased dramatically in past few years. The requirements to create and get most out of "Big Data" environment is classified into 3 tiers Base Layer - DevOps and Infrastructure Mid  Layer - Understanding & manipulating data Front Layer - Analytics, data science I feel the jobs surrounding "Big Data" would also ultimately reflect this. Learning Big Data should be also based on these tiers. Software Suite/Tools Base Layer - Summary This layer forms the core infrastructure of "big data" platform and should be horizontally scalable. OS - Linux is the way forward for big data technologies. RedHat, SuSe, Ubuntu, CentOS Distributed Computing tools/software - Hadoop, Splunk Data Storage - Splunk, MongoDB, Apache Cassandra Configuration management - Ansible, Puppet, Chef Others - Networking knowledge, Version Control (Git) Mid Layer - Summary This

DevOps : Search and replace files with JSON dataset and template Engine

Today's code build & continuous deployment models are highly diverse thus leading to handwritten and complicated perl/awk/sed scripts. DevOps should come out of age old hand-crafted find and replace scripts with much modern template engines.               Of course template engines are available in wide variety. All enterprise configuration management (chef, puppet, ansible ) software are equipped with their own flavour of template engines and playbooks.  This article however concentrate on "Mustache" template  which is logicless template system and work on any text based data (Web pages, scripts, dataset, config files etc..)               The example below focusses on replacing dynamic text using a JSON dataset. Let's define terminologies Source :   Template Parent Directory with all files/directories +  dynamic variables in it Dataset :    JSON based self defining dataset to replace the above source(s) Params :    Extra parameters that are supplied (eg

USB Type C - The new Standard of Data Interface

USB-C aka USB Type-C aka USB 3.1 is the new standard which OEM companies going to adopt from 2014 onwards. This standard has been given a great boost by Apple MacBook 2015 edition thus paving way for next generation data transfer speeds and compatibility. Apple Macbook now has a single external interface ( A SINGLE port !! can you believe it !!) which shows the flexibility of USB3.1 . USB TypeC Port in 2015 Apple Macbook Pros of USB Type-C (USB 3.1) High Data Signalling Speed of 10Gbps (i.e Twice that of USB3.0 and 16x of USB2.0) Industry Standard and adopted by all major OEMs Reversible plug orientation (i.e. you can plug USB cable in any direction) Backward compatible with USB 3.0 and USB 2.0 Various Power Profiles and can deliver upto 100W @20V(5A) Multiple channels of data flow thus be used at both host and device side Can replace multiple other data transfer cables (Thunderbolt, USB TypeB etc) USB Speed comparison Various USB2.0 ports Th

CRUD form using Datatables with dynamic Add & Delete rows

Hope you guys have suffered same issues as me with Web Development to create forms with Create/Read/Update/Delete functionality. I really use "Datatables" ( www.datatables.net ) for displaying data in tabular format.  Datatables has got many advantages and is easily compatible with bootstrap stylings. IE8+ support, major browser support and uses jQuery as its core Can natively map JSON data Searching and Filtering functionality with pagination Support MultiLanguage support, ClientSide Sorting, AJAX based inputs Can filter required Columns at front-end Datatables is free but it's editor (editable tables) are paid. Hence I thought of putting together a simple table to use for editing and publishing data for POST purposes. In Summary, what we are doing here is Creating HTML stub with table headers within a form . Also add "Add Row" and "Delete Row" buttons with onclick actions attached to it and a "Submit" button In your javas

Web Development Test bed

There are quite few test-bed online for your web-development . JSFiddle was on the top but now it is loosing its sheen. I have now moved most of my web-development tests to "plunker" which is much advanced than already available ones. Moreover you can integrate with your github.com id thus keeping version of your code Please find a sample Plunker embedded into google blogger (Sample below) Loading plunk... Code to embed is:

IMDB API's and URL based search parameters

Advanced Search in IMDB I'm a big fan of IMDB and everytime if you really want to do detailed search, you need to use IMDB Advanced Search . But this is not practical each time, but seems you could tweak IMDB URL's to do magic searches. Few examples below. If you look the search tag, everything is self explanatory Most Popular English/Hindi-Language Feature Films Released 1 January 1980 Or Later With User Rating Between 7.5 And 10 and At Least 500 Votes http://www.imdb.com/search/title?languages=en,hi&num_votes=500,&release_date=1980-01-01,&title_type=feature&user_rating=7.5, Highest Rated Feature Films Not In "watchlist" Released 2011 Or Later With At Least 5,000 Votes http://www.imdb.com/search/title?num_votes=5000,&release_date=2011,&sort=user_rating,desc,&title_type=feature&my_ratings=exclude&lists=!watchlist IMDB Top TV Series - Highest Rated TV Series With At Least 2,000 Votes http://www.imdb.com/sea

Certification authority and applying certificate to your domain

Generating a Certificate Signing Request and applying the certificate is  very frequent. Some companies require 2048 bit signatures. The following steps will create certificates with 2048 bit and later apply to relevant httpd (apache) server. Certificate Hierarchy Creating a CSR Values Country Name (2 letter code) [AU]:GB State or Province Name (full name) [Some-State]:London Locality Name (eg, city) []:Croydon Organization Name (eg, company) [Internet Widgits Pty Ltd]:My company Organizational Unit Name (eg, section) []:diaryfolio.com Common Name (eg, YOUR name) []:subdomain.diaryfolio.com Email Address []: Things to remember In most cases, ensure "Email address is left blank". But consult your certificate signing authority on this. Ensure "Common Name" matches exactly the "ServerName" specified in your httpd.conf (or httpd/conf/extra/httpd-ssl.conf) Actual Creation Below creates CSR with 2048 bit # CSR Key generation # Generate a