Wednesday, 25 May 2016

Definitions of Chef, Puppet, Vagrant, and Docker

In my pursuit to help people about DevOps terminologies here are the definitions of Chef, Puppet, Vagrant, and Docker. There are so many others out there. This is my attempt.

Chef: Chef is an automation platform to transform your infrastructure into code. It is also called, Configuration Management Software. You can define the state with different parameters as your config files, s/w, tools, access types and resource types, etc. Also, you can configure different machines with different functionality based on your need.

Puppet: Puppet is a tool that allows you to abstract specific concepts of the target machine and make the configuration process more operating system agnostic. It allows to install a package or run a background service on startup independently. Define a command, packages those need to installed, step dependencies, file content, and other things you would need to get machine up and running without problems.

Vagrant: Vagrant is a project that helps the spawning of virtual machines. It started as a command line of VirtualBox, something similar to Gemfile for VM's. You can choose the base image to start with, network, IP, share folders and put it all in a file that anyone can reuse to spawn the same configured machine. Vagrant has different extensions, provisioning options and VM providers. You can run a VirtualBox, VMware and it is extensible enough to be able to create instances on EC2.

Docker: Docker, allows to package an application with all of its dependencies into a standardized unit of software development. So, it reduces the friction between developer, QA and testing. It dynamically changes your application, adding new capabilities every single day, scaling out services to quickly changing the problem areas. Docker is putting itself in an excited place as the interface to PaaS be it networking, discovery and service discovery with applications not having to care about underlying infrastructure.

Friday, 13 May 2016

Infrastructure as Code

Infrastructure as Code

Picking up my blogs after a very long time, let’s see how long I last this time. So I decided to write something about a topic which I am hearing a lot, seems like it’s a buzz word these days.

“Infrastructure as Code”

Infrastructure as Code/Infrastructure automation is the process of scripting environments, from installing an operating system, to installing and configuring servers on instances, to configuring how the instances and software communicate with one another, and much more. By scripting environments, you can apply the same configuration to a single node or to thousands.

Infrastructure automation brings agility to both development and operations because any authorized team member can modify the scripts while applying good development practices
·       Such as automated testing and versioning to your infrastructure.

Developers can learn a lot from operations, and operations can learn a lot from developers. There are many open source and commercial tools out there which support infrastructure automation. I’ll discuss Chef and Puppet, the two major players in this market and will look at others like Ansible in my other posts.

Chef and Puppet both use a Ruby domain-specific language (DSL) for scripting environments. Chef is expressed as an internal Ruby DSL, and Puppet users primarily use its external DSL, also written in Ruby. These tools tend to be used more often in Linux® system automation, but they have support for Windows as well. Puppet has a larger user base than Chef, and it offers more support for older, outdated operating systems. With Puppet, you can set dependencies on other tasks.

Chef has been around since 2009. It was influenced by Puppet and CFEngine. Chef supports multiple platforms including Ubuntu, Debian, RHEL/CentOS, Fedora, Mac OS X, Windows 7, and Windows Server. It just started supporting AIX infrastructure also. Everything in Chef is defined as a Ruby script and follows a model that developers are used to working in. Chef has a very passionate user base, and the Chef community is rapidly growing while developing cookbooks for others to use.
How it works
In Chef, three core components interact with one another, Chef server, nodes, and Chef workstation.

  • Chef runs cookbooks, which consist of recipes that perform automated steps 
  • Called actions on nodes, such as installing and configuring software or adding files. 
  • The Chef server contains configuration data for managing multiple nodes. The configuration files and resources stored on the Chef server are pulled down by nodes when requested. Examples of resources include file, package, cron, and execute. 
  • Users interact with the Chef server using Chef's command-line interface, called Knife. 
  • Nodes can have one or more roles. 
  • A role defines attributes (node-specific settings) and recipes for a node and can apply them across multiple nodes. 
  • Recipes can run other recipes. 
  • The recipes in a node, called a run list, are executed in the order they are listed. 
  • A Chef workstation is an instance with a local Chef repository and Knife installed on it.

Chef components
Describe node data, such as the IP address and hostname.
Chef client
Does work on behalf of a node. A single Chef client can run recipes for multiple nodes.
Chef Solo
Allows you to run Chef cookbooks in the absence of a Chef server.
Contain all the resources you need to automate your infrastructure and can be shared with other Chef users. Cookbooks typically consist of multiple recipes.
Data bags
Contain globally available data used by nodes and roles.
Used by system administrators to upload configuration changes to the Chef Server. Knife is used for communication between nodes via SSH.
Management console
Chef server's web interface for managing nodes, roles, cookbooks, data bags, and API clients.
Hosts that run the Chef client. The primary features of a node, from Chef's point of view, are its attributes and its run list. Nodes are the component to which recipes and roles are applied.
Detects data about your operating system. It can be used stand-alone, but its primary purpose is to provide node data to Chef.
The fundamental configuration in Chef. Recipes encapsulate collections of resources that are executed in the order defined to configure the nodes.
Repository (Chef repository)
The place where cookbooks, roles, configuration files, and other artifacts for managing systems with Chef are hosted.
A cross-platform abstraction of something you're configuring on a node. For example, users and packages can be configured differently on different OS platforms; Chef abstracts the complexity in doing this away from the user.
A mechanism for grouping similar features of similar nodes.
Server (Chef server)
Centralized repository of your server's configuration.

Following recipe demonstrates the use of the service resource within a recipe that's part of a Tomcat cookbook. You can see that you can use tools like Chef to do platform-specific configuration and manage server configuration.

service "tomcat" do
  service_name "tomcat6"
  case node["platform"]
  when "centos","redhat","fedora"
    supports :restart => true, :status => true
  when "debian","ubuntu"
    supports :restart => true, :reload => true, :status => true
  action [:enable, :start]

Following defines the attributes for the Tomcat cookbook. In this example, I'm defining some external ports for the Tomcat server to make available. Other types of attributes you might see include values for directories, options, users, and other configurations.
default["tomcat"]["port"] = 8080
default["tomcat"]["ssl_port"] = 8443
default["tomcat"]["ajp_port"] = 8009


Puppet has been in use since 2005. Many organizations, including Google, Twitter, Oracle, and Rackspace, use it to manage their infrastructure. Puppet, which tends to require a steeper learning curve than Chef, supports a variety of Windows and various Linux environments. Puppet has a large and active user community. It has been used in thousands of organizations with installations running tens of thousands of instances.

How it works
Puppet uses the concept of a master server, called the Puppet master, which centralizes the configuration among nodes and groups them together based on type. For example, if you had a set of web servers that were all running Tomcat with a Jenkins WAR, you'd group them together on the Puppet master. The Puppet agent runs as a daemon on systems. This enables you to deploy infrastructure changes to multiple nodes simultaneously. It functions the same way as a deployment manager, but instead of deploying applications, it deploys infrastructure changes.

Puppet includes a tool called facter. Facter holds metadata about the system and can be used to filter among servers. For example, you can use facter to determine a node's hostname. MCollective is a deployment tool that integrates with Puppet. You can use MCollective to deploy infrastructure changes across nodes.

Key Puppet components
A daemon process running on a node that collects information about the node and sends it to the Puppet master.
Compilation of facts that specifies how to configure the node.
Data about a node, sent by the node to the Puppet master.
Describes resources and the dependencies among them.
Groups related manifests (in a directory). For example, a module might define how a database like MySQL gets installed, configured, and run.
A host that is managed by the Puppet master. Nodes are defined like classes but contain the host name or fully qualified domain name.
Puppet master
The server that manages all the Puppet nodes.
For example, a package, file, or service.
In the example below a Puppet manifest describes the packages to install on a node. Puppet determines the best approach and order of execution for installing these packages.

class system {
  package { "rubygems": ensure => "installed" }
  package { "gcc": ensure => "installed" }
  package { "gcc-c++": ensure => "installed" }
  package { "ruby-devel": ensure => "installed" }
  package { "libcurl-devel": ensure => "installed" }
  package { "openssl-devel": ensure => "installed" }
  package { "libxml2-devel": ensure => "installed" }
The Puppet manifest snippet below shows examples of different resource types, package and service, that can be used in scripting an infrastructure:

class httpd {
  package { 'httpd-devel':
    ensure => installed,
  service { 'httpd':
    ensure => running,
    enable => true,
    subscribe => Package['httpd-devel'],
Puppet employs a declarative model with explicit dependency management. Because of this, it tends to be one of the first tool considerations by engineers who have more of a systems administration background and are looking to script their environments.

Monday, 17 March 2014

Mutual SSL Handshake .NET, SSL Client Authentication

Few days back I was working on this Mutual SSL Handshake Issue. I was stuck on this for quite sometime. Lack of information and very limited number people actually using this, caused me to work for lot of hours, which was supposed to be a very tiny piece of work.

I was supposed to talk to a webserver from my webserver (IIS 7.5) using the SSL certificate client authentication.

Because of limited information out there I figured few things on my own and thought of creating this blog about it so that people who are working on this can get some benefit out of it.
  1. Client certificate should have following Key Usages. Digital Certificate, Key Encipherment and extended usage of Client Authentication.
  2. The private key should be exportable and not protected.
  3. The root signers of your certificate should be on the target webserver’s trust store.
  4. Server certificate’s root signers should be on your server’s trust store.
  5. You don’t need to import your certificate to your certificate store. Instead I recommend to use the .PFX file instead. Because your IIS server’s users don’t have enough permission to read the private key of your certificate. Whereas by using the PFX file we don’t need any permission whatsoever. PFX file contains all the certificates Root, Intermediate, Shared and Client Certificate.

Once the certificate part is done use the following code to perform the Client authentication in code. HttpWebRequest will take care of everything else. Please feel free to comment.

private static void FromFile()
 var certificate = "CertificateFile.pfx";
 var certpwd = "password";
 var URL = "Host_URL";
 var certs = new X509Certificate2Collection();
 certs.Import(certificate, certpwd, X509KeyStorageFlags.DefaultKeySet);

 var webrequest = WebRequest.Create(URL) as HttpWebRequest;

 ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;
 ServicePointManager.Expect100Continue = false;
 ServicePointManager.ServerCertificateValidationCallback += new RemoteCertificateValidationCallback(AlwaysGoodCert);


 webrequest.Credentials = CredentialCache.DefaultNetworkCredentials;
 webrequest.Method = WebRequestMethods.Http.Get;
 webrequest.ContentType = "application/json; charset=utf-8";

 var responseStream = webrequest.GetResponse().GetResponseStream();

private static bool AlwaysGoodCert(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslpolicyerrors)
 return true;

Tuesday, 1 October 2013

Generating PHP SHA1 Hash in Windows Phone and Windows 8 App

If you want to generate the PHP SHA1 (sometimes called hex digest format) in windows phone or windows 8 app us the following method.

private string HashString(string str)
 string dataHash = string.Empty;
 string hashType = "SHA1";

  HashAlgorithmProvider Algorithm = HashAlgorithmProvider.OpenAlgorithm(hashType);
  IBuffer vector = CryptographicBuffer.ConvertStringToBinary(str, BinaryStringEncoding.Utf8);
  IBuffer digest = Algorithm.HashData(vector);

  if (digest.Length != Algorithm.HashLength)
   throw new System.InvalidOperationException("Failed");
  dataHash = CryptographicBuffer.EncodeToHexString(digest); 
  return dataHash;
 catch (Exception){}

 return null;