Hacking weeds with Fog

If you're working on HPCloud you may want to monitor your assets, and know where your instances are running, and what state there in.  Additionally, you might be concerned with cost, which like many cloud providers, is determined by the size of the instance [read amount of allocated resources] and instance up time.  HPCloud provides you with 3 different ways of working with their OpenStack API.

Python Nova Client
Windows Shell
Ruby Fog - yes please!

If your reading this post you are likely already familiar with the awesomeness of Fog, if not you'll definitely want to take a gander at Fog.io

Fog works with different cloud providers and receives contributions of code changes which they push into their fog gem.  If your working with HPCloud its worth it to work to go ahead and grab HPCloud's HP Fog gem.  For more information check out: https://docs.hpcloud.com/bindings/fog

Assuming you are working with HP Fog version 20, you can connect like so:

	def create_connection(az_id)
	return conn = Fog::Compute.new(
	:provider => "HP",
	:hp_access_key => ENV['HP_ACCESS_KEY'],
	:hp_secret_key => ENV['HP_SECRET_KEY'],
	:hp_auth_uri => ENV['HP_AUTH_URI'],
	:hp_tenant_id => ENV['HP_TENANT_ID'],
	:hp_avl_zone => az_id,
	:hp_use_upass_auth_style => "false",
	:user_agent => "some-script/x.x.x.x",
	:connection_options => { :connect_timeout => 30, :read_timeout => 30, :write_timeout => 30 })
	end

view raw connecting hosted with ❤ by GitHub

Ok - thtats cool so we get a connection back that will allow us to work with the compute service...yawn.

So now lets do something a little more useful.  Lets imagine that every team member has a ppk or security key which is labeled by either the team or instance owners name.  We can group the number of instances using this ppk type for identifying assets or even use it to execute specific tasks on those labeled instances, maybe even throw in some Chef'ness to map specific roles.

	def instances_by_owner(sample_az)
	# Define hash as place holders for data and pull data from HP Console
	total_instance, held_machines = Hash.new(0), Hash.new(0)
	conn = create_connection(sample_az)
	# Get List of Instance 'owners' by PPK name
	result = conn.servers.inject([]) { |result,h| result << h.key_name unless result.include?(h.key_name); result }
	# Create Hash of Total Instances owned by each PPK that are also in active state aka running
	result.each do |v| conn.servers.each do |n|
	held_machines[v] +=1 if v == n.key_name && n.state=~/ACTIVE/
	end
	end
	held_machines.each do |k,v|
	v ||= 0
	push_data("stats.HPCS.#{sample_az}.instances.owner.#{k} #{v} #{@g.time_now}")
	end
	conn = nil
	end

view raw instances_by_owner hosted with ❤ by GitHub

Lastly I did mention earlier that I wanted to understand cost.  Well if you have multiple instances of different sizes, on different avalibilty zones your not going to want to do this by hand.

	def instances_by_owner_size(sample_az)
	# Define hash as place holders for data and pull data from HP Console
	total_instance, held_machines = Hash.new(0), Hash.new(0)
	total_instance_by_type, total_size_types = Hash.new(0), Hash.new(0)
	list_of_results = Array.new
	ppk_owners = Hash.new(0)
	images = Array.new
	conn = create_connection(sample_az)
	# Get List of Instance 'owners' by PPK name
	result = conn.servers.inject([]) { |result,h| result << h.key_name unless result.include?(h.key_name); result }
	# Get List of Instance Sizes types by ID @conn.flavors aka server.flavor id
	flavor_id_list_result = conn.flavors.inject([]) { |flavor_id_list_result,h| flavor_id_list_result.to_a << h.id unless flavor_id_list_result.include?(h.id); flavor_id_list_result }
	flavor_id_list_result.each {|x| images << Image.new( x, '', owners=Hash.new{0})}
	# Get List of Instance Sizes Names @conn.flavors.name
	flavor_name_list = conn.flavors.inject([]) { |flavor_name_list,h| flavor_name_list << h.name unless flavor_name_list.include?(h.name); flavor_name_list }
	flavor_name_list = flavor_name_list.map {|name| name.gsub!('.','_')}
	result.each do |v| conn.servers.each do |n|
	if v == n.key_name then
	images[find_images(n.flavor_id.to_i,images)].owners[v] +=1
	images[find_images(n.flavor_id.to_i,images)].type = convert_typeId_typeName(n.flavor_id.to_i,conn).to_s
	end
	end
	end
	total_instance_by_type.each do |k, v|
	v ||= 0
	push_data("stats.HPCS.#{sample_az}.instances.size.#{k} #{v} #{@g.time_now}")
	end
	send_type_by_owner(images,sample_az)
	conn = nil
	end

view raw instances_by_owner_size hosted with ❤ by GitHub

Even if working with Ruby isn't your thing™, anyone can easily see the value that Fog is giving you.  Script-able infrastructure changes, and asset querying that spans the Compute, Storage, DNS, and CDN services is some crazy useful stuff.


Happy Bootstrapping!

Performance Testing Strategy in 5 steps

When I get put on a new project I'll make certain adjustments but overall my goal is to get to 'Performance Ready' as quick as possible.  This allows me to divide my time on things that really matter.  I can focus about 70% of time on exploratory performance testing and syncing on serious defects with developers; while 20% is spent on analysis, and 10% spent on fun things like front end performance or sleeping.

Below are a few notes on how I get to 'performance ready' in a few stages.

Testing Restful services with Jmeter

Outside of Transaction Controllers, regular expressions and xpath extractors are probably used frequently in your scripts.  Sometimes you have the need to collect multiple variables from one or more responses and store them in one variable that will become a data parameter in a subsequent request.
Generally if you read through Jmeter User Group threads you’ll see lots of people trying to do this or at least asking how do I store variables in an array type of data structure.
In order to do this

Ruby Sparklines

My very bright younger sister was visiting me a few weeks ago. While the rest of the family was enjoying some relax time we decided to mess around with some Ruby.

Jmeter remote execution, log collection via STAF / STAX

One of the biggest challenges after learning your way around Jmeter is figuring out a way to organize and retrieve results in an automated way. There are different documented ways of doing this via Ant scripts or even neat graphing results like Performance Bling. Though my needs were different and after much looking around, coding and testing my performance environment is made up of 3 major areas.

Getting Started in Jmeter

A while back feeling increasingly frustrated by commercial load testing tools I decided to take Goranka Bjedov's advice in her presentation Using Open Source Tools for Performance testing.

I made the switch to

Load STAF quietly- How To

Problem:  Staf doesn't start unless someone logs into the computer or STAFProc window is closed accidentally.


Well don't fret there's a very easy solution to get STAF to start up; and get rid of that pesky window.

You can read....