Small update to rdzr.co pushed

I just pushed a small update to http://rdzr.co that improves error handling, uuid-collisions, and, more importantly, it now has reverse lookup (which can be helpful if you don't trust a rdzd URL). To do so, simply add '/reverse' to the suspect URL.

For example:
You see http://rdzr.co/xyz in a friend's post, but you know the person is a well-known RickRoller so, to foil his or her nefarious plans you simple copy and paste the URL into your browser's location bar, and append /reverse at the end of the URL (i.e. http://rdzr.co/xyz/reverse). WAM. Crisis averted.

Another of my small personal pet projects is up: RDZR

About a three days ago I thought up of a very simple use case to tackle as a personal project: a URL "minifying", "shrinking" service. It so happened that my head was fresh-hot on web2py that I had been using for a work-related project. So, I took on the task to implement it as fast as I could -- given the natural time constraints that come with being a father of two rambunctious kids and a full-time job :) So, having which framework to use figured out, I moved to the next question: where to host it? AWS was a natural choice given that I've been using their services for about one-and-half year with very little problems. However, I  thought I'd give GAE (Google App Engine) a try given than web2py has built-in GAE deployment (including database layer for their data store). But, as it turns out, GAE doesn't offer what's, in my opinion, very basic options (like providing/renting IP addresses and assigning it to an app so that one can have DNS A-records pointing to it), as well as lack of SSL for custom domains. So, after bumping my head against their wall, I went with AWS -- and yes, it was as easy as pie to setup: fire up the instance, associate elastic IP, point Route 53 to that address and presto!  That out of the way, I tackled the code aspect. After, say, couple of hours I was done with that. About forty lines of python (although I'm sure hard-core pythonistas could compress it further through conventions and idioms) and a few tweaks to their routing it was all it took to have the whole service completed -- with error handling and all that jazz. Then came my least favorite part of the process: dealing with the presentation layer (i.e. HTML/CSS). Even though I'm using most of web2py's default layout and CSS, it was still rather annoying adding my own dash html/css to it. But that's done.

So without fanfare I present you RDZR: Yet Another URL ReDuZR. Which can be reached through http://rdzr.co (or its SSL flavor, if you prefer https://rdzr.co)

Technologies used: Python, Web2Py, SQLite, Apache + mod_wsgi + mod _ssl

HTML5 value semantics: a pragmatic perspective.

In recent weeks there has been a healthy and lively debate about value semantics in the HTML5 spec. Divya Manian on her "Our Pointless Pursuit Of Semantic Value" opinion article decries and points out the problems that can arise if we give HTML5 semantics too much importance. A bit later, Jeremy Keith's riposte to Divya's article brought forth the reasons why semantics are important in HTML5.

I will begin by saying that, in my opinion, HTML5 value semantics are purely academic. I don't mean that in a derogatory manner, but rather, the people who ultimately matter (i.e. the end user) could not care less about how semantically correct the markup behind a web page is. Will a semantically correct HTML5 page make you a better designer or developer? Probably not. Will a semantically correct HTML5 pages bring more views to your website? No, by itself it won't. Will your end-users have an improved experience by having semantically correct HTML5 page(s)? In my opinion, no. Will your customer/client approve your work because it is semantically correct or because it looks good and works well? I think we know the answer.

Also to bear in mind is that these semantics are shallow shells. That is to say, behind the scenes, these semantic tags are converted to spans, divs, h1s or other "primitive" html elements by the rendering engine and once painted on the page, they are invisible to the user. So, in many ways, I have to agree with Divya: time spent on figuring out the correct semantics is pointless. Not to say that you shouldn't use them, but they should not be an excuse or an obstacle to your creativity. Semantic markup shouldn't be the basis for design decisions and certainly shouldn't be a reason to hold a meeting about.

In short, life's too short; don't sweat the markup semantics.

 

DIY Basic AWS EC2 Dashboard using Apache, Python, Flask and boto (Part I)

While Amazon Web Services offers a nice web-based UI to handle and manage EC2 instances, it might well be the case that you do wish to give access to some of this functionality to more people in your organization, but you do not with to provide them with full access to the AWS EC2 dashboard or wish to limit the type of API calls they make (for instance, you might want to allow users to start/stop instances, but you do now want them to be able to launch/terminate them). Whatever your use case might be, you can create your own "in-house" EC2 Dashboard with relative ease. Our software stack will consist of:

• Apache (for basic authentication, SSL and WSGI, virtual hosts) with mod_ssl and mod_wsgi.


• OpenSSL (for SSL and certificate generation).


• Python.


• Flask (py-based web services micro-framework).


• boto (py-based AWS API library).


• AWS KeyID/SecretKey credentials.


• Admin/sudo privileges

Please note, if you do not need SSL/htpasswd you can use Flask's bundled web server which is suitable for most in-house deployments; however in this example, I will be using Apache. Also to note: I'm not going to spend time in the installation process of the packages/SSL certicate generation above as it should be fairly straightforward for anyone with minimal dev/sysadmin experience as well as there being many well-written tutorials for the setup of  these tools floating around the 'net.

First we need to make sure your tools are working. Make sure Apache is working, make sure mod_ssl is working, etc. Open up a python prompt and try importing flask, boto and so forth. Once you are fairly confident your tools are good to go then let's get moving.

1. Create an Apache virtual host entry file specifying the SSL certificate location, port number, location of the wsgi file and other important parameters as show below:

<VirtualHost *:443>
        ServerAdmin webmaster@localhost

        DocumentRoot /var/www/[WEB_APP_NAME]
        SSLEngine On
        SSLCertificateFile /path/to/certs/server.crt
        SSLCertificateKeyFile /path/to/certs/server.key
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>

        WSGIDaemonProcess [WEB_APP_NAME] user=[APACHE_USER] group=[APACHE_USER_GROUP] threads=5
        WSGIScriptAlias / /var/www/[WEB_APP_NAME]/[WEB_APP_NAME].wsgi

        <Directory /var/www/[WEB_APP_NAME]>
                WSGIProcessGroup [WEB_APP_NAME]
                WSGIApplicationGroup %{GLOBAL}
                WSGIScriptReloading On
                Options Indexes FollowSymLinks MultiViews
                AllowOverride All #important line for using htpasswd
                Order allow,deny
                allow from all
        </Directory>

		#other settings here

</VirtualHost>

2. We then need to write the wsgi file telling mod_wsgi which app and instance it should start when a request comes along. It's a very simple file. Just make sure its name matches the ones you provided in the vhost file above. Its contents should look something like this:

from [WEB_APP_NAME] import app as application

One thing to bear in mind is that this way of using wsgi requires that your web app be anywhere in the $PYTHONPATH environment variable. Save, close and restart Apache.

3. For this example, as said above, I'm working under the assumption that your authentication needs are very basic and those requirements can be fulfilled with Apache's htpasswd. Assuming you setup Apache correctly with the right mods, you can go to your application's root directory and tell htpasswd to create a password file (.htpasswd) with a username-password pair. You can do so by running the following command:

sudo htpasswd -c .htpasswd [USERNAME]

It will then prompt you for a password which it will then encrypt using basic symmetric encryption algorithms and the create the .htpasswd file.

4. Next step is to setup your .htaccess file so that Apache knows when to ask for the credentials. Your .htaccess should have (at least) the following rules:

AuthUserFile /path/to/[WEB_APP_NAME]/.htpasswd
AuthGroupFile /dev/null
AuthName "EnterPassword"
AuthType Basic
require valid-user

You can test it by pointing your web browser to whatever URL you set up for this site. You should now be prompted with a username/password dialog.

5. Now that we have the basics ready, let us get to the meat and substance. First I highly suggest you give a quick perusal to Flask's "Quickstart" tutorial which can be found here and trying out the first few trivial examples to make sure you have everything setup correctly.

6. Make a new file named [WEB_APP_NAME].py and copy-and-paste the text below:

from flask import Flask, flash, abort, redirect, url_for, request, render_template
from boto.ec2.connection import EC2Connection
import boto.ec2
app = Flask(__name__)
akeyid = '[AWS_KEY_ID]'
seckey = '[AWS_SECRET_KEY]'
conn = EC2Connection(akeyid,seckey)

7. One of the concepts to bear in mind with respect to AWS API connections is regions. There is no global end-point for your AWS API calls and calls made to that region's API only make sense for services within that Region. For instance, you can't "see" your instances in the west coast Region ("us-west-1") from any other region. So, whatever regions you wish to have access to, you need to specify those explicitly. By default, boto connects to the "us-east-1" region.

8. So, our index page will be the Dashboard itself, that is to say, a place where users will be able to see all the instances from all regions. You can choose to limit the regions you wish to show/scour fairly easily, but for the sake of this example I'm going to simply gather all the instance information from all of AWS' regions. I will the create an object data structure with all the data and eventually I'll pass that data to the template rendering engine that Flask comes with.  So, we're going to create an app route for the index page, use boto to create an EC2 connection, retrieve all available regions, get all the instance reservations in that region, get all the instances within each of those regions, then bundle the data in a data structure and pass it to the rendering engine.

a. Create the index route:

@app.route("/")

b.  declare your method:

def my_method_name():

c.  retrieve the list of all AWS available regions:

        allinfo  = []
        regions = conn.get_all_regions()
        for region in regions:

d. connect to each of those regions and retrieve all the instance reservations (something to note: I'm not sure if it's boto's or AWS' boo-boo, but retrieve_all_instances() method does not retrieve all instances per se, instead it retrieves all the instance reservations, which are instance "containers") :

                rconn = EC2Connection(akeyid,seckey,region=region)
                rsvs = rconn.get_all_instances() #read note above

e. loop over the reservations and gather all the instance information:

                   for rsv in rsvs:
                        insts = rsv.instances
                        for inst in insts:
                               #do stuff

f. now all together (including populating our instance info data structure and pass to the rendering engine):

@app.route("/")
def my_method_name():
        allinfo  = []
        regions = conn.get_all_regions()
        for region in regions:
                regioninfo = {}
                regioninfo['Name'] = region.name
                rconn = EC2Connection(akeyid,seckey,region=region)
                rsvs = rconn.get_all_instances()
                instances = []
                for rsv in rsvs:
                        insts = rsv.instances
                        for inst in insts:
                                instances.append({'Id': inst.id, 'Name':inst.tags['Name'],'State':inst.state, 'Type':inst.get_attribute('instanceType')['instanceType']})
                regioninfo['instances'] = instances
                allinfo.append(regioninfo)

        return render_template('index.html',all_info=allinfo)

9. Now that we have the route and the code, we are going to use Flask's nifty template rendering engine (Jinja). To do so we need to create a file that matches the name in the render_template call above.

a. create a file with nano or your favorite text editor named index.html (or whatever name you chose ). This file has to be (by Flask convention) inside a directory called 'templates' and this directory should be at the same level as your web app Py script.

b. copy-paste the following "boilerplate" html:

<!doctype html>
<html>
<head><title>EC2 Dashboard</title>
</head>
<body>
<div class="header">Welcome to EC2 Dashboard</div>
<div class="content">
<div class="region-text">Regions Available</div>
{% for region in all_info %}
	{% if region['instances'] %}
		<div class="region-info"><span style="font-weight:bold">Region: {{region['Name']}}</span>
		<span>Instances Avaliable</span></div>
		<div class="region-content">
			<table><tr><th>Instance Name</th><th>Instance State</th><th>Instance Type</th><th>Instance Id</th><th>Instance Actions</th></tr>
			{% for instance in region['instances'] %}
					<tr>
					<td><span>{{instance['Name']}}</span></td>
					<td><span>{{instance['State']}}</span></td>
					<td><span>{{instance['Type']}}</span></td>
					<td><span>{{instance['Id']}}</span></td>
					<td>
					<a href="/details/{{region['Name']}}/{{instance['Id']}}">See Details</button>
					</td>
					</tr>
			{% endfor %}
			</table>
		</div>
	{% else %}
	<div class="region-info">
		<span style="font-weight:bold">Region: {{region['Name']}}</span>
		<span>No Instances Avaliable in this region</span>
	</div>
	{% endif %}
{% endfor %}
</div>
</body>
</html>

It should be obvious that you can (and should!) use your own html and CSS styles. The above example was to illustrate the rendering engine usage and syntax, which is pretty self explanatory and simple. Flask's and Jinja's documentation is very good and when in doubt you should consult those as your primary source.

This concludes part I of this tutorial. In part II I will then show how to post information, how to use boto for modifying instance information and more on Flask's routing/url facilities.

JavaScript/Canvas based Barnsley's fern generator

As part of my endeavors in the world of HTML5 Canvas, I wrote a Barnsley's Fern fractal generator that's currently hosted at jsdo.it It was a nice learning experience and pleasantly surprised me that most modern web browsers can handle the generation and rendering of 100K data points within a second (this includes mobile webkit browsers).

I was planning on implementing a Koch's curve generator, but I'd need to implement a js/canvas version of "Turtle graphics" before that and I've honestly not had the time to invest on it. But, one of these days I'll get frisky and work on it :)

html5/js-based Analog Clock

Last night I felt both bored and industrious. One of this things I've been meaning to do as of late is to mess around with html5 canvas (2d) object. So, I headed to http://jsdo.it which is a Japanese site with a take on "social" sharing of code. There I saw several "digital" clocks, but saw none that were analog (at least not in the front page or through searching). So, I thought it would be a good exercise to make one. Two hours later it was out for public view/critique. It was really fun and a bit challenging ( figuring out the methods available in the canvas 2d context was a bit of a pain) -- mostly remembering my high school trig :) Once I had the seconds "hand" figured out it was easy peasy thereafter.

Next, I plan to whip out couple of 2d fractal generators ... nothing Mandelbrot or Julia though (those get plenty of limelight and mentions).  If I see the canvas 2d platform lends itself to it and if I feel gumptious enough I'm going to start a math graphing and visualization html5/js library. If you are interested in collaborating in such library, drop me a line via Twitter.