Beta Testers Delight, an AWS EC2 AMI to Chew On

Not so long ago I was on the lowest rung of the Microsoft Partner ladder. Initially I received box loads of DVDs each quarter with the latest development and server software. Often it was several weeks after I needed the install disks, but I never used more than 10% of them. Then Microsoft moved to OTA distribution and I only needed to download those disk images that I actually needed, and I received them as soon as they were available. A big improvement except it tore a big hole in the Internet data cap.

This big distribution improvement did nothing for the next problem which was to find a physical or virtual machine on which to install and test the new software. But now even that chore has vanished. Joe Brockmeier reports on an ideal solution for beta testing new Microsoft servers in a post entitled ‘Amazon Spins Up Microsoft SQL Server 2012 on AWS for Testing’.

Amazon recently announced the availability of Windows Server 2008 R2 in its range of EC2 machine instances, AMIs. Now they have made available an AMI with SQL Server 2012 (Denali) community preview running on Windows Server. All beta testers need do is head to their AWS account and fire up the SQL Server 2012 AMI. In a few minutes they are testing the new version with no massive downloads, no local machine to find, and Internet data caps largely unaffected. Brilliant! Surely all new server software should now be made available this way.

Pity I am a Microsoft Partner no longer! I done ‘gone Google’ a year or so ago.

Time to Store All Cloud Data in Dynamic Memory

It is fairly clear we are at a point when purchasing personal workstations consisting of tablets, ultrabooks and laptops that only solid state drives, SSDs, make sense. There are huge performance benefits and the generally lower capacity of an SSD compared to a hard disk drive, HDD, of similar cost is not a problem with apps and data rapidly migrating to the cloud.

One might imagine that the cloud data itself, though, being so massive, must necessarily be stored on massive arrays of HDDs that is currently the case. It therefore stretches belief to find recent research indicating that the most cost effective way to store cloud data from now on is to use massive arrays of standard dynamic random access memory, DRAM. The article by Scott Fulton entitled ‘Stanford: Move the Cloud from Disk to DRAM’ summarises the paper by Ousterhout et al that introduces the punchy concept of RAMClouds. The argument relies on the 3-year cost of ownership for large datasets taken from the second figure in the paper:

I expect this to be an historic turning point for cloud infrastructure and totally unexpected. RAMClouds will have an absolutely major impact on cloud services from the performance perspective and well as the design and siting of data centers. I can’t wait to see the effects.

Major Shifts in IT Encore

Great diagrams convey information to all readers with utter simplicity. I think the diagram below by Dion Hinchcliffe conveys complex IT trends with pure clarity. I simply have nothing to add except the appearance of cloud in the context of this blog is important.

See Dion’s post for a much more detailed commentary on these weighty matters.

The First Truly Shareable Computer

As Cliff Boodoosingh points out in in his post ‘Get a Library Card & Borrow a Chromebook’:

Here’s a novel idea: go to the library and borrow a Chromebook.

This is what is happening in Hillsborough Library in Hillsborough, New Jersey. With Wi-Fi already available throughout the library the borrower simply opens the Chromebook lid, and within a couple of seconds logins in with a Google account and is connected with the cloud. Even better is at logout all trace of that user leaves the Chromebook ready for the next library patron – the Chromebook is a genuine shareable computer.

It gets better again since no power point is needed, the bane of most libraries because of insurance liability requirements. The Chromebook’s 8-hour battery life is good enough for a day’s heavy usage. At Hillsborough the rental period is 4 hours with up to a 2-hour extension.

I wonder when and if this service will arrive at my local Gold Coast Library?

Code Snippet Archive to Benefit Everyone

I just saw a very useful jQuery code snippet from @aneesha in one of her tweets:

jquery to hide or show columns in a table $('td:nth-child(2)').toggle();

I don’t need this code today but probably could use in the future. Where should I store it for safe keeping? Indeed where should Aneesha store it for all software devs to use?

I have been storing such coding gems in my Evernote repository (great for me), and I can even make my Evernote notes available to everyone (only great if everyone can find them). This is not a good solution for a really useful repository and permanent curation.

However I did come across a much more useful solution when I attended the BrisJS meetup earlier this week. Pete Chappell gave a lightning talk about hashbangs and he put his notes/slides/code examples in the gist repository collection on github. As all developers have long known github has become the popular place to store shared, distributed repositories of code/data/text documents of all kinds.

You don’t have to be a software developer to find these repositories useful, they can be used for web sites, large structured documents, data collections and so on. git and github are supported on all hardware platforms.

github says of gist:

Gist is a simple way to share snippets and pastes with others. All gists are git repositories, so they are automatically versioned, forkable [copied, cloned] and usable as a git repository.

You can create both public and private gists and update them to new versions as time goes by. All the public gists are searchable and there is an immense treasure trove of code and data samples there, both big and small. Pete Chappell’s gist is number 1,258,359 for example.

If you have text-based data of any kind you want to share publically or privately long-term then gists are for you. OK, Aneesha?

Writing Faster Programs about to be Harder and Harder

The computer-buying public have known for a year or two that individual central processing units or cores have ceased increasing in speed. The faster computers on the shelves need multiple cores. The same is becoming true for smartphones and tablets that now need multiple cores to be faster. But heed the warning from Steve Blackburn in his post on ‘Performance anxiety: the end of software’s free ride’. Writing software that runs faster from here on in is going to be very difficult indeed.

As Steve points out we have known how hard it is to write software to keep our multiple cores executing at the same time, ie parallel processing. He cites the simple example:

just as two cars are unlikely to get you to work faster than one, the addition of another core is often unhelpful in completing a computing problem more quickly

In the last 30 years of my computing experience the advances in writing parallel programs have been slight. So this is the first hard problem.

We used to think it was easier to make all the cores the same but it is now becoming clear we need a hierarchy of cores with varying power to control them efficiently – ‘ many designers now believe that we need a more complex combination of simple and powerful cores on each chip’. so ‘ heterogeneity takes us even further from the world of transparent performance improvements’.

Steve then reveals the problem of giving enough power to lots of cores on one chip. There is an obvious physical limit.

In practice, power densities on chip have become so high that we can no longer fully power an entire chip lest we melt the silicon. This radically changes the economics of microarchitecture.

Thus software developers must now optimise for energy consumption, not performance – a completely new approach will be needed. New optimisation algorithms will be needed.

The hard problem of parallelism still lies unsolved after decades. We computer scientists are going to have to do a lot better with the heterogeneity and energy optimisation hard problems if smartphone/tablet/laptop/desktop performance is not to stagnate. A dim prospect indeed unless some new lateral thinking can come to the rescue.