OT: Cloud Computing is coming to ...

[Christopher A. Williams]

On Wed, 2010-07-21 at 03:00 +0100, Marko Vojinovic wrote:
> On Tuesday, July 20, 2010 23:18:11 Phil Meyer wrote:
> > That is the whole point.  Ideally this is how it works from a ptactical
> > point of view:
> > 
> > I am the Dean of Engineering and we need to run a massive simulation of
> > a type 5 tornado on a set of bridge types.  This is not only for
> > instruction is a senior level class, but as our main graduate focus for
> > the next year.
> > 
> > We assess our own needs, and access the University 'cloud' website.  I
> > order 256 2.6 GHZ or better CPUs, 192GB of RAM, and 2TB of disk space,
> > and specify Linux as my OS.
> > 
> > Thirty minutes later I get an email notification with the hostname, IP
> > Address, administrator login, and password for my new compute environment.

<snip...>

> Nope. Thirty minutes later you will get an email saying that the simulation 
> from the Dean of Engineering is using up vast amount of resources, and that 
> there is nowhere nearly enough RAM for what you want.

Nope - That's not true. Resource sharing in a cloud environment doesn't
work that way unless you're really bad at managing things. If you are,
go back to the 101 level course on managing cloud infrastructure. This
is the most basic of operations management for cloud.

> Then you get pissed off, call the Engineering Dean on the phone, and he tells 
> you that his department invested more money for the cloud infrastructure than 
> you, and that his simulation is more important than your database, from the 
> scientific point of view. Then you call University Dean and through him order 
> the cloud maintainers to reduce the resources for the simulation in order to 
> accommodate for your database.
> 
> Dean of Engineering then gets pissed off, and decides to withdraw his share of 
> money and build his own cloud, to be used only by the Engineering department. 
> Soon enough other departments do the same and you end up having a dozen of 
> clouds on the University, and they don't interoperate since people are fighting 
> over resources and who invested more money and whose work is more important.
> 
> This will of course defeat the purpose of having a cloud in the first place, 
> since every department is going to invest into their own equipment, and then 
> have it idling or thrown away after their projects are over.
> 
> And actually, all this has already happened. I've seen it on a couple of 
> Universities. It's just that they were "sharing" (ie. fighting over) *cluster* 
> resources, not *cloud* resources. But that's just terminology difference.

No - It's not just a terminology difference. There truly is a difference
between clusters and cloud environments. In cloud environments, You can
absolutely guarantee resource availability (CPU, RAM, Disk, and Network
resources) to a designated groups of systems, and you can dynamically
scale the environment to efficiently meet the compute needs of all
parties. If anything, it makes capacity planning much simpler.
Traditional clusters simply do not, and cannot, do this.

> When people invest their money into something, they want to be "in charge" of 
> it. And if they are supposed to share it with others, there is bound to be a 
> lot of friction.
> 
> I agree that this cloud environment can be useful in a commercial company 
> where there is only one "money bag". But in University environment, somehow I 
> doubt...

I've faced this issue in more client engagements than I care to count.
It's invariably a red herring. The vast majority of people really just
want reasonable guarantees that they will actually have their
*realistic* expectations met. Maintaining control over one's own little
IT fiefdom for the sake of ego maintenance is something even commercial
organizations can not afford these days. Let's not even start with
universities.

In my professional life, I deal with exactly these kinds of issues
constantly. While most folks just want to understand how things work,
the truly dogmatic objectors turn out with impressive consistency to all
be basically ego driven control freaks, who are also not very adept at
this kind of technology - let alone IT infrastructure.

One short, but true story: Someone who fit this dogmatic category well
(his own coworkers would joke about him) once asked me about how much
storage we were allocating VMs. I said, "On the average we plan for 50GB
of disk per VM," - an industry norm.

Half-chuckling, he said, "Well, that's not enough for us."

"Really?" I replied. "How much disk do you need?"

Leaning back in his chair, he smugly answered, "We usually deal with
storage in a minimum of half-terabyte increments."

"Oh." I said. "I see. ...and how much of that half-terabyte are you
currently using?"

Someone from the other end of the table snickered (I guess they couldn't
help it) and said, "...About 20GB."

...That pretty much ended the meeting. I thought to myself, so in
reality, this guy *wastes* about half a terabyte of disk at a time.

Over the life of that phase of the project, he never came close to using
200GB (although he demanded much more repeatedly - he just couldn't
justify it), and when his disk usage spiked, people jokingly questioned
what part of his personal MP3 collection he was keeping out there - at
which point his storage usage started to go down.

We just made sure he always had the compute resources he really needed,
and also made sure that he was efficiently using what was given to him.
Think of it as a "Eat all of your food, children, or no dessert!"
mantra. 

Cheers,

Chris

-- 

===================================

"The problem with socialism is that
you eventually run out of other people's money."

-- Margaret Thatcher