Shifting apps to cloud no easy task

The uptime, cost and administration benefits of shifting to cloud computing are self-evident, but reworking applications to actually operate on cloud systems can greatly complicate that process, according to open source developers.

At the recent PyCon developer 2010 conference in Sydney, an underlying theme in several presentations was the difficulty in reworking applications to operate effectively in a cloud environment - even if that could ultimately deliver major improvements.

For example, software developer Richard Jones is one of the administrators on the Python Package Index (PyPI) project, which tracks and hosts modules and applications written in the Python language in order to help other developers easily locate useful utilities.

PyPI, informally known amongst Python users as 'The Cheese Shop', has seen a rapid increase in traffic as the development language - widely used by online projects such as YouTube - has grown in popularity.

"Present indications are that it will overtake [Microsoft's] Visual Basic by the end of the year," Jones noted.

PyPI itself had indexed 44,000 package releases by the beginning of June 2010 and sees thousands of updates a month.

"The Cheese Shop is a single server run in the Netherlands," Jones said during his presentation, noting that this was beginning to become a bottleneck on the growth of Python development.

A proposal for mirroring the server system has been active since 2009, but doesn't offer sufficient support for client-side access, Jones said.

"A more agreeable solution may be to move the Cheese Shop off into the cloud."

However, any such migration would be hampered by the way in which the code for the project was originally structured.

"The Cheese Shop has some eccentricities which make it difficult to work in," Jones said, which would complicate any attempt to re-architect it for the cloud.

Mark Streatfield, a software developer specialising in movie projects, also noted similar barriers. He said that the software typically used on motion pictures was often developed in an unstructured and "secretive" way that made re-use and migration difficult, even with a "limited cloud" approach based on internal virtualisation.

"The software is very much a side effect of the process of making a film," Streatfield said. "The software development process is much more ad-hoc than usual, and since everyone has access to the language, the amount of code is staggering."

Basic processes such as documentation and reuse tend to be ignored, making any shift to centralisation extremely difficult.

"The challenge now is trying to control that process," he said.

Coding and process management aren't the only challenges. Small-scale cloud projects typically use public cloud providers such as Amazon EC2 or Google AppEngine, and availability can be an issue when accessing those services.