Super-Cannes: 'actors in our own self-referential drama'

Exponential information growth

On 1 January last year I posted about information trends. I had been reading 'How Much Information? 2003' (School of Information & Management Systems, Berkeley), led to that study by Alex Barnett. (There's a summary of the 2003 findings here.)

Now comes a new study, The Expanding Digital Universe: A Forecast of Worldwide Information Growth Through 2010, also found through Alex — The Expanding Digital Universe. These points are from the executive summary:

YouTube, a company that didn’t exist just a few years ago, hosts 100 million video streams a day. Experts say more than a billion songs a day are shared over the Internet in MP3 format. Digital bits. London's 200 traffic surveillance cameras send 64 trillion bits a day to the command data center. Chevron's CIO says his company accumulates data at the rate of 2 terabytes – 17,592,000,000,000 bits – a day. TV broadcasting is going all-digital by the end of the decade in most countries. More digital bits. …

  • In 2006, the amount of digital information created, captured, and replicated was 1,288 x 1018 bits. In computer parlance, that's 161 exabytes or 161 billion gigabytes … This is about 3 million times the information in all the books ever written.
  • Between 2006 and 2010, the information added annually to the digital universe will increase more than six fold from 161 exabytes to 988 exabytes.
  • Three major analog to digital conversions are powering this growth – film to digital image capture, analog to digital voice, and analog to digital TV.
  • Images, captured by more than 1 billion devices in the world, from digital cameras and camera phones to medical scanners and security cameras, comprise the largest component of the digital universe. They are replicated over the Internet, on private organizational networks, by PCs and servers, in data centers, in digital TV broadcasts, and on digital projection movie screens.
  • IDC predicts that by 2010, while nearly 70% of the digital universe will be created by individuals, organizations (businesses of all sizes, agencies, governments, associations, etc.) will be responsible for the security, privacy, reliability, and compliance of at least 85% of that same digital universe.
  • This rapidly expanding responsibility will put pressure on existing computing operations and drive organizations to develop more information-centric computing architectures.
  • IT managers will see the span of their domains considerably enlarged – as VoIP phones come onto corporate networks, building automation and security migrates to IP networks, surveillance goes digital, and RFID and sensor networks proliferate.
  • Information security and privacy protection will become a boardroom concern as organizations and their customers become increasingly tied together in real-time. This will require the implementation of new security technologies in addition to new training, policies, and procedures.
  • IDC estimates that today, 20% of the digital universe is subject to compliance rules and standards, and about 30% is potentially subject to security applications.
  • The community with access to corporate data will become more diffuse – as workers become more mobile, companies implement customer self service, and globalization diversifies customer and partner relationships and elongates supply chains.
  • The growth of the digital universe is uneven. Emerging economies – Asia Pacific without Japan and the rest of the world outside North America and Western Europe – now account for 10% of the digital universe, but will grow 30%- 40% faster than mature economies.
  • In 2007 the amount of information created will surpass, for the first time, the storage capacity available.

This incredible growth of the digital universe means more than simply the fact that as individuals we will be facing information explosion on an unprecedented scale. It has implications for organizations concerning privacy, security, intellectual property protection, content management, technology adoption, information management, and data center architecture.

The growth and heterogeneous character of the bits in the digital universe mean that organizations worldwide, large and small, whose IT infrastructures transport, store, secure, and replicate these bits, have little choice but to employ ever more sophisticated techniques for information management, security, search, and storage.