Spectra Logic Announces BlackPearl – Tape Archiving For The Internet Generation

Spectra Logic Highlights

• Announces BlackPearl – a deep storage appliance sitting between clients and tape libraries allowing a RESTful interface
• Expands the use of Amazon’s S3 language with its DS3 bulk commands
• Claims significantly lower costs with tape libraries (versus long term disk storage)
• Wins endorsement from Yahoo!, NASCAR, axle Video and EVC
• Makes tape archiving relevant for the Internet generation

Smaller vendors are usually more agile than the large suppliers – which is certainly true of Spectra, whose management team has been briefing a room full of analysts for the last couple of days in Colorado.
I must admit tape has been back of mind for many data centre managers over the last few years and the market has moved downwards since the Credit Crunch in 2008 – beyond the typical peaks and troughs of the long 8-12 year replacement cycle of this type of equipment. The meeting here has revitalised my interest in the area, because Spectra has announced a few things that make tape more appropriate for modern application owners. You’ll want to find out more about the technology, target customers and future.

Launching BlackPearl – A Data Moving Appliance For REST Protocols

We’ve already looked at Spectra’s ‘deep storage’ vision. As part of the strategy it has launched DS3, an interface based on Amazon’s Simple Storage Service (S3) programme. It adds BulkPut and BulkGet commands to S3’s ‘put’ and ‘get’, while introducing the concept of a bucket – a receptacle big enough to store projects with multiple files.
Today it announces its BlackPearl appliance with general availability in December, which sits between clients and tape libraries to make tape archiving much more appealing to those familiar with REST protocols. Developers write code in REST/HTTP(s), which is translated along with appropriate tape management controls via SAS (and coming soon) Fibre Channel connections to tape libraries at the back. Client servers are connected via 10G Ethernet (or Infiniband for HPC customers).
It believes that the complexity of dealing with and programming block storage has been holding back the use of tape – one reason why it introduced its nTier Verde archive-level disk array back in May. That product is connected to clients using CIFS/NFS connections.
BlackPearl is a 2U high rack-mountable appliance containing 4 x 400GB Enterprise MLC SSD. Inside it handles the new commands, stores data in its buffer before parcelling it off in buckets to the attached libraries (which can contain a maximum of 4 tape drives).
The cost of archived data is of course much cheaper that disk. Using list prices for BlackPearl and its own nTier Verde array for comparison it calculates the cost per GB as follows:

• nTier Verde archive grade NAS disk – 1.4PB @ $0.45/GB
• BlackPearl for 6.4PB @ $0.09/GB T950, using IBM TS1140 tape drives 4TB cartridge
• BlackPearl T950 and LTO 2.4OB @ $0.10/GB
• BlackPearl with T380 and LTO 1.9PN @ $0.14/GB

The price of tape has always been compelling for those who need to store large quantities of data for long periods. BlackPearl brings its potential use into the 21^st century – clusters of client servers in the front-end, for instance, can be running Hadoop – a use which Spectra has already captured in client code.
To encourage new developers Spectra is also launching a Web site complete with a BlackPearl simulator and its Hadoop front-end code to download. Spectra may also offer the DS3 Hadoop client for consideration as part of a future Hadoop distribution.
Customers In Media, Entertainment, Public Cloud

While guarding its secret carefully Spectra has shared developments with 30-40 of its closest partners and customers, 4 of whom we were able to meet at the conference. They were very positive about DS3 and BlackPearl. In particular:

• Sam Bogoch of axle Video – a Media Asset Management company – presented a business view of developments including market stats on the size of the storage market; he pointed to the growth of storage spending in broadcast and video production companies, driven by the use of higher resolution and multiple cameras in production; his company has its own ‘Gear’ appliance, which – when added alongside BlackPearl – will allow for easier management, editing, approval, storage and retrieval of media content
• Davy Van Ceursen of EVC – a company involved in live video editing including instant slow motion replay – talked about his Proof of Concept work with the BlackPearl simulator; he noted that its simple interface fits with new trends of broadcast architectures and that the use of REST APIs will be helpful without DS3 having to become a standard
• Kevin Graham from Yahoo! talked about its endorsement of BlackPearl in its Tumblr and Yahoo! Mail business (both of which now offer customers 1TB of personal data); it recognises the need for backup and archiving to tape, but wants to have secondary storage without middleware; Yahoo! is also a major Hadoop user with 100k nodes; BlackPearl gives it the opportunity to link that to tape more easily than before
• Chris Witmayer from NASCAR is involved in media production and talked about the massive amounts of data created in each race event; while maintaining a dual supplier approach at each layer of its architecture it has used Spectra to help it make significant savings in archival ingest times

Those in broadcast, video and audio recording industries naturally think in terms of linear storage – so it’s no surprise to find them endorsing the attempt to make handling data archiving easier. Other appropriate vertical markets include:

• Scientists working on human genome analysis, who will be able to call up projects over night to be placed into disk or SSD
• Museums and others interested in making a digital archive of analogue tape stock, sometimes offsetting the costs through monetisation
• Government agencies who capture lots of mechanical data, but often do not need to access it for years
• Oil and gas companies who capture huge amounts of seismic data before analysing it

Amazon AWS, S3 and AWS Glacier are both the competition and inspiration for DS3 and BlackPearl. Spectra claims that S3 has stored 2 Trillion objects since 2005 (which is the equivalent of 250 objects per person alive on Earth) and that AWS will break $1 Billion in annual revenues before 2014. Glacier offers customers an archiving service at a low monthly charge of $0.01 per GB throughout the life of the data, with penalty charges for early access. Spectra expects DS3 will be broadly adopted as an option in software partner applications in addition to others writing their own clients.

Some Conclusions – Simplifying The Use of Archiving For The Internet Generation

Spectra is a medium-sized storage player making most of its 9-digit annual revenues from tape libraries. It is well connected to customers and partners working in the media, entertainment and broadcast business, where linear storage is a norm. The introduction of DS3 and BlackPearl will increase the relevancy of those devices to 100s of newer companies for whom block storage is a complex and secret science.
It has no ambition to become a primary storage vendor itself, nor does it tie itself exclusively to tape as an archiving medium going forward. It demonstrates a greater agility in its vision and planning than most of the larger companies with which it competes. We expect to see additional features, such as clustering of BlackPearls and the addition of FC and perhaps even 10G Ethernet in the future. It will be interesting to see how the other players react to these announcements, especially in adopting DS3 themselves. We’ll be watching the company closely in the future.