Making the right database choices

Whoever owns the data, owns the organisation and its applications and other IT resources. Little wonder the database market is so competitive.

But databases are becoming commodities. When one product overtakes the next in terms of functionality, it is only a matter of time before it is leapfrogged again. At a high level, any database will probably suit many users` purposes. Along with technological maturity, the players and the different camps are also more or less fixed.

Adding to this climate of sameness, there are things all databases claim: open standards, high performance, reliability, high availability, scalability, failover, security and the rest of the well-known must-haves.

Some vendors, however, offer unique features. Open source, mainly trading on its being "free", is one claim not made by every vendor. However, the issue of total cost of ownership (TCO) goes beyond licences, and support is another factor that complicates open source value. Many vendors still answer its challenges fairly satisfactorily.

Neil Solomon, Oracle technology specialist at T-Systems South Africa, a systems integrator, says players like MySQL are still small, and will probably get snapped up sooner or later, when it starts posing a serious threat. MySQL also offers native XML, something other vendors are just starting to pay lip-service to. Alan Raubenheimer, CEO of Embarcadero Africa, calls MySQL a "rising star", for its low cost, XML and bundling strategy.

Where the differences between offerings start coming in is when one recognises that the purposes for using a database differ extensively. So it is often better and easier to find the database best suited to a company`s needs than trying to establish which is the "best" database, a quest that seems fruitless with the above me-too scenario.

MySQL is a rising star, for its low cost, XML and bundling strategy.

Alan Raubenheimer, CEO, Embarcadero Africa

But "purpose" could mean many different things. It entails, among others, the following questions: Will it serve an enterprise or fit a smaller setting? Is it meant for transactioning or analysis? Does it support the operating infrastructure?

Must it offer a Web development platform? Will it run in a mostly heterogeneous database environment? Will the user have to migrate from current technology and need help from the kinds of tools such as those offered by Embarcadero? Does it support applications that need access to all databases?

The database market is crowded and fiercely competitive. Each vendor will market according to the criteria it feels it is strongest in, and with so much evangelising, things can be downright confusing. The above questions should be asked of vendors. For current purposes, a high-level picture of the main players and offerings, their unique selling points and areas of overlap should provide some insights.

The database market is awash with relational heavyweights like IBM (DB2), Oracle (9i), Microsoft (SQL Server), Sybase (Adaptive Enterprise) and Informix (bought by IBM in 2001).

There are no recent local stats, but IBM leads the worldwide relational market, overtaking Oracle`s lead about two years ago (Gartner). The 2002 showing is worse for Oracle, which now faces serious market share competition from Microsoft on the lower end. Microsoft`s SQL Server became the top Windows-based vendor last year.

Gartner found that the worldwide database market shrank by 7% in 2002 in new-licence revenues to $6.6 billion. IBM still leads, with 36% market share, Oracle is again second (34%), and Microsoft is third, with 18%. Oracle`s new-licence revenues for the overall market were down 20%, with IBM holding firm and Microsoft surging nearly 17%.

One size does not fit all in databases.

Stefan Ferreira, product manager, Sybase

SQL Server is well positioned for a climate of low spending. Its per-processor cost is half of Oracle`s charges, and slightly less than IBM`s price. Hence, it is strong in small and medium-sized businesses, one of the few growth spots in the database sector and a new market for Oracle and IBM.

Oracle is now pitching its products as the upgrade path for SQL Server users who need more features and user capacity, and by putting new energy into Linux.

Object-related database vendors have made a niche for themselves in a much smaller market. They include Computer Associates (Jasmine) and Progress Software.

While relational databases give the ability to query and manipulate data in complex ways, object databases have the advantage of rapid application development, ease of maintenance, reusability of code and software robustness.

Many traditional relational vendors, including Oracle, have added object features to their offerings. Intersystems SA MD Henry Adams makes a critical distinction between this phenomenon and Intersystems technology, saying its databases can be used relationally or as object databases, with both relational and object access to data in a single data file, "next to" each other, rather than object access through SQL.

This fact alone makes Intersystems an interesting choice of database.

Sybase SA product manager Stefan Ferreira says one size does not fit all in databases. "A large enterprise running enterprise applications in a mission-critical environment will require high-specification infrastructure, manage large volumes of data accessed by thousands of users, require maximum performance and need a full-time database administrator (DBA)."

Enterprise databases must satisfy more stringent reliability, availability and scalability (RAS) criteria, allow more customisability and will be more expensive than mobile/embedded databases. The latter may underlie applications with only one user, such as laptop-based insurance broking apps, or be found in branch offices or head office workgroups, or in a wide array of devices, from industrial controllers, set-top boxes or braking systems.

"Embedded and other smaller-than-enterprise databases are self-tuning and self-optimising to reduce maintenance and TCO," says Ferreira. "They typically have a small footprint - generally around 200Kb." The vast majority of such databases are found in financial services, retail or healthcare solutions.

Sybase has an offering called SQL Anywhere/Adaptive Server Anywhere. Other players in this $400 million market include Intersystems (Cach'e 5), Progress Software, Microsoft (SQL Server CE), IBM (DB2 Everyplace) and Oracle (Oracle 9i Lite).

Neither Intersystems nor Progress sell their databases to end-users, but market them as part of an application development toolset (Cach'e and OpenEdge, respectively). Some developers demand this double feature, as there may be technology issues with different-vendor tools and databases.

Nevertheless, proprietary application programming interfaces are a no-no in such offerings. Embeddable databases must provide standard APIs like Open Database Connectivity (ODBC), Java DBC, OLE DB and Active Database Objects.

Customers also demand data replication to let mobile databases synchronise with the enterprise database. Sybase already offers this, while Progress planned to add such features soon, according to late 2002 reports.

Finally, native XML (extensible markup language) database vendors say they see an increasing demand for their products. These include Software AG. And open source databases are making a run for this market, but commentators say the scalability and reliability do not compare with proprietary offerings.

Another issue at the heart of the choice is a database`s base user-operational purpose. Will it be accessed by many thousands of users who constantly update, insert or delete data? Such user-operation is transaction-intensive, and an OLTP engine is needed for it.

Should the database be subject only to periodic or batch updates with a mass of data, and be used to analyse trends, such as how much boerewors Pick `n Pay sold last Christmas, as opposed to turkeys, the answer is an OLAP engine.

With OLAP, you cannot predict the type of analysis the user might do.

Stefan Ferreira, product manager, Sybase

Relational databases are well suited to OLTP environments, whereas OLAP functionality is at basis data warehousing. Since relational databases change in data composition constantly, the database should not be used for analysis. The solution is to "move" the whole or a portion of it out for analysis.

The quickest way to the data in an OLTP database is through indexing. "The few interaction types possible in an OLTP database make it possible to write a few typical indices, but with OLAP, you cannot predict the type of analysis the user might do," says Sybase`s Ferreira.

In an OLAP situation, vendors have taken to storing data in a different way (bitmap indexing) and compressing data. Bitmap indexing is an accepted way of retrieving data quickly, but it`s thought to be too slow for updating data. Intersystems, however, claims its database, Cach'e 5, can store and retrieve bitmap data and perform real-time analytics in OLTP environments.

The functional evolutions of mainstream databases have prompted T-Systems` Solomon to call them stores of technology (security, user-based profiling, etc) rather than just data stores. Oracle Spatial is an example of incorporating maps to deliver databases that suit mining houses.

Starting with IBM`s DB2, the vendor`s main current preoccupations are self-management and automation. This correlates with its vision of e-business on demand, the next step in e-business, depending on standards-based Web services, and the company`s dream of computing as a self-managing and self-healing utility.

New DB2 (v8) features in this regard include multi-dimensional clustering and additional "autonomic" computing features.

Oracle`s early vision of grid/utility computing is embodied in its load-balancing and redundancy features in Oracle 9i, for instance sharing the load of much-used apps between servers. Microsoft writes cluster-aware applications with active-passive failover features built into SQL Server, says Kaylash Bhana, Microsoft SA application platform and development product manager.

SQL Server has active-passive failover features built into it.

Kaylash Bhana, application platform and development product manager, Microsoft SA

Clustering, scalability and support for 64-bit processors also cover the area of availability. This is also Oracle`s main focus, according to SA product manager Deon Roos.

"Instead of offering one production server on stand-by for failover, Oracle`s DataGuard feature makes full use of both servers, writing logs of before- and after-transaction images, saving time and money with backup and restore functions."

Roos says "the 'shared-nothing` architecture of vendors like Microsoft (each server on a different database) means data is in danger of being lost if a primary server fails". Oracle`s shared cluster may lose performance, but the distributed setting of a "shared-nothing" database means it is optimised only for benchmark test-runs, Roos says. "The real world is often a different kettle of fish."

IBM application developers, as with many using other vendors` offerings as database servers, have a variety of options. DB2 UDB packs Borland development tools and vice-versa. UDB also comes with a Development Centre for building server-side pieces of the app in Java and SQL procedural language.

DB2 has a variety of data management products and solutions, including business intelligence (something other vendors, such as Microsoft, have built in), which spans data warehousing and built-in metadata management, enterprise content and records management, as well as federation and information integration.

DB2 OLAP Server provides multidimensional and relational data storage. "The universe of data sources available to DB2 applications has expanded in recent years to include WebSphere MQ message queues and standards-based Web services," the company says.

As the number of sites in a mass-deployment architecture increases, admin costs increase, while most other costs have volume discounts or are "sunken costs".

Aberdeen research report, ,

Pricing is a major differentiator. DB2 pricing is based largely on a per-processor model for simplicity and clarity, and claims half the cost (as does Microsoft) of Oracle. TCO, however, is a multi-step equation, and scarcity of skills (there are more Microsoft skills than others), cost of skills (again Microsoft wins), need for a DBA (embedded databases need fewer skills than mainstream ones), reliability (the traditional enterprise vendors have a track record) all play a part.

It must be noted that relational (SQL) skills are often shared. Oracle has tried to address cost-criticism by offering Web-based training (course material is free), but with questionable chances of success, one integrator says.

Oracle also mentions reliability and licensing. "How much does it cost you not to do business?" Roos asks. "Are you forced to upgrade to every single new version every so often, or are you, as with us, licensed for life?"

In the area of Web services, a single IBM user SQL query can span both DB2 and Web service providers, simplifying programming. And in terms of Web accessibility, most of the mainstream databases have portal offerings.

An Aberdeen research report has found that the lowest "visible" cost of ownership (VCO) for mass-deployment databases belongs to SQL Anywhere Studio from iAnywhere Solutions when compared to both Microsoft and Oracle databases. VCO of the winner over five years is, on average, approximately three times less expensive than Microsoft`s, and eight times less expensive than Oracle`s.

As the number of sites in a mass-deployment architecture increases, administrator costs increase, while most other costs have volume discounts, such as licensing, or are "sunken costs", such as development. "Thus, iAnywhere Solutions` advantages increase as the size of the architecture increases," the report finds.

Data on related markets such as "embedded" or "low-IT" databases indicates that as people costs, such as administrator salaries, continue to rise, effective "zero-administration" solutions such as iAnywhere Solutions` SQL Anywhere Studio ... drive real-world admin costs close to zero".

Research further suggests that all major mass-deployment databases can handle typical mass-deployment demand surges and integrate well with corporate "standard" databases. Therefore, IT shops are likely to see relatively little pain from supporting a separate mass-deployment database and a major gain in cost of ownership.

Having considered specific needs in a database, including the environment in which it will be used and user operations, the choices will probably be scaled down considerably. After also considering TCO, including skills availability and cost, and the cost of losing data, one can next give time to the issues all databases claim to be good in. Finally, focus on matters such as software vendor support, product support and openness, and perhaps the choice to be made won`t seem as daunting after all.