If you missed the tweet… 2+ years ago I predicted here that Teradata would go away from ByNet… and lo and behold they did (see here).
In the same post I predicted that Netezza would go away from FPGAs. This has not come to pass. But I wonder if it might… or if there is a bigger change possible?
With the recent announcements of DB2 BLU and column store I suspect that DB2 will outperform Netezza when the query mix does not fall directly in Netezza’s sweet spot.
I also have a suspicion that the Netezza architecture, with its execution engine split across two different processors, is just hard to engineer. I cannot think of another reason features come so slowly there. Why, for example, is there no columnar support? Greenplum built it on the same Postgres base with less than a handful of engineers in a year. Teradata now offers columnar tables as well.
These concerns… combined with some previous notes on Netezza add up as follows:
- FPGAs no longer provide a performance advantage (per my link above)
- FPGAs limit the ability of the DBMS to use more cores (see here)
- FPGAs limit the ability of the DBMS to manage workload (see here… and especially the comments)
- FPGAs and having a 2-phase split execution environment limits the ability to extend and enhance the code base (a new conjecture)
- Zone Maps and CBTs provide a limited ability to solve for a wide range of queries… they are just an index (see here)
- DB2 Column Store provides a performance boost equal to or greater than zone maps and CBTs (a new conjecture)
- DB2 BLU provides a performance boost well in excess of what Netezza can provide (see here)
The Netezza architecture with FPGAs provided a distinct advantage in 2000 when CPU was the scarce commodity. But multi-core systems and the advance of Moore’s Law soon made processing abundant… and the advantage of FPGA co-processing diminished. Without a distinct advantage the split execution architecture became a disadvantage… and the complexity of that design kept Netezza from developing the advances on top of the Postgres base that were very easy to develop by others.
Architecture counts… and DB2 is a strong product. If, as I suspect, DB2 is now a more capable product than Netezza… I wonder what path IBM may take?
You miss the rumor that a great number of employees left Netezza in the past months. You can’t speedup development without talented people.
You don’t get the point. Netezza is an appliance i.e. nobody should care on how the engine works. The system simply should do where it is designed for. Netezza wants to deliver performance without tuning. It does not aim to be the fastest engine using a myriad of tuning features.
The FPGA is completely invisible for the appliance user (even for the DBA) so if IBM ever decides to remove it they can do it transparently (as expected from an appliance). IBM engineers are capable to manipulate individual atoms and beat Jeopardy champions, so I’m confident they are able to remove it whenever they see fit.
About columnar storage: Netezza does store row based, however compresses column based (btw: you do not need to set this up, the machine just does it) i.e. it works in kind of a mixed mode. This is a very nice approach as even very hot database systems like HBase, the Hadoop columnar database, uses a mixed mode storage paradigm as it stores related columns together in ‘column families’ to mitigate the backdrafts of the pure columnar storage approach.
About the path IBM is taking: the strategy is called ‘Workload Optimized Systems’
Teradata moving away from BYNET? Yes and no. We are moving away from the BYNET fabric, but keeping the BYNET messaging passing interface in S/W, with all of the attendant advantages that implies (point-to-point as well as broadcast messaging, on-the-fly sort / merge processing, etc., etc.). And this has been the direction of travel for a while now, with many members of the Teradata Platform Family running GbE rather than BYNET for several years. But if your argument is that scale-out architectures running on Standard High Volume Server technology are winning the database wars, then no argument from me… Whether the “memory wall” phenomenon will cause us all to re-assess this direction of travel is a subject for another discussion, another time…