@henryccook made an interesting point regarding Netezza workload management this morning… He suggested that once a SPU is engaged by a snippet the work must be completed before another snippet can start. To say this another way… a SPU has no OS and cannot save context for a snippet and start another… then return.
If this is true it means that if a long-running snippet starts… a full file scan of a fact table with no use of the zone map… then that snippet will lock out others queries until it completes.
This is not a very fine-grained approach to workload management and we would expect it to cause difficulties.
Can anyone confirm that this is true? It feels right from an architectural perspective…
No it is not true. Neteeza does have a workload manager and it does context switching allowing short running queries to get in during the execution of a long running querry
I understand that Netezza has a workload manager, Al… and that once the SPU has completed a snippet that another snippet can start… but how do you switch context on an FPGA? The posts I’ve read suggest that once a snippet starts it has to go to completion? Some details, please?
No, Rob – it’s not true (or rather, I think you may have misunderstood Henry’s point). Each snippet (i.e. step of an application plan that is responsible for producing part of an answer set) is run in parallel across all S Blades (although in Netezza v7 a snippet can be directed to a single S Blade if the optimizer knows that’s the only place where it’ll find qualifying rows), & each S Blade is capable of running multiple snippets from different application plans (a configurable number, via Workload Management settings in fact) at the same time. One of the reasons for creation of the S Blade – coupling intel quad core CPUs to FPGAs, with a lot if RAM – was in order to do this well.
Hi Huw… OK… this is cool(er).
Let me restate to see if I have it right? Lets say that we configure the S Blade to run 2 snippets… just to keep it simple. Both snippets are doing full scans without any help from the zone maps… and each scan takes 10 minutes. Then a high priority query hits the workload manager. Are you saying that one running snippet is interrupted and its combined state is saved on both the Intel and the FPGA processors… and it is then resumed once the new query completes?
This also raises the question: If the two snippets are dispatched on a single FPGA how is state saved when one snippet gives up the FPGA processor to wait for an I/O to the disk?
Thanks…
Your question seems to imply that the FPGAs are single threaded, have no cache memory, that the Intel CPUs they are coupled with are not involved in the query execution process, and that IBM haven’t implemented a pre-emptive multi-tasking snippet scheduling subsystem. The reality is that in fact none of those things are true. In terms of state preservation, you’d have to ask Netezza engineering about the precise mechanics, but certainly from personal experience it seems that multi-threaded multi-tasking workload management works just fine at the snippet level and there’s certainly more than enough Intel/FPGA cores, cache, and RAM available in an S-Blade to preserve state. Instead of imagining one query ‘interrupting’ the execution of another, I personally find it more helpful to think of a scheduling subsystem that’s constantly sharing available resources between multiple different queues of scheduled (and executing) snippets to ensure guaranteed minimum/maximum service levels are met all of the time.
You are right Huw… it was not my impression that an FPGA had an operating system that would manage interrupts and context switching. You say that “in fact none of those things are true” but then provide only an anecdote that in your personal experience workload management works fine and point me to Netezza engineering for the facts.
I did some reading on FPGA programming and it supported the idea that an FPGA did not run a pre-emptive OS… but it was a general discussion… not about Netezza… so I’ll wait for new information.
I’m not trying to be difficult… and I sort of expected that you might come back with some facts that would clear this up… in which case this blog would be very supportive of Netezza and help to get the good word out. But I don’t see it as yet?
I believe that Netezza has a capable-enough workload management system… but the point of this blog is that I wonder about how fine grained the management is at the FPGA-level? I still wonder…
Despite Ringerrr’s explanation, it appears you still have some misconceptions of about the role of the FPGA in Netezza query execution.
The FPGA is involved in very little of the computation on a query. Think of the FPGA as a glorified disk controller, while the snippet itself is a process running on a Linux OS on the S-Blade. Yes, the FPGA does some trivial projection and selection, but the main advantage is “free” compression / decompression without a CPU performance penalty. And since the snippets are just processes on a Linux OS (not on the FPGA) the benefit from the full workload management of a normal Linux operating system (plus whatever else NZ has baked in).
This is probably what happens:
The snippet requests data from disk, and the Linux OS of the S-Blade time-slices the read requests among the running processes. The OS doesn’t tell a snippet “read all data off disk from this table until you are done”. Rather, it probably tells the FPGA to read a certain number of blocks off disk at a time. The FPGA will decompress/project/restrict and put the results in RAM for the snippet process on the CPU to do the real work of joins, functions and expressions. Once the FPGA is done with the blocks it was assigned to read, it asks for more work, and the OS is free to fulfill a higher priority process at that time. You don’t need to have a pre-emptive anything on the FPGA with this explanation.
If you want technical explanations, try posting enzeecommunity.com, Netezza engineers monitor the forums there and respond relatively quickly.
Also, to clarify the original questions, snippets within a query run one after another. The next snippet for a query cannot start until all the nodes are done. This is a logical dependency that has nothing to do with multi-tasking. And as mentioned above, by myself and others, this has no impact on having snippets from other queries running at the same time (snippets are just processes on a Linux OS).
I wonder though if some of your confusion is caused by the older Mustang architecture, where each SPU was fused with the disk in a single piece of hardware. I don’t know if that would have the effect you originally mentioned, but for sure it is not the case with the TwinFin architecture. The TwinFin S-Blades work much more like a normal Linux server, and with the FPGAs a resource to be used by the host.
1) A snippet is one step in Netezza SQL processing, which on Netezza is happen to be done with generated and compiled C programs. With other words a snippet is a C program that implements one step of the SQL access plan. You cannot configure an S-Blade to have two snippets. This would mean that you configure a DBMS to create a two step access plan for every SQL thrown at it. Simple SQL statements are done with 1-2 snippets complex ones can be 100+ or even more easily. Snippets must be run sequentaly on every S-Blade (there is some minor change to that on NPS 7.0) and each snippet must complete before the next snippet can start. This has nothing to do with workload management. This is just the way SQL statements are processed on Netezza.
2) I think the best to think of the FPGA as an intelligent disk controller. Its primary role is to filter out rows that are not relevant for the query.
3) S-Blades do have an O/S: Linux. It is possible to connect to them but it is limited to Netezza Engineers.
4) Going back to your original question: I do not know the details of Netezza workload management well enough to answer your question but I am pretty sure that between snippets of an SQL a higher priority SQL’s snippet can get scheduled before a lower priority SQL’s snippet. One snippet might be and very often it is an FTS on any sized table. Whether a running snippet can be stopped and restarted later is possible I do not know. My guess is that the answer is no. Different SQL statements (snippets) can read the same table at the same time so there is no lock out unless it is explicitly requested.
Hi Andras,
There is no question that an S-Blade has a pre-emptible LINUX OS… the question is whether the FPGA can be pre-empted?
I’ll leave my question standing, then. It seems that there are three possible answers:
-Rob