In describing query-based FOR loop, the PostgreSQL documentation states the following:

The query used in this type of FOR statement can be any SQL command that returns rows to the caller: SELECT is the most common case, but you can also use INSERT, UPDATE, or DELETE with a RETURNING clause. Some utility commands such as EXPLAIN will work too.

Looping over a SELECT statement

The query-based FOR loop is most commonly used to process the result set produced by a SELECT statement, for example:
DECLARE

    employee  emp%ROWTYPE;
BEGIN
    FOR employee IN SELECT * FROM emp LOOP
        RAISE NOTICE 'employee - %', employee.empno;
        RAISE NOTICE 'name     - %', employee.ename; 
        RAISE NOTICE 'salary   - %', employee.sal;
    END LOOP;
END;

When you execute a query-based FOR loop over a SELECT statement, PL/pgSQL evaluates the query (SELECT * FROM emp) and then executes the loop body (the three RAISE statements) once for each row returned by the query.  Each time control passes through body of the loop, PL/pgSQL assigns a row to the iterator (the row-shaped variable named ‘employee’).  No big surprises here (although see part 2 for details about how PL/pgSQL handles an iterator whose shape does not match the shape of the result set).  If the query returns 0 rows, PL/pgSQL sets each attribute in employee to NULL (PL/pgSQL does not leave employee untouched, it sets each attribute to NULL).

Looping over an INSERT statement

You can also loop over the result set produced by an INSERT…RETURNING statement:

DECLARE
    employeeName emp.ename%TYPE;
BEGIN
    CREATE TEMP TABLE tempEmp(LIKE emp);

    FOR employeeName IN INSERT INTO tempEmp SELECT * FROM emp RETURNING ename LOOP
        RAISE NOTICE 'copied %', employeeName;
    END LOOP;
END;

In this example, we execute a single INSERT statement (this one happens to copy rows from an existing table into a temporary table) and we have included a RETURNING clause to specify that we want the ename value for each row copied. Remember that a query-based FOR loop iterates through a result set.  In the case of an INSERT statement, the shape of the result set is determined by the RETURNING clause; you can specify a one or more column names, (or ‘*’ to return all columns), or you can specify one or more expressions (such as ‘salary * 1.10′).  The result set will contain one row for each row added to the target table.

As I described in part 2 of this series, you can specify the loop iterator (employeeName in the example above) in any of the following forms:

• A comma-separated list of variables
• A row variable
• A record variable

So what happens if you write the iterator as a comma-separated list of variables but you forget to specify one or more columns in the RETURNING clause?  PL/pgSQL matches the attributes in the iterator against the attributes in the result set, from left to right; if the result set contains more attributes than you’ve specified in the iterator, the extra attributes are ignored.  If the iterator contains more attributes that the result set, the extra iterator variables are set to NULL.  If such an iterator variable is defined as NOT NULL, PL/pgSQL will throw a hissy fit.

Looping over a DELETE statement

Looping over a DELETE statement is similar to looping over an INSERT statement:

DECLARE
    employeeName emp.ename%TYPE;
BEGIN
    FOR employeeName IN DELETE FROM emp RETURNING ename LOOP
        RAISE NOTICE 'copied %', employeeName;
    END LOOP;
END;

When you loop over a DELETE statement, the result set processed by the FOR loop contains one row for each row you delete.

Looping over an UPDATE statement

Like INSERT and DELETE, you can convince UPDATE to return a result set:

DECLARE
    employeeName emp.ename%TYPE;
    newSalary    emp.sal%TYPE; 
BEGIN
    FOR employeeName IN UPDATE emp SET sal = sal * 1.10 RETURNING ename, sal LOOP
        RAISE NOTICE '% - new salary = %', employeeName, newSalary;
    END LOOP;
END;

As described in the PostgreSQL documentation, the result set produced by an UPDATE statement is based on the new values.  As far as I know, there is no way to return the original values.

Looping over other… stuff

I quoted part of the PostgreSQL documentation at the start of this article – that documentation states that you can loop over an EXPLAIN statement as well as some of the utility commands.  The following example shows how to loop over an EXPLAIN statement:

DECLARE
  planStep RECORD;
BEGIN
  FOR planStep IN EXPLAIN SELECT * FROM emp e, dept d WHERE d.deptno = e.deptno LOOP
    RAISE NOTICE '%', planStep."QUERY PLAN";
  END LOOP;
END;

When you loop over an EXECUTE statement, the result set contains one row for each step in the plan and a single TEXT column (named “QUERY PLAN”) that contains the (properly indented) step.

Now what about those other mysterious utility commands mentioned in the PostgreSQL documentation?  Which commands can you loop over?

It turns out that you can loop over three utility commands:  EXPLAIN, SHOW, and FETCH.

As you probably know, the SHOW command produces a list of configuration parameters (aka GUC variables).  The result set produced by a SHOW command contains one row for each GUC variable, each composed of three TEXT columns: name, setting, and description:

DECLARE
  parameter RECORD;
BEGIN
  FOR parameter IN SHOW ALL LOOP
    RAISE NOTICE '% - % - %', parameter.name, parameter.setting, parameter.description;
  END LOOP;
END;

Is that useful?  Probably not – you can get the same result set by selecting from the pg_settings system view; actually, you can get much more information from pg_settings.

Finally, you can loop over a FETCH statement.  You probably knew that you could loop over a cursor, but you may not have known that you can loop over a FETCH statement:

DECLARE
  employee   emp%ROWTYPE;
  executives CURSOR FOR SELECT * FROM emp WHERE mgr IS NULL;
BEGIN
  OPEN executives;

  FOR employee IN FETCH 20 FROM executives LOOP
    RAISE NOTICE '%', employee;
  END LOOP;
END;

In this  last example, we have declared a cursor (executives) and opened that cursor.  Next, we managed to find a new and inefficient way to emulate the LIMIT clause – we execute a FETCH statement to read the first 20 rows from the cursor and then loop through each of those rows.  You can loop over all variants of the FETCH statement; you cannot loop over a MOVE statement.

Is that useful?  I can’t imagine a scenario where you would want to loop over a FETCH statement when you could simply loop over the cursor instead.  But hey, I’m not here to judge.

(Reference: QueryReturnsTuples(), UtilityReturnsTuples())

I promised that part 2 (this article) would describe how to loop through the result sets produced a SELECT, INSERT, and UPDATE statements (along with a few oddities).

I lied.

Part 2 will instead provide a mini-reference (or perhaps, a supplement to the official documentation) for the query-based FOR loop, the cursor-based FOR loop, and the EXECUTE-based FOR loop.

Part 3 (which is already here) will discuss what I had originally planned to cover in part 2 – looping through SELECT, INSERT, UPDATE, DELETE, and other utility commands.

As I mentioned in part 1, the FOR loop comes in a number of different flavors.  In this article, we will examine three of those flavors:  the query-based FOR loop, the cursor-based FOR loop, and the EXECUTE-based FOR loop.  In each case, PL/pgSQL executes a statement and then iterates through the results produced by that statement, executing the body of the loop once for each row.

Query-based FOR loop

The general form of the query-based FOR loop is:

FOR iterator IN query LOOP
    loopBody
END LOOP [ label ];

The query-based FOR loop executes the loopBody once for each row returned by the given query (unless the loop is terminated by an EXIT, scoped CONTINE, RETURN, or unhandled exception).

Unlike an integer-based FOR loop, PL/pgSQL does not declare the iterator for you – you must declare iterator before executing the FOR loop.  iterator can be any of the following:

• A comma-separated list of variables
• A row variable
• A record variable

When PL/pgSQL executes a query-based FOR loop, it evaluates the given query and loops through the result set, assigning one row to the iterator for each iteration.

Now, what happens if the shape of the result set does not match the shape of the iterator? PL/pgSQL matches up the columns in the result set against the columns in the iterator proceeding from left to right (in other words, the first column in the result set gets copied into the first column in the iterator).  If the iterator contains more attributes than the query returns, the FOR loop sets the extra iterator variables to NULL.  If the query returns more attributes than you’ve defined in the iterator, the extra values (returned by the query) are simply ignored.  Of course, if the iterator is a record variable, the shape of the record variable will exactly match the shape of the row returned by the query.  The data type of each column in the iterator must be compatible with the data type of the corresponding column in the result set (“compatible” means that the types must be identical or that an implicit cast is defined that will convert the type in the result set to the type in the iterator).

PL/pgSQL will pre-fetch from the query to improve performance.  When the FOR loop begins, PL/pgSQL fetches up to 10 rows from the result set and then iterates through those values. After processing the first 10 rows, PL/pgSQL fetches the remainder of the result set 50 rows at a time.

When the loop terminates, PL/pgSQL sets the FOUND pre-defined variable to TRUE if the query returns at least one row; otherwise FOUND is set to FALSE.  After the loop terminates, the variables in the iterator contain the values in the last row of the result set.  If the query returns no data, the variables in the iterator are set to NULL. Remember that PL/pgSQL will raise an exception (22004 - null_value_not_allowed) if you try to assign NULL to a variable declared as NOT NULL.

Like any other loop construct, you can terminate the loop early by executing an EXIT statement, a RETURN statement, a scoped CONTINUE statement, or an unhandled exception.

Cursor-based FOR loop

The general form of the cursor-based FOR loop is:

FOR iterator IN cursorVar [ ( argument_values ) ] LOOP
    loopBody
END LOOP [ label ];

When PL/pgSQL executes a cursor-based FOR loop, it opens the cursor and loops through the result set, assigning one row to the iterator for each iteration.   When the loop terminates, PL/pgSQL closes the cursor.

The cursorVar must specify the name of a bound cursor variable (that is, the cursor variable must be declared for a specific query). If the query bound to the cursorVar is a parameterized query, you may specify the query arguments in the argument_values clause.  cursorVar is typically bound to a SELECT query, but it may, in fact, be bound to any command which returns a result set.  For example, you can bind cursorVar to an INSERT…RETURNING, DELETE…RETURNING, or UPDATE…RETURNING statement (see part 3 of this series).

You can refer to the cursor (within the loop) in an UPDATE…WHERE CURRENT OF cursorVar statement or a DELETE…WHERE CURRENT OF cursorVar statement.  Note that, unlike the query-based and EXECUTE-based FOR loops, PL/pgSQL will not prefetch from a cursor-based FOR loop.

The cursor must not be open at the time the FOR loop begins (the FOR loop will automatically open the cursor on your behalf).

PL/pgSQL automatically declares the iterator for you – the iterator is a record variable accessible only within the loopBody. The data type and name of each attribute in the iterator is determined by the types and names in the cursor’s result set.  By default, PostgreSQL assigns the name “?column?” to each computed column in the result set; you may want to alias computed columns. If you don’t, you may end up with more than one “?column?” in the result set and you won’t be able to access any computed column other than the left-most.

When the loop terminates, PL/pgSQL sets the FOUND pre-defined variable to TRUE if the cursor returns at least one row; otherwise FOUND is set to FALSE.

You can terminate the loop early by executing an EXIT statement, a RETURN statement, a scoped CONTINUE statement, or an unhandled exception. In each case, PL/pgSQL closes the cursor.

EXECUTE-based FOR loop

The general form of the EXECUTE-based FOR loop is:

FOR iterator IN EXECUTE expression [ USING expression [, ... ] ] LOOP
    loopBody
END LOOP [ label ];

An execute-based FOR loop is similar to a query-based FOR loop; each loop evaluates a query and executes the loopBody once for each row returned by the query.  The primary difference between an execute-based FOR loop and a query-based FOR loop is that you use an execute-based FOR loop to iterate through the result set of a dynamically generated query.  In an execute-based FOR loop, you specify the query in the form of a string (or an expression of string type).  If the expression contains parameter markers ($1, $2, …), you must specify a USING clause that includes one value for each marker.

The iterator may be any of the following:

• A comma-separated list of variables
• A row variable
• A record variable

If the iterator contains more attributes than the query returns, the FOR loop sets the extra iterator variables to NULL.  If the query returns more attributes than you’ve defined in the iterator, the extra values (returned by the query) are simply ignored.  If the iterator is a record variable, the shape of the record variable will exactly match the shape of the row returned by the query.

PL/pgSQL will pre-fetch from the query to improve performance.  When the FOR loop begins, PL/pgSQL fetches up to 10 rows from the result set and then iterates through those values. After processing the first 10 rows, PL/pgSQL fetches the remainder of the result set 50 rows at a time.

When the loop terminates, PL/pgSQL sets the FOUND pre-defined variable to TRUE if the query returns at least one row; otherwise FOUND is set to FALSE.  After the loop terminates, the variables in the iterator contain the values in the last row of the result set.  If the query returns no data, the variables in the iterator are set to NULL. Remember that PL/pgSQL will raise an exception (22004 – null_value_not_allowed) if you try to assign NULL to a variable declared as NOT NULL.

You can terminate the loop early by executing an EXIT statement, a RETURN statement, a scoped CONTINUE statement, or an unhandled exception.

Next Time

In the last part of this series, I will show you how to loop over the result sets produced by SELECT, INSERT, UPDATE, DELETE, and even FETCH statements.

FOR Loop Flavors

The FOR loop comes in a variety of different flavors.  Which flavor you get is determined by the way you specify the range.

The general form of the FOR loop is:

  FOR iterator IN range LOOP
    loopBody
  END LOOP [ label ];

In this post, I’ll explain some of the intricacies of the integer FOR loop.  In future posts, I will cover cursor-based FOR loops, dynamic FOR loops, and query-based FOR loops.

Integer-based FOR Loops

The most familiar flavor is the integer-based FOR loop.  In an integer-based for loop, the range specifies two integer values and the loop executes once for each integer between those two values (inclusive):

  FOR iterator IN [REVERSE] start-value .. end-value [BY increment] LOOP
    loopBody
  END LOOP [label];

Where iterator is the name of an integer variable and start-value, end-value, and increment are expressions which return an integer value (or a value which can be coerced to an integer).

You should understand that the integer-based FOR loop creates a spanking new iterator variable for you.  If you have variable of the same name in an outer scope, that variable will not be modified by the FOR loop – the loop iterator is a new variable automatically declared in an inner scope.  Within the body of the loop, you can refer to the current value of iterator as iterator or as label.iterator (if you have assigned a label to the FOR loop).  To refer to a variable with the same name declared in an outer scope, you must write outerLabel.iterator, where outerLabel is the label assigned to the outer scope.  Here’s an example that demonstrates how to refer to variables in different scopes:

DO $$
<<outer_scope>> DECLARE
  month  INTEGER := 12;
BEGIN
  RAISE NOTICE 'month = %', month;

  FOR month IN 1..3 LOOP
    RAISE NOTICE 'inner month = %, outer month = %', month, outer_scope.month;
  END LOOP;

  RAISE NOTICE 'month = %', month;
END;
$$;

This anonymous block would give the following result:

NOTICE:  month = 12
NOTICE:  inner month = 1, outer month = 12
NOTICE:  inner month = 2, outer month = 12
NOTICE:  inner month = 3, outer month = 12
NOTICE:  month = 12

Notice that month variable in the outer scope is unchanged by the loop (of course, you could assign a new value to outer_scope.month yourself, but the FOR loop never touches it).

When executing an integer FOR loop, PL/pgSQL begins by evaluating the start-value, end-value, and increment expressions (which must be of type INTEGER or of a type which can be implicitly coerced to INTEGER). If start-value, end-value, or increment evaluates to NULL, PL/pgSQL raises an exception (22004 – null_value_not_allowed).  The increment expression must evaluate to an integer value greater than zero. otherwise, PL/pgSQL raises an exception (22023 – invalid_parameter_value).

It is important to note that PL/pgSQL keeps an internal copy of the end-value, increment, and iterator.  You can change any of those values in the body of the loop, but your change will not affect how many times the loopBody executes.

Once PL/pgSQL has made a copy of the end-value and increment values, it assigns start-value to the loop iterator. If iterator is less than or equal to end-value, PL/pgSQL executes the statements within the loopBody.  That’s an important point – the loopBody only executes if start-value <= end-value.

If the flow of control reaches the END LOOP (remember, an EXIT, scoped CONTINUE, RETURN, or unhandled exception may cause an early termination), PL/pgSQL adds the increment value to iterator and again compares the iterator to the stored copy of end-Value.  If the new value of iterator is less than or equal  to end-Value, PL/pgSQL repeats the loop.

Reverse Integer-based FOR Loops

A plain-vanilla integer-based FOR loop counts up (by increment) from start-value to end-value.  If you want to count down instead (that is, you want want the iterator to decrease in value with each trip through the loop), you must specify the REVERSE keyword.  In a REVERSE for loop, PL/pgSQL subtracts the increment from the iterator before repeating the loop.  The loopBody executes only if iterator is greater than or equal to end-value.  Like a forward integer-based FOR loop, increment must evaluate to a value greater than zero.

Terminating a FOR Loop

PL/pgSQL executes the loop body once for each value (or set of values) found in the given range.  For each trip through the loop body, PL/pgSQL assigns a new value (or set of values) to the iterator.  The loop terminates when any of the following occurs:

• The range is exhausted (in other words, the loop has processed every value in the given range)
• An exception is raised within the loop body and is not handled
• The loop body executes an EXIT statement
• The loop body executes a scoped CONTINUE statement (scoped meaning that the CONTINUE statement refers to a label outside of the loop body)
• The loop body executes a RETURN statement

When the loop terminates, PL/pgSQL assigns TRUE or FALSE to a pre-defined variable named FOUND to indicate whether any values were found within the given range (FOUND = TRUE means that PL/pgSQL executed the loop body at least once; FOUND = FALSE means that no values were found in the given range).

Summary

To summarize the rules:

• The start-value, end-value, and increment expressions must not evaluate to NULL
• The increment expression (if present) must evaluate to an integer value greater than zero (even for a REVERSE loop).
• In  a forward loop, PL/pgSQL will execute the loop body as long as the iterator is less than or equal to the end-value
• In a forward loop,  start-value must be less than or equal to end-value (assuming that you actually want to execute the loopBody)
• In a reverse loop, PL/pgSQL will execute the loop body as long as the iterator is greater than or equal to the end-value
• In a reverse loop,  start-value must be greater than or equal to end-value (assuming that you actually want to execute the loopBody)
• PL/pgSQL automatically creates a new variable (distinct from other variables with the same name declared in an outer scope) to hold the iterator
• You can change the end-value and increment within the loop, but your changes will not affect the number of trips through the loop
• There are five different ways to terminate a loop

Next Time

You probably know that you can write a FOR loop that iterates through the result set of a SELECT statement, but do you know what other kinds of statements you can loop through?  In the next installment of this series, I’ll show you how to walk through all of the rows modified by an UPDATE statement, all rows removed by a DELETE statement, plus a few oddities of questionable use.

Parts 2 and 3 are here and here.

Nyahhh