Skip to content

Commit

Permalink
Merge with PostgreSQL 8.3.23.
Browse files Browse the repository at this point in the history
Everything in PostgreSQL 8.3 is now in Greenplum. PostgreSQL 8.3 has been
end-of-lifed in upstream, so there will be no new upstream minor releases
of this series anymore either.

This includes a lot of new features from PostgreSQL 8.3, and a ton of bug
fixes. See upstream release notes for details. One big user-visible change
included is the removal of implicit cast from other datatypes to text. When
this was done in PostgreSQL, it caused a lot of sloppily written
application queries to break. Which is a good thing in the long run, but it
caused a lot of pain on upgrade.

A few features work slightly differently in GPDB:

* Lazy XIDs feature in upstream reduces XID consumption, by not assigning
  XIDs to read-only transactions. That optimization has not been
  implemented for GPDB distributed transactions, however, so practically
  all queries in GPDB still consume XIDs, whether they're read-only or not.

* temp_tablespaces GUC was added in upstream, but it has been disabled in
  GPDB, in favor of GPDB-specific system to decide which filespace to use
  for temporary files.

* B-tree indexes can now be used to answer "col IS NULL" queries. That
  support has not been implemented for bitmap indexes.

* Support was added for "top N" sorting, speeding up queries with ORDER BY
  and LIMIT. That support was not implemented for tuplesort_mk.c, however,
  so you only get that benefit with enable_mk_sort=off.

* Multi-worker autovacuum support was merged, but autovacuum as whole is
  still disabled in GPDB.

* Planner hook was added. Nothing special there, but we should consider
  refactoring the ORCA glue code to use it.

* Plan cache invalidation was added upstream, which made the
  gp_plpgsql_clear_cache_always option obsolete. We also backported the
  further invalidation improvements from 8.4. See below for more
  information.

In addition to those, there were some big refactorings that were not that
interesting from user point of view, but caused a lot of code churn:
InitPlans are now initialized separately at executor startup, the executor
range table was changed to be "flat", and code related to parse analysis of
DDL commands was moved around. The way UPDATE WHERE CURRENT OF was
implemented in GPDB was refactored to match the new upstream support for
the same (see below for more information).

Plan cache invalidation
-----------------------

One notable feature included in this merge is the support for plan cache
invalidation. GPDB contained a hack to invalidate all plans in master
between transactions, for the same purpose, but the proper plan cache
invalidation is a much better solution. However, because the GPDB hack also
handled dropped/recreated functions in some cases, if we just leave it out
and replace with the 8.3 plan cache invalidation, there will be a
 regression in those cases. To fix, this merge commit includes a backport
the rest of the plan cache invalidation support from PostgreSQL 8.4, which
 handles functions and operators too. The upstream commit for that was:

commit ee33b95
Author: Tom Lane <[email protected]>
Date:   Tue Sep 9 18:58:09 2008 +0000

    Improve the plan cache invalidation mechanism to make it invalidate plans
    when user-defined functions used in a plan are modified.  Also invalidate
    plans when schemas, operators, or operator classes are modified; but for these
    cases we just invalidate everything rather than tracking exact dependencies,
    since these types of objects seldom change in a production database.

    Tom Lane; loosely based on a patch by Martin Pihlak.

As a result of that, the gp_plpgsql_clear_cache_always GUC is removed, as
it's no longer needed. This also includes new test cases in the plpgsql
cache regression test, to demonstrate the improvements.

CURRENT OF code changes
-----------------------

This merge includes the upstream support for UPDATE WHERE CURRENT OF. GPDB
had support for that already, it's been refactored to use the upstream code
as much as possible (execCurrentOf()). Now that we have the upstream code
available, this also refactors the way current position of a cursor is
dispatched. Instead of modifying every CurrentOfExpr node in the plan tree
before dispatching it, store the current positions of each cursor mentioned
in a CurrentOfExpr in a separate CursorPosInfo node, and dispatch those
along with the query plan, in QueryDispatchDesc.

This was a joint effort between me (Heikki) and Daniel Gustafsson.
  • Loading branch information
hlinnaka committed Jun 15, 2016
2 parents 5dc7dda + d13f41d commit a453004
Show file tree
Hide file tree
Showing 1,480 changed files with 129,472 additions and 66,369 deletions.
12 changes: 7 additions & 5 deletions GNUmakefile.in
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ distdir = postgresql-$(VERSION)
dummy = =install=
garbage = =* "#"* ."#"* *~* *.orig *.rej core postgresql-*

dist: $(distdir).tar.gz
dist: $(distdir).tar.gz $(distdir).tar.bz2
ifeq ($(split-dist), yes)
dist: postgresql-base-$(VERSION).tar.gz postgresql-docs-$(VERSION).tar.gz postgresql-opt-$(VERSION).tar.gz postgresql-test-$(VERSION).tar.gz
endif
Expand All @@ -129,6 +129,8 @@ dist:
$(distdir).tar: distdir
$(TAR) chf $@ $(distdir)

.INTERMEDIATE: $(distdir).tar

opt_files = \
src/tools src/tutorial \
$(addprefix src/pl/, plperl plpython tcl)
Expand All @@ -150,7 +152,7 @@ postgresql-test-$(VERSION).tar: distdir

distdir:
-rm -rf $(distdir)* $(dummy)
for x in `cd $(top_srcdir) && find . -name CVS -prune -o -print`; do \
for x in `cd $(top_srcdir) && find . \( -name CVS -prune \) -o \( -name .git -prune \) -o -print`; do \
file=`expr X$$x : 'X\./\(.*\)'`; \
if test -d "$(top_srcdir)/$$file" ; then \
mkdir "$(distdir)/$$file" && chmod 777 "$(distdir)/$$file"; \
Expand All @@ -165,12 +167,12 @@ distdir:
#cp $(distdir)/doc/src/sgml/INSTALL $(distdir)/
#cp $(distdir)/doc/src/sgml/regress_README $(distdir)/src/test/regress/README
$(MAKE) -C $(distdir) distclean
-rm -f $(distdir)/README.CVS
#rm -f $(distdir)/README.git

distcheck: $(distdir).tar.gz
distcheck: dist
-rm -rf $(dummy)
mkdir $(dummy)
$(GZIP) -d -c $< | $(TAR) xf -
$(GZIP) -d -c $(distdir).tar.gz | $(TAR) xf -
install_prefix=`cd $(dummy) && pwd`; \
cd $(distdir) \
&& ./configure --prefix="$$install_prefix"
Expand Down
3 changes: 2 additions & 1 deletion config/c-library.m4
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Macros that test various C library quirks
# $PostgreSQL: pgsql/config/c-library.m4,v 1.33 2008/08/21 13:53:28 petere Exp $
# config/c-library.m4


# PGAC_VAR_INT_TIMEZONE
Expand Down Expand Up @@ -298,6 +298,7 @@ int main()
AC_MSG_RESULT([$pgac_cv_printf_arg_control])
])# PGAC_FUNC_PRINTF_ARG_CONTROL


# PGAC_TYPE_LOCALE_T
# ------------------
# Check for the locale_t type and find the right header file. Mac OS
Expand Down
36 changes: 3 additions & 33 deletions config/docbook.m4
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# config/docbook.m4
# $PostgreSQL: pgsql/config/docbook.m4,v 1.8 2007/08/09 02:33:58 tgl Exp $

# PGAC_PROG_JADE
# --------------
Expand Down Expand Up @@ -55,7 +55,7 @@ AC_CACHE_VAL([pgac_cv_path_stylesheets],
[if test -n "$DOCBOOKSTYLE"; then
pgac_cv_path_stylesheets=$DOCBOOKSTYLE
else
for pgac_prefix in /usr /usr/local /opt /sw; do
for pgac_prefix in /usr /usr/local /opt; do
for pgac_infix in share lib; do
for pgac_postfix in \
sgml/stylesheets/nwalsh-modular \
Expand All @@ -64,8 +64,7 @@ else
sgml/docbook-dsssl \
sgml/docbook/dsssl/modular \
sgml/docbook/stylesheet/dsssl/modular \
sgml/docbook/dsssl-stylesheets \
sgml/dsssl/docbook-dsssl-nwalsh
sgml/docbook/dsssl-stylesheets
do
pgac_candidate=$pgac_prefix/$pgac_infix/$pgac_postfix
if test -r "$pgac_candidate/html/docbook.dsl" \
Expand Down Expand Up @@ -97,32 +96,3 @@ if test -n "$DOCBOOKSTYLE"; then
else
AC_PATH_PROGS(COLLATEINDEX, collateindex.pl)
fi])# PGAC_PATH_COLLATEINDEX


# PGAC_PATH_DOCBOOK2MAN
# ---------------------
# Find docbook2man program from the docbook2X package. Upstream calls
# this program docbook2man, but there is also a different docbook2man
# out there from the docbook-utils package. Thus, the program we want
# is called docbook2x-man on Debian and db2x_docbook2man on Fedora.
#
# (Consider rewriting this macro using AC_PATH_PROGS_FEATURE_CHECK
# when switching to Autoconf 2.62+.)
AC_DEFUN([PGAC_PATH_DOCBOOK2MAN],
[AC_CACHE_CHECK([for docbook2man], [ac_cv_path_DOCBOOK2MAN],
[if test -z "$DOCBOOK2MAN"; then
_AS_PATH_WALK([],
[for ac_prog in docbook2x-man db2x_docbook2man docbook2man; do
ac_path="$as_dir/$ac_prog"
AS_EXECUTABLE_P(["$ac_path"]) || continue
if "$ac_path" --version 2>/dev/null | $GREP docbook2x >/dev/null 2>&1; then
ac_cv_path_DOCBOOK2MAN=$ac_path
break
fi
done])
else
ac_cv_path_DOCBOOK2MAN=$DOCBOOK2MAN
fi])
DOCBOOK2MAN=$ac_cv_path_DOCBOOK2MAN
AC_SUBST(DOCBOOK2MAN)
])# PGAC_PATH_DOCBOOK2MAN
8 changes: 5 additions & 3 deletions config/mkinstalldirs
Original file line number Diff line number Diff line change
Expand Up @@ -105,13 +105,15 @@ esac

for file
do
set fnord `echo ":$file" | sed -ne 's/^:\//#/;s/^://;s/\// /g;s/^#/\//;p'`
# pathcomp starts with leading slash if present, else empty
pathcomp=`echo "$file" | sed -ne 's|^\(.\).*$|\1|;s|[^/]||;p'`
# swap spaces and slashes, then split on spaces
set fnord `echo "$file" | tr ' /' '/ '`
shift

pathcomp=
for d
do
pathcomp="$pathcomp$d"
pathcomp="$pathcomp"`echo "$d" | tr '/' ' '`
case $pathcomp in
-*) pathcomp=./$pathcomp ;;
esac
Expand Down
50 changes: 29 additions & 21 deletions config/programs.m4
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# config/programs.m4
# $PostgreSQL: pgsql/config/programs.m4,v 1.24 2008/08/29 13:02:32 petere Exp $


# PGAC_PATH_BISON
Expand All @@ -10,7 +10,7 @@
AC_DEFUN([PGAC_PATH_BISON],
[# Let the user override the search
if test -z "$BISON"; then
AC_PATH_PROGS(BISON, bison)
AC_CHECK_PROGS(BISON, bison)
fi
if test "$BISON"; then
Expand All @@ -19,15 +19,15 @@ if test "$BISON"; then
if echo "$pgac_bison_version" | $AWK '{ if ([$]4 < 1.875) exit 0; else exit 1;}'
then
AC_MSG_WARN([
*** The installed version of Bison, $BISON, is too old to use with PostgreSQL.
*** Bison version 1.875 or later is required, but this is $pgac_bison_version.])
*** The installed version of Bison is too old to use with PostgreSQL.
*** Bison version 1.875 or later is required.])
BISON=""
fi
fi
if test -z "$BISON"; then
AC_MSG_WARN([
*** Without Bison you will not be able to build PostgreSQL from Git nor
*** Without Bison you will not be able to build PostgreSQL from CVS nor
*** change any of the parser definition files. You can obtain Bison from
*** a GNU mirror site. (If you are using the official distribution of
*** PostgreSQL then you do not need to worry about this, because the Bison
Expand All @@ -42,11 +42,8 @@ AC_SUBST(BISONFLAGS)
# PGAC_PATH_FLEX
# --------------
# Look for Flex, set the output variable FLEX to its path if found.
# Reject versions before 2.5.31, as we need a reasonably non-buggy reentrant
# scanner. (Note: the well-publicized security problem in 2.5.31 does not
# affect Postgres, and there are still distros shipping patched 2.5.31,
# so allow it.) Also find Flex if its installed under `lex', but do not
# accept other Lex programs.
# Avoid the buggy version 2.5.3. Also find Flex if its installed
# under `lex', but do not accept other Lex programs.

AC_DEFUN([PGAC_PATH_FLEX],
[AC_CACHE_CHECK([for flex], pgac_cv_path_flex,
Expand Down Expand Up @@ -78,6 +75,9 @@ else
*** The installed version of Flex, $pgac_candidate, is too old to use with Greenplum DB.
*** Flex version 2.5.4 or later is required, but this is $pgac_flex_version.])
fi
pgac_cv_path_flex=$pgac_candidate
break 2
fi
fi
done
Expand All @@ -88,8 +88,14 @@ fi
])[]dnl AC_CACHE_CHECK
if test x"$pgac_cv_path_flex" = x"no"; then
if test -n "$pgac_broken_flex"; then
AC_MSG_WARN([
*** The Flex version 2.5.3 you have at $pgac_broken_flex contains a bug. You
*** should get version 2.5.4 or later.])
fi
AC_MSG_WARN([
*** Without Flex you will not be able to build PostgreSQL from Git nor
*** Without Flex you will not be able to build PostgreSQL from CVS or
*** change any of the scanner definition files. You can obtain Flex from
*** a GNU mirror site. (If you are using the official distribution of
*** PostgreSQL then you do not need to worry about this because the Flex
Expand All @@ -98,7 +104,7 @@ if test x"$pgac_cv_path_flex" = x"no"; then
FLEX=
else
FLEX=$pgac_cv_path_flex
pgac_flex_version=`$FLEX --version 2>/dev/null`
pgac_flex_version=`$FLEX -V 2>/dev/null`
AC_MSG_NOTICE([using $pgac_flex_version])
fi
Expand All @@ -117,14 +123,15 @@ AC_SUBST(FLEXFLAGS)
AC_DEFUN([PGAC_CHECK_READLINE],
[AC_REQUIRE([AC_CANONICAL_HOST])
AC_CACHE_CHECK([for library containing readline], [pgac_cv_check_readline],
AC_CACHE_VAL([pgac_cv_check_readline],
[pgac_cv_check_readline=no
pgac_save_LIBS=$LIBS
if test x"$with_libedit_preferred" != x"yes"
then READLINE_ORDER="-lreadline -ledit"
else READLINE_ORDER="-ledit -lreadline"
fi
for pgac_rllib in $READLINE_ORDER ; do
AC_MSG_CHECKING([for ${pgac_rllib}])
for pgac_lib in "" " -ltermcap" " -lncurses" " -lcurses" ; do
LIBS="${pgac_rllib}${pgac_lib} $pgac_save_LIBS"
AC_TRY_LINK_FUNC([readline], [[
Expand All @@ -143,11 +150,14 @@ for pgac_rllib in $READLINE_ORDER ; do
]])
done
if test "$pgac_cv_check_readline" != no ; then
AC_MSG_RESULT([yes ($pgac_cv_check_readline)])
break
else
AC_MSG_RESULT(no)
fi
done
LIBS=$pgac_save_LIBS
])[]dnl AC_CACHE_CHECK
])[]dnl AC_CACHE_VAL
if test "$pgac_cv_check_readline" != no ; then
LIBS="$pgac_cv_check_readline $LIBS"
Expand All @@ -163,21 +173,19 @@ fi
# Readline versions < 2.1 don't have rl_completion_append_character

AC_DEFUN([PGAC_VAR_RL_COMPLETION_APPEND_CHARACTER],
[AC_CACHE_CHECK([for rl_completion_append_character], pgac_cv_var_rl_completion_append_character,
[AC_TRY_LINK([#include <stdio.h>
[AC_MSG_CHECKING([for rl_completion_append_character])
AC_TRY_LINK([#include <stdio.h>
#ifdef HAVE_READLINE_READLINE_H
# include <readline/readline.h>
#elif defined(HAVE_READLINE_H)
# include <readline.h>
#endif
],
[rl_completion_append_character = 'x';],
[pgac_cv_var_rl_completion_append_character=yes],
[pgac_cv_var_rl_completion_append_character=no])])
if test x"$pgac_cv_var_rl_completion_append_character" = x"yes"; then
[AC_MSG_RESULT(yes)
AC_DEFINE(HAVE_RL_COMPLETION_APPEND_CHARACTER, 1,
[Define to 1 if you have the global variable 'rl_completion_append_character'.])
fi])# PGAC_VAR_RL_COMPLETION_APPEND_CHARACTER
[Define to 1 if you have the global variable 'rl_completion_append_character'.])],
[AC_MSG_RESULT(no)])])# PGAC_VAR_RL_COMPLETION_APPEND_CHARACTER



Expand Down
4 changes: 2 additions & 2 deletions config/python.m4
Original file line number Diff line number Diff line change
Expand Up @@ -85,13 +85,13 @@ AC_SUBST(python_libdir)[]dnl
AC_SUBST(python_libspec)[]dnl
AC_SUBST(python_additional_libs)[]dnl
# threaded python is not supported on bsd's
# threaded python is not supported on OpenBSD
AC_MSG_CHECKING(whether Python is compiled with thread support)
pythreads=`${PYTHON} -c "import sys; print(int('thread' in sys.builtin_module_names))"`
if test "$pythreads" = "1"; then
AC_MSG_RESULT(yes)
case $host_os in
openbsd*|freebsd*)
openbsd*)
AC_MSG_ERROR([threaded Python not supported on this platform])
;;
esac
Expand Down
Loading

0 comments on commit a453004

Please sign in to comment.