Greenplum 7 first Beta Released!!!
111 views
Skip to first unread message
Ashwin Agrawal
Dec 16, 2022, 12:09:15 PM12/16/22
to Greenplum Developers, Greenplum Users
The Greenplum Team enthusiastically announces that the first Beta
release of Greenplum Database 7 is now available for
[download](https://github.com/greenplum-db/gpdb/releases/tag/7.0.0-beta.0).
This release contains a preview of the features that will be available
when Greenplum Database 7 is made generally available, though details
of the release can change during the Beta period. Greenplum 7 is based
on PostgreSQL 12 and includes a wide array of new functionality
representing five major versions of PostgreSQL development, plus
Greenplum features along and on top.
In the spirit of the open-source Greenplum community, we strongly
encourage you to test the new features of Greenplum Database 7 on your
systems to help us eliminate bugs or other issues that may
exist. While we do not advise you to run Greenplum 7 Beta in
production environments, we encourage you to find ways to run your
typical application workloads against this Beta release.
Your testing and feedback will help the community ensure that the
Greenplum 7 release upholds our standards of delivering a stable,
reliable release of the next major version of Greenplum.
Feature Highlights
===============
Greenplum 7 incorporates several new features from PostgreSQL versions
9.5 through version 12:
- Improve the sorting speed of varchar, text, and numeric fields via
“abbreviated” keys (Postgres 9.5).
- Support for array_position() and array_positions() functions
(Postgres 9.5).
- Support for INSERT ... ON CONFLICT UPDATE, an UPSERT-like operation
that handles conflicts between concurrent data changes on a
table. (Not yet supported by GPORCA, or on append-only tables.)
(PostgreSQL 9.5).
- Support for row-level security that implements true per-row and
per-column data access control (PostgreSQL 9.5). Database
administrators can now set security policies that filter which rows
particular users are permitted to update or view. Refer to About
Configuring Row -Level Security Policies for more information.
- The pg_resetxlog utility was renamed to pg_resetwal.
- Partition the shared hash table freelist to reduce contention on
multi-CPU-socket servers (PostgreSQL 9.6).
- Performance improvement when using atomic operations, rather than a
spinlock, to protect an LWLock’s wait queue (PostgreSQL 9.6).
- Allow setting and revoking default permissions on schemas using the
ALTER DEFAULT PRIVILEGES command (PostgreSQL 10).
- Support for PostgreSQL declarative table partitioning syntax
(PostgreSQL 10).
- Allow multiple functions, operators, and aggregates to be dropped
with a single DROP command (PostgreSQL 10).
- Support for SQL stored procedures that support embedded transactions
(PostgreSQL 11).
- Reduces the WAL overhead when building a GiST, GIN, or SP-GiST
index; less space on disk is now required for these WAL records and
the data replays faster during crash recovery or point-in-time
recovery (PostgreSQL 12).
- Support for the SQL/JSON path language (PostgreSQL 12).
- Support for STORED generated columns (columns whose content is
computed from expressions, including references to other columns in
the same table, rather than being specified by INSERT or UPDATE
commands). Generated columns cannot be used as a distribution key in
Greenplum Database. (PostgreSQL 12)
- Support for allowing some DDL commands to accept the current user
(CURRENT_USER) or the session user (SESSION_USER) in place of a
specific user name. (PostgreSQL 9.5).
- The CREATE/ALTER USER ... PASSWORD commands no longer support the
UNENCRYPTED option (PostgreSQL 10).
- Support for defining multi-column most-common-value (MCV) extended
statistics (via CREATE STATISTICS) to generate better plans for
queries that test several non-uniformly-distributed
columns. Currently supported only by the Postgres Planner, not
GPORCA (PostgreSQL 12).
- Support for automatic (but overridable) inlining of common table
expressions (CTEs). In Greenplum 7, CTEs are automatically inlined
if they have no side effects, are not recursive, and are referenced
only once in the query. You can prevent inlining by specifying
MATERIALIZED, or force inlining for multiple-referenced CTEs by
specifying NOT MATERIALIZED. In previous Greenplum releases, CTEs
were never inlined and were always evaluated before the rest of the
query. (PostgreSQL 12)
- BRIN indexes (Block Range INdexes) are supported in Greenplum
7. GPORCA currently uses the cost model for bitmap indexes to
support BRIN indexes. GPORCA does not currently support the full set
of BRIN datatypes that are supported with the Postgres Planner
(PostgreSQL 9.5).
- Hash indexes are supported in Greenplum 7 with the Postgres planner
(PostgreSQL 10). (Not supported by GPORCA.)
- The built-in full-text search functionality can now search for
phrases (multiple adjacent words) that appear next to each other in
a specific order, or with a specified distance between the words
(PostgreSQL 9.6).
- Greenplum Database now supports ALTER TABLE SET DISTRIBUTED BY for
external tables. However, you must ensure that the contents of the
external/foreign tables satisfy the DISTRIBUTED BY rules.
- Window functions now support all framing options shown in the
SQL:2011 standard, including RANGE distance PRECEDING/FOLLOWING,
GROUPS mode, and frame exclusion options enhancement.
New Server Features
- Greenplum Database now includes a new server configuration parameter
– default_table_access_method – which determines the default table
access method when a CREATE TABLE command does not explicitly
specify an access method.
- The CREATE TABLE SQL command now supports specifying a table access
method with the new USING <access method> clause.
- You can now dynamically update the access method for a table with
the ALTER TABLE command, using the new clause (subcommand) SET
ACCESS METHOD <access_method>.
- You can now dynamically update the following storage parameters for
a table using the ALTER TABLE command: appendoptimized, blocksize,
orientation, compresstype, compresslevel, checksum
- You may now dynamically update an AOCO table’s column encodings,
using the ALTER TABLE command.
- Unique indexes, unique constraints, and primary keys are now
supported on append-optimized tables.
- You may now alter a heap table with a unique index to an
append-optimized table with a unique index.
- Greenplum Database now includes a new server configuration parameter
– enable_partition_pruning – which enables or disables the query
planner’s ability to eliminate a partitioned table’s partitions from
query plans.
- A new gp_toolkit.__gp_aoblkdir(regclass) function that you can use
to obtain each block directory entry for a given AO/AOCO table that
has or has an index.
- The GPORCA Query Optimizer now supports the CUBE grouping set result
set.
- The GPORCA Query Optimizer now supports planning and running queries
that you specify with multiple grouping sets.
- All Greenplum utilities use now Python 3
This is not a full list, there is much more content for Greenplum 7
which we might have missed to call out. Please subscribe to the
Greenplum Database YouTube Channel to learn more about these
capabilities. We continuously upload new content to help Greenplum
users and developers.
Beta Information
=============
Greenplum Beta releases are pre-release testing versions before a new
release is generally available. We release these to the public for
testing before our final release. Greenplum Betas are not meant for
use in production systems. Beta releases are especially helpful for
developers of platforms, drivers, tools, and utilities that depend
upon Greenplum Database in order to prepare for changes in the new
version.
We strongly advise against using Beta versions of Greenplum Database
in production installations or active development projects. While the
Greenplum Database code is continuously subject to several automated
and manual tests, Beta releases might have serious bugs. Features are
subject to changes that are backward incompatible at any time during
the development of the Betas and could possibly be removed altogether.
Testing for Bugs & Compatibility
=========================
The stability of each Greenplum Database release greatly depends on
you, the community, to test the upcoming version with your workloads
and testing tools in order to find bugs and regressions before general
availability. As this is a Beta, changes to database behaviors,
feature details, and APIs are possible. Your feedback and testing will
help determine the final adjustments to the new features, so please
test. The quality of user testing helps determine when we can make a
final release.
You can report bugs using [github
issues](https://github.com/greenplum-db/gpdb/issues).
Links
=====
- [7.0.0-beta.0
Release](https://github.com/greenplum-db/gpdb/releases/tag/7.0.0-beta.0)
- [Greenplum Version 7 Youtube
Playlist](https://www.youtube.com/playlist?list=PL4duir3J-8GX9EmTGRXoBE9vTKFWHsXLC)
- [GPDB7: Unique Indexes For AO/CO
Tables](https://greenplum.org/gp7-unique-indices-aoco)
- [GPDB7: ALTER Your Table’s
Storage](https://greenplum.org/gpdb7-alter-your-tables-storage)
release of Greenplum Database 7 is now available for
[download](https://github.com/greenplum-db/gpdb/releases/tag/7.0.0-beta.0).
This release contains a preview of the features that will be available
when Greenplum Database 7 is made generally available, though details
of the release can change during the Beta period. Greenplum 7 is based
on PostgreSQL 12 and includes a wide array of new functionality
representing five major versions of PostgreSQL development, plus
Greenplum features along and on top.
In the spirit of the open-source Greenplum community, we strongly
encourage you to test the new features of Greenplum Database 7 on your
systems to help us eliminate bugs or other issues that may
exist. While we do not advise you to run Greenplum 7 Beta in
production environments, we encourage you to find ways to run your
typical application workloads against this Beta release.
Your testing and feedback will help the community ensure that the
Greenplum 7 release upholds our standards of delivering a stable,
reliable release of the next major version of Greenplum.
Feature Highlights
===============
Greenplum 7 incorporates several new features from PostgreSQL versions
9.5 through version 12:
- Improve the sorting speed of varchar, text, and numeric fields via
“abbreviated” keys (Postgres 9.5).
- Support for array_position() and array_positions() functions
(Postgres 9.5).
- Support for INSERT ... ON CONFLICT UPDATE, an UPSERT-like operation
that handles conflicts between concurrent data changes on a
table. (Not yet supported by GPORCA, or on append-only tables.)
(PostgreSQL 9.5).
- Support for row-level security that implements true per-row and
per-column data access control (PostgreSQL 9.5). Database
administrators can now set security policies that filter which rows
particular users are permitted to update or view. Refer to About
Configuring Row -Level Security Policies for more information.
- The pg_resetxlog utility was renamed to pg_resetwal.
- Partition the shared hash table freelist to reduce contention on
multi-CPU-socket servers (PostgreSQL 9.6).
- Performance improvement when using atomic operations, rather than a
spinlock, to protect an LWLock’s wait queue (PostgreSQL 9.6).
- Allow setting and revoking default permissions on schemas using the
ALTER DEFAULT PRIVILEGES command (PostgreSQL 10).
- Support for PostgreSQL declarative table partitioning syntax
(PostgreSQL 10).
- Allow multiple functions, operators, and aggregates to be dropped
with a single DROP command (PostgreSQL 10).
- Support for SQL stored procedures that support embedded transactions
(PostgreSQL 11).
- Reduces the WAL overhead when building a GiST, GIN, or SP-GiST
index; less space on disk is now required for these WAL records and
the data replays faster during crash recovery or point-in-time
recovery (PostgreSQL 12).
- Support for the SQL/JSON path language (PostgreSQL 12).
- Support for STORED generated columns (columns whose content is
computed from expressions, including references to other columns in
the same table, rather than being specified by INSERT or UPDATE
commands). Generated columns cannot be used as a distribution key in
Greenplum Database. (PostgreSQL 12)
- Support for allowing some DDL commands to accept the current user
(CURRENT_USER) or the session user (SESSION_USER) in place of a
specific user name. (PostgreSQL 9.5).
- The CREATE/ALTER USER ... PASSWORD commands no longer support the
UNENCRYPTED option (PostgreSQL 10).
- Support for defining multi-column most-common-value (MCV) extended
statistics (via CREATE STATISTICS) to generate better plans for
queries that test several non-uniformly-distributed
columns. Currently supported only by the Postgres Planner, not
GPORCA (PostgreSQL 12).
- Support for automatic (but overridable) inlining of common table
expressions (CTEs). In Greenplum 7, CTEs are automatically inlined
if they have no side effects, are not recursive, and are referenced
only once in the query. You can prevent inlining by specifying
MATERIALIZED, or force inlining for multiple-referenced CTEs by
specifying NOT MATERIALIZED. In previous Greenplum releases, CTEs
were never inlined and were always evaluated before the rest of the
query. (PostgreSQL 12)
- BRIN indexes (Block Range INdexes) are supported in Greenplum
7. GPORCA currently uses the cost model for bitmap indexes to
support BRIN indexes. GPORCA does not currently support the full set
of BRIN datatypes that are supported with the Postgres Planner
(PostgreSQL 9.5).
- Hash indexes are supported in Greenplum 7 with the Postgres planner
(PostgreSQL 10). (Not supported by GPORCA.)
- The built-in full-text search functionality can now search for
phrases (multiple adjacent words) that appear next to each other in
a specific order, or with a specified distance between the words
(PostgreSQL 9.6).
- Greenplum Database now supports ALTER TABLE SET DISTRIBUTED BY for
external tables. However, you must ensure that the contents of the
external/foreign tables satisfy the DISTRIBUTED BY rules.
- Window functions now support all framing options shown in the
SQL:2011 standard, including RANGE distance PRECEDING/FOLLOWING,
GROUPS mode, and frame exclusion options enhancement.
New Server Features
- Greenplum Database now includes a new server configuration parameter
– default_table_access_method – which determines the default table
access method when a CREATE TABLE command does not explicitly
specify an access method.
- The CREATE TABLE SQL command now supports specifying a table access
method with the new USING <access method> clause.
- You can now dynamically update the access method for a table with
the ALTER TABLE command, using the new clause (subcommand) SET
ACCESS METHOD <access_method>.
- You can now dynamically update the following storage parameters for
a table using the ALTER TABLE command: appendoptimized, blocksize,
orientation, compresstype, compresslevel, checksum
- You may now dynamically update an AOCO table’s column encodings,
using the ALTER TABLE command.
- Unique indexes, unique constraints, and primary keys are now
supported on append-optimized tables.
- You may now alter a heap table with a unique index to an
append-optimized table with a unique index.
- Greenplum Database now includes a new server configuration parameter
– enable_partition_pruning – which enables or disables the query
planner’s ability to eliminate a partitioned table’s partitions from
query plans.
- A new gp_toolkit.__gp_aoblkdir(regclass) function that you can use
to obtain each block directory entry for a given AO/AOCO table that
has or has an index.
- The GPORCA Query Optimizer now supports the CUBE grouping set result
set.
- The GPORCA Query Optimizer now supports planning and running queries
that you specify with multiple grouping sets.
- All Greenplum utilities use now Python 3
This is not a full list, there is much more content for Greenplum 7
which we might have missed to call out. Please subscribe to the
Greenplum Database YouTube Channel to learn more about these
capabilities. We continuously upload new content to help Greenplum
users and developers.
Beta Information
=============
Greenplum Beta releases are pre-release testing versions before a new
release is generally available. We release these to the public for
testing before our final release. Greenplum Betas are not meant for
use in production systems. Beta releases are especially helpful for
developers of platforms, drivers, tools, and utilities that depend
upon Greenplum Database in order to prepare for changes in the new
version.
We strongly advise against using Beta versions of Greenplum Database
in production installations or active development projects. While the
Greenplum Database code is continuously subject to several automated
and manual tests, Beta releases might have serious bugs. Features are
subject to changes that are backward incompatible at any time during
the development of the Betas and could possibly be removed altogether.
Testing for Bugs & Compatibility
=========================
The stability of each Greenplum Database release greatly depends on
you, the community, to test the upcoming version with your workloads
and testing tools in order to find bugs and regressions before general
availability. As this is a Beta, changes to database behaviors,
feature details, and APIs are possible. Your feedback and testing will
help determine the final adjustments to the new features, so please
test. The quality of user testing helps determine when we can make a
final release.
You can report bugs using [github
issues](https://github.com/greenplum-db/gpdb/issues).
Links
=====
- [7.0.0-beta.0
Release](https://github.com/greenplum-db/gpdb/releases/tag/7.0.0-beta.0)
- [Greenplum Version 7 Youtube
Playlist](https://www.youtube.com/playlist?list=PL4duir3J-8GX9EmTGRXoBE9vTKFWHsXLC)
- [GPDB7: Unique Indexes For AO/CO
Tables](https://greenplum.org/gp7-unique-indices-aoco)
- [GPDB7: ALTER Your Table’s
Storage](https://greenplum.org/gpdb7-alter-your-tables-storage)
Soumyadeep Chakraborty
Jan 4, 2023, 3:23:24 PM1/4/23
to Ashwin Agrawal, Greenplum Developers, Greenplum Users
To add to the list of features in the Beta:
CLUSTER is now supported on append-optimized tables. Here is a blog
that goes into it in detail:
https://greenplum.org/gp7-clustering-aoco-tables/
Regards,
Soumyadeep (VMware)
CLUSTER is now supported on append-optimized tables. Here is a blog
that goes into it in detail:
https://greenplum.org/gp7-clustering-aoco-tables/
Regards,
Soumyadeep (VMware)
Ashwin Agrawal
Feb 6, 2023, 12:31:29 PM2/6/23
to Greenplum Developers, Greenplum Users
Happy to announce the next beta of GPDB7 has been released.
Available for [download](https://github.com/greenplum-db/gpdb/releases/tag/7.0.0-beta.1)
In this beta releases on top of all existing contents for GPDB7 beta.0,
- REL_12_STABLE branch was merged which includes over ~2000 commits for improvements/fixes from upstream
- Just-in-Time (JIT) compilation, it is possible to use JIT with both the Postgres Planner and GPORCA.
- You may now fetch a subset of columns when using the command COPY TO from a AOCO table.
- The table pg_appendonly no longer records append-only storage options, they are now only listed under pg_class.reloptions, which significantly reduces the size of pg_appendonly catalog table.
- VACUUM can now identify pages containing only already frozen tuples in the table’s visibility map and skips these pages, hence reducing the cost of maintaining large tables which contain mostly unchanging data. The new VACUUM parameter DISABLE_PAGE_SKIPPING forces VACUUM to run against all frozen pages in case the contents of the visibility map are suspect, which should happen only if there is a hardware or software issue causing database corruption.
- The new option SKIP_LOCKED allows VACUUM and ANALYZE to skip relations that cannot lock immediately due to conflicting locks.
The new option INDEX_CLEANUP allows VACUUM to skip index cleanup. Setting the option to false will make VACUUM run as quickly as possible, for example, to avoid imminent transaction ID wraparound.
The new option INDEX_CLEANUP allows VACUUM to skip index cleanup. Setting the option to false will make VACUUM run as quickly as possible, for example, to avoid imminent transaction ID wraparound.
- VACUUM can now avoid unnecessary heap table truncation attempts that require taking an exclusive table lock even when no truncation is possible. This enhancement avoids unnecessary query cancellations on the standby servers.
- The new configuration parameter vacuum_cleanup_index_scale_factor helps minimize unnecessary index scans during VACUUM.
- The new table and partition storage parameter vacuum_index_cleanup lets you control whether, for a given table ot partition, VACUUM attempts to remove index entries pointing to dead tuples.
- [Greenplum Version 7 Youtube
Playlist](https://www.youtube.com/playlist?list=PL4duir3J-8GX9EmTGRXoBE9vTKFWHsXLC)
Playlist](https://www.youtube.com/playlist?list=PL4duir3J-8GX9EmTGRXoBE9vTKFWHsXLC)
Reply all
Reply to author
Forward
0 new messages