State of the merge

[Heikki Linnakangas]

Hi all!

We've completed merging PostgreSQL 9.0 into GPDB master. 9.0 was a
relatively straightforward release. There was a bunch of refactoring
needed, as there always is, this time e.g. around rewriting of VACUUM
FULL in the upstream. See commit message
(https://github.com/greenplum-db/gpdb/commit/e5d17790c185217831828169884f992be32502a6)
for details.

Putting a PM hat on for a second: we've now merged three major releases
in total. We did the 8.3 merge in spring 2016. It took about 6 months.
Since then, we've done a lot of cleanup, refactoring, and we've learned
a lot on how to do this. We did the 8.4 merge in about 3 months, and the
9.0 merge in a bit under 2 months.

Based on these three data points, 2 months per release seems like a good
estimate going forward. Assuming we keep up pace and don't get
distracted by other things:

9.0 in 2018 March (just completed this)
9.1 in 2018 May
9.2 in 2018 July
9.3 in 2018 September
9.4 in 2018 November
9.5 in 2019 January
9.6 in 2019 March
v10 in 2019 May
v11 in 2019 July

Now, this is purely an extrapolation based on current velocity. There
may well be some unforeseen hiccups that take longer than expected, and
we will almost certainly get distracted with something :-). And we'll
surely need to stop at some point to stabilize for a new GPDB release.

We'll see how this works out as we move forward. I am just about to cut
the working branch for 9.1 now, and if all goes well, we'll be done with
that in May.

- Heikki

[Ivan Novick]

WOW!!!!!!!!!!!!!!!

Great job all

       .''.      .        *''*    :_\/_:     .
      :_\/_:   _\(/_  .:.*_\/_*   : /\ :  .'.:.'.
  .''.: /\ :    /)\   ':'* /\ *  : '..'.  -=:o:=-
 :_\/_:'.:::.  | ' *''*    * '.\'/.'_\(/_'.':'.'
 : /\ : :::::  =  *_\/_*     -= o =- /)\    '  *
  '..'  ':::' === * /\ *     .'/.\'.  ' ._____
      *        |   *..*         :       |.   |' .---"|
        *      |     _           .--'|  ||   | _|    |
        *      |  .-'|       __  |   |  |    ||      |
     .-----.   |  |' |  ||  |  | |   |  |    ||      |
 ___'       ' /"\ |  '-."".    '-'   '-.'    '`      |____
jgs~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  &                    ~-~-~-~-~-~-~-~-~-~   /|
 ejm97    )      ~-~-~-~-~-~-~-~  /|~       /_|\
        _-H-__  -~-~-~-~-~-~     /_|\    -~======-~
~-\XXXXXXXXXX/~     ~-~-~-~     /__|_\ ~-~-~-~
~-~-~-~-~-~    ~-~~-~-~-~-~    ========  ~-~-~-~

--

Ivan Novick, Product Manager Pivotal Greenplum

ino...@pivotal.io --  (Mobile) 408-230-6491

https://www.youtube.com/GreenplumDatabase

[Amil Khanzada]

Awesome work with such a small team!! Great we are catching up, as I think the estimate was 5 years or never catching up to Postgres a couple of years back when I spoke to some gpdb folk in Palo Alto!

[CK Tan]

Nice!

On Fri, Mar 9, 2018 at 9:28 AM, Ivan Novick <ino...@pivotal.io> wrote:

--
You received this message because you are subscribed to the Google Groups "Greenplum Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gpdb-dev+unsubscribe@greenplum.org.

[Oak Barrett]

awesome job everyone... 

oak

On Fri, Mar 9, 2018 at 9:09 AM, Heikki Linnakangas <hlinna...@pivotal.io> wrote:

[Venkatesh Raghavan]

Great work!

V

ᐧ

[Yandong Yao]

Wow, this goes far beyond my expectations

--

Best Regards,

Yandong

[Jasper Li]

Awesome! Big thanks for the excellent work!

Best wishes!

Jasper

发件人: Yandong Yao

发送时间: 3月10日星期六 上午8:23

主题: Re: State of the merge

收件人: Venkatesh Raghavan

抄送: Oak Barrett, Heikki Linnakangas, Greenplum Developers

Wow, this goes far beyond my expectations

On Sat, Mar 10, 2018 at 2:22 AM, Venkatesh Raghavan <vrag...@pivotal.io> wrote:

Great work!

V

[Gang Xiong]

Thanks, team. 

Very exciting to see the timetable. Would like to help to keep you guys from being distracted.

[Heikki Linnakangas]

On 10/03/18 04:01, Gang Xiong wrote:
> Thanks, team.
> Very exciting to see the timetable. Would like to help to keep you guys
> from being distracted.

Great, thanks! Various FIXMEs and other issues crop up all the time,
where we really could use help.

For example, we really ought to stop relying on ExecutorEnd being called
on Abort, to clean up the dispatcher and interconnect. That's caused
various issues. I first bumped into that in 2016 [1], and during the 9.0
merge, we could not merge upstream commit 7981c342 [2] properly because
of that. I'm sure it will cause us more trouble as we go forward.

Some refactoring is needed to fix that. I think we should start using
the ResourceOwner mechanism to clean up the dispatcher and the
interconnect on abort. If you could write a PR to do that, that'd be
really helpful!

[1]
https://groups.google.com/a/greenplum.org/d/msg/gpdb-dev/-BTKTbLuP1k/pbzwZAxOEwAJ

[2]
https://github.com/greenplum-db/gpdb/commit/7981c34279fbddc254cfccb9a2eec4b35e692a12

- Heikki