Transitioning...

[red floyd]

I have a codebase written in c90. We have a once in a lifetime
opportunity to refactor, rearchitect, and rewrite. As such, we
are going to rewrite in C++.

Here's my issue. We have two target platforms, one of which has
only a C++03 compiler, the other has a C++11 compiler as well.
Yes I know that both are old language standards, but it's what
we have to work with.

The platform with only C++03 support is old, and may stop being
supported. We're trying to decide whether to write in C++11 and
forget the older platform, or use C++03 and support both, and later
transition to C++11.

We know the LOE to transition to C++. The question is, what is the
effort to transition a C++03 code base to C++11, including the use of
more modern features to enhance code robustness? I know that you can
throw C++03 at a C++11 compiler and it will build, but as this is a one
time only deal, we want to do things as right as possible the first
time.

Any opinions on the level of effort for a "proper" 03 => 11 transition?
Suggestions, etc, are also welcome.

Thanks,

red floyd

[Paavo Helde]

01.05.2020 19:45 red floyd kirjutas:
>
> I have a codebase written in c90.   We have a once in a lifetime
> opportunity to refactor, rearchitect, and rewrite.   As such, we
> are going to rewrite in C++.

Having an opportunity is not a sufficient reason to justify a rewrite
which will inevitable introduce new bugs and destabilize the code base
(unless you have a pretty decent unit test coverage).

A valid reason for a rewrite would be a need to significantly expand the
system later. For that it would help if the existing codebase is written
in ways compatible with extensions (abstract base classes, std::function
callbacks, etc). C++03 is actually pretty good for building such
infrastructure.

However, if you are going to phase out the old platform anyway soon,
will it benefit at all from this rewrite? If not, then it should be left
to use the C solution.

So, the correct answer depends on the actual reason of the rewrite which
you have not disclosed. If this reason is valid also for the old
platform you should rewrite the codebase in C++03 and move on to C++11
later. If the reasons of rewrite do not apply to the old platform you
should leave the old platform to the existing C solution and upgrade the
rest of the system directly to C++11 (or later) as C++ programming has
become much more convenient and safer with C++11.

[red floyd]

On 5/1/2020 10:43 AM, Paavo Helde wrote:
> 01.05.2020 19:45 red floyd kirjutas:
>>
>> I have a codebase written in c90.   We have a once in a lifetime
>> opportunity to refactor, rearchitect, and rewrite.   As such, we
>> are going to rewrite in C++.
>
> Having an opportunity is not a sufficient reason to justify a rewrite
> which will inevitable introduce new bugs and destabilize the code base
> (unless you have a pretty decent unit test coverage).
>
> A valid reason for a rewrite would be a need to significantly expand the
> system later. For that it would help if the existing codebase is written
> in ways compatible with extensions (abstract base classes, std::function
> callbacks, etc). C++03 is actually pretty good for building such
> infrastructure.
>
> However, if you are going to phase out the old platform anyway soon,
> will it benefit at all from this rewrite? If not, then it should be left
> to use the C solution.
>
> So, the correct answer depends on the actual reason of the rewrite which
> you have not disclosed. If this reason is valid also for the old
> platform you should rewrite the codebase in C++03 and move on to C++11
> later. If the reasons of rewrite do not apply to the old platform you
> should leave the old platform to the existing C solution and upgrade the
> rest of the system directly to C++11 (or later) as C++ programming has
> become much more convenient and safer with C++11.

Thanks, Paavo,

I understand your concerns, and am normally a huge proponent of the "if
it ain't broke, don't fix it" approach. However, I believe that it is
"broke".

The platforms currently share a common codebase. The code has become so
fragile, to the point that we are afraid to fix any bugs, because we are
worried that touching $X will break $Y. It's an accumulation of fifteen
years of ad hoc patches. This is ancient legacy code.

We also have some internal framework code that we use, but it's very
rudimentary, and doesn't support all the features we need, which has led
to some very ugly and fragile hacks. The latest version of the
framework, which includes fully fleshed out versions of the features
that we hacked, requires C++.

In all fairness to our management, they are giving us a reasonable
amount of time to do the up front design work.

The main question here was assuming that we are redesigning in C++ --
which, from signs given by management, appears to be a fait accompli --
Is it worth it to design in C++03, and later migrate to C++11, when the
users of the older platform have migrated to the newer platform? What
are people experiences in moving from C++03 to C++11 (or any modern
variant), and what is the level of effort involved?

-- red floyd

[Jorgen Grahn]

On Fri, 2020-05-01, red floyd wrote:
...

> The main question here was assuming that we are redesigning in C++ --
> which, from signs given by management, appears to be a fait accompli --
> Is it worth it to design in C++03, and later migrate to C++11, when the
> users of the older platform have migrated to the newer platform? What
> are people experiences in moving from C++03 to C++11 (or any modern
> variant), and what is the level of effort involved?

As far as I'm concerned, all the basic, useful features are in C++98.
My C++11 code mostly adds 'auto' and lambdas, but these don't affect
the basic design. I'm more likely to use std::find_if due to the easy
access to functors (lambdas), and less likely to add a lot of typedefs
due to auto.

Smart pointers (unique_ptr; shared_ptr if you need those) is another
thing, but as with many library features you can get it from Boost.

It also depends on what the programmers are comfortable with. Maybe
you all write C++17 code and would hate to step back in time 20 years?
Or maybe learning how to use a new C++ dialect well would distract
from the porting work?

/Jorgen

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .

[Paavo Helde]

I would say the most urgent need for you is to develop a comprehensive
automatic test suite. It would help with avoiding $X breaking $Y, it
would help (a lot!) when rewriting the code in C++, and it will help (a
lot!) when later developing the system further.

I know it does not sound pleasant to put all developers (QA, if you have
one, is probably not qualified enough) writing unit/integration tests
for a couple of months, instead of writing new shiny C++ code, but this
is actually the only way to come out of such a situation and save money
in the long run.

>
> We also have some internal framework code that we use, but it's very
> rudimentary, and doesn't support all the features we need, which has led
> to some very ugly and fragile hacks. The latest version of the
> framework, which includes fully fleshed out versions of the features
> that we hacked, requires C++.
>
> In all fairness to our management, they are giving us a reasonable
> amount of time to do the up front design work.
>
> The main question here was assuming that we are redesigning in C++ --
> which, from signs given by management, appears to be a fait accompli --
> Is it worth it to design in C++03, and later migrate to C++11, when the
> users of the older platform have migrated to the newer platform?

This depends on whether these people need feature $Y before migrating to
the newer platform.

>   What
> are people experiences in moving from C++03 to C++11 (or any modern
> variant), and what is the level of effort involved?

You actually don't migrate the codebase from C++03 to C++11, this is not
needed for anything. You just starting writing the new code (and any
local refactoring of old code) with C++11 means as soon as the build
process has switched on -std=c++11 (or equiv). So in this sense the
transition should be smooth.

In C++03, beware to not use things removed in later C++ standards like
throw specifications, 'register' keyword, std::bind1st, etc.






> -- red floyd
>
>

[Paavo Helde]

01.05.2020 21:48 Paavo Helde kirjutas:
> In C++03, beware to not use things removed in later C++ standards like
> throw specifications, 'register' keyword, std::bind1st, etc.

Oh, and avoid std::auto_ptr of course. Use Boost smart pointers or
develop your own, that's not so complicated.

[Jorgen Grahn]

On Fri, 2020-05-01, Paavo Helde wrote:
...

> I would say the most urgent need for you is to develop a comprehensive
> automatic test suite. It would help with avoiding $X breaking $Y, it
> would help (a lot!) when rewriting the code in C++, and it will help (a
> lot!) when later developing the system further.
>
> I know it does not sound pleasant to put all developers (QA, if you have
> one, is probably not qualified enough) writing unit/integration tests
> for a couple of months, instead of writing new shiny C++ code,

Seconded. You feel so stupid when you know how you want to refactor
the code, but have no way to tell if it still works afterwards.

> but this is actually the only way to come out of such a situation
> and save money in the long run.

[Öö Tiib]

On Friday, 1 May 2020 19:45:00 UTC+3, red floyd wrote:
> I have a codebase written in c90. We have a once in a lifetime
> opportunity to refactor, rearchitect, and rewrite. As such, we
> are going to rewrite in C++.
>
> Here's my issue. We have two target platforms, one of which has
> only a C++03 compiler, the other has a C++11 compiler as well.
> Yes I know that both are old language standards, but it's what
> we have to work with.

Platforms that lag behind do often have some quirks in their
implementations of those older standards too. Common code
can be fine but some trick (like used in boost) may refuse to
compile or act in non-conforming manner.

>
> The platform with only C++03 support is old, and may stop being
> supported. We're trying to decide whether to write in C++11 and
> forget the older platform, or use C++03 and support both, and later
> transition to C++11.
>
> We know the LOE to transition to C++. The question is, what is the
> effort to transition a C++03 code base to C++11, including the use of
> more modern features to enhance code robustness? I know that you can
> throw C++03 at a C++11 compiler and it will build, but as this is a one
> time only deal, we want to do things as right as possible the first
> time.
>
> Any opinions on the level of effort for a "proper" 03 => 11 transition?
> Suggestions, etc, are also welcome.

There were next to none silently breaking changes between C++03 and
C++11. Vast majority of C++03 code did build and run as C++11 too.
There can be some undefined behaviors that start to manifest
differently, but it is generally good that these will be noticed.
Additionally there is Clang-Tidy that actively suggests what to
refactor in C++03 code.

So I think that rewriting C90 -> C++03 is large effort because the
idioms and design strategies of those languages are different.
Migrating C++03 -> C++11 is order of magnitude smaller effort.

[Keith Thompson]

What if you transition directly from C90 to C++11 and just don't
support the new codebase on the old platform? Can you continue
supporting only the old C codebase on the old platform until it's
retired, while using the new C++11 codebase on the new platform?
You know the old C90 code still works; is that good enough for
*temporary* support?

On the other hand, supporting two different codebases simultaneously
isn't going to be free.

On the gripping hand, having the old and new codebases in production
use at the same time might help weed out inconsistencies.

How sure are you that the old target platform is really going away?
Is there a risk that you'd be stuck supporting the C90 codebase
indefinitely?

--
Keith Thompson (The_Other_Keith) Keith.S.T...@gmail.com
Working, but not speaking, for Philips Healthcare
void Void(void) { Void(); } /* The recursive call of the void */

[red floyd]

That's one of the pluses of a refactor on this scale. For historical
reasons, and then because of the high coupling factor from the patches,
it was really impossible to have an automated unit test suite. By
refactoring on this scale, we can write the unit tests as we go.

A little more historical background, this was a proof-of-concept that
marketing grabbed and said "OK, it's the product". So it didn't have
the in-depth design and review that it would today.

[Mr Flibble]

Use C++17.

/Flibble

--
"Snakes didn't evolve, instead talking snakes with legs changed into snakes." - Rick C. Hodgin

“You won’t burn in hell. But be nice anyway.” – Ricky Gervais

“I see Atheists are fighting and killing each other again, over who doesn’t believe in any God the most. Oh, no..wait.. that never happens.” – Ricky Gervais

"Suppose it's all true, and you walk up to the pearly gates, and are confronted by God," Byrne asked on his show The Meaning of Life. "What will Stephen Fry say to him, her, or it?"
"I'd say, bone cancer in children? What's that about?" Fry replied.
"How dare you? How dare you create a world to which there is such misery that is not our fault. It's not right, it's utterly, utterly evil."
"Why should I respect a capricious, mean-minded, stupid God who creates a world that is so full of injustice and pain. That's what I would say."

[red floyd]

On 5/1/20 6:41 PM, Mr Flibble wrote:
> On 01/05/2020 17:45, red floyd wrote:

>> Here's my issue.  We have two target platforms, one of which has
>> only a C++03 compiler, the other has a C++11 compiler as well.
>> Yes I know that both are old language standards, but it's what
>> we have to work with.
>>

>

> Use C++17.
>
> /Flibble
>

What part of "only a C++11 and C++03 compiler are available" do you have
a problem understanding?

[Alf P. Steinbach]

It reminds me of the Eiffel approach of compiling to C as an
intermediate language.

After all C was /made for the purpose/ as a high level assembly
language, to ease the porting of early Unix.

Build times for C++ being what they are in 2020 it's probably not on as
a practical proposition, but it could bring the advantages of each new
C++ standard to even the most backward platforms, in essence
cross-compiling to that platform's main C language implementation.

And it's been done before. CFront, Bjarne's first compiler, emitted C
code. The Comeau compiler, that 15+ years ago (not sure about dates) we
used to check code correctness, like a reference implementation of the
language, likewise could compile to C.

Some people, e.g. I found some SO questions about it, assert that one
can use LLVM to translate C++ to equivalent C.


- Alf

[Ian Collins]

We went through this process a few years back and it wasn't too big a
deal. Your C++03 should continue to build with c++11. For the library
improvements in C++11, you can generally use boost in C++03 code and
simply swap boost to std when you upgrade. If I look back in our git,
there are a ton of "de-boosting" commits :)


As someone else said, make sure to avoid auto_ptr!

--
Ian.

[Jorgen Grahn]

On Fri, 2020-05-01, red floyd wrote:

> On 5/1/2020 2:17 PM, Jorgen Grahn wrote:
>> On Fri, 2020-05-01, Paavo Helde wrote:
>> ...
>>> I would say the most urgent need for you is to develop a comprehensive
>>> automatic test suite. It would help with avoiding $X breaking $Y, it
>>> would help (a lot!) when rewriting the code in C++, and it will help (a
>>> lot!) when later developing the system further.
>>>
>>> I know it does not sound pleasant to put all developers (QA, if you have
>>> one, is probably not qualified enough) writing unit/integration tests
>>> for a couple of months, instead of writing new shiny C++ code,
>>
>> Seconded. You feel so stupid when you know how you want to refactor
>> the code, but have no way to tell if it still works afterwards.
>>
>>> but this is actually the only way to come out of such a situation
>>> and save money in the long run.
>
> That's one of the pluses of a refactor on this scale. For historical
> reasons, and then because of the high coupling factor from the patches,
> it was really impossible to have an automated unit test suite. By
> refactoring on this scale, we can write the unit tests as we go.

It's orthogonal though, isn't it? Unit tests won't be that helpful in
keeping the software's behavior the same after the porting, since it
tests interfaces you're replacing. I'd be more interested in
(automated) tests on the whole system, and/or on components.

Of course unit tests have other benefits.

[Sam]

red floyd writes:

> Any opinions on the level of effort for a "proper" 03 => 11 transition?
> Suggestions, etc, are also welcome.

Could be anything, from minimal to massive. This assumes on whoever is
working on it is thoroughly familiar with C++11 language features and is
capable of employing them where needed, of course.

Strangers on the intertubes who have never seen your code have no way of
being more precise than that.

[Mr Flibble]

My answer remains unchanged, use C++17, and by that I mean your reasoning that "that is what you have to work with" is bogus: make changes so that you can use C++17.

[red floyd]

On 5/2/20 6:01 AM, Mr Flibble wrote:
> On 02/05/2020 04:15, red floyd wrote:
>> On 5/1/20 6:41 PM, Mr Flibble wrote:
>>> On 01/05/2020 17:45, red floyd wrote:
>>
>>>> Here's my issue.  We have two target platforms, one of which has
>>>> only a C++03 compiler, the other has a C++11 compiler as well.
>>>> Yes I know that both are old language standards, but it's what
>>>> we have to work with.
>>>>
>>
>>>
>>> Use C++17.
>>>
>>> /Flibble
>>>
>>
>> What part of "only a C++11 and C++03 compiler are available" do you have
>> a problem understanding?
>
> My answer remains unchanged, use C++17, and by that I mean your
> reasoning that "that is what you have to work with" is bogus: make
> changes so that you can use C++17.
>
> Use C++17.
>

OK, let's assume I use C++17. How do you propose I compile it? I
cannot change platforms. I'm all ears. Only the vendor provided
compilers are available, there is no g++.

[red floyd]

I mean, if you can't give me an answer here to my "how do you propose"
question, your answer is simply sausages.

[Mr Flibble]

My answer is simple: change things so you can use a C++17 compiler.

[red floyd]

Thanks for nothing. I do not have the option to change the platform.
You must be lovely to work with.

Your advice is sausages.

[Manfred]

On 5/1/20 7:43 PM, Paavo Helde wrote:
> For that it would help if the existing codebase is written in ways
> compatible with extensions (abstract base classes, std::function
> callbacks, etc). C++03 is actually pretty good for building such
> infrastructure.

Isn't std::function one of the additions of C++11?

[Manfred]

On 5/1/20 6:45 PM, red floyd wrote:
>
> The platform with only C++03 support is old, and may stop being
> supported.  We're trying to decide whether to write in C++11 and
> forget the older platform, or use C++03 and support both, and later
> transition to C++11.
>
> We know the LOE to transition to C++.  The question is, what is the
> effort to transition a C++03 code base to C++11, including the use of
> more modern features to enhance code robustness?   I know that you can
> throw C++03 at a C++11 compiler and it will build, but as this is a one
> time only deal, we want to do things as right as possible the first
> time.

As other have said, the transition from C++03 to C++11 may not be that
hard, however you may want to give a look at
https://en.cppreference.com/w/cpp/11

The point is that C++11 was "the most important update since C++98" (the
next one being probably C++20), and this is evident from the list of new
core language features.

Many of these features do have impact at the design level, so the
decision about using them or not is not only a matter of
maintainability, but it may also have influence on how you want to shape
your design - I am thinking of things like lambdas, std::function, move
semantics, variadic templates, constexpr, (also static_assert is pretty
handy) ...
In other words, upgrading incrementally from C++03 to C++11 may lead to
some loss in design homogeneity.
On the other hand, it is also true that many C++03 implementations
already included a number of C++11 features that were published in
advance as TR's.

Of course all of this depends on your needs, it is very well possible
that none of the features above really matters for your specific
project, keeping things simple is still the golden rule of good design.

My 2c

[Ian Collins]

Yes it is, C++03 code would have to use boost::function.

--
Ian.

[Pavel]

I experienced a similar transition (more precisely, change from C++03 compiler
to C++11) on significant code base about 3 years ago. We did not do any
refactoring, just switched to the newer compiler, set up the command-line
options and started using C++11 features. As the programmers became better
familiar with C++11 features they would sometimes replace a piece of the old
code when there was a need to fix a bug or change or add a feature with the code
using C++11 features.

I think our approach was close to optimal: where we did not need to change code
for a business reason or to fix a bug, maybe it was better to not touch it at
all (based on the principle to not touch what's either working or useless).
Otherwise, we had a chance to incrementally change it later wherever it was most
beneficial.

-Pavel

[red floyd]

Thanks, Pavel.

This is the sort of thing I wa slooking forward. This and Manfred's
note about some C++11-isms being needed at the design stage.

Appreciate your and everyone else's comments!

Thanks everyone!

-- red floyd

[Paavo Helde]

Yes, C++11 is in principle pretty backward compatible with C++03. The
same cannot be said about compiler versions, in general one can expect
more porting problems when upgrading compiler versions.

So take care to not use an old compiler version for you C++03. Use the
latest available compiler in C++03 mode, at least on some platform. If
you have to use an old compiler on the other platform(s), then expect
some porting efforts between the platforms.

[Ian Collins]

On 04/05/2020 18:21, Paavo Helde wrote:
>
> Yes, C++11 is in principle pretty backward compatible with C++03. The
> same cannot be said about compiler versions, in general one can expect
> more porting problems when upgrading compiler versions.
>
> So take care to not use an old compiler version for you C++03. Use the
> latest available compiler in C++03 mode, at least on some platform. If
> you have to use an old compiler on the other platform(s), then expect
> some porting efforts between the platforms.

That's good advice; our embedded product uses gcc for the target but
clang and VS2017 for development and test builds.

One other useful thing that I have done (and currently do!) is use the
latest supported standard for unit tests to familiarise myself and the
other team members with the latest and greatest features. This is a
great way to engage people in writing unit tests!

--
Ian.

[Richard]

[Please do not mail me a copy of your followup]

red floyd <my...@its.invalid> spake the secret code
<r8hjm1$6dk$1...@redfloyd.dont-email.me> thusly:

>I have a codebase written in c90. We have a once in a lifetime
>opportunity to refactor, rearchitect, and rewrite. As such, we
>are going to rewrite in C++.

My advice is to never rewrite. Refactor, yes, but not rewrite. This
is particularly true in this case since you can easily consume C90
code from C++.

Depending on the size of your code base, you could invest tens of man
years in rewriting your system in C++ (easily an expense of a couple
million $) to end up with a system that has the same functionality and
new bugs that you have to remove.

Just think what you could have had instead for those man years if you
built on top of your existing system.

The last time I was on a team that decided to do a big rewrite, they
were all laid off at the end of the adventure. Instead, had they
spent that development effort on brainstorming new products and ideas,
chances are they would have had a new market opportunity on their
hands and would still be employed.

YMMV, but I am not alone in this assessment.

<https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/>
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>