[boost] bcp update #2: namespace renaming
John Maddock
two new options are:
--namespace=newname
Renames namespace boost to "newname" in all the source files copied.
Also changes library names to start with the prefix "newname" or
"libnewname" rather than "boost" or "libboost".
--namespace-alias
Makes namespace boost an alias for the new name.
Using these two options you can get "strong versioning" with two different
boost installs coexisting side by side (in the same application if
necessary). What you can not do at present, is #include two different boost
versions in the same translation unit.
This update is in, and tested against, SVN Trunk only at present.
Enjoy, John.
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Artyom
That is great news, one of the most missing features.
Several questions:
a) Does it rename macros as well? I mean if I have a library
with defined BOOST_ASIO_ENABLE_FOO_BAR would it be renamed
to NEWNAME_ASIO_ENABLE_FOO_BAR ?
b) Doest it updates BJam files as well.
c) How do you deal with various test files?
d) Have you tryed to run some regrssion tests?
I had written once such tool on my own, see
http://lists.boost.org/boost-build/2009/05/21911.php
http://cppcms.svn.sourceforge.net/viewvc/cppcms/framework/branches/refactoring/tools/rename.py?revision=935&view=markup
However I hadn't updated the BJam files. I still
managed to build most of parts of boost and run some regession
test.
I would defenatly test it ASAP.
Artyom
>
> Also changes library names to start with the prefix
> "newname" or "libnewname" rather than "boost" or
> "libboost".
Does it changes include paths as well? So I can include
<newname/shared_ptr.hpp>
> What you can
> not do at present, is #include two different boost versions
> in the same translation unit.
>
When you use alias or in any case?
Actually my tool had allowed this. Why you can't do this?
Artyom.
P.S.: This is very important step!
John Maddock
>
> Several questions:
>
> a) Does it rename macros as well? I mean if I have a library
> with defined BOOST_ASIO_ENABLE_FOO_BAR would it be renamed
> to NEWNAME_ASIO_ENABLE_FOO_BAR ?
No not at present. That's whay I said you can't #include two different
Boost versions in the same TU.
That's something I might look at later.
> b) Doest it updates BJam files as well.
Yes, library names get adjusted.
> c) How do you deal with various test files?
Like any other source file - if they're copied then the namespace get
renamed.
> d) Have you tryed to run some regrssion tests?
Yes, the Trunk test results are the same after the rename.
Cheers, John.
Bjørn Roald
> > That is great news,
I just tested both the two new options in bcp. The modified bcp'ed source code
and the produced libraries after a plain bjam build of a selected subset of
boost looks very good. I have to agree with Artyom, this is great news.
This give both open and closed source library developers a very nice tool
which allow use of boost in their library implementation without risking
polluting the boost namespace for their users. In other words, this make use
of boost more feasible for a large group of C++ developers which otherwise
typically would re-invent what they needed.
> > one of the most missing features.
> >
> > Several questions:
> >
> > a) Does it rename macros as well? I mean if I have a library
> > with defined BOOST_ASIO_ENABLE_FOO_BAR would it be renamed
> > to NEWNAME_ASIO_ENABLE_FOO_BAR ?
>
> No not at present. That's whay I said you can't #include two different
> Boost versions in the same TU.
>
> That's something I might look at later.
This would be useful if full side-by-side header support is provided. But
that is more involved than just renaming macros. Also, I think what is in now
is the more important feature.
Thank you for adding this John.
--
Bjørn
Daniel James
> OK, since folks keep asking for it, bcp will now do namespace renaming, the
> two new options are:
>
> --namespace=newname
>
> Renames namespace boost to "newname" in all the source files copied.
There are a few places where another namespace is used. A quick search
found boost_asio_handler_alloc_helpers, boost_concepts (in the
iterator library), boost_optional_detail, boost_132 (in serialization)
and boost_swap_impl. Will that be a problem?
Daniel
John Maddock
>> > with defined BOOST_ASIO_ENABLE_FOO_BAR would it be renamed
>> > to NEWNAME_ASIO_ENABLE_FOO_BAR ?
>>
>> No not at present. That's whay I said you can't #include two different
>> Boost versions in the same TU.
>>
>> That's something I might look at later.
>
>This would be useful if full side-by-side header support is provided. But
>that is more involved than just renaming macros. Also, I think what is in
>now
>is the more important feature.
I assume that for full side-by-side inclusioin in the same TU we would need
to rename macros and also rename the boost/ directory (and hense all
#includes). Anything else?
Cheers, John.
John Maddock
>> the
>> two new options are:
>>
>> --namespace=newname
>>
>> Renames namespace boost to "newname" in all the source files copied.
>
> There are a few places where another namespace is used. A quick search
> found boost_asio_handler_alloc_helpers, boost_concepts (in the
> iterator library), boost_optional_detail, boost_132 (in serialization)
> and boost_swap_impl. Will that be a problem?
Sigh... probably :-(
Is that a complete list of all the top level namespaces? If not, any ideas
on how to quickly produce one?
Thanks, John.
Artyom
> Sigh... probably :-(
>
> Is that a complete list of all the top level
> namespaces? If not, any ideas on how to quickly
> produce one?
>
Hello,
Indeed this is a problem, but I also remember that not only namespace are
relevant.
In some older versions of boost I remember (I think in boot_thread 1.35)
some function had names like:
extern "C" boost_thread_some_callback_functions
Were used, also, if at some point shared object/dll dynamic loading
would be added they may use resolveable points like
extern "C" boost_foobar_resolvable_sybol.
My be added, outside boost namespace in order be loaded with
dlsym/GetProcAddress.
I suggest different approach I used in my namespace renaming
script:
http://art-blog.no-ip.info/files/rename.py
I had taken each tocken and substituted boost -> newnamespace and
BOOST -> NEWNAMESPACE
Also I required rename directories as well.
This approach is free from the above error that could happen.
If some symbol would be missed it may cause symbol collapse and
some wired bugs when using 2 versions boost in same program,
shared object.
I would recommend you to take a look on this script. It worked very
fine for me, and it is very simple and generic.
Artyom
John Maddock
> relevant.
>
> In some older versions of boost I remember (I think in boot_thread 1.35)
> some function had names like:
>
> extern "C" boost_thread_some_callback_functions
>
> Were used, also, if at some point shared object/dll dynamic loading
> would be added they may use resolveable points like
>
> extern "C" boost_foobar_resolvable_sybol.
>
> My be added, outside boost namespace in order be loaded with
> dlsym/GetProcAddress.
>
> I suggest different approach I used in my namespace renaming
> script:
>
> http://art-blog.no-ip.info/files/rename.py
>
> I had taken each tocken and substituted boost -> newnamespace and
> BOOST -> NEWNAMESPACE
>
> Also I required rename directories as well.
> This approach is free from the above error that could happen.
>
> If some symbol would be missed it may cause symbol collapse and
> some wired bugs when using 2 versions boost in same program,
> shared object.
Indeed, however your script would still miss (all?) of the extern "C"
callbacks in current Trunk.
So far I've found:
asio_detail_posix_thread_function
lw_thread_routine
free_static_mutex
execution_monitor_jumping_signal_handler
execution_monitor_attaching_signal_handler
at_thread_exit
on_process_exit
on_process_enter
on_thread_enter
on_thread_exit
Plus all of the TR1 math functions (which should definitely not be renamed
IMO).
Unfortunately I don't see any way of renaming these automatically without
also clobbering all the system API's which get declared here and there.
Obviously these could be renamed as special cases easily enough... but
maintaining a manual list of API's to rename is a maintainance nightmare.
Any better ideas?
John.
Maurizio Vitale
John> Obviously these could be renamed as special cases easily
John> enough... but maintaining a manual list of API's to rename is
John> a maintainance nightmare.
John> Any better ideas?
wouldn't running nm on non-renamed boostlibs end grepping for visible text
and data symbol give you a good list?
Best regards,
Maurizio
Artyom
>
> Indeed, however your script would still miss (all?) of the
> extern "C"
> callbacks in current Trunk.
You right, my script does not handle them.
I've checked some of them:
> asio_detail_posix_thread_function
Even it is defined as extern "C" it is inline function thus it should
not even appear in object file. Not a probem
> free_static_mutex
> at_thread_exit
> on_process_exit
> on_process_enter
> on_thread_enter
> on_thread_exit
Indeed, bad, but this is specific library bug and should be reported
to their maintiainers.
Library developers should make sure to use proper namespaces in "C like"
functions: i.e. add "boost_" prefix as it is done with #define's
> Plus all of the TR1 math functions (which should definitely
> not be renamed
> IMO).
What do you mean? It is still under boost::tr1:: or am I wrong?
> Obviously these could be renamed as special cases easily
> enough... but
> maintaining a manual list of API's to rename is a
> maintainance nightmare.
>
> Any better ideas?
Report bugs to developers and fix them.
If someone declares
extern "C" void my_func();
It pollutes global namespace -- rename it to boost_my_func
Artyom
Bjørn Roald
> >> > a) Does it rename macros as well? I mean if I have a library
> >> > with defined BOOST_ASIO_ENABLE_FOO_BAR would it be renamed
> >> > to NEWNAME_ASIO_ENABLE_FOO_BAR ?
> >>
> >> No not at present. That's whay I said you can't #include two different
> >> Boost versions in the same TU.
> >>
> >> That's something I might look at later.
> >
> >This would be useful if full side-by-side header support is provided. But
> >that is more involved than just renaming macros. Also, I think what is in
> >now
> >is the more important feature.
>
> I assume that for full side-by-side inclusioin in the same TU we would need
> to rename macros and also rename the boost/ directory (and hense all
> #includes). Anything else?
I think that is what I did when I played with this, and that it will do the
trick together with what you have in now. I think what is also is a possible
issue is:
#include SOME_MACRO
directives. If these are all prefixed with BOOST_ they get a unique name, but
they also need to expand to the correct include file path.
The --namespace-alias option is great when not using boost in interfaces, but
it is not useful in the side-by-side header case. The method I had in mind to
allow boost to be used as normally as possible was to allow replacing boost
namespace with nested namespace mylib::boost. In implementation files a "using
mylib" directive would allow relatively normal use of boost and still have
separate namespace protection for the interfaces.
Supporting replacing boost with a nested namespace easily gets very tricky
without enforcing stricter coding conventions with explicit end-namespace
tagging in boost. So I think allowing replacing boost with nested namespace
is not worth it unless someone come up with a clever and simple way of making
this work correctly.
--
Bjørn
John Maddock
>without enforcing stricter coding conventions with explicit end-namespace
>tagging in boost. So I think allowing replacing boost with nested
>namespace
>is not worth it unless someone come up with a clever and simple way of
>making
>this work correctly.
I'd go so far as to say it's basically impossible with the current code
base - I started out using regexes that matched nested { and } so I could
identify the full boost namespace scope - but although it 99% worked there
were always cases where there were { or } in macros or comments that weren't
balanced :-( In short you would need a full C++ preprocessor and lexor to
handle this.
John.
Bjørn Roald
> >Supporting replacing boost with a nested namespace easily gets very tricky
> >without enforcing stricter coding conventions with explicit end-namespace
> >tagging in boost. So I think allowing replacing boost with nested
> >namespace
> >is not worth it unless someone come up with a clever and simple way of
> >making
> >this work correctly.
>
> I'd go so far as to say it's basically impossible with the current code
> base - I started out using regexes that matched nested { and } so I could
> identify the full boost namespace scope - but although it 99% worked there
> were always cases where there were { or } in macros or comments that
> weren't balanced :-( In short you would need a full C++ preprocessor and
> lexor to handle this.
Probbably more than that, unless you accept that the output is target
dependent. The preprocessor would otherwise strip off code that need to be
processed.
--
Bjørn
Andrey Semashev
>>> OK, since folks keep asking for it, bcp will now do namespace
>>> renaming, the
>>> two new options are:
>>>
>>> --namespace=newname
>>>
>>> Renames namespace boost to "newname" in all the source files copied.
>>
>> There are a few places where another namespace is used. A quick search
>> found boost_asio_handler_alloc_helpers, boost_concepts (in the
>> iterator library), boost_optional_detail, boost_132 (in serialization)
>> and boost_swap_impl. Will that be a problem?
>
> Sigh... probably :-(
>
> Is that a complete list of all the top level namespaces? If not, any
> ideas on how to quickly produce one?
I'm also aware of phoenix in Spirit v1.
John Maddock
>> extern "C"
>> callbacks in current Trunk.
>
> You right, my script does not handle them.
>
> I've checked some of them:
>
>> asio_detail_posix_thread_function
>
> Even it is defined as extern "C" it is inline function thus it should
> not even appear in object file. Not a probem
Are you sure? I thought there would still *possibly* be a definition
available in every object file that included that header - even though it
would have vague linkage.
>> free_static_mutex
>> at_thread_exit
>> on_process_exit
>> on_process_enter
>> on_thread_enter
>> on_thread_exit
>
> Indeed, bad, but this is specific library bug and should be reported
> to their maintiainers.
>
> Library developers should make sure to use proper namespaces in "C like"
> functions: i.e. add "boost_" prefix as it is done with #define's
>
>
>> Plus all of the TR1 math functions (which should definitely
>> not be renamed
>> IMO).
>
> What do you mean? It is still under boost::tr1:: or am I wrong?
No they're extern "C" so they're globally visable (as well as visible in
std::tr1), and that's the whole point, they provide an alternative
implementation of those std functions if the std lib doesn't provide them.
Renaming them would defeat the whole point of them being there.
>> Obviously these could be renamed as special cases easily
>> enough... but
>> maintaining a manual list of API's to rename is a
>> maintainance nightmare.
>>
>> Any better ideas?
>
> Report bugs to developers and fix them.
Fair point. Filed as:
https://svn.boost.org/trac/boost/ticket/3809
https://svn.boost.org/trac/boost/ticket/3810
https://svn.boost.org/trac/boost/ticket/3811
https://svn.boost.org/trac/boost/ticket/3812
John.
John Maddock
> base - I started out using regexes that matched nested { and } so I could
> identify the full boost namespace scope - but although it 99% worked there
> were always cases where there were { or } in macros or comments that
> weren't balanced :-( In short you would need a full C++ preprocessor and
> lexor to handle this.
>Probbably more than that, unless you accept that the output is target
>dependent. The preprocessor would otherwise strip off code that need to be
>processed.
Nod. In short "there's no easy way".
John.
Daniel James
> On 01/01/2010 03:30 PM, John Maddock wrote:
>>>
>>> There are a few places where another namespace is used. A quick search
>>> found boost_asio_handler_alloc_helpers, boost_concepts (in the
>>> iterator library), boost_optional_detail, boost_132 (in serialization)
>>> and boost_swap_impl. Will that be a problem?
>>
>> Sigh... probably :-(
>>
>> Is that a complete list of all the top level namespaces? If not, any
>> ideas on how to quickly produce one?
No. I just found those namespaces by searching for 'namespace boost_'
since most top level namespaces follow that pattern.
> I'm also aware of phoenix in Spirit v1.
Also BOOST_HASH_DETECT_FLOAT_FUNCTIONS in hash. I'm renaming it to be
lower case for consistency. It only contains function declarations, so
it might not create any symbols.
Daniel