Windows

[Joris Dauphin]

Hi,
I have "ported" oclint for windows (Jarod42's fork).
I'm currently use a custom build system based on premake (instead of CMake) using g++ to compile (MinGW) which works :).
I'm not very familiar with CMake :/.

The auto-registration (based on a static variable) doesn't work (in win) unless Ruleset is a sharedLib instead of a static lib.

I tried some stuff with CMake without full success (I used an other cmake generator ("msys makefiles"))
I have in fact some issue somewhere to use CMake correctly with llvm/clang
(and it takes, for my poor PC a very long time to recompile llvm/clang and a long time for the oclint-rules).

[Longyi Qi]

Hi Joris,

Thank you for your efforts in porting OCLint for Windows.

I am not quite familiar with Windows development environment myself, but I am very happy to know/to hear OCLint even can be ported to Windows, I definitely would like to see the benefits that OCLint can bring to Windows developers.

I am open to other build systems as well, but one of CMake's advantages is cross-platform, so I would insist on having it at least be able to build with CMake. Clang has this document http://clang.llvm.org/get_started.html to build llvm/clang with CMake and Visual Studio, I guess if Clang can be built by CMake, building OCLint should be easy afterwards.

For the auto-registration mechanism, I am lack of the knowledge about how ddl works in Windows. I need to study the differences in dynamic libraries between unix-like systems and Windows. Any good web references you can recommend?

I also need to think about a Windows box for continuous integration. You could check on http://pearland.qilongyi.com:5581 to see the current build status. (The machine for this CI is temporary, but I am using a personal url, will change it to oclint domain once we have an official buildbot.)

If you like, you could cherry pick your changes about the <ctime> inclusions, and send only these changes as a pull request now. ;)

Please forgive me and correct me if some of my thoughts sounds so different from Microsoft world.

Regards,

Longyi

--
You received this message because you are subscribed to the Google Groups "OCLint Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to oclint-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Joris Dauphin]

On Friday, July 5, 2013 12:41:42 AM UTC+2, Longyi Qi wrote:

I am open to other build systems as well, but one of CMake's advantages is cross-platform, so I would insist on having it at least be able to build with CMake. Clang has this document http://clang.llvm.org/get_started.html to build llvm/clang with CMake and Visual Studio, I guess if Clang can be built by CMake, building OCLint should be easy afterwards.

I don't suggest to use another build system.
I just says that I have some difficulties to port my configuration into CMake.
(use different generator, remove some flags, change some properties).
And as I said, it take me long time to rebuild each time I change something in CMake stuff :-/
(I know that I have a working clang built with autoconf/gcc, And it take me a day to build it :-/
and it is boring to lost a day for each compilation)

 

For the auto-registration mechanism, I am lack of the knowledge about how dll works in Windows. I need to study the differences in dynamic libraries between unix-like systems and Windows. Any good web references you can recommend?

No, sorry.

from what I can read/experiment,
It seems that:
- static vars from static lib are "duplicated" in each exe/dll.
- static vars from dll are used directly from other dll/exe
- some code from static lib (as auto registration) may be dead code stripped if the code is not reacheable from exe/dll

that means that the static var for autoregistration should be in dll.
Results::_singleton in core may be in static lib since it is used/feeded only in exe
(we may have some empty copies not used/feeded in each rules).
 

I also need to think about a Windows box for continuous integration. You could check on http://pearland.qilongyi.com:5581 to see the current build status. (The machine for this CI is temporary, but I am using a personal url, will change it to oclint domain once we have an official buildbot.)

the link doesn't work currently :-/
will try later

If you like, you could cherry pick your changes about the <ctime> inclusions, and send only these changes as a pull request now. ;)

In fact, main.cpp changes are correct too.
it is just the build config stuff which is not already working.
I will continue to try to fix that.

[Joris Dauphin]

On Friday, July 5, 2013 1:08:06 PM UTC+2, Joris Dauphin wrote:

On Friday, July 5, 2013 12:41:42 AM UTC+2, Longyi Qi wrote:

building OCLint should be easy afterwards.

I just says that I have some difficulties to port my configuration into CMake.
(use different generator, remove some flags, change some properties).

Problem solved.

(I don't know if it is the best way, but it works...)

 

I also need to think about a Windows box for continuous integration. You could check on http://pearland.qilongyi.com:5581 to see the current build status.

 

tested, it seems good.

If you like, you could cherry pick your changes about the <ctime> inclusions, and send only these changes as a pull request now. ;)

 

I will add the "full" :-) pull request as soon as possible.

Then I will port my rules from my personal appli to oclint :-)

[Longyi Qi]

Perfect, thank you, Joris. The changes on github look great! Now, I hope you could help me get started with Windows dev env, ;) Please find my comments on github.

Thanks,

Longyi

--

[Longyi Qi]

Hi Joris,

I tried to set up my Windows environment yesterday, but I don't think I am on the right track.

I got the Visual Studio 2008 installed, and I thought a cygwin might help. I was able to have cmake, git, svn configured and recognized in cmd. However, cmake couldn't find the correct C/C++ compiler. But I thought Visual Studio should comes with C++ compiler, is that right?

May I know your dependency toolset? Am I missing something?

Thanks,

Longyi

[Joris Dauphin]

On Tuesday, July 9, 2013 3:12:28 PM UTC+2, Longyi Qi wrote:

Hi Joris,

I tried to set up my Windows environment yesterday, but I don't think I am on the right track.

I got the Visual Studio 2008 installed, and I thought a cygwin might help. I was able to have cmake, git, svn configured and recognized in cmd. However, cmake couldn't find the correct C/C++ compiler.

Generally, to be able to use command with visual, you have to use a script to setup some environment variables

-- 8< my Visual Studio Command Prompt (2010) link 8< --

%comspec% /k ""C:\Program Files\Microsoft Visual Studio 10.0\VC\vcvarsall.bat"" x86

-- >8 -- 

But I thought Visual Studio should comes with C++ compiler, is that right?

Visual comes with a C++ compiler (named CL.exe)

With only ./cmake

it failed too to find the msvc compiler (I didn't use Visual Studio Command prompt though)

but with cmake -G "Visual Studio 9 2008"

it successfully generates visual solution.

 

May I know your dependency toolset? Am I missing something?

I personally use MinGW with MSYS (So I use -G "MSYS Makefiles" for cmake)

I have also Visual Studio 2010 Express (which I don't use for oclint).

[Longyi Qi]

Thanks Joris for your help. I am working on MinGW now, making progress, but looks like it has some conflits with the Cygwin I have installed. CMake can locate the correct MinGW compiler, but couldn't configure llvm/clang properly. Do you know if there is a way that I can launch MSYS console with Cygwin disabled?

Another question, after this is compiled within a MSYS environment, do end users require to have the same environment set up in order to use the tool? Could you help me with a list of runtime dependencies and compilation dependencies?

Best regards,

Longyi

[Joris Dauphin]

On Wednesday, July 10, 2013 3:35:46 PM UTC+2, Longyi Qi wrote:

Thanks Joris for your help. I am working on MinGW now, making progress, but looks like it has some conflits with the Cygwin I have installed.

I don't have Cygwin, but I know they conflict each other.

CMake can locate the correct MinGW compiler, but couldn't configure llvm/clang properly. Do you know if there is a way that I can launch MSYS console with Cygwin disabled?

I would say, make sure that Cygwin is _not_ in your path.

 

Another question, after this is compiled within a MSYS environment, do end users require to have the same environment set up in order to use the tool? Could you help me with a list of runtime dependencies and compilation dependencies?

By a quick search I found that:

http://stackoverflow.com/questions/6298297/mingw-build-depends-on-dll-not-native-to-windows

So, we may have no dependencies.

[Longyi Qi]

Cool, looks like removing cygwin out of path resolves the conflicts.

Now, move on to the next problem :)

I got the following error when build the compiler-rt, and refer to a discussion on llvm/clang mailing list https://groups.google.com/forum/#!topic/llvm-dev/wuNlxOzuxpg. How did you come through this?

[ 62%] Building C object projects/compiler-rt/lib/CMakeFiles/clang_rt.i386.dir/divxc3.c.obj

[ 62%] Building C object projects/compiler-rt/lib/CMakeFiles/clang_rt.i386.dir/enable_execute_stack.c.obj

c:/oclint/oclint/llvm/projects/compiler-rt/lib/enable_execute_stack.c:13:22: fatal error: sys/mman.h: No such file or directory

compilation terminated.

make[2]: *** [projects/compiler-rt/lib/CMakeFiles/clang_rt.i386.dir/enable_execute_stack.c.obj] Error 1

make[1]: *** [projects/compiler-rt/lib/CMakeFiles/clang_rt.i386.dir/all] Error 2

make: *** [all] Error 2

Thanks,

Longyi

--

[Joris Dauphin]

On Wednesday, July 10, 2013 9:57:10 PM UTC+2, Longyi Qi wrote:

Now, move on to the next problem :)

I got the following error when build the compiler-rt, and refer to a discussion on llvm/clang mailing list https://groups.google.com/forum/#!topic/llvm-dev/wuNlxOzuxpg. How did you come through this?

I simply didn't compile it (as they suggest in your link):

First time, I had an too old CMake (just for compiler-rt), and compilation succeeded without this compound.

After a cmake upgrade, I made:

make edit_cache

and uncheck compiler-rt

[Longyi Qi]

Alright, I can try the same thing. But without compiler-rt compiled, did you by any chance run tests on Windows?

For compiling oclint core, do you keep using mingw gcc compiler or you also switch to use clang?

Thanks,

Longyi

[Joris Dauphin]

On Thursday, July 11, 2013 5:14:43 AM UTC+2, Longyi Qi wrote:

Alright, I can try the same thing. But without compiler-rt compiled, did you by any chance run tests on Windows?

Test runs under windows without compiler-rt.

In fact, I think we use some useless llvm libs currently:

in

 LLVM_MAP_COMPONENTS_TO_LIBRARIES(REQ_LLVM_LIBRARIES ...)

most of the lib are for specific target, and I think they could be removed. (but it is out topic)

For compiling oclint core, do you keep using mingw gcc compiler or you also switch to use clang?

 

I always use the same compiler to build executable with its libs.

in that case g++.

I don't think g++ ABI is compatible with the clang one.

(the g++ ABI is generally incompatible between version (at least for minGW)).

If you want to use clang, I suggest to (re)build llvm/clang with clang first

and then oclint...

[Longyi Qi]

I agree there are useless libs there, would you help clean them? Or feel free to create a github issue for this matter.

And I made big progress on Windows build. I can now compile and test oclint-core module, and ready to move on to the rest of the modules. But when I build google mock/test, I had some problems with the pthread, I ended up with manually changing the configuration to disable pthread at all. Did you have the same issue? May I know how you handled this? Or you have pthread for windows installed?

Thanks,

Longyi

[Joris Dauphin]

On Thursday, July 11, 2013 8:22:03 PM UTC+2, Longyi Qi wrote:

I agree there are useless libs there, would you help clean them? Or feel free to create a github issue for this matter.

Ok, will add an issue for these. 

 

And I made big progress on Windows build. I can now compile and test oclint-core module, and ready to move on to the rest of the modules.

:-)

 

But when I build google mock/test, I had some problems with the pthread, I ended up with manually changing the configuration to disable pthread at all. Did you have the same issue? May I know how you handled this? Or you have pthread for windows installed?

I disabled all thread stuff of gtest too.

[Joris Dauphin]

Any progress or issues ?

[Longyi Qi]

Sorry for my lack of responsiveness. 

First, I have compiled the project entirely and run the tool as the screenshot attached. Thank you for your effort. Frankly, I never thought oclint will be appended with .exe, and you make the magic happen!

I have spent some time with automating the build process, but the progress is so disappointing due to I am not familiar with Windows. Do you think you would like to help on this? If Windows is supported, then we definitely need the automated scripts and CI box to ensure the stability of the project.

Many thanks, this is a big milestone to the tool.

Longyi

On Wed, Jul 17, 2013 at 12:29 PM, Joris Dauphin <joris....@gmail.com> wrote:

Any progress or issues ?

[Joris Dauphin]

On Friday, July 19, 2013 12:06:25 AM UTC+2, Longyi Qi wrote:

Sorry for my lack of responsiveness. 

First, I have compiled the project entirely and run the tool as the screenshot attached.

Thank you for your effort. Frankly, I never thought oclint will be appended with .exe, and you make the magic happen!

 

:-)

I have spent some time with automating the build process, but the progress is so disappointing due to I am not familiar with Windows. Do you think you would like to help on this? If Windows is supported, then we definitely need the automated scripts and CI box to ensure the stability of the project.

 

I can help,

but I don't see what you want me to do :-/

Explain, and I will try to help.

[Longyi Qi]

Glad to hear you are willing to help, thank you in advance.

What we want to achieve eventually is to automate the entire build process. For example, now, if a user wants to build the tool in Linux, he or she just need to type three commands, and the scripts will handle the rest. Similarly, we would like to have something like this for Windows. This can help us set up a continuous integration server that always checks the stability of the codebase whenever someone makes modifications to the code, this also ensures the future changes can be guaranteed to work on Windows machine.

I also recommend you take a look at the current oclint-scripts folder, it has all the scripts for unix-like systems. Even though the goal is the same, but I don't think we need to follow the same thing for Windows. You can create a new folder, maybe oclint-mingw-scripts, and put your scripts there.

Feel free to suggest new ideas.

Thanks,

Longyi

[Joris Dauphin]

On Saturday, July 20, 2013 12:04:03 AM UTC+2, Longyi Qi wrote:

What we want to achieve eventually is to automate the entire build process.

I also recommend you take a look at the current oclint-scripts folder, it has all the scripts for unix-like systems.

Even though the goal is the same, but I don't think we need to follow the same thing for Windows.

You can create a new folder, maybe oclint-mingw-scripts, and put your scripts there.

I will commit soon (simple) mingw scripts in that folder.

Feel free to suggest new ideas.

I'm not familiar with CMake,

but why don't you have a CmakeList.txt in oclint root to "generate" all oclint stuff ?

[Longyi Qi]

I don't know if having root CMakeLists.txt in root folder provides any extra values. The reason behind current modularization is for reuse purposes. This project is derived from a research project. Modules, like metrics and core, are used by other projects.

[Joris Dauphin]

On Saturday, July 20, 2013 10:40:47 PM UTC+2, Longyi Qi wrote:

I don't know if having root CMakeLists.txt in root folder provides any extra values. The reason behind current modularization is for reuse purposes.

It should enable some factorization in scripts, and I don't suggest to remove the ones in submodule.

(So it doesn't break modularization).

But if you have to keep the module scripts, I may understand

 

This project is derived from a research project. Modules, like metrics and core, are used by other projects.

So, I understand more why it is like that.

To return on the windows subject,

I have committed some changes in oclint-scripts to enable compilation under minGW of oclint modules.

Before I continue, I want to have your feed back.

Here is the missing stuff

TodoList:

- buildClang.sh : mingw + disable compiler-rt

- buildGoogleTest.sh : mingw + disable thread

- dogfooding: should be OK

- test*: mingw,

 but if I remember correctly, the coverage reporting is problematic under minGW :/ (but generation is ok).

(the paths are incompatible in perl script).

 I read that reporting should work under CygWin.

And coverage should be identical between win, *nix,

so it may be unnecessary to enable it under windows.

Do I miss something ?

[Longyi Qi]

Hi Joris,

I have taken a look at the latest change on github, and they are great, thank you for your help.

For disabling compiler-rt, I would suggest you modify the checkoutLLVMClang.sh script, and simply don’t check out the compiler-rt code, it might be easier for you?

Don’t worry about the code coverage for now, we are on the direction of migrating to C++11, and will use llvm-cov to replace gcov. 

And feel free to add new scripts when necessary, I think probably disabling pthread in google test may need some work.

May I also know the C++11 support status in MinGW?

Thanks,

Longyi

[Joris Dauphin]

On Monday, July 22, 2013 3:04:49 PM UTC+2, Longyi Qi wrote:

Hi Joris,

I have taken a look at the latest change on github, and they are great, thank you for your help.

Ok, So I continue :) 

 

For disabling compiler-rt, I would suggest you modify the checkoutLLVMClang.sh script, and simply don’t check out the compiler-rt code, it might be easier for you?

I think we can pass some variable to CMake to "disable" it. 

Don’t worry about the code coverage for now, we are on the direction of migrating to C++11, and will use llvm-cov to replace gcov. 

The problem is not gcov, but lcov and genhtml...

I will ignore that stuff.

And feel free to add new scripts when necessary, I think probably disabling pthread in google test may need some work.

I think it can be solved too with some variable passed to CMake

 

May I also know the C++11 support status in MinGW?

 

I don't use c++11 yet,

but simple test cases work.

I don't know if g++ supports all c++11 features.

[Longyi Qi]

On Mon, Jul 22, 2013 at 8:37 AM, Joris Dauphin <joris....@gmail.com> wrote:

On Monday, July 22, 2013 3:04:49 PM UTC+2, Longyi Qi wrote:

Hi Joris,

I have taken a look at the latest change on github, and they are great, thank you for your help.

Ok, So I continue :) 

 

For disabling compiler-rt, I would suggest you modify the checkoutLLVMClang.sh script, and simply don’t check out the compiler-rt code, it might be easier for you?

I think we can pass some variable to CMake to "disable" it. 

Yep, which you think is easier for you. 

 

Don’t worry about the code coverage for now, we are on the direction of migrating to C++11, and will use llvm-cov to replace gcov. 

The problem is not gcov, but lcov and genhtml...

I will ignore that stuff.

You are right. But I would like to give llvm-cov a try, to reduce the development dependencies. 

 

And feel free to add new scripts when necessary, I think probably disabling pthread in google test may need some work.

I think it can be solved too with some variable passed to CMake

 

May I also know the C++11 support status in MinGW?

 

I don't use c++11 yet,

but simple test cases work.

I don't know if g++ supports all c++11 features.

The mingw-gcc version I have is 4.7.2, so I expect C++11 is supported.

[Joris Dauphin]

last commit should fix scripts for clang, googletest.

test scripts are "fixed", except lcov calls.

CMake counter part is not ready now for TestBuild.

Back to CMake issues :/ (It works in my alternative build system (which I know) though)

- link to "useless" ${PROFILE_RT_LIBS}

- "--coverage" is only in compiler flags and not in linker flags.

- *.dll is not generated along *.exe, so I have to copy them manually.

Can you check that ${PROFILE_RT_LIBS} is really useful.

my local "fixes" for these problems are mostly hacks currently (and so, not committed).

May I drop code coverage completely for windows ?

[Longyi Qi]

Thank you for your great help, Joris. Overall speaking, I don't mind dropping code coverage completely for Windows if it is a block and requires large effort. Hope it's a relief. :-) Two comments below.

On Mon, Jul 22, 2013 at 1:01 PM, Joris Dauphin <joris....@gmail.com> wrote:

last commit should fix scripts for clang, googletest.

test scripts are "fixed", except lcov calls.

CMake counter part is not ready now for TestBuild.

Back to CMake issues :/ (It works in my alternative build system (which I know) though)

Is this a problem if we completely drop the code coverage for Windows?

 

- link to "useless" ${PROFILE_RT_LIBS}

- "--coverage" is only in compiler flags and not in linker flags.

- *.dll is not generated along *.exe, so I have to copy them manually.

Well, PROFILE_RT_LIBS is quite useful (again) for code coverage, and these libraries are generated by compiler-rt. So, probably we could discard this part for windows build. 

 

Can you check that ${PROFILE_RT_LIBS} is really useful.

my local "fixes" for these problems are mostly hacks currently (and so, not committed).

May I drop code coverage completely for windows ?

[Joris Dauphin]

On Monday, July 22, 2013 8:31:47 PM UTC+2, Longyi Qi wrote:

Thank you for your great help, Joris. Overall speaking, I don't mind dropping code coverage completely for Windows if it is a block and requires large effort. Hope it's a relief. :-) Two comments below.

Ok, with some search, I understand what is ${PROFILE_RT_LIBS}:

It is mostly the equivalent of -lgcov.

Instead of specifying library, we can use the flag "--coverage" (which works with gcc, and it seems that clang support it too)

So I would say it is more portable (but I can't test it)

Can you check if that work for you ?

I think I have a solution for code coverage for windows :)

(except lcov part: but we can use lcov on generated data produced by win).

[Longyi Qi]

If I remember correctly, the `--coverage` won't help if Clang is used as compiler. That's why we ended up with that complex CMake configurations, just to find the correct "gcov" replacement library based on various systems and architectures.

I guess what we can try instead is to keep `--coverage` for the compilation flag, and `SET(PROFILE_RT_LIBS gcov)` to ask linker use gcov.

Longyi

[Joris Dauphin]

On Monday, July 22, 2013 11:31:42 PM UTC+2, Longyi Qi wrote:

If I remember correctly, the `--coverage` won't help if Clang is used as compiler. That's why we ended up with that complex CMake configurations, just to find the correct "gcov" replacement library based on various systems and architectures.

Hope that clang will fix that.

 

I guess what we can try instead is to keep `--coverage` for the compilation flag, and `SET(PROFILE_RT_LIBS gcov)` to ask linker use gcov.

mingw/gcc part fixed :)

(I have added '--coverage' and not 'gcov').

The last issue is the "missing dll" when launching exe.

Possibilities I see:

- copy dll along the executables

- Add dll path into the PATH

I prefer to copy dll...

Will see how to do it in CMake.

[Longyi Qi]

On Tue, Jul 23, 2013 at 11:16 AM, Joris Dauphin <joris....@gmail.com> wrote:

On Monday, July 22, 2013 11:31:42 PM UTC+2, Longyi Qi wrote:

If I remember correctly, the `--coverage` won't help if Clang is used as compiler. That's why we ended up with that complex CMake configurations, just to find the correct "gcov" replacement library based on various systems and architectures.

Hope that clang will fix that.

 

I guess what we can try instead is to keep `--coverage` for the compilation flag, and `SET(PROFILE_RT_LIBS gcov)` to ask linker use gcov.

mingw/gcc part fixed :)

(I have added '--coverage' and not 'gcov').

In theory, should behave the same, adding `--coverage` to linker will be converted to `-lgcov` for gcc compilers.

 

The last issue is the "missing dll" when launching exe.

Possibilities I see:

- copy dll along the executables

- Add dll path into the PATH

I prefer to copy dll...

Will see how to do it in CMake.

Is it possible to modify `buildRelease.sh` file and use `COPY` DOS command? Or create a new script that simply copy all DLLs and EXEs into a proper location? 

[Joris Dauphin]

On Tuesday, July 23, 2013 6:54:36 PM UTC+2, Longyi Qi wrote:

On Tue, Jul 23, 2013 at 11:16 AM, Joris Dauphin <joris....@gmail.com> wrote:

On Monday, July 22, 2013 11:31:42 PM UTC+2, Longyi Qi wrote:

(I have added '--coverage' and not 'gcov').

In theory, should behave the same, adding `--coverage` to linker will be converted to `-lgcov` for gcc compilers.

I find --coverage clearer.

 

The last issue is the "missing dll" when launching exe.

Possibilities I see:

- copy dll along the executables

- Add dll path into the PATH

I prefer to copy dll...

Will see how to do it in CMake.

Is it possible to modify `buildRelease.sh` file and use `COPY` DOS command? Or create a new script that simply copy all DLLs and EXEs into a proper location? 

Done in CMake 

(why use DOS copy instead of msys cp ? ).

So, now, everything should work with minGW (except the html coverage report).

Tell me if I miss something.

My TODO list for this task is empty.

Once integrated, I will work on some rules I have in the pipe.

[Longyi Qi]

Hi Joris, did you try building llvm/clang on Windows these days? I was in a stage that almost everything works, but when I tried to build everything again from scratch, ld.exe always stop responding when it comes to 97% with libclang.dll with a returned code 5. Any ideas? Thanks.

--

[Joris Dauphin]

On Thursday, August 1, 2013 1:12:53 AM UTC+2, Longyi Qi wrote:

Hi Joris, did you try building llvm/clang on Windows these days?

Which version ?

I use 3.3 here without any problem.

And i had now problem when I built/used a trunk version a month ago.

 

I was in a stage that almost everything works, but when I tried to build everything again from scratch, ld.exe always stop responding when it comes to 97% with libclang.dll with a returned code 5. Any ideas? Thanks.

any error message ?

Does the dll already exist and is in use (and so write protected) ? ...

I may try on my side when you give me version or revision.

[Longyi Qi]

I pulled the latest svn version of clang.

Clang: svn rev187497.

OS: MINGW32_NT-6.1 LQI-PC 1.0.18(0.48/3/2) 2012-11-21 22:34 i686 Msys

mingw32-gcc: 4.7.2-1

The error I got was-

[ 97%] Built target clangFormat

Linking CXX shared library ../../../../bin/liblibclang.dll

collect2.exe: error: ld returned 5 exit status

make[2]: *** [bin/liblibclang.dll] Error 1

make[1]: *** [tools/clang/tools/libclang/CMakeFiles/libclang.dir/all] Error 2

make: *** [all] Error 2

And a screenshot is attached. Thanks for helping me out.

Longyi

--

[Joris Dauphin]

On Thursday, August 1, 2013 5:43:49 PM UTC+2, Longyi Qi wrote:

I pulled the latest svn version of clang.

Clang: svn rev187497.

OS: MINGW32_NT-6.1 LQI-PC 1.0.18(0.48/3/2) 2012-11-21 22:34 i686 Msys

mingw32-gcc: 4.7.2-1

 Just tested (with gcc (tdm-1) 4.7.1 and same version of llvm/clang) and it works (except the missing $EXTRA to disable compiler-rt (fixed in trunk)).

Note also that I deleted llvm build directories before launching the script.

With a quick search about ld error you mention,

I saw that when it arrives in other project, they tell to do a "real" clean and then recompile.

[Longyi Qi]

What I have done was to shift+delete the build folder completely, but still got the same ld error.

I was wondering if the compiler versions make the differences. Any idea on how to use an older version of gcc?

Thanks,

Longyi

[Joris Dauphin]

On Sunday, August 4, 2013 8:13:25 AM UTC+2, Longyi Qi wrote:

What I have done was to shift+delete the build folder completely, but still got the same ld error.

Maybe ask to llvm/clang dev...

Does it happen too if you use the release tag version (3.3) ?

Are you sure that llvm and clang are at the same version ?

I was wondering if the compiler versions make the differences. Any idea on how to use an older version of gcc?

I'm using the mingw that came with Codelite 5.2.

For msys, you have to change "$ROOT_OF_MSYS"\msys\1.0\etc\fstab to point to the wanted mingw folder.

[Longyi Qi]

Joris,

Thanks a lot for baring with me.

Good news: gcc 4.7.1 successfully compiles the latest llvm/clang codebase for me.

New issue: as shown in the screenshot attached, I think my compiler treats itself as a Linux one. 

Another interesting thing I have noticed is, when I try to compile oclint-driver/main.cpp, it cannot find dlfcn.h, where it should look for windows.h instead.

[100%] Building CXX object CMakeFiles/oclint-0.8.dir/main.cpp.obj

c:/oclint/oclint/oclint-driver/main.cpp:4:20: fatal error: dlfcn.h: No such file or directory

compilation terminated.

make[2]: *** [CMakeFiles/oclint-0.8.dir/main.cpp.obj] Error 1

make[1]: *** [CMakeFiles/oclint-0.8.dir/all] Error 2

make: *** [all] Error 2

I also have my merged branch with the latest c++11 changes at https://github.com/lqi/oclint/tree/windows for you. 

Thanks for the help,

Longyi

[Joris Dauphin]

On Tuesday, August 6, 2013 8:00:38 PM UTC+2, Longyi Qi wrote:

Joris,

Thanks a lot for baring with me.

Good news: gcc 4.7.1 successfully compiles the latest llvm/clang codebase for me.

:-) 

 

New issue: as shown in the screenshot attached, I think my compiler treats itself as a Linux one. 

I also have my merged branch with the latest c++11 changes at https://github.com/lqi/oclint/tree/windows for you. 

I got the same error after check-out your branch...

Investigating on my side.

(maybe with c++11 we need some more define/things to be portable). 

Another interesting thing I have noticed is, when I try to compile oclint-driver/main.cpp, it cannot find dlfcn.h, where it should look for windows.h instead.

Not tested yet.

[Joris Dauphin]

On Wednesday, August 7, 2013 3:05:50 PM UTC+2, Joris Dauphin wrote:

On Tuesday, August 6, 2013 8:00:38 PM UTC+2, Longyi Qi wrote: 

New issue: as shown in the screenshot attached, I think my compiler treats itself as a Linux one. 

I also have my merged branch with the latest c++11 changes at https://github.com/lqi/oclint/tree/windows for you. 

I got the same error after check-out your branch...

Investigating on my side.

(maybe with c++11 we need some more define/things to be portable). 

I found:

http://stackoverflow.com/questions/6312151/stricmp-with-mingw-and-c0x-not-existent

which talk about googletest...

Possible solutions:

* use -std=gnu++11 instead of c++11

* undef __STRICT_ANSI__

* wait that googletest fix that

I add ADD_DEFINITIONS(-U__STRICT_ANSI__) before BUILD_TEST(CanaryTest) in oclint-core/test/CMakeList.txt

and it compiles.

[Joris Dauphin]

On Tuesday, August 6, 2013 8:00:38 PM UTC+2, Longyi Qi wrote: 

Another interesting thing I have noticed is, when I try to compile oclint-driver/main.cpp, it cannot find dlfcn.h, where it should look for windows.h instead.

I got the same thing,

Fixed in my branch using _WIN32 instead of WIN32.

I don't know how it worked before.

These two problems should be solved now (the first one isn't commited).

[Longyi Qi]

Great help, Joris. Thank you!

I prefer using gnu++11 for MinGW environment if you agree with me, too.

https://github.com/lqi/oclint/commit/473136b3ec1316b99eef3b34838e1605d362a842 

Using _WIN32 is not quite necessary if we use gnu++11, however, I think we are good for now. In the future, we probably need to have separate files that port to different environments.

We are at a very good track, thanks Joris. I have few other questions for you, but they are pretty minor now-

https://github.com/lqi/oclint/blob/windows/oclint-rules/test/CMakeLists.txt#L21

This works for oclint-core module, however, it cannot find OCLintRuleSet target here in oclint-rules module.

https://github.com/lqi/oclint/blob/windows/oclint-rules/rules/CMakeLists.txt#L9

https://github.com/lqi/oclint/blob/windows/oclint-reporters/reporters/CMakeLists.txt#L9

We have forced the libraries to be stored in rules.dl and reporters.dl folders respectively. But it is not working here, not sure if it's working on your machine. We have scripts to copy DLLs from these two folders and bundle them together with the release, https://github.com/lqi/oclint/blob/windows/oclint-scripts/buildRelease.sh#L29.

Look forward to your suggestions.

Thanks,

Longyi

--

[Joris Dauphin]

On Friday, August 9, 2013 1:48:48 AM UTC+2, Longyi Qi wrote:

Great help, Joris. Thank you!

I prefer using gnu++11 for MinGW environment if you agree with me, too.

https://github.com/lqi/oclint/commit/473136b3ec1316b99eef3b34838e1605d362a842 

I found your solution cleaner but less local than mine. I'm ok with that.

I suggest to add a comment to tell that googleTest doesn't support c++11 with mingw, so we use gnu++11 instead.

We are at a very good track, thanks Joris. I have few other questions for you, but they are pretty minor now-

https://github.com/lqi/oclint/blob/windows/oclint-rules/test/CMakeLists.txt#L21

This works for oclint-core module, however, it cannot find OCLintRuleSet target here in oclint-rules module.

I changed from $<TARGET_FILE:OCLintRuleSet> to "${OCLINT_BUILD_DIR}/bin/libOCLintRuleSet.dll"

(committed in my branch)

You may prefer "*.dll"

https://github.com/lqi/oclint/blob/windows/oclint-rules/rules/CMakeLists.txt#L9

https://github.com/lqi/oclint/blob/windows/oclint-reporters/reporters/CMakeLists.txt#L9

We have forced the libraries to be stored in rules.dl and reporters.dl folders respectively. But it is not working here, not sure if it's working on your machine. We have scripts to copy DLLs from these two folders and bundle them together with the release, https://github.com/lqi/oclint/blob/windows/oclint-scripts/buildRelease.sh#L29.

Look forward to your suggestions.

I'll look at it.

[Joris Dauphin]

It seems that the "correct" properties for dll is

RUNTIME_OUTPUT_DIRECTORY ${PROJECT_BINARY_DIR}/reporters.dl

(commited in my branch)

 

[Longyi Qi]

These helps are great. I can automate the entire test and build process without any human interfere. In addition, all tests pass. This is fantastic. Thank you, Joris.

I believe this should be the last issue (I hope so) - I am trying to use oclint to analyze its own codebase (and other codebase). However, when the rule set is large, I couldn't load every rules into the main program. 

For one example:

lqi@LQI-PC /c/oclint/oclint/oclint-core

$ ../build/oclint-release/bin/oclint.exe lib/Results.cpp

8

oclint: error: cannot open dynamic library: c:\oclint\oclint\build\oclint-release\bin/../lib/oclint/rules/libDeadCodeRule.dll

Deal code rule is actually okay if I load it individually or together with a small rule set.

Not sure if number of DLLs or the size of rules being loaded matters. So, I tried to print out the error code, which turns out to be `8`. I looked it up online, and followed suggestions like http://stackoverflow.com/questions/507853/system-error-code-8-not-enough-storage-is-available-to-process-this-command to change my SharedSection to 1024,20480,1024, but no luck in the end.

I hope you could help again on this. And you can check out my latest code from https://github.com/lqi/oclint/tree/windows, hope it could ease your work. Now, you can simply run buildAll.sh, and it should build everything, and organize the bundle in place at build/oclint-release folder.

Thanks,

Longyi

--

[Joris Dauphin]

On Saturday, August 10, 2013 10:42:07 PM UTC+2, Longyi Qi wrote:

These helps are great. I can automate the entire test and build process without any human interfere. In addition, all tests pass. This is fantastic. Thank you, Joris.

nice :)

 

I believe this should be the last issue (I hope so) - I am trying to use oclint to analyze its own codebase (and other codebase). However, when the rule set is large, I couldn't load every rules into the main program. 

For one example:

lqi@LQI-PC /c/oclint/oclint/oclint-core

$ ../build/oclint-release/bin/oclint.exe lib/Results.cpp

Work fine here (In release mode)

Not sure if number of DLLs or the size of rules being loaded matters.

Since in release it works for me, I would say that size may be a problem...

In release, the total size of rules is around 278 Mb (+ 4Mb with reporters + 15Mb with exe -> total ~300Mb).

Not yet compiled in debug, but the size should be a lot bigger :-/ . 

What are the size in *nix/mac in debug/release ?

And you can check out my latest code from https://github.com/lqi/oclint/tree/windows, hope it could ease your work.

I see that you use switch on uname now.

which was the error with *nix/mac ?

[Longyi Qi]

On Sat, Aug 10, 2013 at 7:17 PM, Joris Dauphin <joris....@gmail.com> wrote:

On Saturday, August 10, 2013 10:42:07 PM UTC+2, Longyi Qi wrote:

These helps are great. I can automate the entire test and build process without any human interfere. In addition, all tests pass. This is fantastic. Thank you, Joris.

nice :)

 

I believe this should be the last issue (I hope so) - I am trying to use oclint to analyze its own codebase (and other codebase). However, when the rule set is large, I couldn't load every rules into the main program. 

For one example:

lqi@LQI-PC /c/oclint/oclint/oclint-core

$ ../build/oclint-release/bin/oclint.exe lib/Results.cpp

Work fine here (In release mode)

Not sure if number of DLLs or the size of rules being loaded matters.

Since in release it works for me, I would say that size may be a problem...

In release, the total size of rules is around 278 Mb (+ 4Mb with reporters + 15Mb with exe -> total ~300Mb).

Not yet compiled in debug, but the size should be a lot bigger :-/ . 

What are the size in *nix/mac in debug/release ?

This makes sense to me, current oclint-release folder takes more than 7Gb in debug mode, which is crazy. I am building everything in release mode now, and see if I can load all rules. But I still hope it would hope on debug mode, eventually we would like to have code coverage work on MinGW as well.

Mac debug 68Mb, release 7.4Mb

Linux debug 217Mb, release 11Mb

 

And you can check out my latest code from https://github.com/lqi/oclint/tree/windows, hope it could ease your work.

I see that you use switch on uname now.

which was the error with *nix/mac ?

Everything works in Mac and Linux with bash, but they don't work in Debian nor Ubuntu since [[ is not supported in dash, meanwhile, the =~ is not supported either in dash.

[Longyi Qi]

Everything in release mode works fine, which is great. 

This is already good to merge, but pardon me to be a little bit greedy, is there a way to make debug model also works? I am okay with modifying the Windows system configuration.

In addition, I was wondering if we should use IF (MINGW) instead of IF (${CMAKE_SYSTEM_NAME} MATCHES "Win") in CMakeLists.txt? MINGW is shorter, and it is more precise. Let me know what you think.

Many thanks,

Longyi

[Joris Dauphin]

On Monday, August 12, 2013 3:08:23 PM UTC+2, Longyi Qi wrote:

Everything in release mode works fine, which is great. 

This is already good to merge,

:-)

Nice to see the job merged.

but pardon me to be a little bit greedy, is there a way to make debug model also works? I am okay with modifying the Windows system configuration.

I am investigating to reduce dll size...

With my build system, I have a little smaller size (~80%), but not enough...

We can also strip symbol to gain a little more (but it is not compatible with debug...).

It seems that "rules*.dll" include "some" llvm/clang symbols...

I wonder how to move those "common" symbols outside of the "rules*.dll"

In addition, I was wondering if we should use IF (MINGW) instead of IF (${CMAKE_SYSTEM_NAME} MATCHES "Win") in CMakeLists.txt? MINGW is shorter, and it is more precise. Let me know what you think.

I don't know if what we have done works also with msvc... 

Anyway, I'm ok with the use of MINGW.

[Joris Dauphin]

On Monday, August 12, 2013 4:32:50 PM UTC+2, Joris Dauphin wrote:

On Monday, August 12, 2013 3:08:23 PM UTC+2, Longyi Qi wrote:

but pardon me to be a little bit greedy, is there a way to make debug model also works? I am okay with modifying the Windows system configuration.

I am investigating to reduce dll size...

With my build system, I have a little smaller size (~80%), but not enough...

We can also strip symbol to gain a little more (but it is not compatible with debug...).

It seems that "rules*.dll" include "some" llvm/clang symbols...

I wonder how to move those "common" symbols outside of the "rules*.dll"

By creating a dll for oclint-rules/include/oclint/*.h (so create cpp files)

I succeeded to have a total dll size lesser than 30Mb in release (and half size once stripped).

My way to do it is currently a little hacky and I have problems to compile 5 rules with duplicated/undefined functions.

So, if the ratio for debug is the same, the result size should be around 700Mb...

I hope it would be sufficient.

[Longyi Qi]

This is significant. I am interested in hearing how you achieve this. I also would like to know which rules are blocking you. If there are code changes to automate this process, I guess you could attach a patch to the email. Thank you!

[Joris Dauphin]

On Wednesday, August 14, 2013 4:36:51 PM UTC+2, Longyi Qi wrote:

This is significant. I am interested in hearing how you achieve this.

I add -s linker flag in release (to strip symbols).

I turn oclint libs to be dynamic (SHARED).

I add a library for AbstractAST* file in rules/lib/

Under windows, I merge all lib from rules/lib/ (may also be done for other OS ?)

(I find this part not very clean, but I don't know what you want to do for other OS

 and how to handle properly sub directories for source in CMake).

I also would like to know which rules are blocking you. If there are code changes to automate this process, I guess you could attach a patch to the email. Thank you!

The rules that blocked me was in size/ and unused/, but now, they links correctly :-)

Attached my current patch.

(name should be improved, buildRelease.sh should copy the new dll)

Total rules dll size (in release) is around 15 Mb :-)

[Joris Dauphin]

Have you found time to watch my patch ? Any feedback ?

[Longyi Qi]

Hi Joris,

Sorry for the lack of response recently. I was travelling to California for the past a few days. I am about the set up the CI machine for Windows, and merge the code soon.

Longyi

On Mon, Aug 26, 2013 at 5:31 AM, Joris Dauphin <joris....@gmail.com> wrote:

Have you found time to watch my patch ? Any feedback ?

--

[Joris Dauphin]

On Wednesday, August 28, 2013 1:39:55 AM UTC+2, Longyi Qi wrote:

Hi Joris,

Sorry for the lack of response recently. I was travelling to California for the past a few days. I am about the set up the CI machine for Windows, and merge the code soon.

Nice to see oclint for windows in trunk :-)

Don't forget to review the patch to optimize dll size in previous post.

Attached patch on this post is to fix compilation on windows with recent change.

[Longyi Qi]

Great, thanks Joris. Would you help create two separate pull requests for these two patches respectively?

BTW, I guess the email address you are using for git's user.email is not added to your github account, so that github cannot recognize you as a project contributor....

--

[Joris Dauphin]

On Monday, September 9, 2013 8:33:20 PM UTC+2, Longyi Qi wrote:

Great, thanks Joris. Would you help create two separate pull requests for these two patches respectively?

BTW, I guess the email address you are using for git's user.email is not added to your github account, so that github cannot recognize you as a project contributor....

Ok, I work on these.

 

[Joris Dauphin]

On Tuesday, September 10, 2013 10:28:47 AM UTC+2, Joris Dauphin wrote:

On Monday, September 9, 2013 8:33:20 PM UTC+2, Longyi Qi wrote:

Great, thanks Joris. Would you help create two separate pull requests for these two patches respectively?

I have create 1 pull request (to fi compilation).

I don't succeed to create the second the correct way (it appears in my own repository :) )

but you can see the change in my repository there reduce_dll_size

BTW, I guess the email address you are using for git's user.email is not added to your github account, so that github cannot recognize you as a project contributor....

Should be fixed.

[Longyi Qi]

Cool, let's work together to have the first pull request merged, then we can start with the second one. 

Thanks,

Longyi