Building plan of C++ SDK on windows platform may not be correct

[KarelG]

Hi all


I'm back as some people may remember me. I had some issues when installing the requirements (discussion thread). After studying the cmake process and the documentation, i finally have a working set of requirements.
I had to edit the batch command files, and not sure if you ( Kaa developers ) are aware of it, but this page needs some revision. It's not really correct.
I'm going to write a discussion thread of my experiences later, to advise/show what could be changed so that future users would be able to generate a Kaa SDK C++ simply for windows.

Currently I have 3 questions in this email.

I'm currently at step 12 of the installation page (see above link). Generating the makefiles happens without big errors, however, if you check the cmake output (see appendix1 here below), it mentions about looking for "pthread.h", but that's not suitable for Windows environment.
But pthread.h is used in UNIX environment, while Windows uses windows.h for threading.

Question1: Can I ignore it ? Or should i NOT ignore it ?

In the output, you can see that it's able to find the dependencies. However, when I issue nmake (next step) to generate nmake files, it throws errors.

D:\kaa\kaa-cpp-ep-sdk-aXrMkhHEE2BrPBehV_Vbxym2MfU\build>nmake

Microsoft (R) Program Maintenance Utility Version 14.00.23506.0
Copyright (C) Microsoft Corporation.  All rights reserved.

Scanning dependencies of target kaacpp_o
[  1%] Building CXX object CMakeFiles/kaacpp_o.dir/impl/event/registration/EndpointRegistrationManager.cpp.obj
EndpointRegistrationManager.cpp
D:\kaa\kaa-cpp-ep-sdk-aXrMkhHEE2BrPBehV_Vbxym2MfU\kaa/KaaDefaults.hpp(24): fatal error C1083: Cannot open include file: 'botan/base64.h': No such file or directory
NMAKE : fatal error U1077: 'C:\PROGRA~2\VISUAL~1.0\VC\bin\cl.exe' : return code '0x2'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Visual Studio 14.0\VC\BIN\nmake.exe"' : return code '0x2'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Visual Studio 14.0\VC\BIN\nmake.exe"' : return code '0x2'
Stop.

But the base64.h from botan isn't on the normal location, but deep in their file structure. They have probably changed the file structure. I don't know since it's my first experience with Kaa.
I have set the link to botan's include to the following path in the cmakelists.txt file:

set(BOTAN_HOME D:/kaa/kaaBuild/botan-1.11.28) <--- the location of botan.exe and botan.dll
set(BOTAN_LIBRARY D:/kaa/kaaBuild/botan-1.11.28/src/lib) <--- location of folders with .h files.

That folder contains all .h files. When digging deeper, you can find the required h file:

---> location of right file
D:\kaa\kaaBuild\botan-1.11.28\src\lib\codec\base64
---> proof
PS D:\kaa\kaaBuild\botan-1.11.28\src\lib\codec\base64> Get-ChildItem -name
base64.cpp
base64.h
info.txt

Question2: Could it be that the build tool is not correct for C++ on Windows ?
Or should i move the .h file manually to the right location ? (That would be risky ...)


Also, the link in the text "For additional cmake options, see C++ endpoint SDK" link in the install instruction page (first line above "Demo applications") points to wrong page. I am not able to find the list of cmake options.
Question3: Can someone pass me the link to the page ? (if the page exists)

Thank you for replying.
Sincerely
- kg


Appendix1: Cmake output in console

D:\kaa\kaa-cpp-ep-sdk-aXrMkhHEE2BrPBehV_Vbxym2MfU\build>cmake .. -G "NMake Makefiles" -DKAA_INSTALL_PATH="D:\kaa\kaaSdkCpp"
-- The C compiler identification is MSVC 19.0.23506.0
-- The CXX compiler identification is MSVC 19.0.23506.0
-- Check for working C compiler: C:/Program Files (x86)/Visual Studio 14.0/VC/bin/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Visual Studio 14.0/VC/bin/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Visual Studio 14.0/VC/bin/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Visual Studio 14.0/VC/bin/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
==================================
Adding avrogen exe location
CMake Warning (dev) at CMakeLists.txt:73 (if):
  Policy CMP0054 is not set: Only interpret if() arguments as variables or
  keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  Quoted variables like "MSVC" will no longer be dereferenced when the policy
  is set to NEW.  Since the policy is not set the OLD behavior will be used.
This warning is for project developers.  Use -Wno-dev to suppress it.

==================================
KAA_MAX_LOG_LEVEL=4
EVENTS ENABLED
NOTIFICATIONS ENABLED
CONFIGURATION ENABLED
LOGGING ENABLED
OPERATION_TCP_CHANNEL ENABLED
OPERATION_LONG_POLL_CHANNEL ENABLED
OPERATION_HTTP_CHANNEL ENABLED
BOOTSTRAP_HTTP_CHANNEL ENABLED
CONNECTIVITY_CHECKER ENABLED
KAA_THREADSAFE ENABLED
==================================
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
-- Boost version: 1.60.0
-- Found the following Boost libraries:
--   log
--   system
--   thread
--   date_time
--   log_setup
--   filesystem
--   regex
--   chrono
--   atomic

Looking for Avro C++ headers and libraries
-- Found Avro: D:/kaa/kaaBuild/avro-cpp-1.7.7/build.win/Debug
-- Include directories: D:/kaa/kaaBuild/avro-cpp-1.7.7/build.win/Debug
-- Libraries: D:/kaa/kaaBuild/avro-cpp-1.7.7/build.win/Debug
-- FindBotan check
-- BOTAN_HOME is not empty: "D:/kaa/kaaBuild/botan-1.11.28"
-- Looking for botan in D:/kaa/kaaBuild/botan-1.11.28
-- Include directory: D:/kaa/kaaBuild/botan-1.11.28
-- Library: D:/kaa/kaaBuild/botan-1.11.28/src/lib
-- KAA WILL BE INSTALLED TO C:/Program Files (x86)/Kaa-cpp
-- Configuring done
WARNING: Target "kaacpp" requests linking to directory "D:/kaa/kaaBuild/botan-1.11.28/src/lib".  Targets may link only to libraries.  CMake is dropping the item.
WARNING: Target "kaacpp" requests linking to directory "D:/kaa/kaaBuild/avro-cpp-1.7.7/build.win/Debug".  Targets may link only to libraries.  CMake is dropping the item.
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    KAA_INSTALL_PATH


-- Build files have been written to: D:/kaa/kaa-cpp-ep-sdk-aXrMkhHEE2BrPBehV_Vbxym2MfU/build

EndOfMail

[Denis Kimcherenko]

Hi Karel,

First of all, sorry for a long delay, but I really had no time to answer.

Let's start from your question:

1. I don't know. Did you install MinGW or Cygwin? Maybe these programs conflicts with MSVC and CMake (but it is only my assumptions!).

My build ouput looks like this:

C:\KAA_APP\thirdparty\archive\kaa-cpp-sdk\build>cmake -G "NMake Makefiles" -DKAA_MAX_LOG_LEVEL=0 -DKAA_DEBUG_ENABLED=1 -DCMAKE_INSTALL_PREFIX=%KAA_APP_THIRDPARTY_DEBUG% .. && nmake install

-- The C compiler identification is MSVC 19.0.23506.0
-- The CXX compiler identification is MSVC 19.0.23506.0

-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/cl.exe -- works

-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done

-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/cl.exe -- works

-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
==================================

CMake Warning (dev) at CMakeLists.txt:73 (if):
  Policy CMP0054 is not set: Only interpret if() arguments as variables or
  keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.


  Quoted variables like "MSVC" will no longer be dereferenced when the policy
  is set to NEW.  Since the policy is not set the OLD behavior will be used.
This warning is for project developers.  Use -Wno-dev to suppress it.


==================================

KAA_MAX_LOG_LEVEL=0

EVENTS ENABLED
NOTIFICATIONS ENABLED
CONFIGURATION ENABLED
LOGGING ENABLED
OPERATION_TCP_CHANNEL ENABLED
OPERATION_LONG_POLL_CHANNEL ENABLED
OPERATION_HTTP_CHANNEL ENABLED
BOOTSTRAP_HTTP_CHANNEL ENABLED
CONNECTIVITY_CHECKER ENABLED
KAA_THREADSAFE ENABLED
==================================

-- Boost version: 1.60.0
-- Found the following Boost libraries:
--   log
--   system
--   thread

Looking for Avro C++ headers and libraries

-- Root dir: C:\KAA_APP\thirdparty\install\debug
-- Found Avro: C:/KAA_APP/thirdparty/install/debug/lib/avrocpp.lib
-- Include directories: C:/KAA_APP/thirdparty/install/debug/include
-- Libraries: C:/KAA_APP/thirdparty/install/debug/lib/avrocpp.lib
-- FindBotan check
-- Looking for botan in C:\KAA_APP\thirdparty\install\debug
-- Include directory: C:/KAA_APP/thirdparty/install/debug/include
-- Library: C:/KAA_APP/thirdparty/install/debug/lib/botan.lib
-- Configuring done
-- Generating done
-- Build files have been written to: C:/KAA_APP/thirdparty/archive/kaa-cpp-sdk/build

Microsoft (R) Program Maintenance Utility Version 14.00.23506.0
Copyright (C) Microsoft Corporation.  All rights reserved.

.....

==============

2. I re-built Botan-1.11 by following command:

C:\KAA_APP\thirdparty\archive\Botan-1.11.28>

python configure.py --cc=msvc --cpu=i386 && nmake install

and found all heares in 

C:\KAA_APP\thirdparty\archive\Botan-1.11.28\build

So this line should be fixed. I will create the bug on this item on Kaa JIRA (on Monday).

By the way, your are free to open new feature stories and bugs as well by yourself.

==============

3. I have updated the link. As for Kaa 0.8.0, it must points to the "Build configuration for C++ endpoint SDK". 

Now this documentation is part of the Linux page, but in the future it will be better to create a separate page to describe C++ SDK build options.

==============

MY STORY

Let me tell you my story of building Kaa C++ SDK on Windows. Maybe some steps would be helpful for you.

First of all, I didn't use build_sdk_thirdparty.bat. I only pick out commads for building Avro and Botan libraries.

Why? Because I wanted to create my custom third-parties storage. See "Artifacts location" for details.

==================

Artifacts location

==================

The result what I wanted to achive after building all Kaa C++ SDK third-parties is to separate debug and release headers/libraries.

So I created two directories:

C:\KAA_APP\thirdparty\install\debug

C:\KAA_APP\thirdparty\install\release

Each of these directories has the following structure:

bin\

include\

lib\

Also I created several environment variables to have easy access to these 'release' and 'debug' directories.

Name: KAA_APP_THIRDPARTY_HOME 

Value: C:\KAA_APP\thirdparty\install

Name: KAA_APP_THIRDPARTY_DEBUG 

Value: %KAA_APP_THIRDPARTY_HOME%\debug

Name: KAA_APP_THIRDPARTY_RELEASE 

Value: %KAA_APP_THIRDPARTY_HOME%\release

==================

env.bat script

==================

As I wanted to use both release and debug versions of libraries, I need ability to switch between these modes.

So I created two env.bat script - both for release and debug modes and put it to %KAA_APP_THIRDPARTY_HOME%

See the attachments.

Each time I need something to re-build, I open new console window and run the following:

Relese:

"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" && cd %KAA_APP_THIRDPARTY_HOME% && release.bat

Debug:

"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat"&& cd %KAA_APP_THIRDPARTY_HOME% && debug.bat

In that way I have no conflicts with libraries' versions.

==================

Build tools

==================

I installled latest CMake, Python 2.7 and 7-Zip and ensure that I can run these tools through the command line.

==================

Building artifacts

==================

All downloaded Kaa C++ SDK third-party I saved to the C:\KAA_APP\thirdparty\archive directory and unpack them manually through GUI 7-Zip Manager.

Boost 1.60

I downloaded a pre-built package from here and installed it to C:\KAA_APP\thirdparty\archive.

Then I manually copied  Boost headers directory (C:\KAA_APP\thirdparty\archive\boost_1_60_0\boost) to %KAA_APP_THIRDPARTY_DEBUG%\include and %KAA_APP_THIRDPARTY_RELEASE%\include.

Also I manually copied all libraries with -gd- suffix from C:\JBTC\agent-thirdparty\archive\boost_1_60_0\lib32-msvc-14.0 to %KAA_APP_THIRDPARTY_DEBUG%\lib and non-gd libraries to %KAA_APP_THIRDPARTY_DEBUG%\lib

Avro 1.7.5

Instead of using all-languages source package, I prefer to use only C++ assembly.

So I unpack this archive and run the following commads from the Avro root directory::

Release:

mkdir build_release && cd build_release

cmake .. -DCMAKE_INSTALL_PREFIX:PATH=%KAA_APP_THIRDPARTY_RELEASE% -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Release

nmake install

Debug:

mkdir build_debug && cd build_debug

cmake .. -DCMAKE_INSTALL_PREFIX:PATH=%KAA_APP_THIRDPARTY_DEBUG% -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Debug 

nmake install

Botan 1.11

Unpack a downloaded archive and run the following commads from the Botan root directory:

Release:

python configure.py --cc=msvc --cpu=i386 --prefix=%KAA_APP_THIRDPARTY_RELEASE% 

nmake install

Debug:

python configure.py --cc=msvc --cpu=i386 --prefix=%KAA_APP_THIRDPARTY_DEBUG% --with-debug-info --no-optimizations

nmake install

Note: Each time I need to re-build Botan, I removed the old directory and re-unpack the downloaded archive ( but I think it is paranoia :)  )

==================

Kaa

==================

Unpack C++ SDK archive and run the following commads a from SDK root directory:

Release:

mkdir build_release && cd build_release

cmake -G "NMake Makefiles" -DKAA_DEBUG_ENABLED=0 -DCMAKE_INSTALL_PREFIX=%KAA_APP_THIRDPARTY_RELEASE% ..

nmake install

Debug:

mkdir build_debug && cd build_debug

cmake -G "NMake Makefiles" -DKAA_DEBUG_ENABLED=1 -DCMAKE_INSTALL_PREFIX=%KAA_APP_THIRDPARTY_DEBUG% .. 

nmake install

==================

Results

==================

After all this, I have the following

--- %KAA_APP_THIRDPARTY_RELEASE%

------ bin

----------- avrogencpp.exe

----------- .....

------ docs

----------- .....

------ include

----------- avro

----------- boost

----------- botan

----------- kaa

------ lib

----------- avrocpp.dll

----------- avrocpp.lib

----------- libboost....lib

----------- boost....lib

----------- boost....dll

----------- botan.dll

----------- botan.lib

----------- kaacpp_s.lib

----------- kaacpp.dll

------ ssl

--- %KAA_APP_THIRDPARTY_DEBUG%

----------- The same as in %KAA_APP_THIRDPARTY_RELEASE% except Boost libraries will have -gd suffix in theirs names.

Sorry for inconvenience and I hope this helps :)

Best regards,

Denis

[Denis Kimcherenko]

Also I manually copied all libraries with -gd- suffix from C:\JBTC\agent-thirdparty\archive\boost_1_60_0\lib32-msvc-14.0 to %KAA_APP_THIRDPARTY_DEBUG%\lib and non-gd libraries to %KAA_APP_THIRDPARTY_DEBUG%\lib

I did a mistake in the description. The right answer is: 

".... and non-gd libraries to %KAA_APP_THIRDPARTY_RELEASE%\lib"