[CPSAT - JAVA] solver.setLogCallback() isn't working when numWorkers > 1

[Luca Diedolo]

Hi everyone,

I'm experiencing a problem using the log callback of the CP solver in Java.

When the number of workers is 1 the log callback is working as expected.

When I try to increase the number of workers to 8 (also two is enough to reproduce the bug) the log callback makes the CP solver crash with a runtime error (JNI related).

I'm not an expert of JNI but I understood that probably the log callback isn't thread safe inside the C++ implementation.

Has anyone experienced something similar? If someone is interested in helping me I will provide the example code.

At the moment I tried to reproduce just the System.out log which is the simplest way to log information, as it is done by the CP solver by default (without the callback) but the error is still present. Also other implementations with a Thread-safe collection of String isn't working. I also tried with an empty log callback and i noticed that this is enough to reproduce the bug ( solver.setLogCallback(s -> {}); )

Thank you in advance!

Luca

[Laurent Perron]

Actually, the swig wrapper of the log callback stores the jenv, which is thread local. 

--
You received this message because you are subscribed to the Google Groups "or-tools-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to or-tools-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/or-tools-discuss/5914a44f-576d-4929-9931-bda78cce4a2dn%40googlegroups.com.

[Laurent Perron]

And it should be fixed on master. 

[Luca Diedolo]

Thank you for your quick response!

Is there any quick-and-dirty workaround to solve my problem in the meantime?

Thank you in advance.

[Mizux Seiha]

Fixed on master by: https://github.com/google/or-tools/commit/e5a5b8e18fdfa3231662be5b54896763ac5689b0#diff-f78edb1256c3b7424a6e2fe5caaa661adf2145b4dc18e7d4857a9f7c5ec8027d

[Laurent Perron]

You need to recompile from source.

Laurent Perron | Operations Research | lpe...@google.com | (33) 1 42 68 53 00

To view this discussion on the web visit https://groups.google.com/d/msgid/or-tools-discuss/3f301858-7422-44e8-987a-3394c5676123n%40googlegroups.com.

[Luca Diedolo]

Thank you for the fix. Really appreciated.

But currently I'm not able to compile following this guide on the official website. It gives me some error after make third_party, I understood something related to a missing path:

copy dependencies\sources\Cbc-2.10.4\Cbc\MSVisualStudio\v10\x64-v142-Release\*.lib dependencies\install\lib\coin
The system cannot find the path specified.
make: *** [makefiles/Makefile.third_party.win.mk:355: dependencies/install/bin/cbc.exe] Error 1

Is that familiar to you? At first time I ignored this message and I run make java but it terminates early with an error related to CoinOR and CoinUtils:

makefiles/Makefile.third_party.win.mk:67: *** Third party CoinUtils files was not found! did you run 'make third_party' or set WINDOWS_COINUTILS_DIR ?. Stop.

Waiting for your kindly response.

Thank you in advance.

[Corentin "Mizux" Le Molgat]

Can we have the output of:

```cmd

make --version

make detect

```

and if you have a bash like term (like git bash)

```sh

find dependencies/sources/Cbc-2.10.4 -type f -iname "*.lib"

```

I'm curious to know which version of Visual Studio you have installed and if whether CBC manage to compile BUT with a different name/location...

[Luca Diedolo]

.\tools\make.exe --version
GNU Make 4.3
Built for Windows32
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

.\tools\make.exe detect
makefiles/Makefile.port.mk:135: "Only 64 bit compilation is supported"
Relevant info on the system:
SYSTEM = win
OS = Windows
PLATFORM = WIN64
PTRLENGTH = 64
PORT = VisualStudio2019-64bit
SHELL = cmd
OR_TOOLS_TOP = C:\\Users\\Luca\\Desktop\\or-tools-stable
OR_TOOLS_VERSION = 9.0.9999
OR_TOOLS_SHORT_VERSION = 9.0
GIT_REVISION = 9999
GIT_HASH = "not_on_git"
CMAKE = C:\Program Files\CMake\bin\cmake.EXE
CMAKE_PLATFORM = "Visual Studio 16 2019" -A x64
SWIG_BINARY = "dependencies\\install\\swigwin-4.0.2\\swig.exe"
SWIG_INC = -DUSE_GLOP -DUSE_BOP -DABSL_MUST_USE_RESULT -I"dependencies/install/include" -I"dependencies/install/include" -I"dependencies\\install/include" -I"dependencies/install/include" -I"dependencies/install
/include/coin" -I"dependencies/install/include" -I"dependencies/install/include/coin" -I"dependencies/install/include" -I"dependencies/install/include/coin" -DUSE_CLP -I"dependencies/install/include" -I"dependencies
/install/include/coin" -I"dependencies/install/include" -I"dependencies/install/include/coin" -DUSE_CBC -I"dependencies/install/include" -DUSE_SCIP -DNO_CONFIG_HEADER

Relevant info on third party:
WINDOWS_ZLIB_DIR = dependencies/install
ZLIB_INC = /I"dependencies\\install\\include"
ZLIB_LNK = "dependencies\\install\lib\zlib.lib"
WINDOWS_PROTOBUF_DIR = dependencies/install
PROTOBUF_INC = /I"dependencies\\install\\include"
PROTOBUF_LNK = "dependencies\\install\lib\libprotobuf.lib"
ABSL_INC = /I"dependencies\\install\\include"
ABSL_LNK = "dependencies\\install\lib\absl_*.lib"
WINDOWS_CBC_DIR = dependencies/install
CBC_INC = /I"dependencies\\install\\include" /I"dependencies\\install\\include\\coin" /DUSE_CBC
CBC_LNK = "dependencies\\install\lib\coin\libCbcSolver.lib" "dependencies\\install\lib\coin\libCbc.lib"
WINDOWS_CLP_DIR = dependencies/install
CLP_INC = /I"dependencies\\install\\include" /I"dependencies\\install\\include\\coin" /DUSE_CLP
CLP_LNK = "dependencies\\install\lib\coin\libClp.lib" "dependencies\\install\lib\coin\libOsiClp.lib"
WINDOWS_SCIP_DIR = dependencies/install
SCIP_INC = /I"dependencies\\install\\include" /DUSE_SCIP /DNO_CONFIG_HEADER
SCIP_LNK = "dependencies\\install\lib\libscip.lib"

Relevant info for the C++ build:
PROTOC = "dependencies/install\\bin\\protoc.exe"
CCC = cl /std:c++17 /EHsc /MD /nologo -nologo /D__WIN32__ /DPSAPI_VERSION=1 /DNOMINMAX /DWIN32_LEAN_AND_MEAN=1 /D_CRT_SECURE_NO_WARNINGS
CFLAGS = /O2 -DNDEBUG /I. /Iortools/gen /DUSE_GLOP /DUSE_BOP /I"dependencies\\install\\include" /I"dependencies\\install\\include" /I"dependencies\\install\\include" /I"dependencies\\install\\include" /I"depende
ncies\\install\\include\\coin" /I"dependencies\\install\\include" /I"dependencies\\install\\include\\coin" /I"dependencies\\install\\include" /I"dependencies\\install\\include\\coin" /DUSE_CLP /I"dependencies\\insta
ll\\include" /I"dependencies\\install\\include\\coin" /I"dependencies\\install\\include" /I"dependencies\\install\\include\\coin" /DUSE_CBC /I"dependencies\\install\\include" /DUSE_SCIP /DNO_CONFIG_HEADER /DOR_TOOL
S_MAJOR=9 /DOR_TOOLS_MINOR=0
LDFLAGS =
LINK_CMD = lib
DEPENDENCIES_INC = /I. /Iortools/gen /DUSE_GLOP /DUSE_BOP /I"dependencies\\install\\include" /I"dependencies\\install\\include" /I"dependencies\\install\\include" /I"dependencies\\install\\include" /I"dependenci
es\\install\\include\\coin" /I"dependencies\\install\\include" /I"dependencies\\install\\include\\coin" /I"dependencies\\install\\include" /I"dependencies\\install\\include\\coin" /DUSE_CLP /I"dependencies\\install\
\include" /I"dependencies\\install\\include\\coin" /I"dependencies\\install\\include" /I"dependencies\\install\\include\\coin" /DUSE_CBC /I"dependencies\\install\\include" /DUSE_SCIP /DNO_CONFIG_HEADER
DEPENDENCIES_LNK = psapi.lib ws2_32.lib shlwapi.lib "dependencies\\install\lib\zlib.lib" "dependencies\\install\lib\libprotobuf.lib" "dependencies\\install\lib\absl_*.lib" "dependencies\\install\lib\coin\libCbcSo
lver.lib" "dependencies\\install\lib\coin\libCbc.lib" "dependencies\\install\lib\coin\libCgl.lib" "dependencies\\install\lib\coin\libClp.lib" "dependencies\\install\lib\coin\libOsiClp.lib" "dependencies\\install\lib
\coin\libOsi.lib" "dependencies\\install\lib\coin\libCoinUtils.lib" "dependencies\\install\lib\libscip.lib"
SRC_DIR = .
GEN_DIR = ortools/gen
CC_EX_DIR = examples/cpp
OBJ_DIR = objs
LIB_DIR = lib
BIN_DIR = bin
prefix = C:\\Program Files\\or-tools
OR_TOOLS_LNK = lib\\ortools.lib
OR_TOOLS_LDFLAGS =
OR_TOOLS_LIBS = lib/ortools.lib

Relevant info for the Python build:
WINDOWS_PATH_TO_PYTHON = "C:\Program Files\Python39"
PYTHON_COMPILER = python.exe
PYTHON_EXECUTABLE = "C:\Program Files\Python39\python.exe"
PYTHON_VERSION = 39
PYTHON3 = true
PYTHON_INC = /I"C:\Program Files\Python39\\include"
PYTHON_LNK = "C:\Program Files\Python39\\libs\\python39.lib"
PYTHON_LDFLAGS =
SWIG_BINARY = "dependencies\\install\\swigwin-4.0.2\\swig.exe"
SWIG_INC = -DUSE_GLOP -DUSE_BOP -DABSL_MUST_USE_RESULT -I"dependencies/install/include" -I"dependencies/install/include" -I"dependencies\\install/include" -I"dependencies/install/include" -I"dependencies/install
/include/coin" -I"dependencies/install/include" -I"dependencies/install/include/coin" -I"dependencies/install/include" -I"dependencies/install/include/coin" -DUSE_CLP -I"dependencies/install/include" -I"dependencies
/install/include/coin" -I"dependencies/install/include" -I"dependencies/install/include/coin" -DUSE_CBC -I"dependencies/install/include" -DUSE_SCIP -DNO_CONFIG_HEADER
SWIG_PYTHON3_FLAG = -py3 -DPY3
SWIG_PYTHON_LIB_SUFFIX = dll
SET_PYTHONPATH = "set PYTHONPATH=C:\\Users\\Luca\\Desktop\\or-tools-stable;C:\\Users\\Luca\\Desktop\\or-tools-stable\\dependencies\\sources\\protobuf-v3.15.8\\python &&"
MYPY_OUT = ""

tools\win\which.exe: no javac in (\bin)
tools\win\which.exe: no jar in (\bin)
tools\win\which.exe: no java in (\bin)
Relevant info for the Java build:
JAVA_INC = /I"\\include" /I"\\include\\win32"
JNIFLAGS = /O2 -DNDEBUG /I. /Iortools/gen /DUSE_GLOP /DUSE_BOP /I"dependencies\\install\\include" /I"dependencies\\install\\include" /I"dependencies\\install\\include" /I"dependencies\\install\\include" /I"depen
dencies\\install\\include\\coin" /I"dependencies\\install\\include" /I"dependencies\\install\\include\\coin" /I"dependencies\\install\\include" /I"dependencies\\install\\include\\coin" /DUSE_CLP /I"dependencies\\ins
tall\\include" /I"dependencies\\install\\include\\coin" /I"dependencies\\install\\include" /I"dependencies\\install\\include\\coin" /DUSE_CBC /I"dependencies\\install\\include" /DUSE_SCIP /DNO_CONFIG_HEADER /DOR_TO
OLS_MAJOR=9 /DOR_TOOLS_MINOR=0 /I. /Iortools/gen /DUSE_GLOP /DUSE_BOP /I"dependencies\\install\\include" /I"dependencies\\install\\include" /I"dependencies\\install\\include" /I"dependencies\\install\\include" /
I"dependencies\\install\\include\\coin" /I"dependencies\\install\\include" /I"dependencies\\install\\include\\coin" /I"dependencies\\install\\include" /I"dependencies\\install\\include\\coin" /DUSE_CLP /I"dependenci
es\\install\\include" /I"dependencies\\install\\include\\coin" /I"dependencies\\install\\include" /I"dependencies\\install\\include\\coin" /DUSE_CBC /I"dependencies\\install\\include" /DUSE_SCIP /DNO_CONFIG_HEADER
JAVA_HOME =
JAVAC_BIN =
CLASS_DIR = classes
JAR_BIN =
JAVA_BIN =
JAVAFLAGS = -Djava.library.path=lib
JAVA_ORTOOLS_JAR = lib/com.google.ortools.jar
SWIG_BINARY = "dependencies\\install\\swigwin-4.0.2\\swig.exe"
SWIG_INC = -DUSE_GLOP -DUSE_BOP -DABSL_MUST_USE_RESULT -I"dependencies/install/include" -I"dependencies/install/include" -I"dependencies\\install/include" -I"dependencies/install/include" -I"dependencies/install
/include/coin" -I"dependencies/install/include" -I"dependencies/install/include/coin" -I"dependencies/install/include" -I"dependencies/install/include/coin" -DUSE_CLP -I"dependencies/install/include" -I"dependencies
/install/include/coin" -I"dependencies/install/include" -I"dependencies/install/include/coin" -DUSE_CBC -I"dependencies/install/include" -DUSE_SCIP -DNO_CONFIG_HEADER
MVN_BIN = C:\Program Files\apache-maven-3.8.2\bin\mvn.cmd
JAVA_ORTOOLS_PACKAGE = com.google.ortools
JAVA_ORTOOLS_NATIVE_PROJECT = ortools-win32-x86-64
JAVA_ORTOOLS_PROJECT = ortools-java

Relevant info for the dotnet build:
DOTNET_BIN =
NUGET_BIN = "" nuget
PROTOC = "dependencies/install\\bin\\protoc.exe"
DOTNET_SNK =
DOTNET_ORTOOLS_SNK = ./ortools/dotnet/or-tools.snk
SWIG_BINARY = "dependencies\\install\\swigwin-4.0.2\\swig.exe"
SWIG_INC = -DUSE_GLOP -DUSE_BOP -DABSL_MUST_USE_RESULT -I"dependencies/install/include" -I"dependencies/install/include" -I"dependencies\\install/include" -I"dependencies/install/include" -I"dependencies/install
/include/coin" -I"dependencies/install/include" -I"dependencies/install/include/coin" -I"dependencies/install/include" -I"dependencies/install/include/coin" -DUSE_CLP -I"dependencies/install/include" -I"dependencies
/install/include/coin" -I"dependencies/install/include" -I"dependencies/install/include/coin" -DUSE_CBC -I"dependencies/install/include" -DUSE_SCIP -DNO_CONFIG_HEADER
SWIG_DOTNET_LIB_SUFFIX = dll
DOTNET_ORTOOLS_NATIVE = google-ortools-native
DOTNET_ORTOOLS_RUNTIME_ASSEMBLY_NAME =Google.OrTools.runtime.win-x64
DOTNET_ORTOOLS_RUNTIME_NUPKG = packages/Google.OrTools.runtime.win-x64.9.0.9999.nupkg
DOTNET_ORTOOLS_ASSEMBLY_NAME = Google.OrTools
DOTNET_ORTOOLS_NUPKG = packages/Google.OrTools.9.0.9999.nupkg
FSHARP_ORTOOLS_ASSEMBLY_NAME = Google.OrTools.FSharp
FSHARP_ORTOOLS_NUPKG = packages/Google.OrTools.FSharp.9.0.9999.nupkg
FSHARP_ORTOOLS_TESTS_ASSEMBLY_NAME = Google.OrTools.FSharp.Tests

Relevant info for the archive build:
TEMP_ARCHIVE_DIR = temp_archive
INSTALL_DIR = or-tools_VisualStudio2019-64bit_v9.0.9999
TEMP_FZ_DIR = temp_fz_archive
FZ_INSTALL_DIR = or-tools_flatzinc_VisualStudio2019-64bit_v9.0.9999
TEMP_DATA_DIR = temp_data
DATA_INSTALL_DIR = or-tools_data_v9.0.9999
ARCHIVE_EXT = .zip

SOURCE =
SOURCE_PATH =
SOURCE_NAME =
SOURCE_SUFFIX =

Visual Studio about

Microsoft Visual Studio Community 2019

Version 16.11.3

VisualStudio.16.Release/16.11.3+31702.278

Microsoft .NET Framework

Version 4.8.04084

Installed Version: Community

Visual C++ 2019   00435-60000-00000-AA718

Microsoft Visual C++ 2019

ASP.NET and Web Tools 2019   16.11.75.64347

ASP.NET and Web Tools 2019

C# Tools   3.11.0-4.21403.6+ae1fff344d46976624e68ae17164e0607ab68b10

C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

IntelliCode Extension   1.0

IntelliCode Visual Studio Extension Detailed Info

Linux Core Dump Debugging   1.0.9.31613

Enables debugging of Linux core dumps.

Microsoft JVM Debugger   1.0

Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

Microsoft MI-Based Debugger   1.0

Provides support for connecting Visual Studio to MI compatible debuggers

Microsoft Visual C++ Wizards   1.0

Microsoft Visual C++ Wizards

Microsoft Visual Studio VC Package   1.0

Microsoft Visual Studio VC Package

NuGet Package Manager   5.11.0

NuGet Package Manager in Visual Studio. For more information about NuGet, visit https://docs.nuget.org/

ProjectServicesPackage Extension   1.0

ProjectServicesPackage Visual Studio Extension Detailed Info

Test Adapter for Boost.Test   1.0

Enables Visual Studio's testing tools with unit tests written for Boost.Test.  The use terms and Third Party Notices are available in the extension installation directory.

Test Adapter for Google Test   1.0

Enables Visual Studio's testing tools with unit tests written for Google Test.  The use terms and Third Party Notices are available in the extension installation directory.

TypeScript Tools   16.0.30526.2002

TypeScript Tools for Microsoft Visual Studio

Visual Basic Tools   3.11.0-4.21403.6+ae1fff344d46976624e68ae17164e0607ab68b10

Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Visual C++ for Linux Development   1.0.9.31613

Visual C++ for Linux Development

Visual Studio Code Debug Adapter Host Package   1.0

Interop layer for hosting Visual Studio Code debug adapters in Visual Studio

Visual Studio Tools for CMake   1.0

Visual Studio Tools for CMake

find dependencies/sources/Cbc-2.10.4 -type f -iname "*.lib"

dependencies/sources/Cbc-2.10.4/Cbc/MSVisualStudio/v10/Win32-v142-Release/libCbc.lib

dependencies/sources/Cbc-2.10.4/Cbc/MSVisualStudio/v10/Win32-v142-Release/libCbcSolver.lib

dependencies/sources/Cbc-2.10.4/Cbc/MSVisualStudio/v10/Win32-v142-Release/libCgl.lib

dependencies/sources/Cbc-2.10.4/Cbc/MSVisualStudio/v10/Win32-v142-Release/libClp.lib

dependencies/sources/Cbc-2.10.4/Cbc/MSVisualStudio/v10/Win32-v142-Release/libCoinUtils.lib

dependencies/sources/Cbc-2.10.4/Cbc/MSVisualStudio/v10/Win32-v142-Release/libOsi.lib

dependencies/sources/Cbc-2.10.4/Cbc/MSVisualStudio/v10/Win32-v142-Release/libOsiClp.lib

[Laurent Perron]

You should build the master branch.

Laurent Perron | Operations Research | lpe...@google.com | (33) 1 42 68 53 00

To view this discussion on the web visit https://groups.google.com/d/msgid/or-tools-discuss/70a444d6-e94d-4fc3-bf9e-3feafdaf0fbbn%40googlegroups.com.

[Corentin "Mizux" Le Molgat]

OK so we are looking at:

dependencies\sources\Cbc-2.10.4\Cbc\MSVisualStudio\v10\x64-v142-Release\*.lib

while on your system it is located at:

dependencies/sources/Cbc-2.10.4/Cbc/MSVisualStudio/v10/Win32-v142-Release/libCbc.lib

=> Need to find what's happen inside the CBC build system

[Corentin "Mizux" Le Molgat]

Seems related to value of CBC_PLATFORM

https://github.com/google/or-tools/blob/90f1f7dd8d8b62335930a3f63b956f902adfd378/makefiles/Makefile.third_party.win.mk#L355-L356

[Corentin "Mizux" Le Molgat]

You need to build using a 64bit (x86_64, amd64, x64, win64) command prompt, and here it seems you are using a x86 (win32) command prompt...

ref: https://github.com/google/or-tools/blob/stable/makefiles/Makefile.port.mk#L129-L167

[Luca Diedolo]

Thanks to you I managed to compile third_party! I wrongly used power shell and visual studio integrated terminal. Now I understood I have to use the x64 native tool.

Now I miss the java part. It runs a little bit and then says it cannot find "jni.h". I tried to find it in my computer and it is not present. Any advice?

Thank you all for your patience, have a nice day!

Luca