YCM does not find all C++ function declarations nor all class definitions from include files.

[Tom Bjerck]

This is in an  application using the spatial database translation library libgdal, which is used to convert between different database formats containing geospatial map data. The application compiles with no errors with clang++, but ycm complains about "member access into incomplete type 'GDALDataset'" and "use of undeclared identifier 'GetGDALDriverManager'". The latter is an extern C function. Both are declared in <gdal/gdal_priv.h>, which is included at the top of <gdal/ogrsf_frmts.h>.

I'd appreciate any hints or explanations of what could lead to this erroneous error reporting by ycm. Also I'd like to know if anybody can replicate this.

How to reproduce:
1) You need the gdal library from http://gdal.org/ installed (libgdal-dev on Ubuntu).
2) Minimal test program (save as test.cpp):

    #include <gdal/ogrsf_frmts.h>

    int main(int argc, char *argv[])
    {
        const char *driver_name = "ESRI Shapefile";
        GDALDriver *driver;

        GDALAllRegister();

        driver = GetGDALDriverManager()->GetDriverByName(driver_name);
        if (driver == NULL) {
            fprintf(stderr, "%s driver not available.\n", driver_name);
            exit(1);
        }

        GDALDataset *DS;
        const char *shapefile = "out.shp";
        DS = driver->Create(shapefile, 0, 0, 0, GDT_Unknown, NULL);
        if (DS == NULL) {
            fprintf(stderr, "Creation of dataset %s failed.\n", shapefile);
            exit(1);
        }

        return 0;
    }

3) Edit test.cpp in vim with the flags from the .ycm_extra_conf.py included below.

4) Compile with:
   clang++ -Wall -Wextra -std=c++11 -x c++ -o test.o -c test.cpp

##################################################################################

Details of my configuration:

You Complete Me was compiled by both "./install.py --clang-completer" and against a
custom compiled downloaded clang 4.0.1. Both versions give the same error.

###################################################################################

Output of "echo | clang++ -v -E -x c++ -":

########################################################################################

clang version 4.0.0-1ubuntu1 (tags/RELEASE_400/rc1)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/i686-linux-gnu/6
Found candidate GCC installation: /usr/bin/../lib/gcc/i686-linux-gnu/6.3.0
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/4.7
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/4.7.4
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/5
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.1
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/6
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/6
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/6.3.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.7
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.7.4
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.4.1
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6.3.0
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/usr/lib/llvm-4.0/bin/clang" -cc1 -triple x86_64-pc-linux-gnu -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -v -dwarf-column-info -debugger-tuning=gdb -resource-dir /usr/lib/llvm-4.0/bin/../lib/clang/4.0.0 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/x86_64-linux-gnu/c++/6.3.0 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/x86_64-linux-gnu/c++/6.3.0 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/backward -internal-isystem /usr/include/clang/4.0.0/include/ -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-4.0/bin/../lib/clang/4.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir /home/tom/programming/C++/gdal/readtest -ferror-limit 19 -fmessage-length 126 -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o - -x c++ -
clang -cc1 version 4.0.0 based upon LLVM 4.0.0 default target x86_64-pc-linux-gnu
ignoring nonexistent directory "/include"
ignoring duplicate directory "/usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/x86_64-linux-gnu/c++/6.3.0"
ignoring duplicate directory "/usr/include/clang/4.0.0/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0
 /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/x86_64-linux-gnu/c++/6.3.0
 /usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/backward
 /usr/include/clang/4.0.0/include
 /usr/local/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 329 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2

##############################################################################################

My .ycm_extra_conf.py Only the compile flags are changed from the example config.:

########################################################################################

# This file is NOT licensed under the GPLv3, which is the license for the rest
# of YouCompleteMe.
#
# Here's the license text for this file:
#
# This is free and unencumbered software released into the public domain.
#
# Anyone is free to copy, modify, publish, use, compile, sell, or
# distribute this software, either in source code form or as a compiled
# binary, for any purpose, commercial or non-commercial, and by any
# means.
#
# In jurisdictions that recognize copyright laws, the author or authors
# of this software dedicate any and all copyright interest in the
# software to the public domain. We make this dedication for the benefit
# of the public at large and to the detriment of our heirs and
# successors. We intend this dedication to be an overt act of
# relinquishment in perpetuity of all present and future rights to this
# software under copyright law.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
# IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.
#
# For more information, please refer to <http://unlicense.org/>

import os
import ycm_core

# These are the compilation flags that will be used in case there's no
# compilation database set (by default, one is not set).
# CHANGE THIS LIST OF FLAGS. YES, THIS IS THE DROID YOU HAVE BEEN LOOKING FOR.
flags = [
'-Wall',
'-Wextra',
#
# You 100% do NOT need -DUSE_CLANG_COMPLETER in your flags; only the YCM
# source code needs it.
##'-DUSE_CLANG_COMPLETER',
# THIS IS IMPORTANT! Without a "-std=<something>" flag, clang won't know which
# language to use when compiling headers. So it will guess. Badly. So C++
# headers will be compiled as C headers. You don't want that so ALWAYS specify
# a "-std=<something>".
# For a C project, you would set this to something like 'c99' instead of
# 'c++11'.
'-std=c++11',
# ...and the same thing goes for the magic -x option which specifies the
# language that the files to be compiled are written in. This is mostly
# relevant for c++ headers.
# For a C project, you would set this to 'c' instead of 'c++'.
'-x',
'c++',
'-I',
'.',
'-isystem',
'/usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0',
'-isystem',
'/usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/x86_64-linux-gnu/c++/6.3.0',
'-isystem',
'/usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0/backward',
'-isystem',
'/usr/include/clang/4.0.0/include',
'-isystem',
'/usr/local/include',
'-isystem',
'/usr/include/x86_64-linux-gnu',
'-isystem',
'/usr/include',
]


# Set this to the absolute path to the folder (NOT the file!) containing the
# compile_commands.json file to use that instead of 'flags'. See here for
# more details: http://clang.llvm.org/docs/JSONCompilationDatabase.html
#
# You can get CMake to generate this file for you by adding:
#   set( CMAKE_EXPORT_COMPILE_COMMANDS 1 )
# to your CMakeLists.txt file.
#
# Most projects will NOT need to set this to anything; you can just change the
# 'flags' list of compilation flags. Notice that YCM itself uses that approach.
compilation_database_folder = ''

if os.path.exists( compilation_database_folder ):
  database = ycm_core.CompilationDatabase( compilation_database_folder )
else:
  database = None

SOURCE_EXTENSIONS = [ '.cpp', '.cxx', '.cc', '.c', '.m', '.mm' ]

def DirectoryOfThisScript():
  return os.path.dirname( os.path.abspath( __file__ ) )


def MakeRelativePathsInFlagsAbsolute( flags, working_directory ):
  if not working_directory:
    return list( flags )
  new_flags = []
  make_next_absolute = False
  path_flags = [ '-isystem', '-I', '-iquote', '--sysroot=' ]
  for flag in flags:
    new_flag = flag

    if make_next_absolute:
      make_next_absolute = False
      if not flag.startswith( '/' ):
        new_flag = os.path.join( working_directory, flag )

    for path_flag in path_flags:
      if flag == path_flag:
        make_next_absolute = True
        break

      if flag.startswith( path_flag ):
        path = flag[ len( path_flag ): ]
        new_flag = path_flag + os.path.join( working_directory, path )
        break

    if new_flag:
      new_flags.append( new_flag )
  return new_flags


def IsHeaderFile( filename ):
  extension = os.path.splitext( filename )[ 1 ]
  return extension in [ '.h', '.hxx', '.hpp', '.hh' ]


def GetCompilationInfoForFile( filename ):
  # The compilation_commands.json file generated by CMake does not have entries
  # for header files. So we do our best by asking the db for flags for a
  # corresponding source file, if any. If one exists, the flags for that file
  # should be good enough.
  if IsHeaderFile( filename ):
    basename = os.path.splitext( filename )[ 0 ]
    for extension in SOURCE_EXTENSIONS:
      replacement_file = basename + extension
      if os.path.exists( replacement_file ):
        compilation_info = database.GetCompilationInfoForFile(
          replacement_file )
        if compilation_info.compiler_flags_:
          return compilation_info
    return None
  return database.GetCompilationInfoForFile( filename )


def FlagsForFile( filename, **kwargs ):
  if database:
    # Bear in mind that compilation_info.compiler_flags_ does NOT return a
    # python list, but a "list-like" StringVec object
    compilation_info = GetCompilationInfoForFile( filename )
    if not compilation_info:
      return None

    final_flags = MakeRelativePathsInFlagsAbsolute(
      compilation_info.compiler_flags_,
      compilation_info.compiler_working_dir_ )

    # NOTE: This is just for YouCompleteMe; it's highly likely that your project
    # does NOT need to remove the stdlib flag. DO NOT USE THIS IN YOUR
    # ycm_extra_conf IF YOU'RE NOT 100% SURE YOU NEED IT.
##    try:
##      final_flags.remove( '-stdlib=libc++' )
##    except ValueError:
##      pass
  else:
    relative_to = DirectoryOfThisScript()
    final_flags = MakeRelativePathsInFlagsAbsolute( flags, relative_to )

  return {
    'flags': final_flags,
    'do_cache': True
  }

###########################################################################################

Output of :YcmDebugInfo :

########################################################################################

Printing YouCompleteMe debug information...
-- Client logfile: /tmp/ycm_1_cfskua.log
-- Server Python interpreter: /usr/bin/python
-- Server Python version: 2.7.13
-- Server has Clang support compiled in: True
-- Clang version: clang version 4.0.1 (tags/RELEASE_401/final)
-- Extra configuration file found and loaded
-- Extra configuration path: /home/tom/programming/C++/gdal/readtest/.ycm_extra_conf.py
-- C-family completer debug information:
--   Compilation database path: None
--   Flags: ['-Wall', '-Wextra', '-std=c++11', '-x', 'c++', '-I', '/home/tom/programming/C++/gdal/readtest/.', '-isystem', '/usr
/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/6.3.0', '-isystem', '/usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/.
./../../../include/x86_64-linux-gnu/c++/6.3.0', '-isystem', '/usr/bin/../lib/gcc/x86_64-linux-gnu/6.3.0/../../../../include/c++/
6.3.0/backward', '-isystem', '/usr/include/clang/4.0.0/include', '-isystem', '/usr/local/include', '-isystem', '/usr/include/x86
_64-linux-gnu', '-isystem', '/usr/include', '-resource-dir=/home/tom/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../clang_in
cludes', '-fspell-checking']
-- Server running at: http://127.0.0.1:43105
-- Server process ID: 3138
-- Server logfiles:
--   /tmp/ycmd_43105_stdout_wb3v3thj.log
--   /tmp/ycmd_43105_stderr_er4ljb5i.log

#########################################################################################

Log files:

########################################################################################

Client log file is empty.

Server stdout log:
serving on http://127.0.0.1:43105

Server stderr log:
2017-07-26 16:45:58,932 - DEBUG - No global extra conf, not calling method YcmCorePreload
2017-07-26 16:45:59,018 - INFO - Received ready request
2017-07-26 16:45:59,093 - INFO - Received event notification
2017-07-26 16:45:59,094 - DEBUG - Event name: BufferVisit
2017-07-26 16:45:59,100 - INFO - Received event notification
2017-07-26 16:45:59,100 - DEBUG - Event name: FileReadyToParse
2017-07-26 16:45:59,100 - INFO - Adding buffer identifiers for file: /home/tom/programming/C++/gdal/readtest/test.cpp
2017-07-26 16:46:04,375 - INFO - Received filetype completion available request
2017-07-26 16:46:21,118 - INFO - Received debug info request
2017-07-26 16:48:17,944 - INFO - Received debug info request

########################################################################################

My ycm vimrc configuration options:

########################################################################################

        let g:ycm_server_python_interpreter = '/usr/bin/python'
        let g:ycm_collect_identifiers_from_tags_files = 1
        let g:ycm_seed_identifiers_with_syntax = 1
        "let g:ycm_server_use_vim_stdout = 1
        let g:ycm_server_log_level = 'debug'
        let g:ycm_extra_conf_globlist = ['~/tmp/*','~/workspace/*','~/programming/C++/*','~/programming/C/*','!~/*']
        let g:ycm_semantic_triggers =  { 'cpp' : ['->', '.', '::','re!gl'], 'c' : ['->', '.','re!xml']}

[Tom Bjerck]

Looking at :YcmDiags helped. As it turns out, the problem was an ELF binary named "map" in my working directory, combined with an .ycm_extra_conf.py that included this directory. When one of the #include files in my project included C++ <map>, it chose the ELF binary, and the error occurred.

I somehow expected that any such output, like the output from :YcmDiags, would be included in one of the log files, but it isn't.