Documentation or example of C++ API in irods

[yuren wang]

I wonder where to find a detailed explaination or illustration of usage of irods C++ Client API(aka, rc_xxxx api from #include <rodsClient.h>).Especially some runnable examples, the doxygen documentation seems little help.

Thanks a lot.

[Kory Draughn]

Hi,

The following are likely the best sources for seeing how to use the C/C++ APIs:

• https://github.com/irods/irods/tree/4-2-stable/unit_tests/src
• https://github.com/irods/irods_library_examples
  
• https://github.com/irods/irods_client_icommands/tree/4-2-stable/src
  

Hope that helps.

Kory Draughn

Chief Technologist

iRODS Consortium

On Mon, May 9, 2022 at 12:16 AM yuren wang <wangyu...@gmail.com> wrote:

I wonder where to find a detailed explaination or illustration of usage of irods C++ Client API(aka, rc_xxxx api from #include <rodsClient.h>).Especially some runnable examples, the doxygen documentation seems little help.

Thanks a lot.

--
--
The Integrated Rule-Oriented Data System (iRODS) - https://irods.org
 
iROD-Chat: http://groups.google.com/group/iROD-Chat
---
You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/26cf5b7e-428d-4314-94b8-07618b0978dan%40googlegroups.com.

[yuren wang]

Thanks Kory, it is exactly what I want.

[yuren wang]

By the way, where to find some analysis documentation of irods source code, especially the architecture and  implementation  of the irods rule engine ?

Thanks. 

[Kory Draughn]

The best documentation for understanding the Rule Engine Plugin Framework is the source. The following links should help with where to start:

• https://github.com/irods/irods/blob/313db31d74080b2a683d9177e374eb847514452c/server/core/src/rsApiHandler.cpp#L38
  
• This is where API requests are serviced
• https://github.com/irods/irods/blob/313db31d74080b2a683d9177e374eb847514452c/lib/core/include/apiHandler.hpp#L120
  
• Defines the order of execution for pre, post, except, finally PEPs and the actual API operation (e.g. rsDataObjPut())
• https://github.com/irods/irods/blob/313db31d74080b2a683d9177e374eb847514452c/server/re/include/irods_re_plugin.hpp#L26-L55
  
• Important notes which document the general Rule Engine behavior
• https://github.com/irods/irods/blob/4-2-stable/server/re/include/irods_re_plugin.hpp
  
• https://github.com/irods/irods/blob/313db31d74080b2a683d9177e374eb847514452c/server/re/include/irods_re_plugin.hpp#L459
  
• The continuation logic for PEPs
• https://github.com/irods/irods/blob/313db31d74080b2a683d9177e374eb847514452c/server/re/include/irods_re_plugin.hpp#L660
  
• The continuation logic for irule invocations
• https://github.com/irods/irods/blob/313db31d74080b2a683d9177e374eb847514452c/server/re/include/irods_re_plugin.hpp#L687
  
• The continuation logic for delay()/remote() rules

Other sources:

• https://docs.irods.org/4.2.11/system_overview/process_model/
  
• https://docs.irods.org/4.2.11/plugins/pluggable_rule_engine/
  
• https://docs.irods.org/4.2.11/doxygen/
  
• https://slides.com/irods
  

If you're interested in implementing your own Rule Engine Plugin, I've listed two examples below. The metadata_guard is a lot simpler than logical quotas. They are both implemented using C++, but there's no reason you couldn't do the same thing using Python or any other language.

• https://github.com/irods/irods_rule_engine_plugin_metadata_guard/tree/4-2-stable
• https://github.com/irods/irods_rule_engine_plugin_logical_quotas/tree/4-2-stable

Thanks,

Kory Draughn

Chief Technologist

iRODS Consortium

To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/55b5135a-92c3-4b04-b971-5cf280c88cd6n%40googlegroups.com.

[yuren wang]

Kori, that's really helpful! Thanks a lot.

I wrote a simple C++ API test program as https://github.com/irods/irods_library_examples said like printing all the resource names in C++17 API.

'''

void print_all_resource_names(rcComm_t& _conn) {
for(auto&& row : irods::query<rcComm_t>{&_conn, "select RESC_NAME"}) {
std::cout << row[0] << std::endl;
}

'''

But it cannot pass the link(I wrote a naive CMakeLists.txt like follows[just the library link part], I haved done "yum install irods-dev")

'''

target_link_libraries(xxx PRIVATE /usr/lib/libRodsAPIs.a)
target_link_libraries(xxx PRIVATE /opt/irods-externals/fmt6.1.2-1/lib/libfmt.so)
target_link_libraries(xxx PRIVATE ${IRODS_EXTERNALS_FULLPATH_BOOST}/lib/libboost_filesystem.so
${IRODS_EXTERNALS_FULLPATH_BOOST}/lib/libboost_program_options.so
${IRODS_EXTERNALS_FULLPATH_BOOST}/lib/libboost_system.so)
target_link_libraries(xxx PRIVATE /opt/irods-externals/clang-runtime6.0-0/lib/libc++.so.1 /opt/irods-externals/clang-runtime6.0-0/lib/libc++abi.so.1)

'''

I feel very confused by the link process. irods-dev just provides a single static library right?[libRodsAPIs.a], but it seems the official repo link irods libs like

https://github.com/irods/irods_client_rest_cpp/blob/bd5393556f20533e44c338b6f47b95077a12b3c5/CMakeLists.txt#L125 (aka irods_common irods_client, and so on)

And if I want to use the irods client library, I have to link boost、fmt、spdlog and all the irods external dependencies(shared libraries).

I wonder If there is a better or standard way to link the libraries.

Thanks again.

[Kory Draughn]

libRodsAPIs.a is the static library.

Try linking against the shared libraries. Those are provided by the irods-runtime package. If you install irods-runtime, you will have libirods_client.so, libirods_common.so, etc. under the /usr/lib directory.

Pay close attention to the following CMake directives in the C++ REST API file you referenced:

• find_package
• include
• target_* directives

Keep in mind the version of iRODS you're trying to compile against. If you have 4.2 libraries, you'll need to look at the 4-2-stable branch or that specific version/release tag. The 4.3 branch is different from the 4.2 branch. This applies to reference material as well. If the C++ REST API has more than one branch, you should switch to the branch that matches the target iRODS version you'll be compiling against.

We'll look into providing a CMake quickstart project for the community to simplify the getting-started process.

Kory Draughn

Chief Technologist

iRODS Consortium

To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/0a2ea6ad-252d-43b6-b172-14c3fb393405n%40googlegroups.com.

[yuren wang]

Thanks for your reply, Kori, it's extremely helpful.

I checked my naive CMakeLists.txt file according to your advice but still cannot pass the link process.

The CMakeLists.txt file:

'''

cmake_minimum_required(VERSION 3.10)
project(xxx VERSION 1.0.0 LANGUAGES C CXX)
find_package(IRODS 4.2.11 EXACT REQUIRED)

set(CMAKE_CXX_STANDARD 17)
add_compile_options(-fpermissive)

set(IRODS_EXTERNALS_FULLPATH_BOOST /opt/irods-externals/boost1.67.0-0)
set(IRODS_EXTERNALS_FULLPATH_FMT /opt/irods-externals/fmt6.1.2-1)

add_executable(xxx main.cpp src/scanner.cpp src/irodsClientApiTest.cpp include/irodsClientApiTest.h) 
include_directories(include /opt/rh/devtoolset-8/root/usr/include) #for <string_view>
target_include_directories(xxx PRIVATE ${IRODS_INCLUDE_DIRS} ${IRODS_EXTERNALS_FULLPATH_BOOST}/include ${IRODS_EXTERNALS_FULLPATH_FMT}/include)

target_link_libraries(xxx PRIVATE
irods_common irods_client

${IRODS_EXTERNALS_FULLPATH_BOOST}/lib/libboost_filesystem.so
${IRODS_EXTERNALS_FULLPATH_BOOST}/lib/libboost_program_options.so
${IRODS_EXTERNALS_FULLPATH_BOOST}/lib/libboost_system.so

${IRODS_EXTERNALS_FULLPATH_FMT}/lib/libfmt.so)

'''

everything is from ` yum install ` and irods-dev is installed, I cannot figure out the linking error.

The error report is:

'''

undefined reference to `irods::connection_pool::connection_pool(int, std::string const&, int, std::string const&, std::string const&, int)'
CMakeFiles/xxx.dir/src/irodsClientApiTest.cpp.o: In function `irods::query<RcComm>::query(RcComm*, std::string const&, std::vector<std::string, std::allocator<std::string> > const*, std::string const&, unsigned long, unsigned long, irods::query<RcComm>::query_type, int)':
/usr/include/irods/irods_query.hpp:452: undefined reference to `irods::exception::exception(long, std::string const&, std::string const&, unsigned int, std::string const&)'
CMakeFiles/xxx.dir/src/irodsClientApiTest.cpp.o: In function `irods::query<RcComm>::iterator::advance_query()':
/usr/include/irods/irods_query.hpp:401: undefined reference to `irods::exception::exception(long, std::string const&, std::string const&, unsigned int, std::string const&)'
CMakeFiles/xxx.dir/src/irodsClientApiTest.cpp.o: In function `irods::query<RcComm>::gen_query_impl::gen_query_impl(RcComm*, int, int, std::string const&, std::string const&, int)':
/usr/include/irods/irods_query.hpp:171: undefined reference to `irods::exception::exception(long, std::string const&, std::string const&, unsigned int, std::string const&)'
CMakeFiles/xxx.dir/src/irodsClientApiTest.cpp.o: In function `std::string fmt::v6::internal::grouping<char>(fmt::v6::internal::locale_ref)':
/opt/irods-externals/fmt6.1.2-1/include/fmt/format.h:815: undefined reference to `std::string fmt::v6::internal::grouping_impl<char>(fmt::v6::internal::locale_ref)'
CMakeFiles/xxx.dir/src/irodsClientApiTest.cpp.o: In function `irods::query<RcComm>::spec_query_impl::~spec_query_impl()':
/usr/include/irods/irods_query.hpp:264: undefined reference to `irods::error::error(bool, long long, std::string, std::string, int, std::string)'
CMakeFiles/xxx.dir/src/irodsClientApiTest.cpp.o: In function `irods::query<RcComm>::gen_query_impl::~gen_query_impl()':
/usr/include/irods/irods_query.hpp:188: undefined reference to `irods::error::error(bool, long long, std::string, std::string, int, std::string)'
collect2: error: ld returned 1 exit status

'''

where is the missing part?

Thanks a lot.

[Kory Draughn]

I noticed that your cmake file is missing the following directives:

• include(${IRODS_TARGETS_PATH})
• include(GNUInstallDirs)
• include(UseLibCXX)

The following lines can be removed because iRODS 4.2.11 defaults to C++17 and the compiler provides std::string_view:

• set(CMAKE_CXX_STANDARD 17)
  
• include_directories(include /opt/rh/devtoolset-8/root/usr/include) #for <string_view>

Try comparing your cmake file to the one located here:

• https://github.com/irods/irods_rule_engine_plugin_metadata_guard/blob/4.2.11.0/CMakeLists.txt

If your code is posted on github, I can try compiling it. Can you share the link to it?

Kory Draughn

Chief Technologist

iRODS Consortium

To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/5535b53e-79cb-4f3d-b08c-f5b94acf3cc7n%40googlegroups.com.

[yuren wang]

Thanks for your reply Kori.

The code is very simple and I just want to learn the irods client API and try to customize our irods interface.

the code is https://github.com/yr-wang-hit/irods_api_test

The linking process is normal now but irods::io::odstream cannot be established and report error when trying to write to a irods dataobject in my envionment.

The error is:

remote addresses: 172.25.27.84 ERROR: procApiRequest: apiTableLookup of apiNumber 20003 failed

Maybe there is wrong client API usage?

Thanks again, Kori 

[Kory Draughn]

That error message means you haven't loaded the client api libraries (i.e. the client-side api plugins).

You need to call load_client_api_plugins() before calling any RPC API. For example, see istream here:

• https://github.com/irods/irods_client_icommands/blob/c5f663c8bb1f932a8e66131cb8d4ccd39a7e98da/src/istream.cpp#L80

load_client_api_plugins() is defined in rodsClient.h.

[yuren wang]

Ah, Thanks Kori.

Evething goes well now.

[Kory Draughn]

Excellent!

Let us know what you think of the libraries and good luck!