The firebase-admin-java 9.4.0 does not work with NIO (Non-blocking I/O).

120 views
Skip to first unread message

stager0909

unread,
Oct 28, 2024, 9:27:58 AM10/28/24
to Firebase Google Group
Hello,
I am testing the firebase-admin-java SDK 9.4.0 with the following configuration:

```
FirebaseOptions options = FirebaseOptions.builder()
    .setCredentials(googleCredentials(firebaseCredential, httpTransport))
    .setDatabaseUrl(firebaseDatabaseUrl)
    .setHttpTransport(httpTransport)
    .setConnectTimeout(FIREBASE_CONNECT_TIMEOUT)
    .setReadTimeout(FIREBASE_READ_TIMEOUT)
    .setThreadManager(new CustomFirebaseThreadManager(
        firebaseTaskExecutor.getThreadPoolExecutor()))
    .build();
```
The reason I explicitly added ThreadManager is to monitor the ThreadPoolExecutor. Without this configuration, the ThreadPoolExecutor generated by FirebaseThreadManagers.DefaultThreadManager.doInit() will be used, which has a queue size of Integer.MAX_VALUE. If FCM responses are delayed, the queue can pile up, potentially causing an OOM (OutOfMemoryError).

Below is the configuration for H2AsyncClientBuilder:

```
H2AsyncClientBuilder h2AsyncClientBuilder = H2AsyncClientBuilder.create()
    .setDefaultConnectionConfig(connectionConfig)
    .evictIdleConnections(TimeValue.of(Duration.ofMinutes(10)))
    .setDefaultRequestConfig(requestConfig())
    .setIOReactorConfig(IOReactorConfig.custom()
        .setSoTimeout(FIREBASE_READ_TIMEOUT, TimeUnit.MILLISECONDS)
        .build())
    .addRequestInterceptorFirst(
        (HttpRequest request, EntityDetails entity, HttpContext context) -> {
            activeStream.incrementAndGet();
            if (context != null) {
                context.setAttribute("startTime", System.nanoTime());
            }
        })
    .addResponseInterceptorFirst(
        (HttpResponse response, EntityDetails entity, HttpContext context) -> {
            if (context == null || context.getAttribute("startTime") == null) {
                return;
            }
            long startTime = Long.parseLong(context.getAttribute("startTime").toString());
            long endTime = System.nanoTime();
            if (startTime < endTime) {
                Timer.builder(FCM_RESPONSE_TIMER_NAME)
                    .publishPercentiles(0.50, 0.95, 0.99)
                    .tags(
                        Lists.newArrayList(
                            Tag.of("host", ((HttpRoute) context.getAttribute("http.route")).getTargetHost().getHostName()),
                            Tag.of("http_version", context.getProtocolVersion().toString())
                        )
                    )
                    .register(meterRegistry)
                    .record(endTime - startTime, TimeUnit.NANOSECONDS);
            }
            activeStream.decrementAndGet();
        })
    .disableCookieManagement()
    .setH2Config(H2Config.custom().setMaxConcurrentStreams(500 * 10).build())
    .disableRedirectHandling()
    .disableAutomaticRetries();
```
Here is the code for sending FCM messages. The postProcessExecutor is simply responsible for logging the send results:

```
ApiFuture<String> responseApiFuture = firebaseMessaging.sendAsync(fcmMessage);
responseApiFuture.addListener(() -> {
    try {
        responseApiFuture.get();
        message.setResponseCode(FcmResult.SUCCESS.getCode());
        postProcessor.postProcess(MessageStatus.SENT, message);
    } catch (Exception e) {
        log ~~~
    }
}, postProcessorExecutor);
```
When testing this, if I set a breakpoint inside addRequestInterceptorFirst (or addResponseInterceptorFirst) in the second code block, the thread executing addRequestInterceptorFirst is the httpclient-dispatch thread.
At this point, firebaseThreadPoolTask.getActiveCount() (as set in FirebaseOptions.builder().setThreadManager()) is 1.
In other words, it seems like the firebaseThread is staying active while the httpclient-dispatch thread is running.

I would like the behavior to work as follows:

The firebaseThread performs tasks (such as SDK validation) before the HTTP call.
After handing over to httpclient-dispatch, the firebaseThread is released.
The httpclient-dispatch makes the request to the FCM server, and the IO selector detects when a response is received.
Once the IO selector detects the response, it hands over to httpclient-dispatch, which processes operations like interceptors and then passes it to postProcessorExecutor.
This behavior should prevent the firebaseThread from getting blocked when the FCM server is down or latency is high.
If the firebaseThread gets blocked, it means that the queue in the thread pool fills up, which can lead to OOM or RejectedExecutionException.

If I have misunderstood anything, please let me know.
If my understanding is correct, how can I resolve this issue?

Joe Spiro

unread,
Oct 28, 2024, 9:30:45 AM10/28/24
to Firebase Google Group
Hello,

Thank you for getting it touch. While I have not confirmed the behaviour, I appreciate your having brought it to our attention. Please file an issue on the Firebase Admin SDK Github repo where the maintainers/developers can have a closer look at it.

Thank you!
- Joe from Firebase
Reply all
Reply to author
Forward
0 new messages