Integrating OpenJDK with OSv to make Java boot and run faster for FaaS
62 views
Skip to first unread message
Yuanqi Li
Mar 19, 2020, 9:30:30 AM3/19/20
to OSv Development
Hi all,
We are a system research group from UCLA focusing on managed runtimes. We have some experience in kernel/JVM codesign. In our last project we modified Linux kernel to expose virtual memory subsystem to JVM to improve its GC performance under resource disaggregated datacenters. It will be subimitted to OSDI this year.
Recently we've been looking for new directions. One of the ideas is to merge JVM and kernel together, so that kernel can expose more system information (e.g., memory usage and layout, scheduling, network stack) to JVM. Some potential benefits may include better Java application performance, better GC performance, faster bootstrapping time, etc. The main target will be serverless functions (e.g., AWS Lambda) where most tasks are short lived function invocations. A slow start of JVM is prohibitive. An insight is that both the guest OS and JVM are virtual machines essentially, a lot of language and security checks in JVM may be redundant given that the guest OS is already well isolated. Memory allocation and reclaim may also work better since OS can see semantic information of memory pages. So our first try will be making JVM and kernel share the same memory management subsystem.
For the new project, I am doing preliminary experiments to collect performance numbers and estimate potential benefits of the design. OSv is a very interesting and promising kernel that we'd like to dig more. Its boot time is blazing fast with Firecracker. I also saw from the original ATC paper and source code that it has a JVM ballooning mechanism. But I didn't see how it could work with unmodified JVM. In addition, if I want to compile both OSv and OpenJDK together from source into a single bootable image, is there any solutions or resources? I didn't see such documentation in OSv's github wiki.
I know that OSv is designed to support arbitrary unmodifed Linux binaries. So our idea diverges from OSv's goal. But given the popularity of FaaS and microservices, a dedicated bootable JVM worths a try. Or we may later find some low-intrusive disign that doesn't conflict with OSv's goal.
I know that academic projects often sound unrealistic and not get concerned by open source communities and industrial level projects. But I'd like to hear your opionions and ideas in this design.
Thanks,
Yuanqi
Pekka Enberg
Mar 19, 2020, 10:06:00 AM3/19/20
to Yuanqi Li, Waldek Kozaczuk, Nadav Har'El, OSv Development
Hi,
On Thu, Mar 19, 2020 at 3:30 PM Yuanqi Li <yuanqi...@gmail.com> wrote:
> Recently we've been looking for new directions. One of the ideas is to merge JVM and kernel together, so that kernel can expose more system information (e.g., memory usage and layout, scheduling, network stack) to JVM. Some potential benefits may include better Java application performance, better GC performance, faster bootstrapping time, etc. The main target will be serverless functions (e.g., AWS Lambda) where most tasks are short lived function invocations. A slow start of JVM is prohibitive. An insight is that both the guest OS and JVM are virtual machines essentially, a lot of language and security checks in JVM may be redundant given that the guest OS is already well isolated. Memory allocation and reclaim may also work better since OS can see semantic information of memory pages. So our first try will be making JVM and kernel share the same memory management subsystem.
Yeah, I think this is an interesting direction! Merging the JVM and
the OS was one of the things we were looking to do with OSv, as you
can see from the JVM memory balloon feature.
> For the new project, I am doing preliminary experiments to collect performance numbers and estimate potential benefits of the design. OSv is a very interesting and promising kernel that we'd like to dig more. Its boot time is blazing fast with Firecracker. I also saw from the original ATC paper and source code that it has a JVM ballooning mechanism. But I didn't see how it could work with unmodified JVM. In addition, if I want to compile both OSv and OpenJDK together from source into a single bootable image, is there any solutions or resources? I didn't see such documentation in OSv's github wiki.
I don't remember if we ever built JVM from sources (I'm CC'ing Calle
for this), but it shouldn't be too hard to do.
The way it works in OSv is that you pass the "image" option to
"./scripts/build" to build an image with OSv and some applications:
https://github.com/cloudius-systems/osv#building-osv-kernel-and-creating-images
What you need to do is define a OSv "module" or "application" that
builds JVM from sources. You can find an example of Python being built
from sources here:
https://github.com/cloudius-systems/osv-apps/tree/master/python27-fromsource
Please note that OSv still consumes the applications as ELF
executables and shared libraries, and not combine them into one binary
like other unikernels like Mirage do.
> I know that OSv is designed to support arbitrary unmodifed Linux binaries. So our idea diverges from OSv's goal. But given the popularity of FaaS and microservices, a dedicated bootable JVM worths a try. Or we may later find some low-intrusive disign that doesn't conflict with OSv's goal.
We wanted to run Linux executables out-of-the-box, but did not rule
out OSv-specific APIs. If you allow yourself to modify the JVM
sources, then there's really no problem in adding more OSv "system
calls" (they're really functions) to do what you need to do.
Hope this helps!
- Pekka
On Thu, Mar 19, 2020 at 3:30 PM Yuanqi Li <yuanqi...@gmail.com> wrote:
> Recently we've been looking for new directions. One of the ideas is to merge JVM and kernel together, so that kernel can expose more system information (e.g., memory usage and layout, scheduling, network stack) to JVM. Some potential benefits may include better Java application performance, better GC performance, faster bootstrapping time, etc. The main target will be serverless functions (e.g., AWS Lambda) where most tasks are short lived function invocations. A slow start of JVM is prohibitive. An insight is that both the guest OS and JVM are virtual machines essentially, a lot of language and security checks in JVM may be redundant given that the guest OS is already well isolated. Memory allocation and reclaim may also work better since OS can see semantic information of memory pages. So our first try will be making JVM and kernel share the same memory management subsystem.
the OS was one of the things we were looking to do with OSv, as you
can see from the JVM memory balloon feature.
> For the new project, I am doing preliminary experiments to collect performance numbers and estimate potential benefits of the design. OSv is a very interesting and promising kernel that we'd like to dig more. Its boot time is blazing fast with Firecracker. I also saw from the original ATC paper and source code that it has a JVM ballooning mechanism. But I didn't see how it could work with unmodified JVM. In addition, if I want to compile both OSv and OpenJDK together from source into a single bootable image, is there any solutions or resources? I didn't see such documentation in OSv's github wiki.
for this), but it shouldn't be too hard to do.
The way it works in OSv is that you pass the "image" option to
"./scripts/build" to build an image with OSv and some applications:
https://github.com/cloudius-systems/osv#building-osv-kernel-and-creating-images
What you need to do is define a OSv "module" or "application" that
builds JVM from sources. You can find an example of Python being built
from sources here:
https://github.com/cloudius-systems/osv-apps/tree/master/python27-fromsource
Please note that OSv still consumes the applications as ELF
executables and shared libraries, and not combine them into one binary
like other unikernels like Mirage do.
> I know that OSv is designed to support arbitrary unmodifed Linux binaries. So our idea diverges from OSv's goal. But given the popularity of FaaS and microservices, a dedicated bootable JVM worths a try. Or we may later find some low-intrusive disign that doesn't conflict with OSv's goal.
out OSv-specific APIs. If you allow yourself to modify the JVM
sources, then there's really no problem in adding more OSv "system
calls" (they're really functions) to do what you need to do.
Hope this helps!
- Pekka
Pekka Enberg
Mar 19, 2020, 10:14:00 AM3/19/20
to Yuanqi Li, Waldek Kozaczuk, Nadav Har'El, OSv Development, Carl Wilund
On Thu, Mar 19, 2020 at 4:05 PM Pekka Enberg <pen...@scylladb.com> wrote:
>
> Hi,
>
> On Thu, Mar 19, 2020 at 3:30 PM Yuanqi Li <yuanqi...@gmail.com> wrote:
> > Recently we've been looking for new directions. One of the ideas is to merge JVM and kernel together, so that kernel can expose more system information (e.g., memory usage and layout, scheduling, network stack) to JVM. Some potential benefits may include better Java application performance, better GC performance, faster bootstrapping time, etc. The main target will be serverless functions (e.g., AWS Lambda) where most tasks are short lived function invocations. A slow start of JVM is prohibitive. An insight is that both the guest OS and JVM are virtual machines essentially, a lot of language and security checks in JVM may be redundant given that the guest OS is already well isolated. Memory allocation and reclaim may also work better since OS can see semantic information of memory pages. So our first try will be making JVM and kernel share the same memory management subsystem.
>
> Yeah, I think this is an interesting direction! Merging the JVM and
> the OS was one of the things we were looking to do with OSv, as you
> can see from the JVM memory balloon feature.
>
> > For the new project, I am doing preliminary experiments to collect performance numbers and estimate potential benefits of the design. OSv is a very interesting and promising kernel that we'd like to dig more. Its boot time is blazing fast with Firecracker. I also saw from the original ATC paper and source code that it has a JVM ballooning mechanism. But I didn't see how it could work with unmodified JVM. In addition, if I want to compile both OSv and OpenJDK together from source into a single bootable image, is there any solutions or resources? I didn't see such documentation in OSv's github wiki.
>
> I don't remember if we ever built JVM from sources (I'm CC'ing Calle
> for this), but it shouldn't be too hard to do.
(Really CC'ing Calle this time....)
>
> Hi,
>
> On Thu, Mar 19, 2020 at 3:30 PM Yuanqi Li <yuanqi...@gmail.com> wrote:
> > Recently we've been looking for new directions. One of the ideas is to merge JVM and kernel together, so that kernel can expose more system information (e.g., memory usage and layout, scheduling, network stack) to JVM. Some potential benefits may include better Java application performance, better GC performance, faster bootstrapping time, etc. The main target will be serverless functions (e.g., AWS Lambda) where most tasks are short lived function invocations. A slow start of JVM is prohibitive. An insight is that both the guest OS and JVM are virtual machines essentially, a lot of language and security checks in JVM may be redundant given that the guest OS is already well isolated. Memory allocation and reclaim may also work better since OS can see semantic information of memory pages. So our first try will be making JVM and kernel share the same memory management subsystem.
>
> Yeah, I think this is an interesting direction! Merging the JVM and
> the OS was one of the things we were looking to do with OSv, as you
> can see from the JVM memory balloon feature.
>
> > For the new project, I am doing preliminary experiments to collect performance numbers and estimate potential benefits of the design. OSv is a very interesting and promising kernel that we'd like to dig more. Its boot time is blazing fast with Firecracker. I also saw from the original ATC paper and source code that it has a JVM ballooning mechanism. But I didn't see how it could work with unmodified JVM. In addition, if I want to compile both OSv and OpenJDK together from source into a single bootable image, is there any solutions or resources? I didn't see such documentation in OSv's github wiki.
>
> I don't remember if we ever built JVM from sources (I'm CC'ing Calle
> for this), but it shouldn't be too hard to do.
Nadav Har'El
Mar 19, 2020, 12:38:11 PM3/19/20
to Yuanqi Li, OSv Development
On Thu, Mar 19, 2020 at 3:30 PM Yuanqi Li <yuanqi...@gmail.com> wrote:
Hi all,We are a system research group from UCLA focusing on managed runtimes. We have some experience in kernel/JVM codesign. In our last project we modified Linux kernel to expose virtual memory subsystem to JVM to improve its GC performance under resource disaggregated datacenters. It will be subimitted to OSDI this year.Recently we've been looking for new directions. One of the ideas is to merge JVM and kernel together, so that kernel can expose more system information (e.g., memory usage and layout, scheduling, network stack) to JVM. Some potential benefits may include better Java application performance, better GC performance, faster bootstrapping time, etc. The main target will be serverless functions (e.g., AWS Lambda) where most tasks are short lived function invocations. A slow start of JVM is prohibitive. An insight is that both the guest OS and JVM are virtual machines essentially, a lot of language and security checks in JVM may be redundant given that the guest OS is already well isolated. Memory allocation and reclaim may also work better since OS can see semantic information of memory pages. So our first try will be making JVM and kernel share the same memory management subsystem.
For the new project, I am doing preliminary experiments to collect performance numbers and estimate potential benefits of the design. OSv is a very interesting and promising kernel that we'd like to dig more. Its boot time is blazing fast with Firecracker. I also saw from the original ATC paper and source code that it has a JVM ballooning mechanism. But I didn't see how it could work with unmodified JVM. In addition, if I want to compile both OSv and OpenJDK together from source into a single bootable image, is there any solutions or resources? I didn't see such documentation in OSv's github wiki.
Pekka gave you a few pointers. We have a bunch of Java versions already set up to be built into OSv, e.g.,
scripts/build image=scripts/build modules=openjdk-zulu-9-and-above
Builds the OSv kernel, downloads the latest Azul JVM (binary), and builds an image with the two of them.
The definition of this "openjdk-zulu-9-and-above" module is in apps/opendjk-zulu-9-and-above/, and is slightly
complicated by the fact we have modules for a dozen different Java versions and setup so there's a whole hierarchy
of them. You can find much simpler "modules" (instructions how to build files to put in the Osv image) in apps/
I know that OSv is designed to support arbitrary unmodifed Linux binaries. So our idea diverges from OSv's goal. But given the popularity of FaaS and microservices, a dedicated bootable JVM worths a try. Or we may later find some low-intrusive disign that doesn't conflict with OSv's goal.
I agree. Historically, running the JVM was our primary goal for OSv, even though technically it could run any Linux binary.
Three years ago I wrote this blog post on why I think OSv is a good match for FaaS: http://blog.osv.io/blog/2017/06/12/serverless-computing-with-OSv/
I know that academic projects often sound unrealistic and not get concerned by open source communities and industrial level projects. But I'd like to hear your opionions and ideas in this design.
I don't think there's anything unrealistic about what you're trying to do, but you should come with the right expectations.
One thing you shouldn't expect to happen is that OSv will give your applications significantly better steady-state performance. We hoped this would be the case, and in the paper we were able to demonstrate some improvements, but these weren't huge improvements (e.g., 20%) and always required heavy optimization sessions to find unoptimized parts remaining in OSv and optimizing.
But you're right that startup time can be significantly better with OSv. But this won't help with JVM startup time - for that you'll need to modify the JVM itself. I'm not at all sure if OSv's single-address-space and zero-overhead-system-calls will really be relevant or helpful for these modifications, though. But definitely modifying the small OSv should be easier than modifying the huge and fast-moving Linux.
Thanks,Yuanqi
Waldek Kozaczuk
Mar 19, 2020, 12:55:47 PM3/19/20
to OSv Development
Hi,
On Thursday, March 19, 2020 at 9:30:30 AM UTC-4, Yuanqi Li wrote:
Hi all,We are a system research group from UCLA focusing on managed runtimes. We have some experience in kernel/JVM codesign. In our last project we modified Linux kernel to expose virtual memory subsystem to JVM to improve its GC performance under resource disaggregated datacenters. It will be subimitted to OSDI this year.Recently we've been looking for new directions. One of the ideas is to merge JVM and kernel together, so that kernel can expose more system information (e.g., memory usage and layout, scheduling, network stack) to JVM. Some potential benefits may include better Java application performance, better GC performance, faster bootstrapping time, etc. The main target will be serverless functions (e.g., AWS Lambda) where most tasks are short lived function invocations. A slow start of JVM is prohibitive. An insight is that both the guest OS and JVM are virtual machines essentially, a lot of language and security checks in JVM may be redundant given that the guest OS is already well isolated. Memory allocation and reclaim may also work better since OS can see semantic information of memory pages. So our first try will be making JVM and kernel share the same memory management subsystem.For the new project, I am doing preliminary experiments to collect performance numbers and estimate potential benefits of the design. OSv is a very interesting and promising kernel that we'd like to dig more. Its boot time is blazing fast with Firecracker. I also saw from the original ATC paper and source code that it has a JVM ballooning mechanism. But I didn't see how it could work with unmodified JVM. In addition, if I want to compile both OSv and OpenJDK together from source into a single bootable image, is there any solutions or resources? I didn't see such documentation in OSv's github wiki.
OSv supports both modified and unmodified JVM. Originally OSv required special Java wrapper - OSv "friendly" version of bin/java - to boostrap JVM. You can find more info about running Linux executables including JVM in this wiki - https://github.com/cloudius-systems/osv/wiki/Running-unmodified-Linux-executables-on-OSv and https://github.com/cloudius-systems/osv/wiki/OSv-Linux-ABI-Compatibility#executable-formats. We have many apps/modules examples as well - https://github.com/cloudius-systems/osv/tree/master/modules/openjdk8-from-host, https://github.com/cloudius-systems/osv-apps#osv-applications, all openjdk* apps like https://github.com/cloudius-systems/osv-apps/tree/master/openjdk-zulu-9-and-above
As far as ballooning feature goes unfortunately it is broken at this point, please see - https://github.com/cloudius-systems/osv/issues/1038 and https://github.com/cloudius-systems/osv/issues/398. Please note that ballooning requires java-wrapper as it relies on JNI integration. You also need to uncomment some code - https://github.com/cloudius-systems/osv/blob/master/modules/java-base/java.cc#L175-L188. Besides fixing ballooning it would be also nice to have it integrate with JVM with mechanisms like JVMTI instead of JNI which I think would not require java wrapper but an agent code instead.
I know that OSv is designed to support arbitrary unmodifed Linux binaries. So our idea diverges from OSv's goal. But given the popularity of FaaS and microservices, a dedicated bootable JVM worths a try. Or we may later find some low-intrusive disign that doesn't conflict with OSv's goal.
I do no think your idea diverges from OSv goal. It is just OSv comes with a dynamic linker which I think is a way more flexible mechanism than compiling and linking both app and kernel into single executable. They still share same memory space and system calls are replaced with local function calls.
For even faster startup have you looked into GraalVM native image - OSv supports it as well - please see https://github.com/cloudius-systems/osv-apps/tree/master/graalvm-example and other graalvm-* examples?
I know that academic projects often sound unrealistic and not get concerned by open source communities and industrial level projects. But I'd like to hear your opionions and ideas in this design.
Lastly one thing that should help with faster JVM startup would be speeding up loading JVM binaries from disk. For that we have been working on virtio-fs DAX feature (please see https://virtio-fs.gitlab.io/) that allows direct virtual memory mapping of files between OSv and Linux host.
Thanks,Yuanqi
Waldek Kozaczuk
Mar 19, 2020, 1:00:31 PM3/19/20
to OSv Development
One more thing: OSv provides pretty sophisticated tracing, profiling and debugging tools - if you want to start playing with it more:
Calle Wilund
Mar 23, 2020, 5:32:11 AM3/23/20
to Pekka Enberg, Yuanqi Li, Waldek Kozaczuk, Nadav Har'El, OSv Development
Den 2020-03-19 kl. 15:13, skrev Pekka Enberg:
> On Thu, Mar 19, 2020 at 4:05 PM Pekka Enberg <pen...@scylladb.com> wrote:
>> Hi,
>>
>> On Thu, Mar 19, 2020 at 3:30 PM Yuanqi Li <yuanqi...@gmail.com> wrote:
>>> Recently we've been looking for new directions. One of the ideas is to merge JVM and kernel together, so that kernel can expose more system information (e.g., memory usage and layout, scheduling, network stack) to JVM. Some potential benefits may include better Java application performance, better GC performance, faster bootstrapping time, etc. The main target will be serverless functions (e.g., AWS Lambda) where most tasks are short lived function invocations. A slow start of JVM is prohibitive. An insight is that both the guest OS and JVM are virtual machines essentially, a lot of language and security checks in JVM may be redundant given that the guest OS is already well isolated. Memory allocation and reclaim may also work better since OS can see semantic information of memory pages. So our first try will be making JVM and kernel share the same memory management subsystem.
>> Yeah, I think this is an interesting direction! Merging the JVM and
>> the OS was one of the things we were looking to do with OSv, as you
>> can see from the JVM memory balloon feature.
that. :-)
>>> For the new project, I am doing preliminary experiments to collect performance numbers and estimate potential benefits of the design. OSv is a very interesting and promising kernel that we'd like to dig more. Its boot time is blazing fast with Firecracker. I also saw from the original ATC paper and source code that it has a JVM ballooning mechanism. But I didn't see how it could work with unmodified JVM. In addition, if I want to compile both OSv and OpenJDK together from source into a single bootable image, is there any solutions or resources? I didn't see such documentation in OSv's github wiki.
>> I don't remember if we ever built JVM from sources (I'm CC'ing Calle
>> for this), but it shouldn't be too hard to do.
> (Really CC'ing Calle this time....)
We (OSv) did not. Up until jdk10 or so, building the JDK was massively
>> I don't remember if we ever built JVM from sources (I'm CC'ing Calle
>> for this), but it shouldn't be too hard to do.
> (Really CC'ing Calle this time....)
horrible. Now it is just prohibitively so. The ballooning was iirc just
some JNI coupled with native agent to create objects that could be
partially unmapped. I can't remember if we simply detected memory access
to determine if HS was trying to move it, or if Glauber tried using
JVMTI. The latter would have been potentially expensive...
Reply all
Reply to author
Forward
0 new messages