Next releases - 0.56 and onward - proposal

[Waldek Kozaczuk]

Hi,

Here is a list of things, not necessarily in any importance order, I am proposing we try to accomplish for next releases:

• Prune the list of issues
• Close anything that is irrelevant or possibly completed
• One subcategory I can think of, are issues about improving java wrapper like this one - https://github.com/cloudius-systems/osv/issues/169. Given we can run unmodified /usr/bin/java without need of a wrapper and given the wrapper (java.so) does not support (or is even compatible) with many features of Java 9 and higher, I suggest we close any issues like 169 which are about improving java.so - I do not think it is worth any time investment (the only real benefit of java.so is ballooning feature which is broken - https://github.com/cloudius-systems/osv/issues/796 and https://github.com/cloudius-systems/osv/issues/1038 [duplicate of 796?] - but if anyone is interested in fixing it I think we might support it without java.so anyway). 
• Better annotate existing issues and especially select ones as '"good first issue" (see https://github.blog/2020-01-22-browse-good-first-issues-to-start-contributing-to-open-source/)
• Better support Linux executables built for Musl-based distributions like Alpine (see https://wiki.musl-libc.org/projects-using-musl.html)
• These may work even better given OSv libc implementation is to great extent based on musl
• Tooling
• • Improve capstan
  • • Better support QEMU on OSX (new accelerators)
    • Support pulling artifacts from GitHub nightly releases repo
    • Support running Firecracker, QEMU microvm
    • Support Virtio-FS
  • Create plugins to deploy Java, Node, etc apps
  • • IntelliJ
    • Gradle
    • Maven
  • Run Docker images (possibly musl-based like Alpine are better)
  • There is already existing example - https://github.com/cloudius-systems/osv-apps/tree/master/openjdk12-jre-from-docker
• Review the lazy app thread stack patch aimed to reduce memory consumption
  
• https://groups.google.com/d/msg/osv-dev/tZnwiScmjZY/GkY0hV9EAwAJ
• Modularization/"Librarization"
• See the discussion - https://groups.google.com/d/msg/osv-dev/BHwN6Stm3n4/arvdOwl3AAAJ
• Hide most symbols in the kernel
• • Issue #97 - Be more selective on symbols exported from the kernel
  • See related discussion - https://groups.google.com/d/msg/osv-dev/fs-jmyaXY_E/vWUk-wDDAAAJ
• Reduce kernel size:
• • See the discussion - https://groups.google.com/d/msg/osv-dev/hCYGRzytaJ4/D23S_ibNAgAJ and https://groups.google.com/d/msg/osv-dev/JsJfnhTX_EY/PKUvYm9-AQAJ
• Expose C++ APIs as C to hide C++ completely 
• Keep improving Virtio-FS (#1079)
• • Integrate with pagecache
  • Improve DAX
• Improve ARM support
• See this Firecracker patch - https://groups.google.com/d/msg/osv-dev/3TrFxpW9Bhk/AdilzZT5AQAJ
• Improve run.py
• Reach out to Spirent to see if any more networking related bugs fixed can be brought over from the https://github.com/SpirentOrion/osv fork
• Add support of Virtio GPU/VGA
• Upgrade musl
• I kind of remember Geraldo Netto started on this
• Test newer runtimes
• • Golang >= 1.13
  • Java > 12
  • DENO 1.0
• Support .NET Core executables
• • See what needs to be finished in this thread - https://groups.google.com/d/msg/osv-dev/k69cHw7qvTg/9NqPq70IAwAJ
• Refresh look and feel of http://osv.io

Other interesting existing issues:

• Better deal with small amount of memory - https://github.com/cloudius-systems/osv/issues/1013
  
• #28 - Optimize atomic operations on UP (single-VCPU)
• #1016 - Add support for smb2/3 (samba) file system T
• There is already a patch sent to the list - https://groups.google.com/d/msg/osv-dev/QToC22-QHiA/ejSOMGe7BwAJ
• #1009 - Make ZFS optional as a shared library
• #212 - Support static executables
• #1069 - Add vsock support
• #1041 - Idea for easily creating OSv images from list of Ubuntu packages
  
• does manifest_from_host.sh provide most of it?
• Fix networking bugs
• • https://github.com/cloudius-systems/osv/issues/420
  • https://github.com/cloudius-systems/osv/issues/454
• Faster VFS - https://github.com/cloudius-systems/osv/issues/450
• https://github.com/cloudius-systems/osv/issues/664
• https://github.com/cloudius-systems/osv/issues/589
• Improve XEN support
• https://github.com/cloudius-systems/osv/issues/498
• https://github.com/cloudius-systems/osv/issues/345
• https://github.com/cloudius-systems/osv/issues/264
• https://github.com/cloudius-systems/osv/issues/235
• Speed-up realloc() by mremap() - https://github.com/cloudius-systems/osv/issues/184
• “Steal-time” to improve scheduler - https://github.com/cloudius-systems/osv/issues/181
• Investigate virtio-net “slow path” - https://github.com/cloudius-systems/osv/issues/177
• Try GOLD linker - https://github.com/cloudius-systems/osv/issues/151
• • Gold becoming obsolete?
• Better memory classification
• Investigate what is left - https://github.com/cloudius-systems/osv/issues/87
• Shall we close it - https://github.com/cloudius-systems/osv/issues/84?
• https://github.com/cloudius-systems/osv/issues/70
• The link does not work.
  
• Add NVme driver
• • Please see this email with a link to the branch - https://groups.google.com/d/msg/osv-dev/-vLNjsXrZcI/q2ci4hQ2BAAJ that implements it partially
• Test if still happens with iperf3 - https://github.com/cloudius-systems/osv/issues/232
• Fast restart to run new app
• • How different from ELF namespaces?
  • Reboot to run new app - would use “runscript"

Please give me feedback about any of the above and if you think we should add something I have missed.

Regards,

Waldek

[Nadav Har'El]

On Sat, Jun 6, 2020 at 7:45 PM Waldek Kozaczuk <jwkoz...@gmail.com> wrote:

Hi,

Here is a list of things, not necessarily in any importance order, I am proposing we try to accomplish for next releases:

• Prune the list of issues
• Close anything that is irrelevant or possibly completed
• One subcategory I can think of, are issues about improving java wrapper like this one - https://github.com/cloudius-systems/osv/issues/169. Given we can run unmodified /usr/bin/java without need of a wrapper

Can you please remind me (I haven't been paying enough attention to this Java stuf...) - is this what we do by default? Are there any downsides to not using the wrapper?

If we don't use the wrapper by default and there is no reason to go back to using it, I guess you're right that there is no need to fix any bugs in that wrapper.

 

• and given the wrapper (java.so) does not support (or is even compatible) with many features of Java 9 and higher, I suggest we close any issues like 169 which are about improving java.so - I do not think it is worth any time investment (the only real benefit of java.so is ballooning feature which is broken - https://github.com/cloudius-systems/osv/issues/796 and https://github.com/cloudius-systems/osv/issues/1038 [duplicate of 796?] - but if anyone is interested in fixing it I think we might support it without java.so anyway). 

I agree.

• Better annotate existing issues and especially select ones as '"good first issue" (see https://github.blog/2020-01-22-browse-good-first-issues-to-start-contributing-to-open-source/)
• Better support Linux executables built for Musl-based distributions like Alpine (see https://wiki.musl-libc.org/projects-using-musl.html)
• These may work even better given OSv libc implementation is to great extent based on musl

If you're talking about dynamically-linked executables, what does it matter if the shared library in a distribution is glibc or musl?

What is preventing "Alpine" executables from running on OSv? Do we have an issue about it?

The longer we don't have this feature, the less interesting it seems to be. Before you start working on this issue seriously, I think you need a good reason to do this.

 

• #1069 - Add vsock support
• #1041 - Idea for easily creating OSv images from list of Ubuntu packages
  
• does manifest_from_host.sh provide most of it?
• Fix networking bugs
• • https://github.com/cloudius-systems/osv/issues/420
  • https://github.com/cloudius-systems/osv/issues/454
• Faster VFS - https://github.com/cloudius-systems/osv/issues/450
• https://github.com/cloudius-systems/osv/issues/664
• https://github.com/cloudius-systems/osv/issues/589
• Improve XEN support
• https://github.com/cloudius-systems/osv/issues/498
• https://github.com/cloudius-systems/osv/issues/345
• https://github.com/cloudius-systems/osv/issues/264
• https://github.com/cloudius-systems/osv/issues/235
• Speed-up realloc() by mremap() - https://github.com/cloudius-systems/osv/issues/184
• “Steal-time” to improve scheduler - https://github.com/cloudius-systems/osv/issues/181
• Investigate virtio-net “slow path” - https://github.com/cloudius-systems/osv/issues/177
• Try GOLD linker - https://github.com/cloudius-systems/osv/issues/151
• • Gold becoming obsolete?

Yes, it seems it is. See for example https://www.phoronix.com/forums/forum/software/general-linux-open-source/1082277-gnu-s-gold-linker-is-stagnating-fedora-looking-to-punt-it-off-into-a-separate-package

 

 

• Better memory classification
• Investigate what is left - https://github.com/cloudius-systems/osv/issues/87
• Shall we close it - https://github.com/cloudius-systems/osv/issues/84?
• https://github.com/cloudius-systems/osv/issues/70
• The link does not work.
  

Updated it. I don't know anything about this issue.

• Add NVme driver
• • Please see this email with a link to the branch - https://groups.google.com/d/msg/osv-dev/-vLNjsXrZcI/q2ci4hQ2BAAJ that implements it partially
• Test if still happens with iperf3 - https://github.com/cloudius-systems/osv/issues/232
• Fast restart to run new app
• • How different from ELF namespaces?
  • Reboot to run new app - would use “runscript"

Please give me feedback about any of the above and if you think we should add something I have missed.

Regards,

Waldek

--
You received this message because you are subscribed to the Google Groups "OSv Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/844f3209-ede5-47eb-a6ad-4f93d201e4dco%40googlegroups.com.

[Roman Shaposhnik]

This is such an impressive list! Would love to see it all!

Thanks,
Roman.