Instructlab weekly updates

[Jaideep Rao]

hi all 

This week:

PR reviews:
- reviewed 10 PRs on the CLI repo
https://github.com/instructlab/instructlab/pulls?q=is%3Apr+reviewed-by%3Ajaideepr97+-author%3Ajaideepr97+updated%3A%3E2024-10-18

Updates:

- Started investigating https://github.com/instructlab/instructlab/issues/2523 as part of granite 8B language support, trying to determine if we can solve this for GGUF and safetensor models in the same effort or if they may need to be phased

- Continued some follow up work on my open PRs: (https://github.com/instructlab/instructlab/pull/1895, https://github.com/instructlab/instructlab/pull/2480)

- Pushed the first model to instructlab's quay org https://quay.io/organization/instructlab 

- Fixed a broken functional test in the CI related to model downloads (by doing the aforementioned) to unblock PR merges

- Facilitated conversations around enhancements to the CLI to allow multiple model downloads through the CLI & config file 

Have a great weekend folks!

--

Thanks and regards,
Jaideep Rao

He/Him

Software Engineer

Red Hat

jr...@redhat.com   

[Jaideep Rao]

Hi all 

Last week:

PR Reviews:
reviewed 9 PRs, working on improving that number each week
List of PRs can be found at https://github.com/instructlab/instructlab/pulls?q=is%3Apr+reviewed-by%3Ajaideepr97+-author%3Ajaideepr97+updated%3A%3E2024-10-25 

Updates:

8b support:

- Syced with @Mustafa Eyceoz regarding various changes needed for 8b support 

- Have a working branch for supporting bring-your-own-chat template workflow for 8b support and exposure of the `use-dolomite` flag as backup in case of performance issues with the granite architecture during training. This is currently undergoing testing. I'm currently running into issues testing this due to how we have set up device detection which adds a lot of friction towards testing hardware specific code paths.

This week:

- Plan to get my initial 8b support PR merged 

- sync with @Mustafa Eyceoz to find a mechanism to support easy addition of system prompts going forwards

Have a good week all! 

--

Thanks and regards,
Jaideep Rao

He/Him

Sr. Software Engineer

Red Hat

jr...@redhat.com   

[Jaideep Rao]

hey all

This week:

PR reviews:

Reviewed 11 PRs on CLI:
https://github.com/instructlab/instructlab/pulls?q=is%3Apr+reviewed-by%3Ajaideepr97+-author%3Ajaideepr97+updated%3A%3E2024-11-01+ 

Reviewed 2 PRs on training:
https://github.com/instructlab/training/pulls?q=is%3Apr+reviewed-by%3Ajaideepr97+-author%3Ajaideepr97+updated%3A%3E2024-11-01+

Updates:

Granite-3.0 support:

- Finished implementation for 8b support (draft PR: https://github.com/instructlab/instructlab/pull/2592). I was able to manually test serving, chatting and data generation against the 8b model on my laptop with llamacpp and gguf models as well as on an aws instance with vLLM. There is still some pending work in terms of using the right system prompts with GGUF models and enabling conversion of granite-3.0 models to GGUF format, along with fixing unit tests and getting CI to pass

- merged couple PRs to SDG to prepare SDG to handle granite-3.0 (https://github.com/instructlab/sdg/pull/339 and https://github.com/instructlab/sdg/pull/341)

- raised a PR against training to allow importing chat templates from there easily, which should be merged soon and included in a training release (https://github.com/instructlab/training/pull/324)

Tasks for next week:

- get https://github.com/instructlab/instructlab/pull/2592 passing CI and merged 

- figure out model architecture extraction from GGUF files to pick the correct system prompt for them

- work on https://github.com/instructlab/instructlab/issues/2584 to enable GGUF conversion of granite-3.0 models

Have a great weekend!

[Jaideep Rao]

hey all

This week:

PR reviews:

Reviewed 12 PRs in CLI:
https://github.com/instructlab/instructlab/pulls?q=is%3Apr+reviewed-by%3Ajaideepr97+-author%3Ajaideepr97+updated%3A%3E2024-11-08

Reviewed 2 PRs in training:
https://github.com/instructlab/training/pulls?q=is%3Apr+reviewed-by%3Ajaideepr97+-author%3Ajaideepr97+updated%3A%3E2024-11-08

Reviewed 1 PR in sdg
https://github.com/instructlab/sdg/pulls?q=is%3Apr+reviewed-by%3Ajaideepr97+-author%3Ajaideepr97+updated%3A%3E2024-11-08

Filed a maintainer nomination for @Ben Browning (https://github.com/instructlab/instructlab/issues/2654)

Updates:

granite-arch-support:
- merged https://github.com/instructlab/training/pull/336 in training for 8b-support

- got 8b-support PR merged in CLI (https://github.com/instructlab/instructlab/pull/2592)

- raised a follow up PR to fix model architecture detection for GGUF models (https://github.com/instructlab/instructlab/pull/2660)

- fixed a bug in CI that unblocked the large/nightly job (https://github.com/instructlab/instructlab/pull/2622)

context-aware-chunking:

- Worked with Ben and Aakanksha to get logic for reading docling models into memory reviewed and merged in SDG

Next week:

- hoping to get around to working on https://github.com/instructlab/instructlab/issues/2351 

- Get back to working on https://github.com/instructlab/instructlab/issues/1548

Thanks all, have a great weekend! 

[Carol Chen]

Thank you Jaideep for the weekly updates to the instructlab-dev list!

It would be great to see more from the dev team doing the same :)

Cheers,

 Carol.

--
You received this message because you are subscribed to the Google Groups "dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev+uns...@instructlab.ai.
To view this discussion visit https://groups.google.com/a/instructlab.ai/d/msgid/dev/CAD7TP7T%2BpOu8TxqHKAi_2E%3Ddi1xUZsXqzwS8225i%2BW3WQJryyQ%40mail.gmail.com.