Model always outputting empty string in prompt API

[Upekshe Jayasekera]

Hey everyone,
This is about one of the behaviours I noticed few days ago and few questions popped up in my head due to that. 
First about what I noticed, few days ago the response on Prompt API (`await session.prompt("hi")`) was always returning an empty string (at least this behavior was there for 2 days), but when I try today, it is working without an issue. The part I missed earlier was to check the remaining token count on the newly created session when it was returning empty strings. 
Important things to notice is I rarely destroy any of the session I created. So it got me thinking whether this could be due to any uncollected sessions.
So I would like to know,

How are these dangling sessions handled, if those are not specifically destroyed by the program?

Is there way to reset everything, basically clearing any open sessions?
Is there a way to redownload the model and reset the chrome so we reset everything to the starting point?
Additional question,
Is it possible to train this model using any unpublished APIs. (I am looking for a way to personalize this one)

Thanks in advance 

Regards,
Upekshe

[Thomas Steiner]

Hi Upekshe,

This was tracked as https://issues.chromium.org/issues/382864244 and is now fixed in the latest Canary.

Cheers,

Tom

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/3ca8c450-4dc5-40b1-9312-3a4827a0b412n%40chromium.org.

--

Thomas Steiner, PhD—Developer Relations Engineer (blog.tomayac.com, toot.cafe/@tomayac)

Google Germany GmbH, ABC-Str. 19, 20354 Hamburg, Germany
Geschäftsführer: Paul Manicle, Liana Sebastian
Registergericht und -nummer: Hamburg, HRB 86891

----- BEGIN PGP SIGNATURE -----

Version: GnuPG v2.4.3 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck

0fjumBl3DCharaCTersAttH3b0ttom.xKcd.cOm/1181.

----- END PGP SIGNATURE -----

[Upekshe Jayasekera]

Hi Thomas,
Thank you for the quick response!.
Yes, I see my chrome browser (canary) has also updated to the same version, which explains how it started to work again.
but, I would really like to know the answers to the following if possible.

• How are these dangling sessions handled, if those are not specifically destroyed by the program?

• Is there a way to reset any open sessions (if applicable)?
• Is there a way to re-download the model and reset the chrome so we reset everything to the starting point?

[Thomas Steiner]

Sure, let me dive in. 

On Tue, Dec 10, 2024 at 5:03 PM Upekshe Jayasekera <upeks...@gmail.com> wrote:

Hi Thomas,
Thank you for the quick response!.
Yes, I see my chrome browser (canary) has also updated to the same version, which explains how it started to work again.
but, I would really like to know the answers to the following if possible.

• How are these dangling sessions handled, if those are not specifically destroyed by the program?

Dangling sessions are garbage collected eventually, but it's definitely a good practice to do so, for example, as I do here in the playground. 

• Is there a way to reset any open sessions (if applicable)?

There is. See this function in the playground. 

 

• Is there a way to re-download the model and reset the chrome so we reset everything to the starting point?

Not yet via the browser UI, but you can just delete the model manually. Depending on your operating system the model is somewhere different in your Chrome profile. The easiest is to search your Chrome profile for files >1GB and you should be able to identify it.

[Upekshe Jayasekera]

Hi Thomas,

This is very helpful!

Thank you very much.

Best regards, 

Upekshe 

[Upekshe Jayasekera]

Hi Thomas,
About the following question I raised.

Is there a way to reset any open sessions (if applicable)?

I see my question wasn't clear enough. But by "any open sessions" I meant any "unknown" sessions, for example an session I have created earlier but left unclosed(not destroyed).
I think that is not possible atm. Because we are depending on the chrome gc to destroy them eventually and that also means we are not keeping any references internally, which would block go from collecting them.
Thanks
Upekshe

[Thomas Steiner]

I see my question wasn't clear enough. But by "any open sessions" I meant any "unknown" sessions, for example an session I have created earlier but left unclosed(not destroyed).
I think that is not possible atm. Because we are depending on the chrome gc to destroy them eventually and that also means we are not keeping any references internally, which would block go from collecting them.

If there're unknown sessions, you may have leaky code. You can keep track of all the sessions you create and then just check if they're "unused" by looking at the used tokens (if no tokens are used, the session is stale). But ideally you only create a session when your user is likely to engage with it.