Layer-wise adapters

[Surangika Ranathunga]

Hi team

Sarubi (cced here) needs some guidance on connecting adapters layer-wise.

Please send information you have for this.

[Pasindu Udawatta]

Hi Sarubi,

Are you trying to connect adapters to a new model architecture or to integrate adapter modules into an existing model architecture of HuggingFace Transformers?

--
You received this message because you are subscribed to the Google Groups "Lingua" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lingua-cse-1...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lingua-cse-18/CAAFKZ_xDPUr0SsW8h%3DoeUjf7YGkHaSFm27a9wv93m3MisPFeTw%40mail.gmail.com.

--

Pasindu Udawatta

(CIMA Dip MA)

Undergraduate

Department of Computer Science & Engineering

University of Moratuwa

+94 714 793 047

[Sarubi Thillainathan]

Hi Pasindu, 

The later one, integrate to existing model architecture. 

We fine tune using mbart with adapters. 

Thank you in advance,

Sarubi. 

--

Sarubi Thillainathan, 

Doctoral Student, 

Department of Language Science and Technology, 

Saarland University | Universität des Saarlandes.

(m) +94 (0) 76 684 9101 | (e) sar...@coli.uni-saarland.de, sts...@gmail.com

[Sarubi Thillainathan]

Hi Pasindu,

Appreciate your guidance on this! 

[Chathulanka Gamage]

Hi Sarubi,

Sorry for the late reply.

load_adapter(adapter_name_or_path: str, config: Union[dict, str] = None, version: str = None, model_name: str = None, load_as: str = None, source: str = None, with_head: bool = True, custom_weights_loaders: Optional[List[transformers.adapters.loading.WeightsLoader]] = None, leave_out: Optional[List[int]] = None, id2label=None, set_active: bool = False, **kwargs) → str

The "leave_out" parameter allows you to specify a list of layers where you do not want the adapters to be applied. You can find more details on the MBart page on the AdapterHub documentation. 

If you need further assistance, feel free to reach out.

To view this discussion on the web visit https://groups.google.com/d/msgid/lingua-cse-18/CADL2bGPtX-unG5VyzyhvDMrh4EPPfTJFRP9K710xB4vH2hK32Q%40mail.gmail.com.

[Sarubi Thillainathan]

Thank you for sharing it. 

Sure!