[vim/vim] Convert arbitrary utf-8 string to base64 using `base64

Maxim Kim

unread,

Jan 1, 2025, 1:15:16 AM1/1/25

to vim/vim, Subscribed

I am currently using python -m base64 to convert text in vim buffer to base64.

I have tried to replace it with a new built-in function but it looks like it can't handle some of utf-8:

echo base64_encode(list2blob(str2list("tab:›\ ,nbsp:␣,trail:·,extends:…,precedes:… showbreak=↪", true)))

image.png (view on web)

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.

zeertzjq

unread,

Jan 1, 2025, 2:16:09 AM1/1/25

to vim/vim, Subscribed

Isn't the problem here that you can't easily convert a String to a Blob?

—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.

Maxim Kim

unread,

Jan 1, 2025, 4:48:03 AM1/1/25

to vim/vim, Subscribed

Isn't the problem here that you can't easily convert a String to a Blob?

It is, however the goal is to convert buffer text to base64. List to blob function may work as designed but not necessarily fit for the base64 encode function.

—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.

Yegappan Lakshmanan

unread,

Jan 1, 2025, 9:50:37 AM1/1/25

to vim...@googlegroups.com, reply+ACY5DGDDJ2DAF6YXAY...@reply.github.com

On Tue, Dec 31, 2024 at 10:15 PM Maxim Kim <vim-dev...@256bit.org> wrote:

I am currently using python -m base64 to convert text in vim buffer to base64.

Do you use “!python -m base64” to base64 encode the text in the current buffer?

I have tried to replace it with a new built-in function but it looks like it can't handle some of utf-8:
echo base64_encode(list2blob(str2list("tab:›\ ,nbsp:␣,trail:·,extends:…,precedes:… showbreak=↪", true)))
image.png (view on web)

I do have a patch to add the str2blob() and blob2str() function. But the problem is it currently works properly only for ASCII characters.

Regards,

Yegappan

vim-dev ML

unread,

Jan 1, 2025, 9:51:05 AM1/1/25

to vim/vim, vim-dev ML, Your activity

On Tue, Dec 31, 2024 at 10:15 PM Maxim Kim ***@***.***>
wrote:

> I am currently using python -m base64 to convert text in vim buffer to
> base64.
>

Do you use “!python -m base64” to base64 encode the text in the current
buffer?

I have tried to replace it with a new built-in function but it looks like
> it can't handle some of utf-8:
>
> echo base64_encode(list2blob(str2list("tab:›\ ,nbsp:␣,trail:·,extends:…,precedes:… showbreak=↪", true)))
>
> image.png (view on web)

> <https://github.com/user-attachments/assets/a5599a4a-9f76-4599-a07a-c8c5db287aaa>

>
>
I do have a patch to add the str2blob() and blob2str() function. But the
problem is it currently works properly only for ASCII characters.

Regards,
Yegappan

>

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

Yegappan Lakshmanan

unread,

Jan 1, 2025, 12:52:43 PM1/1/25

to vim...@googlegroups.com, reply+ACY5DGDDJ2DAF6YXAY...@reply.github.com

I have now made these functions work with Unicode strings. I will send out a PR later.

- Yegappan

vim-dev ML

unread,

Jan 1, 2025, 12:53:09 PM1/1/25

to vim/vim, vim-dev ML, Your activity

On Wed, Jan 1, 2025 at 6:50 AM Yegappan Lakshmanan ***@***.***>
wrote:

>
>
> On Tue, Dec 31, 2024 at 10:15 PM Maxim Kim ***@***.***>
> wrote:
>
>> I am currently using python -m base64 to convert text in vim buffer to
>> base64.
>>
>

> Do you use “!python -m base64” to base64 encode the text in the current
> buffer?
>

> I have tried to replace it with a new built-in function but it looks like
>> it can't handle some of utf-8:
>>
>> echo base64_encode(list2blob(str2list("tab:›\ ,nbsp:␣,trail:·,extends:…,precedes:… showbreak=↪", true)))
>>
>> image.png (view on web)

>> <https://github.com/user-attachments/assets/a5599a4a-9f76-4599-a07a-c8c5db287aaa>

>>
>>
> I do have a patch to add the str2blob() and blob2str() function. But the
> problem is it currently works properly only for ASCII characters.
>
>
I have now made these functions work with Unicode strings. I will send out
a PR later.

- Yegappan

>
>>

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

matveyt

unread,

Jan 2, 2025, 1:04:37 AM1/2/25

to vim/vim, vim-dev ML, Comment

I am currently using python -m base64 to convert text in vim buffer to base64.

I have tried to replace it with a new built-in function but it looks like it can't handle some of utf-8:
echo base64_encode(list2blob(str2list("tab:›\ ,nbsp:␣,trail:·,extends:…,precedes:… showbreak=↪", true)))

:echo iconv('tab:›\ ,nbsp:␣,trail:·,extends:…,precedes:… showbreak=↪', 'latin1', &encoding)->str2list()->list2blob()->base64_encode()

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you commented.

matveyt

unread,

Jan 2, 2025, 1:39:09 AM1/2/25

to vim/vim, vim-dev ML, Comment

On Wed, Jan 1, 2025 at 6:50 AM Yegappan Lakshmanan @.***> wrote:
I have now made these functions work with Unicode strings. I will send out a PR later. - Yegappan

function! Str2Blob(str) abort
    return iconv(a:str, 'latin1', &encoding)->str2list()->list2blob()
endfunction

function! Blob2Str(blob) abort
    return blob2list(a:blob)->list2str()->iconv(&encoding, 'latin1')
endfunction

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you commented.

Christian Brabandt

unread,

Jan 2, 2025, 4:22:23 AM1/2/25

to vim/vim, vim-dev ML, Comment

That is a clever trick to force reading byte values. But I would think having separate functions is a bit cleaner.
And it is a bit of an overhead to invoke the iconv() function, when it is not needed yet. And it seems the iconv() function may not always be available. :h +iconv, mentions this is system dependent.

—
Reply to this email directly, view it on GitHub.
You are receiving this because you commented.

matveyt

unread,

Jan 2, 2025, 4:46:14 AM1/2/25

to vim/vim, vim-dev ML, Comment

But I would think having separate functions is a bit cleaner.

Rather an improved version of str2list / list2str. Having also an extra function, such as str2blob is really optional.

—
Reply to this email directly, view it on GitHub.
You are receiving this because you commented.

Maxim Kim

unread,

Jan 2, 2025, 7:34:12 PM1/2/25

to vim/vim, vim-dev ML, Comment

Do you use “!python -m base64” to base64 encode the text in the current buffer?

More or less: https://asciinema.org/a/GLIRVXXjvhfKzCxoSDA1pZem9

I have now made these functions work with Unicode strings. I will send out a PR later.

Great, thank you!

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you commented.

Yegappan Lakshmanan

unread,

Jan 3, 2025, 12:10:16 AM1/3/25

to vim...@googlegroups.com, reply+ACY5DGDDJ2DAF6YXAY...@reply.github.com

I have created the PR https://github.com/vim/vim/pull/16373 to add the str2blob() and blob2str() functions.

Please try these out and let me know if you have any problems in using these functions.

Regards,

Yegappan

vim-dev ML

unread,

Jan 3, 2025, 12:10:42 AM1/3/25

to vim/vim, vim-dev ML, Your activity

On Wed, Jan 1, 2025 at 9:52 AM Yegappan Lakshmanan ***@***.***>
wrote:

>
>
> On Wed, Jan 1, 2025 at 6:50 AM Yegappan Lakshmanan ***@***.***>
> wrote:

>
>>
>>
>> On Tue, Dec 31, 2024 at 10:15 PM Maxim Kim ***@***.***>
>> wrote:
>>
>>> I am currently using python -m base64 to convert text in vim buffer to
>>> base64.
>>>
>>

>> Do you use “!python -m base64” to base64 encode the text in the current
>> buffer?
>>

>> I have tried to replace it with a new built-in function but it looks like
>>> it can't handle some of utf-8:
>>>
>>> echo base64_encode(list2blob(str2list("tab:›\ ,nbsp:␣,trail:·,extends:…,precedes:… showbreak=↪", true)))
>>>
>>> image.png (view on web)

>>> <https://github.com/user-attachments/assets/a5599a4a-9f76-4599-a07a-c8c5db287aaa>

>>>
>>>
>> I do have a patch to add the str2blob() and blob2str() function. But the

>> problem is it currently works properly only for ASCII characters..

>>
>>
> I have now made these functions work with Unicode strings. I will send out
> a PR later.
>

I have created the PR https://github.com/vim/vim/pull/16373 to add the
str2blob() and blob2str() functions.
Please try these out and let me know if you have any problems in using
these functions.

Regards,
Yegappan

—
Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.

Maxim Kim

unread,

Jan 3, 2025, 12:36:31 AM1/3/25

to vim/vim, vim-dev ML, Comment

Please try these out and let me know if you have any problems in using these functions.

Works fine, however a tiny bit different to the python -m base64 -- probably no big deal -- it adds newlines whereas base64_encode() doesn't.

https://asciinema.org/a/wybRstcNg6jHX0x4T1mFXmBxl

—
Reply to this email directly, view it on GitHub.
You are receiving this because you commented.

Maxim Kim

unread,

Jan 3, 2025, 12:42:21 AM1/3/25

to vim/vim, vim-dev ML, Comment

it adds newlines whereas base64_encode()

ok, this is expected

—
Reply to this email directly, view it on GitHub.
You are receiving this because you commented.

Christian Brabandt

unread,

Jan 18, 2025, 3:08:27 AM1/18/25

to vim/vim, vim-dev ML, Comment

Closed #16355 as completed.

—
Reply to this email directly, view it on GitHub.
You are receiving this because you commented.

Christian Brabandt

unread,

Jan 18, 2025, 3:08:27 AM1/18/25

to vim/vim, vim-dev ML, Comment

This is fixed

—
Reply to this email directly, view it on GitHub.
You are receiving this because you commented.

Reply all

Reply to author

Forward

[vim/vim] Convert arbitrary utf-8 string to base64 using `base64_encode()` (Issue #16355)

Maxim Kim

zeertzjq

Maxim Kim

Yegappan Lakshmanan

vim-dev ML

Yegappan Lakshmanan

vim-dev ML

matveyt

matveyt

Christian Brabandt

matveyt

Maxim Kim

Yegappan Lakshmanan

vim-dev ML

Maxim Kim

Maxim Kim

Christian Brabandt

Christian Brabandt