Add an image attribute to popup_create() / popup_setoptions() that renders a pixel buffer inside the popup window. Terminal backends emit the buffer as DEC sixel DCS sequences (FEAT_IMAGE_SIXEL); on the MS-Windows GUI the same data is BitBlt'd through a GDI device bitmap (FEAT_IMAGE_GDI). The popup auto-sizes its cell box from the image's pixel dimensions, so the caller does not have to compute minwidth/maxwidth/minheight/maxheight by hand.
Vim itself does not link against libpng, libjpeg, libwebp or any image-decoding library. The image attribute takes an already-decoded raw RGB / RGBA pixel buffer; format decoding is left to the script caller, who can pipe the file through any external tool (GraphicsMagick, ImageMagick, ffmpeg, custom converter, ...) and pass the resulting bytes via a Blob. This keeps Vim's build footprint unchanged while letting users render PNG, JPEG, GIF, WebP, AVIF or proprietary formats with whatever decoder they already have.
The image dict accepts data (a Blob of width*height*3 for RGB or width*height*4 for RGBA), width and height. RGBA buffers are composited over the popup's background. A companion script function getbgcolor() returns the current background colour as [r, g, b] (from the terminal's OSC 11 response, or the GUI's Normal highlight) so scripts that want to pre-composite their pixels can match the popup's actual backdrop -- handy when generating an image with anti-aliased edges that need to blend cleanly into the editor.
Minimal RGB example — a 4×4 red square popped at the cursor:
let pixels = repeat([0xff, 0x00, 0x00], 4 * 4)->list2blob() call popup_create('', #{ \ image: #{ data: pixels, width: 4, height: 4 }, \ line: 'cursor+1', col: 'cursor', \ })
Real-world example — pipe a PNG through GraphicsMagick and pop the resulting raw RGB bytes:
let png = 'cat.png' let dim = split(system('gm identify -format "%w %h" ' .. shellescape(png))) let [w, h] = [str2nr(dim[0]), str2nr(dim[1])] call system('gm convert ' .. shellescape(png) .. ' -depth 8 rgb:/tmp/cat.rgb') let blob = readblob('/tmp/cat.rgb') call popup_create('', #{ \ image: #{ data: blob, width: w, height: h }, \ line: 1, col: 1, border: [], padding: [0, 0, 0, 0], \ })
Sixel encoding is deferred to popup_adjust_position() so the encoder knows the final winrow and can crop the bottom row of the image to one cell above the screen edge, avoiding sixel-induced terminal scrolling. popup_getoptions() returns the same dict back -- data is a fresh blob copy independent of the popup's internal buffer.
Other refinements:
has('image'), has('image_sixel'), has('image_gdi') are registered as known feature names so the test framework's CheckFeature works and has('image') is no longer a runtime no-op.Tests in test_popupwin.vim cover image acceptance for both RGB and RGBA, blob round-trip through popup_getoptions, the validation paths (wrong data length / missing keys), and the existing dimension-update case (whose blob lengths were corrected -- the previous test silently no-op'd because has('image') returned 0).
With this in place, Vim's expressive range grows considerably: anything that can be rendered to an RGB/RGBA buffer can now sit next to your text, anchored to a textprop and tracking the buffer as it scrolls. A few demos of what this enables straight from a markdown buffer:
PlantUML diagram preview, anchored to a fenced ```plantuml block:
https://github.com/user-attachments/assets/8c6e3973-ff16-4605-91be-51bfe77e4308
LaTeX math rendered through the CodeCogs API and pinned next to the formula:
image.png (view on web)QR code generated for the URL under the cursor, ready to scan with a phone:
https://github.com/user-attachments/assets/352c8398-78b0-4331-8aca-a8d6c010b85e
And -- because the popup just consumes a raw pixel buffer -- you can generate each frame yourself in Vim script and feed it back via popup_setoptions() on a timer. That alone is enough to build an xeyes-style dynamic application, drawn pixel by pixel and updated in place, running entirely inside Vim.
https://github.com/user-attachments/assets/de5ca08a-b0f0-47db-9ac3-ad9ae9d7c89a
https://github.com/vim/vim/pull/20136
(24 files)
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
Wow, this is amazing work
do you plan on supporting the kitty graphics protocol? Thanks
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 2 commits.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@64-bitman Now supported kitty renderer.
https://github.com/user-attachments/assets/fe2b6354-d13c-4b78-883d-aecaee368cef
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
I just took a quick look at the commits, but it seems that detection for the kitty graphics protocol is done via environment variables. However support for it can be queried from the terminal: https://sw.kovidgoyal.net/kitty/graphics-protocol/#querying-support-and-available-transmission-mediums. Not sure if that is feasible though
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
Now supported cairo renderer (gtk3)
image.png (view on web)
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 28 commits.
—
View it on GitHub or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@64-bitman commented on this pull request.
In src/os_unix.c:
> @@ -4527,6 +4527,76 @@ mch_get_shellsize(void)
return OK;
}
+/*
+ * Synchronously ask the terminal for its window pixel dimensions via the
+ * xterm CSI 14 t query and parse the CSI 4 ; H ; W t response. Returns OK
+ * and fills *win_w and *win_h on success. Returns FAIL on any error
+ * (non-tty, no response within ~200ms, malformed reply).
+ */
+ static int
+query_terminal_pixel_size(int *win_w, int *win_h)
Is it possible to do this asynchronously? Although I'm guessing it will be very complicated
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
> + } + buf[n] = 0; + tcsetattr(fd_in, TCSANOW, &old_t); + + // expected: ESC [ 4 ; H ; W t + p = (char *)vim_strchr((char_u *)buf, '\033'); + if (p == NULL || p[1] != '[' || p[2] != '4' || p[3] != ';') + return FAIL; + p += 4; + semi = (char *)vim_strchr((char_u *)p, ';'); + if (semi == NULL) + return FAIL; + end = (char *)vim_strchr((char_u *)semi, 't'); + if (end == NULL) + return FAIL; + hpx = atoi(p);
maybe use strtol instead of atoi?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@64-bitman commented on this pull request.
In src/popupwin.c:
> + return FALSE;
+ new_t = old_t;
+ new_t.c_lflag &= ~(ICANON | ECHO);
+ new_t.c_cc[VMIN] = 0;
+ new_t.c_cc[VTIME] = 3; // 300ms grace per read()
+ if (tcsetattr(fd_in, TCSANOW, &new_t) != 0)
+ return FALSE;
+
+ if (write(fd_out, query, sizeof(query) - 1) != sizeof(query) - 1)
+ {
+ tcsetattr(fd_in, TCSANOW, &old_t);
+ return FALSE;
+ }
+
+ // Read until the DA1 reply (`\e[?...c`) lands or we time out.
+ while (n < (int)sizeof(buf) - 1)
Is it possible to check for the response later when it is received, and during the meantime just make it so that kitty graphics protocol isn't available?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
> + return ok;
+}
+# endif
+
+/*
+ * Pick a terminal image backend. Cached on first call. Returns
+ * IMAGE_BACKEND_KITTY when the host terminal advertises kitty graphics
+ * support, otherwise IMAGE_BACKEND_SIXEL.
+ *
+ * Detection order:
+ * 1. $VIM_IMAGE_BACKEND override ("kitty" | "sixel")
+ * 2. $KITTY_WINDOW_ID (kitty itself)
+ * 3. $GHOSTTY_BIN_DIR / $GHOSTTY_RESOURCES_DIR (Ghostty)
+ * 4. $WEZTERM_EXECUTABLE / $WEZTERM_PANE (WezTerm)
+ * 5. $TERM contains "kitty" / "ghostty" / "wezterm"
+ * 6. $TERM_PROGRAM equals one of the above
I don't think detecting backends based on environment variables is a good idea. Either we can add an option like 'keyprotocol' or just rely on querying the terminal instead. It also seems like Konsole supports the kitty graphics protocol as well.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@mattn commented on this pull request.
In src/os_unix.c:
> @@ -4527,6 +4527,76 @@ mch_get_shellsize(void)
return OK;
}
+/*
+ * Synchronously ask the terminal for its window pixel dimensions via the
+ * xterm CSI 14 t query and parse the CSI 4 ; H ; W t response. Returns OK
+ * and fills *win_w and *win_h on success. Returns FAIL on any error
+ * (non-tty, no response within ~200ms, malformed reply).
+ */
+ static int
+query_terminal_pixel_size(int *win_w, int *win_h)
The query is lazy and cached — it only runs the first time something asks for cell pixel size, and the 200 ms timeout bounds the worst case, so it's a one-time cost on the first image popup.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn commented on this pull request.
In src/popupwin.c:
> + return FALSE;
+ new_t = old_t;
+ new_t.c_lflag &= ~(ICANON | ECHO);
+ new_t.c_cc[VMIN] = 0;
+ new_t.c_cc[VTIME] = 3; // 300ms grace per read()
+ if (tcsetattr(fd_in, TCSANOW, &new_t) != 0)
+ return FALSE;
+
+ if (write(fd_out, query, sizeof(query) - 1) != sizeof(query) - 1)
+ {
+ tcsetattr(fd_in, TCSANOW, &old_t);
+ return FALSE;
+ }
+
+ // Read until the DA1 reply (`\e[?...c`) lands or we time out.
+ while (n < (int)sizeof(buf) - 1)
Same answer as the CSI 14 t one — the probe is lazy and one-shot per session, cached on first call to popup_image_backend(). Async would be a worthwhile follow-up if the cost actually shows up.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 19 commits.
—
View it on GitHub or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 1 commit.
—
View it on GitHub or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 20 commits.
—
View it on GitHub or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
@mattn pushed 5 commits.
—
View it on GitHub or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
Now popup image support clipwindow
https://github.com/user-attachments/assets/2dbe7168-52de-41ff-9a25-d269b1d2f7e5
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
@github-advanced-security[bot] commented on this pull request.
In src/popupwin.c:
> +
+ VIM_CLEAR(wp->w_popup_image_seq);
+
+ // The sixel/kitty encoders read data tightly packed as width*height
+ // pixels. When the source row width changes (left or right clipped),
+ // or when rows must be skipped from the top, build a contiguous cropped
+ // buffer. A pure bottom crop (target_h reduced, full source width) can
+ // reuse the source buffer as-is.
+ if (crop_left_px > 0 || crop_right_px > 0 || crop_top_px > 0)
+ {
+ int bpp = wp->w_popup_image_alpha ? 4 : 3;
+ int src_stride = wp->w_popup_image_w * bpp;
+ int dst_stride = target_w * bpp;
+ int y;
+
+ crop_buf = alloc(target_h * dst_stride);
Multiplication result may overflow 'int' before it is converted to 'size_t'.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()