on implementing IMR_DOCUMENTFEED

[johnsonj]

I have played IMR_DOCUMENTFEED.

IME : win32 japanese ime, google japanes ime

Editor : Em editor, Sakura editor, ms word 2016, firefox

I implemented it into scintilla and compared them.

but I can not tell any diffenence.

Here  I just leave my implentation,

for Anyone interested in to test.

-------------------  source from now on --------------------

LRESULT ScintillaWin::ImeOnDocumentFeed(LPARAM lParam) {
// Limit feed within one line without eol.
// Look around:   lineStart |<--  |compStart| - caret - compEnd|  -->| lineEnd.

const int curPos = CurrentPosition();
const Sci::Line curLine = pdoc->SciLineFromPosition(curPos);
const Sci::Position lineStart = pdoc->LineStart(curLine);
const Sci::Position lineEnd = pdoc->LineEnd(curLine);

const int codePage = CodePageOfDocument();

const std::wstring rcFeed = StringDecode(RangeText(lineStart, lineEnd), codePage);
const int rcFeedLen = static_cast<int>(rcFeed.length()) * sizeof(wchar_t);
const int rcSize = sizeof(RECONVERTSTRING) + rcFeedLen + sizeof(wchar_t);

RECONVERTSTRING *rc = static_cast<RECONVERTSTRING *>(PtrFromSPtr(lParam));
if (!rc)
return rcSize;

wchar_t *rcFeedStart = reinterpret_cast<wchar_t*>(rc + 1);
memcpy(rcFeedStart, &rcFeed[0], rcFeedLen);

IMContext imc(MainHWND());
if (!imc.hIMC)
return 0;

int compStrLen = imc.GetCompositionString(GCS_COMPSTR).size();
int imeCaretPos = imc.GetImeCaretPos();
Sci::Position compStart = pdoc->GetRelativePositionUTF16(curPos, -imeCaretPos);
int compStrOffset = pdoc->CountUTF16(lineStart, compStart);

// Fill in reconvert structure.
// Let Ime to decide what the target is.
rc->dwVersion = 0; //constant
rc->dwStrLen = static_cast<DWORD>(rcFeed.length());
rc->dwStrOffset = sizeof(RECONVERTSTRING); //constant
rc->dwCompStrLen = static_cast<DWORD>(compStrLen);
rc->dwCompStrOffset = static_cast<DWORD>(compStrOffset) * sizeof(wchar_t);
rc->dwTargetStrLen = rc->dwCompStrLen;
rc->dwTargetStrOffset = rc->dwCompStrOffset;

return rcSize;
}

[johnsonj]

What is documentfeed?
best example :
https://d-toybox.com/studio/weblog/show.php?id=2009121800

But I could not reproduce the above case.
I do not know how to make it work even on firefox.

Here is my findings from implementing documentfeed.
1. windows Ime defaults documentfeed option on (see "predictive input" item in config box)
2. documentfeed convokes only in composition mode.
3. reconvertstring triggers only when hitting reconversion key.
4. composition mode and selection can not go with together.

but sources in action each has a different implement.
It is unclear to me.
so I give up.

I leave lots of resources about implementing IMR_DOCUMENTFEED
japanese document:

https://ttssh2.osdn.jp/manual/4/ja/usage/tips/ime.html
https://sakura-editor.github.io/bbslog/sf/unicode/1317.html
https://wwwcfe.hatenablog.com/entry/20100512/textbox_imr_documentfeed
https://jp.emeditor.com/forums/topic/atok%E3%81%AE%E5%89%8D%E5%BE%8C%E5%8F%82%E7%85%A7%E5%A4%89%E6%8F%9B/
https://topiyama.hatenadiary.org/entry/20070703/p1


sources in action:

https://learn.microsoft.com/en-us/windows/win32/intl/imr-documentfeed
https://chromium.googlesource.com/experimental/chromium/src/+/refs/wip/bajones/webvr/ui/base/ime/input_method_win.cc
https://github.com/sakura-editor/sakura/blob/master/sakura_core/view/CEditView_Ime.cpp
https://searchfox.org/mozilla-central/source/widget/windows/IMMHandler.cpp

[Neil Hodgson]

johnsonj:

> I have played IMR_DOCUMENTFEED.

> I implemented it into scintilla and compared them.
> but I can not tell any diffenence.

I tried again and couldn't see any of the desired effects,
including in the Sakura editor. The patch appears to behave sensibly
with IMR_DOCUMENTFEED and there aren't any runtime errors. There are
some compiler warnings on 64-bit builds: curPos should be
Sci::Position; compStrLen should be size_t; compStrOffset should be
Sci::Position.

Perhaps we should include this and just hope it does something for
someone. I'd much prefer being able to test it but that may require
systems set up better for Japanese.

Neil

[johnsonj]

I think Sci::Position is for scintilla for large document.

but compStrOffset is just short size, not for scintilla.

Thank you, I will try again.

2023년 5월 4일 목요일 오후 5시 37분 36초 UTC+9에 Neil님이 작성:

[Neil Hodgson]

johnsonj:

> I think Sci::Position is for scintilla for large document.
> but compStrOffset is just short size, not for scintilla.

Large documents can work with discovery of local context as the
context is a short distance from the current position.

compStrOffset should be a short size but CountUTF16 returns
Sci::Position so assign it to a Sci::Position or cast the result so
there is no warning.

Neil

[johnsonj]

see documentfeed in action:

 https://www.youtube.com/watch?v=9Ti8EhK4eRw

I sincerely accept your instruction.

patch attached as you point out.

[johnsonj]

document feed in action on scintillawin:

https://www.youtube.com/watch?v=Kj3lTrCcS7M

2023년 5월 5일 금요일 오후 2시 40분 39초 UTC+9에 johnsonj님이 작성:

[Neil Hodgson]

johnsonj:

> patch attached as you point out.

OK, works for me. Committed with
https://sourceforge.net/p/scintilla/code/ci/dcbeefaf7d8c0b7af430eaa5381971ab53388339/
I made a few minor edits: added const, more type harmony, changed comments.

Neil

[johnsonj]

I sincerly appreciate you for your hard works and instructions.

2023년 5월 6일 토요일 오후 6시 48분 47초 UTC+9에 Neil님이 작성:

[johnsonj]

It does not support "look around"

it woks by looking behind, but not looking ahead.

I think it is likely related with supporting windowed input.

https://youtu.be/8-qaEHzaW24

2023년 5월 6일 토요일 오후 7시 50분 22초 UTC+9에 johnsonj님이 작성:

[Neil Hodgson]

johnsonj:

> It does not support "look around"
> it woks by looking behind, but not looking ahead.

The implementation only retrieves the current line so any
contribution from other lines is IME memory from when they were typed
or other sources such as dictionaries.

The GTK implementation also retrieves the current line but the Qt
implementation retrieves the current paragraph. The macOS interface is
more complex and allows the system to ask for any range of text.

Neil