Fwd: Yet another question for the Google Group!

[Charles Aschmann]

Forwarding this for Michael, who still cannot access the group.

Charles Aschmann

I have a question. I'm noticing that Felix does not perform particularly well if I use relatively large translation memories. My client sent me a TM with around 150,000 translation units (only 90 MB), which I have connected to my current project. However, every time I moved to a new segment, I have to wait quite a long time, which is getting really annoying, especially when I doing many small segments, which would otherwise be very fast. 

Am I doing something wrong, or is there any way to speed up lookup? I tried putting everything in a Memory Serves database, but it was just a slow. Do you have any plans to implement a faster database engine at any point? I know from my experience with TMLookup, e.g., that SQlite is extremely fast with very large databases.

I did have an idea though. Would it be possible to implement some kind of split lookup system, which would basically work as follows: 

When you move to the next segment (and Felix starts looking in your translation memories for anything relevant), Felix looks in two places simultaneously:

(1) in a small project memory, so that lookup is much faster, allowing you to quickly finish the segment and move to the next segment if you want, and

(2) in a so-called background memory, which could be displayed in a secondary window. This lookup/window would have no effect on the perceived speed at which you can move on to the next segment.

Does this make any sense?

Best wishes,

Michael

******************************************************
MICHAEL BEIJER
Dutch-English translator
Hastings, UK.
Tel. +44 (0)1424 430250
Mob. +44 (0)747 5771720
Email: mic...@beijer.uk
Skype/Twitter: michaelbeijer
Beijer.uk (translation/terminology work)
******************************************************

<This email was dictated using Dragon Professional 14. Please excuse any typos!>

[Ryan Ginstrom]

Thanks for forwarding this, Charles.

> On Behalf Of Charles Aschmann

> Forwarding this for Michael, who still cannot access the group.
>

> I have a question. I'm noticing that Felix does not perform particularly well if I
> use relatively large translation memories. My client sent me a TM with around
> 150,000 translation units (only 90 MB), which I have connected to my current
> project. However, every time I moved to a new segment, I have to wait quite a
> long time, which is getting really annoying, especially when I doing many small
> segments, which would otherwise be very fast.

One quick fix is to raise the match threshold. In the Felix memory window, select Tools >> Preferences, then in the Memory tab, set the "Minimum Match Score" to 70% or higher. In testing I've found that going from 10% to 70% can increase lookup speed by as much as 50x.

As for speedups in the engine themselves, there were two areas I was investigating before I got sick:
1. Using sqlite as the memory engine (added benefits are automatic background saves and ability to search tm using SQL)
2. Making searches multi-threaded, where each search happens in a different thread. This will greatly speed up lookups if you have multiple tms and a multi-core machine (true of almost every PC today).

The time I can spend on development is limited (although much better than when I first got sick), and there are some more minor issues that need to be addressed before I can tackle big ones like these. But they are on the radar!

Regards,
Ryan

[Charles Aschmann]

Once again posting for Michael Beijer.

Charles Aschmann

On 2/11/2016 12:00 PM, Ryan Ginstrom wrote:

Thanks for forwarding this, Charles.

On Behalf Of Charles Aschmann
Forwarding this for Michael, who still cannot access the group.

I have a question. I'm noticing that Felix does not perform particularly well if I
use relatively large translation memories. My client sent me a TM with around
150,000 translation units (only 90 MB), which I have connected to my current
project. However, every time I moved to a new segment, I have to wait quite a
long time, which is getting really annoying, especially when I doing many small
segments, which would otherwise be very fast.

One quick fix is to raise the match threshold. In the Felix memory window, select Tools >> Preferences, then in the Memory tab, set the "Minimum Match Score" to 70% or higher. In testing I've found that going from 10% to 70% can increase lookup speed by as much as 50x.

​OK, I tried this, but it didn't seem to improve the speed any. Not sure why not.​

I actually also tried using a RAM disk program (SoftPerfect Ram Disk), thinking that it might help, but it also did not speed up look-up. I'm basically stuck here, trying to get my 500 MB legal TM to work with Felix at the moment.

As for speedups in the engine themselves, there were two areas I was investigating before I got sick:
1. Using sqlite as the memory engine (added benefits are automatic background saves and ability to search tm using SQL)
2. Making searches multi-threaded, where each search happens in a different thread. This will greatly speed up lookups if you have multiple tms and a multi-core machine (true of almost every PC today).

The time I can spend on development is limited (although much better than when I first got sick), and there are some more minor issues that need to be addressed before I can tackle big ones like these. But they are on the radar!

Regards,
Ryan

That sounds interesting indeed!​

I had another idea this morning, based on something that CafeTran can do (using its ‘Total Recall’ system). Would it be possible to somehow extract only the relevant portions of my very large TMX(s) (and save them to a smaller TMX) before starting on a project in Felix? We basically need a way to scan a document against a huge TMX, and extract only those TUs with matches of say 70% and above, and then save these into a smaller TM. As I said, CafeTran can already do this, and I am currently trying it as we speak, in order to try to produce a smaller version of my legal TMX, containing only the most relevant to TUs. CafeTran's Total recall system relies on SQLite.

By the way, are there any maximum .tmx and .ftm sizes that Felix can handle comfortably? Both in number of translation units and megabytes?

Regards,

Michael