error compiling unicode into stunt?

1 view
Skip to first unread message

Nathan Smith

unread,
Jan 17, 2019, 10:18:38 PM1/17/19
to MOO Talk
Hi all,
I decided to try compile the unicoe patch into stunt.
I used the standard unicode patch for lambdamoo, and just did it by hand.
Unfortunately, I get this error:
utf8.c: In function ‘const char* utf8_substr(const char*, int32, int32)’:
utf8.c:91:11: error: expected unqualified-id before ‘new’
     char* new = NULL;
           ^
utf8.c:91:11: error: expected initializer before ‘new’
utf8.c:185:9: error: expected type-specifier before ‘=’ token
     new = mymalloc(numbytes + 1, M_STRING);
         ^

I've tried googling around, but nothing really seems to fit.
I see nothing wrong with the syntax... I'd appreciate help if possible. Thanks!

Littlefield, Tyler

unread,
Jan 17, 2019, 11:52:18 PM1/17/19
to Nathan Smith, MOO Talk
You can't name variables after c++ keywords.
--
You received this message because you are subscribed to the Google Groups "MOO Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to MOO-talk+u...@googlegroups.com.
To post to this group, send email to MOO-...@googlegroups.com.
Visit this group at https://groups.google.com/group/MOO-talk.
For more options, visit https://groups.google.com/d/optout.


Nathan Smith

unread,
Jan 18, 2019, 11:56:03 AM1/18/19
to MOO Talk
Oops. That fixed it.
I had to also convert void * to char * in two instances due to mymaloc returning int not char *, for char * nnew=NULL;
So, after those were sorted, the server actually compiled and runs fine.
The only problem is it seems to actually have had no affect.
Somewhere it looks as though stunt is deleting a backslash character, here is some sample output:
output1: player:tell("hi\nhow are you.");
hinhow are you
Output 2: player:tell("hi\\nhow are you")
hi\nhow are you.

See what I mean?
Both times it dropped a backslash, and neither times realised that there was a \n in there.

Is there a place in stunt I need to modify to get this working?
Thanks!

Nathan Smith

unread,
Jan 18, 2019, 1:21:25 PM1/18/19
to MOO Talk
oopsies! That would be my bad, then. Is there a patch  floating around to allow this for stunt? I see one for lambda, but that requires other patches, and modifies db_save_lambda, which is no longer a thing in stunt

Littlefield, Tyler

unread,
Jan 18, 2019, 2:35:28 PM1/18/19
to Nathan Smith, MOO Talk
Unicode is not the same as adding line feed characters...
--
You received this message because you are subscribed to the Google Groups "MOO Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to MOO-talk+u...@googlegroups.com.
To post to this group, send email to MOO-...@googlegroups.com.
Visit this group at https://groups.google.com/group/MOO-talk.
For more options, visit https://groups.google.com/d/optout.

Jesús Pavón

unread,
Dec 14, 2019, 7:47:31 PM12/14/19
to MOO Talk
Can anyone tell me where to start? I need to implement unicode in stunt, but I don't understand how it works the patch, patch it is blocked from applying.
What should I modify? I'm pretty lost.
Thanks!

Michael Munson

unread,
Dec 16, 2019, 3:16:29 AM12/16/19
to Jesús Pavón, MOO Talk
  1. Change the representation of the MOO data type string (which is a char *) into a std::string, this includes changing every representation of char * to std::string, and making sure all the places that use const char * convert that into a std::string before using it.
  2. Thats pretty much it.

--
You received this message because you are subscribed to the Google Groups "MOO Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to MOO-talk+u...@googlegroups.com.

Michael Munson

unread,
Dec 16, 2019, 3:45:31 AM12/16/19
to Jesús Pavón, MOO Talk
I will add that you also need to change all the MOO string functions to instead use std::string equivalent functions or recode them. If you're making really heavy use of Chinese it might be more memory efficient to use std::u32string, but for the most part std::string handles utf8 very well.

Lastly you'd still need to disallow carriage return characters based on the way the MOO database works.

Jesús Pavón

unread,
Dec 23, 2019, 12:31:13 PM12/23/19
to MOO Talk
Thanks you!
I don't understand this. "Lastly you'd still need to disallow carriage return characters based on the way the MOO database works."

El lunes, 16 de diciembre de 2019, 9:45:31 (UTC+1), Michael Munson escribió:
I will add that you also need to change all the MOO string functions to instead use std::string equivalent functions or recode them. If you're making really heavy use of Chinese it might be more memory efficient to use std::u32string, but for the most part std::string handles utf8 very well.

Lastly you'd still need to disallow carriage return characters based on the way the MOO database works.

On Mon, Dec 16, 2019 at 1:16 AM Michael Munson <michael...@gmail.com> wrote:
  1. Change the representation of the MOO data type string (which is a char *) into a std::string, this includes changing every representation of char * to std::string, and making sure all the places that use const char * convert that into a std::string before using it.
  2. Thats pretty much it.

On Sat, Dec 14, 2019 at 5:47 PM Jesús Pavón <galo...@gmail.com> wrote:
Can anyone tell me where to start? I need to implement unicode in stunt, but I don't understand how it works the patch, patch it is blocked from applying.
What should I modify? I'm pretty lost.
Thanks!


El viernes, 18 de enero de 2019, 4:18:38 (UTC+1), Nathan Smith escribió:
Hi all,
I decided to try compile the unicoe patch into stunt.
I used the standard unicode patch for lambdamoo, and just did it by hand.
Unfortunately, I get this error:
utf8.c: In function ‘const char* utf8_substr(const char*, int32, int32)’:
utf8.c:91:11: error: expected unqualified-id before ‘new’
     char* new = NULL;
           ^
utf8.c:91:11: error: expected initializer before ‘new’
utf8.c:185:9: error: expected type-specifier before ‘=’ token
     new = mymalloc(numbytes + 1, M_STRING);
         ^

I've tried googling around, but nothing really seems to fit.
I see nothing wrong with the syntax... I'd appreciate help if possible. Thanks!

--
You received this message because you are subscribed to the Google Groups "MOO Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to MOO-...@googlegroups.com.

Michael Munson

unread,
Jan 15, 2020, 8:32:20 AM1/15/20
to Jesús Pavón, MOO Talk
The current MOO database is a flat text file and it is formatted using carriage returns (new lines) to separate data types. If you allow any carriage returns to be stored in any property as a value then when the MOO checkpoints the carriage returns will be written to the database. Then the next time you try to restart the database it will try to read and parse the value that should be stored on one line over two lines and it will fail.

So, the only options are either: 1) change the MOO database (this is the best but most work intensive solution.) You could use a modern binary database of some kind, or even XML or JSON. 2) Disallow carriage returns/new line characters in the MOO. 3) Change the way the flat text file is parsed so that instead of using carriage returns to separate data you use some other character or group of characters that is unique... but then you'd have to disallow these characters/bytes too if they could be produced by the MOO somehow.



To unsubscribe from this group and stop receiving emails from it, send an email to MOO-talk+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/MOO-talk/1f423dbf-3d71-4d48-8612-d2d137eed39a%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages