[Samba] Samba 4 DC log.smd flooded with Conversion error
Kinglok, Fong
Following the Samba AD Howto and running Samba 4.0.3 successfully but with one pretty serious problem. When I try access the folder with 1000 files, the speed is *VERY* slow.
After employ log level to 3, log.smbd is flooded with:
=============================================
[2013/02/09 23:44:05.910717, 3] ../source3/locking/share_mode_lock.c:408(fetch_share_mode_unlocked)
Could not fetch share entry
[2013/02/09 23:44:05.911631, 3] ../source3/smbd/dir.c:1136(smbd_dirptr_get_entry)
smbd_dirptr_get_entry mask=[*] found ./行政 fname=行政 (行政)
[2013/02/09 23:44:05.912607, 3] ../lib/util/charset/convert_string.c:316(convert_string_handle)
convert_string_internal: Conversion error: Illegal multibyte sequence(行政)
[2013/02/09 23:44:05.913517, 3] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(??政)
[2013/02/09 23:44:05.914467, 3] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(?政)
[2013/02/09 23:44:05.915412, 3] ../lib/util/charset/convert_string.c:316(convert_string_handle)
convert_string_internal: Conversion error: Illegal multibyte sequence(政)
[2013/02/09 23:44:05.916356, 3] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(??)
==================================================================
I have googled the mailing list and I am sure that the iconv library is installed correctly and also pass the test for ACL in filesystem in linux listed here (https://wiki.samba.org/index.php/Samba_4/OS_Requirements#Testing_your_filesystem). Also, the folder permission is 770 and owned by root:root.
Can anyone help?
smb.conf
==============
# Global parameters
[global]
workgroup = YAUOICHURCH
realm = SAMBA4.YAUOI.ORG
netbios name = FILE
server role = active directory domain controller
dns forwarder = 192.168.107.1
log level = 2
[netlogon]
path = /usr/local/samba/var/locks/sysvol/samba4.yauoi.org/scripts
read only = No
[sysvol]
path = /usr/local/samba/var/locks/sysvol
read only = No
[printers]
comment = All Printers
path = /usr/local/samba/var/spool
browseable = Yes
read only = No
printable = Yes
[print$]
comment = Point and Print Printer Drivers
path = /usr/local/samba/var/print
read only = No
[profiles]
path = /usr/local/samba/var/profiles
read only = No
[Personal]
path = /home/personal/
read only = No
[Public]
path = /home/group
read only = No
=============
Log level = 10 is as follows:
=============
http://kinglok.org/log.smbd
=============
--
To unsubscribe from this list go to the following URL and read the
instructions: https://lists.samba.org/mailman/options/samba
Jeremy Allison
> My machine is running samba 4.0.3 inside a DomU of Debian Wheezy.
>
> Following the Samba AD Howto and running Samba 4.0.3 successfully but with one pretty serious problem. When I try access the folder with 1000 files, the speed is *VERY* slow.
>
> After employ log level to 3, log.smbd is flooded with:
> =============================================
> [2013/02/09 23:44:05.910717, 3] ../source3/locking/share_mode_lock.c:408(fetch_share_mode_unlocked)
> Could not fetch share entry
> [2013/02/09 23:44:05.911631, 3] ../source3/smbd/dir.c:1136(smbd_dirptr_get_entry)
> smbd_dirptr_get_entry mask=[*] found ./行政 fname=行政 (行政)
> [2013/02/09 23:44:05.912607, 3] ../lib/util/charset/convert_string.c:316(convert_string_handle)
> convert_string_internal: Conversion error: Illegal multibyte sequence(行政)
> [2013/02/09 23:44:05.913517, 3] ../lib/util/charset/convert_string.c:297(convert_string_handle)
> convert_string_internal: Conversion error: Incomplete multibyte sequence(??政)
> [2013/02/09 23:44:05.914467, 3] ../lib/util/charset/convert_string.c:297(convert_string_handle)
> convert_string_internal: Conversion error: Incomplete multibyte sequence(?政)
> [2013/02/09 23:44:05.915412, 3] ../lib/util/charset/convert_string.c:316(convert_string_handle)
> convert_string_internal: Conversion error: Illegal multibyte sequence(政)
> [2013/02/09 23:44:05.916356, 3] ../lib/util/charset/convert_string.c:297(convert_string_handle)
> convert_string_internal: Conversion error: Incomplete multibyte sequence(??)
> ==================================================================
Is that filename '行政' in utf8 format on the disk ?
Or is it in a previous encoding format such as big5 or
Guobiao ?
Without a parameter setting Samba defaults to utf8
encoding, and things can go wrong if filenames on
disk aren't actually utf8.
Jeremy.
TAKAHASHI Motonobu
parameter if you do not use UTF-8 on Linux.
Kinglok, Fong
I execute some commands to make sure the locale is in UTF-8 by
dpkg-reconfigure locales and even adding setting in /etc/environment
and using utility like convmv to turn all file and folder into UTF-8 (in fact, they were in UTF-8 already.)
I add option in smb.conf
unix charset = UTF8
dos charset is omitted as default (dos charset = CP850)
However, when I run
/usr/local/samba/bin/smbclient //localhost/Public -UAdministrator%'verysecurepasswd' -c 'ls'
The same error in my log floods……
Is it a bug?
Kinglok, Fong
Kinglok, Fong
I have created another test case for the problem.
Rather testing in domU, running Debian Wheezy, I have constructed another machine without running xen and installed with Debian squeeze.
I have successfully setup samba 4.0.3 and create a folder called "中港矛盾" in share named "test". And I have used convmv to make sure the name is in UTF-8
================================
root@file:/home/test# convmv * -t utf8
Your Perl version has fleas #37757 #49830
Starting a dry run without changes...
Skipping, already UTF-8: ./中港矛盾
No changes to your files done. Use --notest to finally rename the files.
================================
When I issue a command:
/usr/local/samba/bin/smbclient //localhost/test -UAdministrator%'verysecurepassword' -c 'ls'
The log.smbd with log level = 3 is as follows: (the same conversion error!)
=================================
[2013/02/11 22:19:15.472365, 3] ../source3/smbd/vfs.c:1118(check_reduced_name)
check_reduced_name [*] [/home/test]
[2013/02/11 22:19:15.472461, 3] ../source3/smbd/vfs.c:1248(check_reduced_name)
check_reduced_name: * reduced to /home/test/*
[2013/02/11 22:19:15.472597, 3] ../source3/smbd/dir.c:663(dptr_create)
creating new dirptr 256 for path ., expect_close = 1
[2013/02/11 22:19:15.472886, 3] ../source3/locking/share_mode_lock.c:408(fetch_share_mode_unlocked)
Could not fetch share entry
[2013/02/11 22:19:15.472956, 3] ../source3/smbd/dir.c:1136(smbd_dirptr_get_entry)
smbd_dirptr_get_entry mask=[*] found ./. fname=. (.)
[2013/02/11 22:19:15.473110, 3] ../source3/locking/share_mode_lock.c:408(fetch_share_mode_unlocked)
Could not fetch share entry
[2013/02/11 22:19:15.473173, 3] ../source3/smbd/dir.c:1136(smbd_dirptr_get_entry)
smbd_dirptr_get_entry mask=[*] found ./.. fname=.. (..)
[2013/02/11 22:19:15.473285, 3] ../lib/util/charset/convert_string.c:316(convert_string_handle)
convert_string_internal: Conversion error: Illegal multibyte sequence(�+#"1�㈐∮¤ギ<9f><9b>�+.<9b>¾)
[2013/02/11 22:19:15.473347, 3] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(�"1�㈐∮¤ギ<9f><9b>�+.<9b>¾)
[2013/02/11 22:19:15.473404, 3] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(�㈐∮¤ギ<9f><9b>�+.<9b>¾)
[2013/02/11 22:19:15.473462, 3] ../lib/util/charset/convert_string.c:316(convert_string_handle)
convert_string_internal: Conversion error: Illegal multibyte sequence(�)A"1"4+.<9f><9b>�+.<9b>¾)
[2013/02/11 22:19:15.473535, 3] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(�"1"4+.<9f><9b>�+.<9b>¾)
[2013/02/11 22:19:15.473592, 3] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(�"4+.<9f><9b>�+.<9b>¾)
[2013/02/11 22:19:15.473649, 3] ../lib/util/charset/convert_string.c:316(convert_string_handle)
convert_string_internal: Conversion error: Illegal multibyte sequence(�+.<9f><9b>�+.<9b>¾)
[2013/02/11 22:19:15.473705, 3] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(<9f><9b>�+.<9b>¾)
[2013/02/11 22:19:15.473762, 3] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(<9b>�+.<9b>¾)
[2013/02/11 22:19:15.473819, 3] ../lib/util/charset/convert_string.c:316(convert_string_handle)
convert_string_internal: Conversion error: Illegal multibyte sequence(�+.<9b>¾)
[2013/02/11 22:19:15.473875, 3] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(<9b>¾)
[2013/02/11 22:19:15.473964, 3] ../source3/locking/share_mode_lock.c:408(fetch_share_mode_unlocked)
Could not fetch share entry
======================================
the smb.conf
======================================
# Global parameters
[global]
workgroup = PLKLSP2
realm = SAMBA4.PLKLSP.EDU.HK
netbios name = FILE
server role = active directory domain controller
dns forwarder = 192.168.107.1
log level = 3
[netlogon]
path = /usr/local/samba/var/locks/sysvol/samba4.plklsp.edu.hk/scripts
read only = No
[sysvol]
path = /usr/local/samba/var/locks/sysvol
read only = No
[test]
path = /home/test
read only = No
========================================
As far as I know, I think the conversion error is not caused by Xen and debian version.
How can I offer more help for testing / debugging this issue?
Kinglok, Fong
TAKAHASHI Motonobu
Date: Sun, 10 Feb 2013 09:40:49 +0800
> Thank you for your help but…
>
> I execute some commands to make sure the locale is in UTF-8 by
> dpkg-reconfigure locales and even adding setting in /etc/environment
>
> and using utility like convmv to turn all file and folder into UTF-8 (in fact, they were in UTF-8 already.)
>
> I add option in smb.conf
> unix charset = UTF8
> dos charset is omitted as default (dos charset = CP850)
>
> However, when I run
> /usr/local/samba/bin/smbclient //localhost/Public -UAdministrator%'verysecurepasswd' -c 'ls'
>
> The same error in my log floods……
environment, same errors occur unless I set "dos charset = CP932", which
means Japanese. It seems that you use Chinese.
---
TAKAHASHI Motonobu <mo...@monyo.com> / @damemonyo
facebook.com/takahashi.motonobu
Kinglok, Fong
After setting dos charset = CP950 (which is the codepage for traditional chinese), the same error still remains.
Furthermore, I have tried testing in a real production environment by the following steps:
1. In smb.conf,
I create a share called Chinese and increase the log level to 10.
[global]
workgroup = YAUOICHURCH
realm = SAMBA4.YAUOI.ORG
netbios name = FILE
server role = active directory domain controller
dns forwarder = 192.168.107.1
log level = 10
unix charset = UTF8
dos charset = CP950
[netlogon]
path = /usr/local/samba/var/locks/sysvol/samba4.yauoi.org/scripts
read only = No
[sysvol]
path = /usr/local/samba/var/locks/sysvol
read only = No
...
[Chinese]
path = /home/chinese
read only = No
2. In the file server, I have created a directory /home/chinese and also, inside it, I have open a folder called "中文測試" and I have tested the name is in UTF-8.
3. I made use of Windows 7 64-bit Traditional Chinese as client to try to browse the share Chinese and then the folder (the client has joined the domain already) and I install wireshark to capture the packet.
The wireshark capture is here:
http://kinglok.org/wireshark2.pcapng
4. the log.smbd still show
[2013/02/13 18:12:11.869648, 3, pid=6810, effective(0, 100), real(0, 0)] ../lib/util/charset/convert_string.c:316(convert_string_handle)
convert_string_internal: Conversion error: Illegal multibyte sequence(�+#"1�㈐<96><87>�)A"1￢����)
[2013/02/13 18:12:11.870614, 3, pid=6810, effective(0, 100), real(0, 0)] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(�"1�㈐<96><87>�)A"1￢����)
[2013/02/13 18:12:11.871426, 3, pid=6810, effective(0, 100), real(0, 0)] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(�㈐<96><87>�)A"1￢����)
[2013/02/13 18:12:11.872210, 3, pid=6810, effective(0, 100), real(0, 0)] ../lib/util/charset/convert_string.c:316(convert_string_handle)
convert_string_internal: Conversion error: Illegal multibyte sequence(�)A<96><87>�)A"1￢����)
[2013/02/13 18:12:11.872970, 3, pid=6810, effective(0, 100), real(0, 0)] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(<96><87>�)A"1￢����)
[2013/02/13 18:12:11.873757, 3, pid=6810, effective(0, 100), real(0, 0)] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(<87>�)A"1￢����)
[2013/02/13 18:12:11.874520, 3, pid=6810, effective(0, 100), real(0, 0)] ../lib/util/charset/convert_string.c:316(convert_string_handle)
convert_string_internal: Conversion error: Illegal multibyte sequence(�)A"1￢����)
[2013/02/13 18:12:11.875503, 3, pid=6810, effective(0, 100), real(0, 0)] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(�"1￢����)
[2013/02/13 18:12:11.876199, 3, pid=6810, effective(0, 100), real(0, 0)] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(￢����)
[2013/02/13 18:12:11.876953, 3, pid=6810, effective(0, 100), real(0, 0)] ../lib/util/charset/convert_string.c:316(convert_string_handle)
convert_string_internal: Conversion error: Illegal multibyte sequence(�+2"m"C)
[2013/02/13 18:12:11.877743, 3, pid=6810, effective(0, 100), real(0, 0)] ../lib/util/charset/convert_string.c:297(convert_string_handle)
convert_string_internal: Conversion error: Incomplete multibyte sequence(�"m"C)
The whole log is here:
http://kinglok.org/log.smbd.2
Should I file a bug for it?
Kinglok, Fong