[sqlite] sqlite3_column_text() returning partial results

1 view
Skip to first unread message

Jacob A. Camp

unread,
Dec 20, 2011, 4:37:22 PM12/20/11
to sqlite...@sqlite.org
Hello,

I've been looking into an issue that a few of our programmers have looked at as well and it left us all wondering. Basically, our database has a VARCHAR column that has an XML file written to it when the object is manipulated. I can use a tool to view the database file and I can ensure that the entire field is filled out correctly and the XML is correctly formed.

The field in the database contains 8955 characters and when I execute the sqlite3_column_text() on that specific column to access the data, the const unsigned char* that's returned only contains 2030 characters and the XML file that I'm trying to reconstruct from it then becomes unusable. I even attempted digging into the SQLite class and it seems that this value is obtained from the official function calls.

In sqlite3.c:

static Mem *columnMem(sqlite3_stmt *pStmt, int i){
Vdbe *pVm;
int vals;
Mem *pOut;

pVm = (Vdbe *)pStmt;
if( pVm && pVm->pResultSet!=0 && i<pVm->nResColumn && i>=0 ){
sqlite3_mutex_enter(pVm->db->mutex);
vals = sqlite3_data_count(pStmt);
pOut = &pVm->pResultSet[i];

After the last line has been executed pOut contains members z and zMalloc which both contain the same memory location that points to the char* that contains the first 2030 characters and none of the rest that are stored in the VARCHAR field.

Is there some limit to the amount of data that can be returned by this function? I read over most of the documentation and didn't see it mentioning any sort of restriction.

Thanks in advance,
--Jake

**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mastercam.com
**********************************************************************
_______________________________________________
sqlite-users mailing list
sqlite...@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Igor Tandetnik

unread,
Dec 21, 2011, 8:20:02 AM12/21/11
to sqlite...@sqlite.org
Jacob A. Camp <Jacob...@mastercam.com> wrote:
> The field in the database contains 8955 characters and when I execute the sqlite3_column_text() on that specific column to access
> the data, the const unsigned char* that's returned only contains 2030 characters

What do you mean by that? sqlite3_column_text doesn't provide any length delimiter. What's in p[2030], p[2031] and so on (where p is a pointer obtained from sqlite3_column_text)?

Are you examining the string immediately after calling sqlite3_column_text? Some sqlite3_* calls invalidate the pointer (by reusing or freeing the memory behind it).

What does sqlite3_column_bytes return for this column?
--
Igor Tandetnik

nobre

unread,
Dec 21, 2011, 8:31:52 AM12/21/11
to sqlite...@sqlite.org

Is there any chance you are storing a \0 char inside the xml ?

--
View this message in context: http://old.nabble.com/sqlite3_column_text%28%29-returning-partial-results-tp33016613p33016758.html
Sent from the SQLite mailing list archive at Nabble.com.

Simon Slavin

unread,
Dec 21, 2011, 8:34:06 AM12/21/11
to General Discussion of SQLite Database

On 21 Dec 2011, at 1:31pm, nobre wrote:

> Is there any chance you are storing a \0 char inside the xml ?

Or that you are mixing 8-bit and 16-bit Unicode in such a way that one of your routines thinks that it has read a 0x00 termination character ?

Simon.

Jake

unread,
Dec 21, 2011, 10:04:33 AM12/21/11
to sqlite...@sqlite.org
Simon Slavin <slavins@...> writes:

>
>
> On 21 Dec 2011, at 1:31pm, nobre wrote:
>
> > Is there any chance you are storing a \0 char inside the xml ?
>
> Or that you are mixing 8-bit and 16-bit Unicode in such a way that one of your
routines thinks that it has read a
> 0x00 termination character ?
>
> Simon.
>


From what I looked at so far, by taking the text and putting it into a basic
text editor so I could see any special characters that would have been added. I
didn't see any \0 or other odd things.

I would check the mixing 8-bit and 16-bit Unicode but I need to research how to
even figure something like that out.

For the previous post asking what occurs after 2030, there's nothing because the
returned value is only allocated for those characters. And regarding the
recycling of the memory the actual return result of the function call after its
complete is the same as when i'm viewing it while debugging.

I can post the stored value and the value the call to sqlite3_column_text
returns if that would be helpful (its just long so I wont post it).

Jake

Igor Tandetnik

unread,
Dec 21, 2011, 10:28:23 AM12/21/11
to sqlite...@sqlite.org
Jake <jacob...@mastercam.com> wrote:
> For the previous post asking what occurs after 2030, there's nothing

What do you mean, nothing? What exactly happens when you try to access p[2030]?

> because the
> returned value is only allocated for those characters.

What makes you believe this?
--
Igor Tandetnik

Jacob A. Camp

unread,
Dec 21, 2011, 10:41:16 AM12/21/11
to General Discussion of SQLite Database
I have a line of code that executes that line:

const unsigned char * temp2 = sqlite3_column_text(state, 0);

This queries the database and after the call is complete I pass this value to another function. This function then fails because temp2 points to a location that contains the incomplete text. I then can view this text using my debugger and I can see that the value has been clipped and the XML is invalid.

Trying to access memory locations outside whatever is allocated by the return of sqlite3_column_text sounds like it would result in undefined behavior?

sqlite3_column_bytes returns 8960 if that's helpful.

**********************************************************************


This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mastercam.com
**********************************************************************

Black, Michael (IS)

unread,
Dec 21, 2011, 11:00:12 AM12/21/11
to General Discussion of SQLite Database
What does strlen() tell you on the XML before you put it in the table?

Can you boil all this down to one record in a table, and some code so we can all see what's going on?

Do a .dump of the table and post the code.

See if you can reproduce it in a simple example.

Michael D. Black

Senior Scientist

Advanced Analytics Directorate

Advanced GEOINT Solutions Operating Unit

Northrop Grumman Information Systems

________________________________
From: sqlite-use...@sqlite.org [sqlite-use...@sqlite.org] on behalf of Jacob A. Camp [Jacob...@mastercam.com]
Sent: Wednesday, December 21, 2011 9:41 AM
To: 'General Discussion of SQLite Database'
Subject: EXT :Re: [sqlite] sqlite3_column_text() returning partial results

www.mastercam.com<http://www.mastercam.com/>


**********************************************************************
_______________________________________________
sqlite-users mailing list
sqlite...@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Pavel Ivanov

unread,
Dec 21, 2011, 11:19:22 AM12/21/11
to General Discussion of SQLite Database
On Wed, Dec 21, 2011 at 10:41 AM, Jacob A. Camp
<Jacob...@mastercam.com> wrote:
> I have a line of code that executes that line:
>
> const unsigned char * temp2 = sqlite3_column_text(state, 0);
>
> This queries the database and after the call is complete I pass this value to another function. This function then fails because temp2 points to a location that contains the incomplete text. I then can view this text using my debugger and I can see that the value has been clipped and the XML is invalid.

Let me ask Igor's questions once again. How does your function
understand that text is incomplete? Your debugger can show you
incomplete string because temp2 is not a zero-terminated string, it
can contain zeros in the middle.

> Trying to access memory locations outside whatever is allocated by the return of sqlite3_column_text sounds like it would result in undefined behavior?

How do you know how much memory sqlite3_column_text allocated for the return?

> sqlite3_column_bytes returns 8960 if that's helpful.

Hint: the above number means that sqlite3_column_text allocated
exactly 8960 bytes for the text it returned. So whatever you use to
define that returned text is incomplete gives you incorrect
information.


Pavel

Jacob A. Camp

unread,
Dec 21, 2011, 1:34:58 PM12/21/11
to General Discussion of SQLite Database
So delving further into this has gotten me nothing else; this is the best I could do to simplify this to understandable steps.

(I'm new to this service so no idea if attachments are allows so sorry about the length in advance)

I have this code:

sqlite3_stmt* state = NULL;
CString sql = _T("SELECT RelationshipHierarchyXML FROM TlAssembly WHERE id=?");
//Query the database
RawSQLTlID (sql, id, &state);
bytes = sqlite3_column_bytes(state, 0); //This line returns 8955
//Try getting data as blob
const void * temp = sqlite3_column_blob(state, 0);
//Set the size for simplification
char chararray[9000];
//Copy value into the char array
memcpy( chararray , temp , bytes);
CString testing;
//Iterate through all elements
for (int x = 0; x < 8999; x++)
{
int temp = chararray[x];
char temp2 = chararray[x];

//if the char is null or not supportted don't append it to CString
if (temp < 1 || temp > 255)
{
//This gets hit a few times with negative int values
temp = temp;
}
else
{
//Append the char to the XML string
testing.AppendChar(temp2);
}
}

return success;

At this point the CString testing contains on a portion of the characters of the XML field which makes it unusable.

Supporting Code:

bool TlToolLibrarySQL::RawSQLTlID (const CString& sql, const TlID& id, sqlite3_stmt** statement, UINT expectedResult)
{
bool found = false;

// Pointer to the uncompiled portion of the SQL query
const char* unused = NULL;

// Prepare the SQL statement
int result = sqlite3_prepare_v2 (m_db, sql, -1, statement, &unused);

// Bind the GUID ID to the statement
GUID tlid = id;
int result1 = sqlite3_bind_blob (*statement, 1, &tlid, sizeof(tlid), SQLITE_TRANSIENT);

// If we prepared and got a row back, we are successful!
if (SQLITE_OK == result && SQLITE_OK == result1)
{
if (expectedResult == sqlite3_step (*statement))
{
found = true;
}
else
{
//Try again with a String GUID instead
sqlite3_reset (*statement);
sqlite3_clear_bindings (*statement);
result1 = sqlite3_bind_text (*statement, 1, id.ToString(), -1, SQLITE_TRANSIENT);
if (SQLITE_OK == result1 && expectedResult == sqlite3_step (*statement))
{
found = true;
}
}
}
return found;
}

Value of CString testing after iterating though the entire char array:

"<?xml version="1.0" encoding="utf-8"?>
<Machine xmlns:i="http://www.w3.org/2001/XMLSchema-instance" z:Id="i1" xmlns:z="http://schemas.microsoft.com/2003/10/Serialization/" xmlns="http://schemas.datacontract.org/2004/07/MDM.MachineDefinition.Models">
<IsVisible>true</IsVisible>
<ModelingTransformation z:Id="i2">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</ModelingTransformation>
<Name>Machine</Name>
<Tag i:nil="true" />
<Transformation z:Id="i3">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</Transformation>
<Components>
<Component z:Id="i4" i:type="ToolProfile">
<IsVisible>true</IsVisible>
<ModelingTransformation z:Id="i5">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</ModelingTransformation>
<Name> 3/8 DRILL</Name>
<Tag i:nil="true" />
<Transformation z:Id="i6">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>

CSV Dump of the TlAssembly Table:

ID,Name,Description,MainHolder,MainTool,ToolNumber,MachineGroup,RelationshipHierarchyXML,Stickout
{9B6CB288-E5E0-4A8A-88C8-20E3A09A7429}, 3/8 DRILL Assembly,"",{351FDBA2-B09F-4264-8227-8FE41C26865C},{02BBEE7C-7856-4E3C-9FDB-FFFB17050197},128,0,"?<?xml version=""1.0"" encoding=""utf-8""?>
<Machine xmlns:i=""http://www.w3.org/2001/XMLSchema-instance"" z:Id=""i1"" xmlns:z=""http://schemas.microsoft.com/2003/10/Serialization/"" xmlns=""http://schemas.datacontract.org/2004/07/MDM.MachineDefinition.Models"">
<IsVisible>true</IsVisible>
<ModelingTransformation z:Id=""i2"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</ModelingTransformation>
<Name>Machine</Name>
<Tag i:nil=""true"" />
<Transformation z:Id=""i3"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</Transformation>
<Components>
<Component z:Id=""i4"" i:type=""ToolProfile"">
<IsVisible>true</IsVisible>
<ModelingTransformation z:Id=""i5"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</ModelingTransformation>
<Name> 3/8 DRILL</Name>
<Tag i:nil=""true"" />
<Transformation z:Id=""i6"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</Transformation>
<Components>
<Component z:Id=""i7"" i:type=""Group"">
<IsVisible>true</IsVisible>
<ModelingTransformation z:Id=""i8"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>3</M42>
<M43>0</M43>
<M44>1</M44>
</ModelingTransformation>
<Name>Target</Name>
<Tag>DefaultTarget</Tag>
<Transformation z:Id=""i9"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</Transformation>
<Components />
</Component>
<Component z:Id=""i10"" i:type=""LinearAxis"">
<IsVisible>true</IsVisible>
<ModelingTransformation z:Id=""i11"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</ModelingTransformation>
<Name>LinearAxis</Name>
<Tag i:nil=""true"" />
<Transformation z:Id=""i12"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</Transformation>
<Components>
<Component z:Id=""i13"" i:type=""ToolProfile"">
<IsVisible>true</IsVisible>
<ModelingTransformation z:Id=""i14"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</ModelingTransformation>
<Name>-- tp_tool Default Holder --</Name>
<Tag i:nil=""true"" />
<Transformation z:Id=""i15"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</Transformation>
<Components>
<Component z:Id=""i16"" i:type=""Group"">
<IsVisible>true</IsVisible>
<ModelingTransformation z:Id=""i17"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>1</M42>
<M43>0</M43>
<M44>1</M44>
</ModelingTransformation>
<Name>Target</Name>
<Tag>DefaultTarget</Tag>
<Transformation z:Id=""i18"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</Transformation>
<Components />
</Component>
</Components>
<HexColor>#6699cc</HexColor>
<Opacity>1</Opacity>
<ProfileType>Holder</ProfileType>
<StickOut>0</StickOut>
<ToolID>351fdba2-b09f-4264-8227-8fe41c26865c</ToolID>
</Component>
</Components>
<DirectionX>0</DirectionX>
<DirectionY>1</DirectionY>
<DirectionZ>0</DirectionZ>
<InitialPosition>0</InitialPosition>
<Max>3</Max>
<MaxFeedrate z:Id=""i19"">
<Inch i:nil=""true"" />
<Metric i:nil=""true"" />
</MaxFeedrate>
<Min>2.5</Min>
<Position>3</Position>
<AxisType>Y</AxisType>
</Component>
</Components>
<HexColor>#6699cc</HexColor>
<Opacity>1</Opacity>
<ProfileType>Tool</ProfileType>
<StickOut>3</StickOut>
<ToolID>00000000-0000-0000-0000-000000000000</ToolID>
</Component>
</Components>
<Control></Control>
<Family></Family>
<IsMetric>false</IsMetric>
<MachineToolType>VerticalLathe</MachineToolType>
<Manufacturer></Manufacturer>
<Model></Model>
<Series></Series>
<SubModel></SubModel>
</Machine>",
{792708B6-2352-4B1D-89C4-FAA65E4EA3F1},EPIC 1/4 FLAT ENDMILL Assembly,"",{D22D6E4C-BE03-471E-BCAC-0E5D12852FB2},{76188D13-D397-4F06-8354-5019A8DD1D05},235,0,"?<?xml version=""1.0"" encoding=""utf-8""?>
<Machine xmlns:i=""http://www.w3.org/2001/XMLSchema-instance"" z:Id=""i1"" xmlns:z=""http://schemas.microsoft.com/2003/10/Serialization/"" xmlns=""http://schemas.datacontract.org/2004/07/MDM.MachineDefinition.Models"">
<IsVisible>true</IsVisible>
<ModelingTransformation z:Id=""i2"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</ModelingTransformation>
<Name>Machine</Name>
<Tag i:nil=""true"" />
<Transformation z:Id=""i3"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</Transformation>
<Components>
<Component z:Id=""i4"" i:type=""ToolProfile"">
<IsVisible>true</IsVisible>
<ModelingTransformation z:Id=""i5"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</ModelingTransformation>
<Name>MAGIC 1/4 FLAT ENDMILL</Name>
<Tag i:nil=""true"" />
<Transformation z:Id=""i6"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</Transformation>
<Components>
<Component z:Id=""i7"" i:type=""Group"">
<IsVisible>true</IsVisible>
<ModelingTransformation z:Id=""i8"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>2.5</M42>
<M43>0</M43>
<M44>1</M44>
</ModelingTransformation>
<Name>Target</Name>
<Tag>DefaultTarget</Tag>
<Transformation z:Id=""i9"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</Transformation>
<Components />
</Component>
<Component z:Id=""i10"" i:type=""LinearAxis"">
<IsVisible>true</IsVisible>
<ModelingTransformation z:Id=""i11"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</ModelingTransformation>
<Name>LinearAxis</Name>
<Tag i:nil=""true"" />
<Transformation z:Id=""i12"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</Transformation>
<Components>
<Component z:Id=""i13"" i:type=""ToolProfile"">
<IsVisible>true</IsVisible>
<ModelingTransformation z:Id=""i14"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</ModelingTransformation>
<Name>Real Holder</Name>
<Tag i:nil=""true"" />
<Transformation z:Id=""i15"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</Transformation>
<Components>
<Component z:Id=""i16"" i:type=""Group"">
<IsVisible>true</IsVisible>
<ModelingTransformation z:Id=""i17"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>6.1209999999999987</M42>
<M43>0</M43>
<M44>1</M44>
</ModelingTransformation>
<Name>Target</Name>
<Tag>DefaultTarget</Tag>
<Transformation z:Id=""i18"">
<Command>I</Command>
<M11>1</M11>
<M12>0</M12>
<M13>0</M13>
<M14>0</M14>
<M21>0</M21>
<M22>1</M22>
<M23>0</M23>
<M24>0</M24>
<M31>0</M31>
<M32>0</M32>
<M33>1</M33>
<M34>0</M34>
<M41>0</M41>
<M42>0</M42>
<M43>0</M43>
<M44>1</M44>
</Transformation>
<Components />
</Component>
</Components>
<HexColor>#6699cc</HexColor>
<Opacity>1</Opacity>
<ProfileType>Holder</ProfileType>
<StickOut>0</StickOut>
<ToolID>d22d6e4c-be03-471e-bcac-0e5d12852fb2</ToolID>
</Component>
</Components>
<DirectionX>0</DirectionX>
<DirectionY>1</DirectionY>
<DirectionZ>0</DirectionZ>
<InitialPosition>0</InitialPosition>
<Max>2.5</Max>
<MaxFeedrate z:Id=""i19"">
<Inch i:nil=""true"" />
<Metric i:nil=""true"" />
</MaxFeedrate>
<Min>0.75</Min>
<Position>1.3118700546473194</Position>
<AxisType>Y</AxisType>
</Component>
</Components>
<HexColor>#6699cc</HexColor>
<Opacity>1</Opacity>
<ProfileType>Tool</ProfileType>
<StickOut>1.3118700546473194</StickOut>
<ToolID>00000000-0000-0000-0000-000000000000</ToolID>
</Component>
</Components>
<Control></Control>
<Family></Family>
<IsMetric>false</IsMetric>
<MachineToolType>VerticalLathe</MachineToolType>
<Manufacturer></Manufacturer>
<Model></Model>
<Series></Series>
<SubModel></SubModel>
</Machine>",

-----Original Message-----
From: sqlite-use...@sqlite.org [mailto:sqlite-use...@sqlite.org] On Behalf Of Pavel Ivanov
Sent: Wednesday, December 21, 2011 11:20 AM
To: General Discussion of SQLite Database

Subject: Re: [sqlite] sqlite3_column_text() returning partial results

**********************************************************************


This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mastercam.com
**********************************************************************

Pavel Ivanov

unread,
Dec 21, 2011, 2:32:14 PM12/21/11
to General Discussion of SQLite Database
Now, this is a very good test case and explanation what happens, except...

How did you check value of CString testing after iterating though the
entire char array? Did you look at it in your debugger? Maybe your
debugger is not capable of showing CString contents longer than 2048
symbols? The value of string testing you gave with Windows-style line
ends gives me 2040 bytes, indentation of the next line is 8 bytes, so
if total limit is 2048 bytes then you won't see any contents of the
next line...

Can you output contents of CString testing using printf() and see if
it shows the same 2040 bytes?


Pavel

Jacob A. Camp

unread,
Dec 21, 2011, 4:33:02 PM12/21/11
to General Discussion of SQLite Database
There it is, when I printed this value out to a txt file everything was there.

Apparently there is some sort of character limit when displaying strings in debug mode in Visual Studio 2010, news to me...

Thanks everyone!

-----Original Message-----
From: sqlite-use...@sqlite.org [mailto:sqlite-use...@sqlite.org] On Behalf Of Pavel Ivanov
Sent: Wednesday, December 21, 2011 2:33 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] sqlite3_column_text() returning partial results


Pavel

**********************************************************************


This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mastercam.com
**********************************************************************

Tony McC

unread,
Dec 24, 2011, 7:28:20 AM12/24/11
to sqlite...@sqlite.org
On Wed, 21 Dec 2011 21:33:02 +0000
"Jacob A. Camp" <Jacob...@mastercam.com> wrote:

> There it is, when I printed this value out to a txt file everything
> was there.
>
> Apparently there is some sort of character limit when displaying
> strings in debug mode in Visual Studio 2010, news to me...
>
> Thanks everyone!

That's what the 2010 in the name of the product means: displayed strings
are limited to that many characters (Studio is an acronym meaning Stop
Displaying Under Debugger If Over [2010]). You were lucky to get a few
more, but that is undocumented behaviour. If you upgrade next year
though you should be able to display a couple more characters.

Tony

Matt Young

unread,
Dec 25, 2011, 6:52:28 PM12/25/11
to General Discussion of SQLite Database
I used the strcpy_s function from Microsoft, the so called safe version
that includes a char count. I used it under the Studio debugger. Set a
buffer of 200 chars to zero, set the char count to 20 in strcpy_s, and the
debugger wrote in the top 180!!

I freaked, didn't see that as safe at all!

Igor Tandetnik

unread,
Dec 25, 2011, 8:29:17 PM12/25/11
to sqlite...@sqlite.org
Matt Young <young...@gmail.com> wrote:
> I used the strcpy_s function from Microsoft, the so called safe version
> that includes a char count. I used it under the Studio debugger. Set a
> buffer of 200 chars to zero, set the char count to 20 in strcpy_s, and the
> debugger wrote in the top 180!!

As per documentation: "The debug versions of these functions first fill the buffer with 0xFD. To disable this behavior, use _CrtSetDebugFillThreshold." This is to make buffer overruns easier to spot under debugger. If you see a variable unexpectedly overwritten with 0xFD pattern, chances are high you are passing incorrect buffer length to strcpy_s or similar, and the buffer happens to be located in memory right before that varible.
--
Igor Tandetnik

Reply all
Reply to author
Forward
0 new messages