Getting a Byte Array from a Variant
Hugh Jones
I want to convert a variant to a byte array. Please note that I am NOT
referring to variant arrays.
Alternatively a pointer to the Variant's data, and its size would be cool.
Is there a snappy efficient way to do this ?
Thanks all
Hugh
Remy Lebeau (TeamB)
"Hugh Jones" <hugh@nospam_37ChapelLane.co.uk> wrote in message
news:482a...@newsgroups.borland.com...
> I want to convert a variant to a byte array.
What does the Variant actually hold to begin with? The only way a Variant
can hold a byte array is by using a SAFEARRAY (Variant type = VT_ARRAY or
VT_UI1). Is that what you are actually working with?
Gambit
Peter Below (TeamB)
You may be able to get some info from this older post:
> Does anyone know how to write the content of a Variant to a Stream?
> I have a picture loaded into a variant (using some third party
components),
> and needs to transfer those data to a stream so I could use the
> LoadFromStream/SaveToFile methods...
> I know I probably have to use methods like VarArrayLock() etc, but I
haven't
> managed to get it running.
you are on the right track. Assuming the variant holds a 1-dimensional
array
of bytes (the usual method of passing binary data in variants) it would
be
something like this (untested!):
Procedure StreamVariant( Var aVariant: Variant; aStream: TStream );
Var
p: PByte;
size: Integer;
Begin
Assert( Assigned( aStream ));
if VarArrayDimCount( aVariant ) <> 1 Then
raise Exception.Create(
'StreamVariant: passed variant does not contain a '+
'1-dimensional array!' );
p:= VarArrayLock( aVariant );
If p <> Nil Then
try
size := (VarArrayHighBound( aVariant, 1 ) -
VarArrayLowBound( aVariant, 1 ) + 1 ) *
TVarData( aVariant ).VArray^.ElementSize;
aStream.WriteBuffer( p^, size );
finally
VarArrayUnlock( aVariant );
end;
End;
Note that this method is not applicable to *all* kinds of 1D variant
arrays,
it would fail on arrays of variants, for example. So some more checks
on the
actual type of the data stored in the variant may be appropriate. See
VarType.
--
Peter Below (TeamB)
Don't be a vampire (http://slash7.com/pages/vampires),
use the newsgroup archives :
http://www.tamaracka.com/search.htm
http://groups.google.com
Hugh Jones
is an Oracle SqlLoader object which expects Fields to be loaded using
just a Pointer and a Size. (And yes it might be easier using queries,
but performance is paramount - this looks like the fastest option)
I guess I could pass in a TField and have a ladder
if ThisField is TStringField then
...
else if ThisField...
But, now that I have discovered I dont understand quite how Variants
work, I have become annoyed, intransigent and stubborn, and Delphi MUST
be made to bend to my will. :)
Do you see ?
Hugh Jones
>
> You may be able to get some info from this older post:
>
>> Does anyone know how to write the content of a Variant to a Stream?
>> I have a picture loaded into a variant (using some third party
> components),
Thanks, Peter. Yes I have seen some posts on the subject, but my need
is slightly different.
What I need to do is handle a Variant of ANY Type and turn it into a
byte array - well a pointer and a size to be precise. Even a ShortInt
would become a 1 element array - if you see what I mean.
There are other ways of doing what I need to do i admit ...
Remy Lebeau (TeamB)
> What I need to do is handle a Variant of ANY Type and turn
> it into a byte array - well a pointer and a size to be precise.
TField has an AsVariant property that handles such conversions for you.
If you are not using TField to access the DB columns, then you will just
have to break down the Variant manually, ie:
var
V: Variant;
P: Pointer;
S: Integer;
begin
V := ...;
case VarType(V) of
varShortInt:
begin
P := @V.VShortInt;
S := SizeOf(ShortInt);
end;
varByte:
begin
P := @V.VByte;
S := SizeOf(Byte);
end;
varSmallint:
begin
P := @V.VSmallInt;
S := SizeOf(Smallint);
end;
varWord:
begin
P := @V.VWord;
S := SizeOf(Word);
end;
varInteger:
begin
P := @V.VInteger;
S := SizeOf(Integer);
end;
varLongWord:
begin
P := @V.VLongWord;
S := SizeOf(LongWord);
end;
// etc...
end;
end;
Gambit
Peter Below (TeamB)
> What I need to do is handle a Variant of ANY Type
Look at the TVarData type in the online help. This type describes the
memory layout of a variant. For some of the variant data types the
actual value is stored inside the record (up to 8 bytes), so for those
the pointer to the data would be @TVarData(V).VInteger (all the data
fields have the same offset, that is: address). The size depends on
VarType(V), you could handle that with a Case statement. For some types
the address of the buffer would be TVarData(V).VAny (the pointer held
in that field). That applies to varOleString, for example. The size of
the data is then dependent on what this pointer points at, for
varOleString you would use
size := lStrLenW(TVarData(V).VOleStr) * Sizeof(Widechar);
For BLOB fields you will get a variant that contains a variant array of
bytes, for those you can use the code in the sample I posted to
calculate the size, and the address is returned by VarArrayLock.
If the variants you have to convert come from a dataset that should
cover all the bases, unless you have to deal with nested tables.
Dennis Passmore
var
Cnt: integer;
function VarToString(const V: variant): string;
var
Vt: integer;
begin
Vt := VarType(V);
case Vt of
{02}varSmallint,
{03}varInteger : result := Inttostr(integer(V));
{04}varSingle,
{05}varDouble,
{06}varCurrency : result := Floattostr(Double(V));
{07}varDate : result := usgFormatDateTime(TDateTime(V));
{08}varOleStr : result := WideString(V);
{0B}varBoolean : result := TF[Boolean(V)];
{0C}varVariant : result := VariantToString(Variant(V));
{11}varByte : result := char(byte(V));
{100}varString : result := String(V);
{2000}varArray : result := VariantToString(Variant(V));
end; {case }
end;
begin
result := '';
if (vartype(vArray) and VarArray)=0 then
result := '|'+VarToString(vArray)
else
for Cnt := VarArrayLowBound(vArray, 1) to
VarArrayHighBound(vArray, 1) do
result := result+'|'+VarToString(vArray[Cnt]);
end;
you can see how to modify the code to convert whatever it is
into your byte array as required
Remy Lebeau (TeamB)
"Dennis Passmore" <some...@someplace.com> wrote in message
news:n9uo24phnm5bmi2aj...@4ax.com...
> function VariantToString(const vArray: variant): string;
The RTL already supports assigning a Variant to a String natively:
var
S: String;
V: Variant;
begin
V := ...;
S := V;
end;
Gambit
Jouni Aro
There is also VarToStr, which also works for Unassigned (varEmpty) and
Null (varNull) values (the above raise an exception, varToStr returns '').
But there is one difference: the automatic variant conversions assume
that '.' is used as decimal separator - FloatToStr correctly uses the
locale specific DecimalSeparator. This can be quite annoying in some
occasions.
Arrays also require your own conversion.
Hugh Jones
> TField has an AsVariant property that handles such conversions for you.
>
Getting a variant was where I started - Fields[n].Value seemed the
obvious way in.
> If you are not using TField to access the DB columns, then you will just
> have to break down the Variant manually, ie:
>
> var
> V: Variant;
> P: Pointer;
> S: Integer;
> begin
> V := ...;
> case VarType(V) of
> varShortInt:
> begin
> P := @V.VShortInt;
> S := SizeOf(ShortInt);
> end;
> varByte:
> begin
In the end I skinned my cat a little differently (not fully tested).
The pointers and sizes had to persist so that I could load large numbers
of records in a 'hit'.
I would appreciate it if anyone could see how the technique could be
speeded up in any way. Creating a stream seems like a lot of hard work ...
Also, My casting and de-referencing :
PSmallint(PBuf)^ := ThisField.AsInteger;
Is that OK ? Could it be quicker ?
for Columns := 0 to AIBTable.FieldCount - 1 do begin
Size := 0;
PBuf := nil;
ThisField := AIBTable.Fields[Columns];
case ThisField.DataType of
ftUnknown : begin // Unknown or undetermined
raise Exception.Create('FieldType ftUnknown Not Handled');
end;
ftString : begin // Character or string field
Size := Length(ThisField.AsString);
PBuf := AllocMem(Size);
with TStringStream.Create(ThisField.AsString) do try
Read(PBuf^, Size);
finally
Free;
end;
end;
ftSmallint : begin // 16-bit integer field
Size := SizeOf(SmallInt);
PBuf := AllocMem(Size);
PSmallint(PBuf)^ := ThisField.AsInteger;
end;
Hugh Jones
Hugh Jones
> For BLOB fields you will get a variant that contains a variant array of
> bytes, for those you can use the code in the sample I posted to
> calculate the size, and the address is returned by VarArrayLock.
>
> If the variants you have to convert come from a dataset that should
> cover all the bases, unless you have to deal with nested tables.
>
Remy, Peter - thanks very much. I am getting there, I think, though it
is quite a laborious task.
Fortunately I have no nested tables or BLOBs, but the are some LONG
fields, for which I guess I will have to use the VarArray Logic.
Hugh Jones
Cheers
Remy Lebeau (TeamB)
> In the end I skinned my cat a little differently (not fully tested). The
> pointers and sizes had to persist so that I could load large numbers of
> records in a 'hit'.
Why not just have an array of Variants?
var
Values: array of Variant;
SetLength(Values, AIBTable.FieldCount);
for Columns := 0 to AIBTable.FieldCount - 1 do
Values[Columns] := AIBTable.Fields[Columns].AsVariant;
// use Values as needed...
> Creating a stream seems like a lot of hard work ...
Not really, though it is completely unnecessary the way you are using it.
Try this instead:
var
S: String;
//...
for Columns := 0 to AIBTable.FieldCount - 1 do
begin
Size := 0;
PBuf := nil;
S := '';
ThisField := AIBTable.Fields[Columns];
case ThisField.DataType of
//...
ftString : begin // Character or string field
S := ThisField.AsString;
Size := Length(S) * SizeOf(Char);
PBuf := AllocMem(Size);
Move(Pointer(S)^, PBuf^, Size);
end;
//...
end;
end;
> Also, My casting and de-referencing :
> PSmallint(PBuf)^ := ThisField.AsInteger;
> Is that OK ?
Yes.
Gambit
Hugh Jones
> "Hugh Jones" <hugh@nospam_37ChapelLane.co.uk> wrote in message
> news:482d...@newsgroups.borland.com...
>
>
>
I am using a list of records with Size/PBuf fields so that I can free
the memory after each bulk-write.
Do you think an array would be quicker ?
>
>> Creating a stream seems like a lot of hard work ...
>
> Not really, though it is completely unnecessary the way you are using it.
> Try this instead:
>
> var
> S: String;
> //...
>
> for Columns := 0 to AIBTable.FieldCount - 1 do
> begin
> Size := 0;
> PBuf := nil;
> S := '';
>
> ThisField := AIBTable.Fields[Columns];
>
> case ThisField.DataType of
> //...
>
> ftString : begin // Character or string field
> S := ThisField.AsString;
> Size := Length(S) * SizeOf(Char);
> PBuf := AllocMem(Size);
> Move(Pointer(S)^, PBuf^, Size);
> end;
>
> //...
> end;
> end;
>
Move() is how I did it in a previous version, and yes it should be
quicker - silly me.
I think I will drop the '* SizeOf(Char)'. I understand why you do that
but this is a throw/away job, but every clock-cycle is worth saving, I
think.
Maybe it gets optimised out any way ?
Thanks Remy
Remy Lebeau (TeamB)
> I think I will drop the '* SizeOf(Char)'.
I would not recommend that. Char is changing from AnsiChar to WideChar in
the next Delphi version. If you drop that calculation, your code will not
function properly in the next version.
Gambit
Hugh Jones
code, so if I can save a clock-cycle or two that is more important.