C# buffer creation and reading back

[develo...@gmail.com]

Hi,

did i something wrong?or why i'am not able to read out the byte array correctly?

creation of the flatbuffer:

FlatBufferBuilder fbb = new FlatBufferBuilder(1);
 int coloroffset = XFBType.Colour.CreateColour(fbb, m_ColourAmbient.R, m_ColourAmbient.G, m_ColourAmbient.B, m_ColourAmbient.A); 
int sceneoffset = XFBType.Scene.CreateScene(fbb, coloroffset);
fbb.Finish(sceneoffset);

if i try to read out only the byte array it doesn't work (this should work or?):

ByteBuffer byteBuffer = new ByteBuffer(fbb.DataBuffer().Data);
 var m_FBDataNOT = XFBType.Scene.GetRootAsScene(byteBuffer); // read //error NULL
XFBType.Colour colourNOT = m_FBDataNOT.ColourAmbient();

if i read the flatbuffer directly after creation it works:

var m_FBData = XFBType.Scene.GetRootAsScene(fbb.DataBuffer()); // read //data OK
 XFBType.Colour colour = m_FBData.ColourAmbient()

and 
Byte[] test2 = fbb.SizedByteArray(); //error index out of bounds

is chrashing with index out of bounds...

[Justin S.]

I've recently been playing around with Flatbuffers, specifically the .NET implementation, and ran into this as well.  From the documentation:

"Importantly, the valid data does not start from offset 0 in this buffer, but from fbb.dataBuffer().position() (this is because the data was built backwards in memory). It ends at fbb.capacity()."

If you look at the generated constructor for ByteByffer, you'll see it assumes the data starts at index 0 (position=0) and there isn't a method generated to modify the position data member.

If you're going to be pulling the byte[] out of the ByteBuffer, you need to point to .position() as the beginning of the byte[], with the length being '.Length - .position()".

I believe there was a response to another question back in December of 2014 on this group stating the .NET port would likely be modified to make it more .NET-like (I assume turning the accessors into properties and the like).  Hopefully part of those mods will include making this a little cleaner.

[K. H.]

so... in the current state it's not possible to get the byte data(without a set position method)?....

i could swear  " Byte[] test = fbb.DataBuffer().Data;" worked before the newly added "Create" functions (that i realy needed).

[Pengwyn]

In my little test app I created a byte[bb.Length - bb.position ()] foo then used System.Buffer.BlockCopy (bb.Data(), bb.position (), foo, 0, bb.Length - bb.position ()).  I was then able to feed foo into a new ByteBuffer, which I then fed to my object's GetRootAsObject method.

On Jan 12, 2015 10:56 AM, "K. H." <develo...@gmail.com> wrote:

so... in the current state it's not possible to get the byte data(without a set position method)?....

i could swear  " Byte[] test = fbb.DataBuffer().Data;" worked before the newly added "Create" functions (that i realy needed).

--
You received this message because you are subscribed to a topic in the Google Groups "FlatBuffers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/flatbuffers/-TAa87iDyCg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to flatbuffers...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[K. H.]

wow, nice thx it works. I can continue to work with flatbuffers in c#.
but yeah, it's not realy .NET-like . :)  reminds me on memcpy

[Justin S.]

Yep.  I also looked into that SizedByteArray() method.  If you have the same code base that I do for that method, it's doing exactly what I posted, except it's passing in _bb.Length for number of bytes to be copied, not _bb.Length - _bb.position().

-Justin

[Wouter van Oortmerssen]

Oops. Indeed, that's wrong. Will fix.

--
You received this message because you are subscribed to the Google Groups "FlatBuffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers...@googlegroups.com.

[Hamza Lazâar]

Hi,

I want to make sure this fixed already ? 

As I'm having an ArgumentOutOfRangeException when trying to read size of array of tables from a table that I read back from a file. Maybe you could help me out here ! I just started using FlatBuffers in a game I have been developing with Unity and maybe it's too late as my game data has become too complex.

public static void SaveGameData(string path, Dictionary<string, GameInfo> games) {
       int[] gamesVector = new int[games.Count];
       FlatBufferBuilder fbb = new FlatBufferBuilder(1);
       int i = 0;
       foreach (GameInfo game in games.Values) {
           gamesVector[i++] = game.ToFlatBuff(fbb);
       }
       int saveData = SaveData.CreateGamesVector(fbb, gamesVector);
       fbb.Finish(saveData);
       File.WriteAllBytes(path, fbb.SizedByteArray());
}

public static Dictionary<string, GameInfo> LoadGameData(string path) {
    Dictionary<string, GameInfo> games = new Dictionary<string, GameInfo>();
    if (File.Exists(path)) {
        byte[] data = File.ReadAllBytes(path);
        ByteBuffer bb = new ByteBuffer(data);
        SaveData save = SaveData.GetRootAsSaveData(bb);
        for (int i = 0; i < save.GamesLength; i++) { // ArgumentOutOfRange 
            games.Add(save.GetGames(i).GameId, GameInfo.GetFromFlatBuff(save.GetGames(i)));
        }
    }
    return games;
}

And here's the stacktrace :

ArgumentOutOfRangeException: Argument is out of range.

FlatBuffers.ByteBuffer.AssertOffsetAndLength (Int32 offset, Int32 length) (at Assets/Scripts/FlatBuffers/ByteBuffer.cs:125)

FlatBuffers.ByteBuffer.ReadLittleEndian (Int32 offset, Int32 count) (at Assets/Scripts/FlatBuffers/ByteBuffer.cs:100)

FlatBuffers.ByteBuffer.GetInt (Int32 index) (at Assets/Scripts/FlatBuffers/ByteBuffer.cs:395)

FlatBuffers.Table.__vector_len (Int32 offset) (at Assets/Scripts/FlatBuffers/Table.cs:57)

CustomFlatBuffs.SaveData.get_GamesLength () (at Assets/Scripts/CustomFlatBuffs/SaveData.cs:15)

SaveManager.LoadGameData (System.String path) (at Assets/Scripts/Managers/SaveManager.cs:75)

GamesListManager.Load () (at Assets/Scripts/Managers/GamesListManager.cs:242)

Also please find attached my schema file.

[Wouter van Oortmerssen]

Yes, that previously mentioned bug was fixed.

Your code looks correct to me.

The exception you're getting however is not that a vector access is out of bounds, but that the length field itself is out of bounds of the underlying array. Now a length field of a vector is usually preceded by the the table it sits in, and followed by all the vector data, so it being out of bounds is highly unusual, and would indicate that the entire vector is out of bounds.

Can you get some more numbers? In the debugger, tell me what the values for "offset" and "length" in AssertOffsetAndLength. What is the value for "offset" in __vector_len? Also get the values for the interals of the bytebuffer (the size of the array, and position).

[Hamza Lazâar]

Since save file is corrupt each time, I'm only testing using one element in the array (start a game then save and quit).

AssertOffsetAndLength

offset : 8196

length : 4

Length : 296

Position : 0

Since I did not understand what I should provide exactly as debug info I took 2 screenshots that you can find in the attached archive file.

I want to mention few things I think they're important to investigating this issue :

• I'm always using the same FlatBufferBuilder in creation and reading of all FlatBuffers and always initialized to 1.
  
• The data I'm saving is "complex" (IMO) : is some classes there is a reference to container class...class can contain array of other classes... I don't know if, when and how to use nested_tables.
  
• In the source code of the classes that are converted to/from FlatBuffers, in each class there are two methods : GetFromFlatBuff() and ToFlatBuff() I think they're worth a review.
  

[Wouter van Oortmerssen]

So clearly the offset is way outside the buffer.

I now see the error in your original code: you pass the offset of the created vector to fbb.Finish. The root of a FlatBuffer must be a table, not a vector.

so instead of:

 int saveData = SaveData.CreateGamesVector(fbb, gamesVector);

You need something like:

 int vec = SaveData.CreateGamesVector(fbb, gamesVector);

 SaveData.StartSaveData(fbb);

 SaveData.AddGames(fbb, vec);

 int saveData = SaveData.EndSaveData(fbb);