git submodule update, or, note about "DMA address tag check failed" error

[Luke Gorrie]

Howdy,

Our deps/luajit git submodule now points to a newer version of LuaJIT.

If your build does not pick this up then you will see a scary looking error like this:

Invalid DMA address: 0x500438aedf00
core/memory.lua:95: DMA address tag check failed

and you will need to update the submodule and rebuild.

In a perfect world we would pick this up automatically. If you have an idea for how to do this, please raise your hand :-).

Meanwhile it should work to something like this:

cd snabbswitch

git checkout master

git pull

git submodule update

make clean

make

or simply create a new checkout in the usual way.

Over on the LuaJIT mailing list we have had excellent support from Mike Pall and established with certainty that this error is caused by a specific LuaJIT bug that was fixed in November. Ref: http://www.freelists.org/post/luajit/Confusing-problem-involving-uint64-t-bitops,7

Cheers,

-Luke

[Nikolay Nikolaev]

Hello,

I'd like to add to what Luke says. When the machine uses 1G Huge pages this command needs to be issued:

sudo sysctl -w kernel.shmmax=1073741824

Or respectively set the value in /etc/sysctl.conf.

regards,

Nikolay Nikolaev

--
You received this message because you are subscribed to the Google Groups "Snabb Switch development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to snabb-devel...@googlegroups.com.
To post to this group, send email to snabb...@googlegroups.com.
Visit this group at http://groups.google.com/group/snabb-devel.

[Luke Gorrie]

On 10 February 2015 at 10:18, Nikolay Nikolaev <n.nik...@virtualopensystems.com> wrote:

I'd like to add to what Luke says. When the machine uses 1G Huge pages this command needs to be issued:

sudo sysctl -w kernel.shmmax=1073741824

Or respectively set the value in /etc/sysctl.conf.

Good point!

I created Github Issue #366 to track this: https://github.com/SnabbCo/snabbswitch/issues/366.

Note: You can use any huge page size you like with Snabb Switch now. For NFV we previously recommended 1GB huge pages to speed up software address translation (TLB) involving virtual machine memory. However, the straightline merge includes a new/simple/fast solution to this problem that works for all page sizes.

(Gory details @ https://groups.google.com/d/msg/snabb-devel/Sp6xwTw0gdY/RTV6BKk1UtoJ)

The NFV deployment guidelines need to be updated with simpler and more relaxed instructions for the next release (stop demanding 1G page size).

Cheers,

-Luke

[Alexander Gall]

On Tuesday, February 10, 2015 at 10:12:13 AM UTC+1, Luke Gorrie wrote:

Howdy,

Our deps/luajit git submodule now points to a newer version of LuaJIT.

If your build does not pick this up then you will see a scary looking error like this:

Invalid DMA address: 0x500438aedf00
core/memory.lua:95: DMA address tag check failed

and you will need to update the submodule and rebuild.

In a perfect world we would pick this up automatically. If you have an idea for how to do this, please raise your hand :-).

Meanwhile it should work to something like this:

cd snabbswitch

git checkout master

git pull

git submodule update

make clean

make

I'm probably missing something.  While this procedure does compile a new luajit executable in the deps/luajit/src tree, it doesn't get copied into deps/luajit/usr/local/bin, which is the path that's used in the Snabb Makefile (src/Makefile). I have to do a contortion using "make install PREFIX=..." (and PREFIX needs to be an absolute path) within deps/luajit to get it working. What am I doing wrong?

--
Alex

[Luke Gorrie]

On 10 February 2015 at 10:12, Luke Gorrie <lu...@snabb.co> wrote:

Our deps/luajit git submodule now points to a newer version of LuaJIT.

[...]

 

In a perfect world we would pick this up automatically. If you have an idea for how to do this, please raise your hand :-).

I have attempted to add "submodule update mistake-proofing" to our Makefiles.

src/Makefile now knows exactly which versions of luajit and ljsyscall are needed and can detect if the wrong ones are built (e.g. because you have pulled new versions but not built them yet).

In this case it will flag an error with a friendly message telling you how to fix it:

SUBMODULES

Error: Submodule version mismatch

luajit:

  require: v2.0.3-328-g04dc64b

  found:   v2.0.3-228-g392b6c9

ljsyscall:

  require: v0.10-65-g7081d97

  found:   v0.10-65-g7081d97 

Please update and rebuild submodules.

Cheat sheet:

  cd snabbswitch

  git submodule update

  make clean

  make -j

Just performing those "Cheat Sheet" commands should make everything right. (If not then please report that as a bug and reclone the repo as a workaround.)

You should expect to see this message if you update from v2015.02 to master.

Bonus: The top-level makefile will now automatically clone the luajit and ljsyscall submodules on demand, so you don't have to clone with "--recursive" or run "git submodule update --init" anymore.

See commit https://github.com/SnabbCo/snabbswitch/commit/e1a2b509c5bc384578b65372e5a983d2903a0497.

Cheers,

-Luke