The Tundra backlog
Jukka Jylänki
decision last August to start working on a unified server+client codebase
and choosing to implement a custom protocol (the "ECSync" mechanism) on
top of a generic application transport layer library were easily the
single biggest impacting factors on the whole project. I had on my list a
task to write about the status of Tundra, and I continually kept
postponing, but now, here we go.
There's already been a few releases of Tundra (get v1.0.2 from
http://code.google.com/p/realxtend-naali/downloads/detail?name=realXtend-Tundra-1.0.2.msi
). For those who haven't yet gotten familiar with it, here's a quick
summary of what Tundra is:
- An application development framework. Instead of providing a single
end-user application like what Naali is, Tundra aims to be a SDK/engine
for developers. This means that no in-world functionality is assumed or
hardcoded (world sizes, scene content, client appearance or behavior, user
interface), but the server you connect to defines all these aspects.
- To some confusion, you'll sometimes hear developers say "Naali is
Tundra". This is due that they both share the same codebase. Tundra is
sometimes referred to as only the server part of the system (replacing the
Taiga server), and sometimes the whole platform (server+client replacing
Naali+Taiga). Perhaps the most appropriate definition is that Tundra is a
branch that diverged off from Naali, and it implements both a server and a
client, instead of being only a client.
- Tundra consists of a Core API which comes with the platform. Optional
added functionality can be implemented in terms of modules (some of the
core systems are implemented as modules as well), scene components and
scripts. See the Core API documentation here
http://www.realxtend.org/doxygen/ . The most often used parts of the API
are the following two:
- The heart of Tundra is the Scene API, which contains the notions Scene,
Entity, Component, Attribute and Action. A generic server<->client
synchronization mechanism (the "ECSync" protocol) hides the content
developers from the details of scene replication.
- Scene objects can refer to Assets (textures, meshes, animations, ui
files, js/py scripts, etc.) in a generic manner, which can be handled
through the Asset API. The system does not specify a fixed list of asset
transfer mechanisms or asset types, but a plugin system is used to allow
writing new asset providers and asset types in C++.
- Full scenes (==applications) are now handled very much like normal
document files. Tundra has built-in .txml & .tbin file formats that store
scenes (but can also store individual objects for script-driven "prefab"
instantiation cases). You can copy scenes around in this manner, and
modify them in the text editor even, so your data is more easily
accessible. See example here http://clb.demon.fi/dump/Tundra/TestScene.xml
. (.txml/.tbin files do not embed asset data, but only refer to them)
- For a demo, see http://www.youtube.com/watch?v=Wg6SAQPW-9k .
Now that the architecture has grown enough, it is no longer the case that
everyone has to be messing with the core parts of it to be able to do
anything, but instead new features can be adequately implemented just by
writing new modules, scene components, or application/entity scripts.
I can't keep the wall up any more, I have to clean it up. (Here's the
original, if you can make out the text:
http://clb.demon.fi/dump/Tundra/Wall.jpg )
I'm putting out the grand list of things that have been discussed since
the inception of the project, categorized in a form of a backlog.
Hopefully this will give more insight to what's been on the radar at the
companies working on Tundra and what might be possible with it. Perhaps it
will give you some ideas to consider, if you're interested in working on
some functionality. The items are very technical and not even direct
visible end-user features, but then again, this is the developer mailing
list.
The list is too big to be sorted in any manner, so I'm just taking off a
note at a time from the wall, and writing it down here.
Config API.
- Remove the use of templates altogether in the Config API. This is
because the API is not generic for arbitrary typename T, and due to the
templates the API is not script-friendly or library-boundary-friendly.
(Remove ConfigurationManager-templates.h)
- Remove the use of Poco XML in the Config API. We don't use Poco XML
anywhere in the system, so replacing this with tinyxml or Qt's xml will
allow us to remove on dynamic .dll dependencies.
- Move all the configuration keys behind a common "application key
string". Now if we have two binary shipped applications "Tundra" and
"Tundra2", we cannot configure them differently, since they'll be looking
at the configuration in the same place.
- Refactor the ConfigurationManager.h et al. to a coherent Config API
library.
- Write script bindings for the Config API.
- Rewrite documentation to match the new system.
Config API registry editor UI.
- In the core Config API, a generic for-developers feature for being able
to see all configuration key-values, in the style of "about:config" in
firefox would be very useful.
- Support clearing the existing config keys through this UI. (A source of
bugs in the past has been that installing on clean machines vs installing
on machines that had old config resulted in different runtime behavior,
this would allows simulating an install on a clean system).
- Support fuzzing the config keys to bad values to detect application
crashes.
UI API type-specific context menus.
- Extend the currently existing UI API to contain a generic mechanism of
adding and spawning context menus acting on items of arbitrary types. This
could be done through a generic extensible set of categories (strings),
like "Entity", "Asset", "Component". The UI API would signal whenever a
context menu is opened for an item of specific type (e.g. a context menu
on an Asset item), to which extensions can react on and provide their own
generic functionality, e.g. a custom tool script could inject a new menu
item on the context menu of all assets, entities, or so on.
- Using this same mechanism, one could provide a set of default actions
on items (c.f. the windows shell API that is based on context-dependent
verbs) to allow generic hooking on clicks and double-clicks of items,
independent of which UI they appear in.
Build System. Rewrite the build system to be SDK-friendly.
- Should support hosting the core SDK in one directory (unmodified), and
custom-written modules/plugins in another. Now all modules need to be
created in the same directory as the SDK itself is hosted.
Console API.
- Rewrite the SupportModules project as the new Console API.
- Clean up (the same functionality can be implemented in 1/5th of the
amount of code that is currently there).
- Support remembering a lastlog of previously inputted items. (in Config
API keys so they persist between runs)
- Support using up/down to scroll the lastlog.
- Implement autocomplete/suggestions.
- Integrate the UI-based Qt object signal/slot invokation system into the
Console API as console commands using a set of canonical predetermined set
of names, like "entity(id).placeable.someqtfunction(params)". The
FunctionInvoker.h was written to support this for the UI-side, and can be
used for this. Support autocomplete for these. Support setting attributes
through console.
Easy scene query helpers + optimizations.
- Currently querying different scene entities/components/attributes is
cumbersome, especially on the C++ side. We should implement an utility
that can automate the process of multi-selecting (and multi-acting on)
collections of interesting objects, something like
--
Scene("scenename").AllEntitiesWithComponent("myComponentType").SetAttribute("myAttribute",
value);
-- currentScene.entity("myCameraEntity").camera.Activate();
- Implement optimizations for the most used forms of queries. At least
the Scene.AllEntitiesWithComponent is very often used in current codebase,
which would deserve a precomputed cache of buckets of entities by their
component. What other kinds of common queries are there?
- These helpers function to automate (hide) the tedious chains of tests
for query successes, e.g. (scene = GetScene("myScene") if (scene) entity =
GetEntity("myEntityName") if (entity) component =
GetComponent("EC_Placeable") if (component) (component as
placeable).position = somevalue. Would be so much nicer to write
scene("myScene").entity("myEntityName").placeable.position = somevalue;
and have the whole chain gracefully fail if a component of it doesn't
exist (without having to do try-catches around each expression either).
Math libraries improvement.
- The current math code we are using is not adequate. The Vector3 and
Quaternion have minimal useful functionality, but lack more higher-level
features, like LookAt, TurnTo, RotateTowards, Slerp (and other
interpolations), and other conversions.
- There is no proper matrix library and matrix generation code. The
current Matrix code (Matrix4.h from Irrlicht) is unsuitable due to it is
assuming V*M convention whereas the Ogre convention is opposite.
- Ogre math is used in several parts of the code. These are not
sophisticated enough and are not script-exposed. Qt math is script-exposed
but inadequate in comparison to Ogre. A ideal solution would have 1) Same
API on C++ and script side, 2) Consistent math conventions, 3) extensive
pre-existing code for common operations, which seamlessly operate with the
following task item:
Scene geometry processing helpers.
- Provide a set of common geometric primitives for C++ and scripts to use:
-- points, ray/finite line, triangle, plane, box/OBB/AABB, sphere,
capsule, polyhedron, polytope.
--- Intersection tests, point-on-surface, corner/face extractions.
Extensions: Asset editor plugins.
A nice feature with the current asset system is that we can support
in-memory modifications of assets, meaning that one can change an asset
temporarily without changing it back to source. The Asset API also has a
built-in support for uploading assets (with the asset provider abstraction
it can support uploads to arbitrary asset destinations, like local disk,
http upload, or other protocol upload). The Asset API does not assume that
the client would act as "just-a-viewer" for the assets, but the assets can
be modified through the Asset API. So,
- Rewrite the old inventory-hardcoded asset previewers/editors as asset
editor plugins. Use the previously mentioned mechanism to inject these
into handling the specific asset types. (Right-click on an asset, choose
Edit/View), or double-click. The old Naali system contains (currently
unused) code for the following:
- Texture asset previewer. Shows Ogre-decoded images.
- improve this to show mipmap levels, 3D textures and cube textures.
- improve this to allow basic manipulation tasks (format conversions,
size changes).
- Mesh asset previewer.
- rewrite this to show the mesh properly. The current one is very
buggy (doesn't scale the object properly in view, doesn't rotate properly
around the object).
- Allow showing wireframe/shaded/custom material version of the object.
- Scene entity previewer.
- This new editor would render a scene saved as a .txml/.tbin file
into a new preview window. This would allow examining entities without
having to instantiate them.
- Audio asset previewer. Refactor the old audio clip previewer to open up
for audio assets in the assets menu.
- Visual material editor. The old Naali didn't have a material previewer,
but it did have a text-based material editor and a material generator
wizard that could ease the generating of Naali SuperShader template-based
materials. These could all be fused into one new material asset editor,
which with the new Asset API can support live editing of materials.
- Particle system previewer, UI asset previewer and animation sequence
previewer. Tundra has code for supporting these asset types, but no
previewers or editors for them exist. The plugin-based system could
support these though.
Assets dialog improvements.
- Improve the Assets dialog to show current asset status. An example:
- show an asterisk (a common convention in document editors)when an
asset is modified in memory, but not saved to source.
- show in UI when an asset was generated programmatically and is a
temporary one that does not have a disk source/originating address.
- Allow cloning/duplicating assets in the UI.
- Allow renaming assets in the UI in such a way that the references in
the current scene are re-tracked to the new asset address. (also allow
manually invoking this kind of search/replace assetref modification
operation without any underlying asset address change)
- Show asset sizes in the dialog.
Asset bundles.
- Implement a method for automatically bundling/unbundling .txml and
accompanied assets to a single .zip file.
- Implement built-in support for understanding these kinds of .zip files
in the different dialogs (as asset type, show inside the .zip in
treeviews, or allow previewing the .zip in an entity/prefab previewer)
- Implement a mechanism for allowing to preload asset bundles into memory
before loading a scene. In this manner, it is possible to show a loading
screen. This helps with the "objects appear one at a time" -issue.
ECEditor improvements.
- Utilise the SceneInteract helper object everywhere in ECEditor instead
of performing manual mouse click tracking.
- Replace the use of (or greatly extend) QtPropertyEditor with a more
robust one. Reasons below.
- Cannot implement artist-friendly sliders for float data (drag left/right
to smoothly edit values, like in Photoshop)
- Need to support customizing attribute types/editable properties better.
(Tab cycles don't work as expected, aggregate editors show up clumsy, one
line per attribute)
- Actual editors never work until clicked (editing a bool requires two
clicks)
- Implement support for buttons/other icons for custom attribute types.
E.g. Assetref attribute should be able to have a "..." (for assets lookup)
icon as well as a magnifier "Preview"/Edit icon.
- Implement support for drag-n-dropping content to ECEditor.
- Assetrefs, EntityRefs.
- Implement visual editing of translate/rotate/scale attributes in
ECEditor (choose an attribute in editor -> shows a gizmo in world).
.txml/.tbin files.
- Write an XML schema file (.xsd) for .txml files.
- Specify a MIME convention for Tundra files.
- Implement a fuzzer script to test for malformed (but valid xml) scene
content files to improve data handling robustness.
- Implement magic markers to .tbin files to detect proper structure on
the files.
Scene API.
- Implement multiple simultaneous scene support. This is partly done, but
lacks finish.
- Implement support for rendering specific loaded scenes to custom render
targets/windows. This will allow creating a separate scene for the avatar
editor, or can also be used to preview full .txml scenes/meshes before
instantiation etc.
- Implement protocol level subscription mechanism for client->server
specification of which scenes to replicate.
- Implement per-action trigger for listening on EC Actions.
Rendering optimizations:
- Implement Ogre LODded mesh support. Implement support for specifying LOD
level distances.
- Implement support for dynamic terrain geometry generation in ECTerrain.
Network optimizations:
- Implement a separate network synchronization mechanism for rigid body
linear and angular movement messages. This has been profiled to be the
single most limiting factor in scaling up the number of moving objects in
the scene.
- Instead of static 20Hz netrate, implement a dynamic/adaptive netrate for
rigid body linear and angular movement messages, which only sends updates
when necessary.
- Implement in server-side interest management in SyncManager.
- Utilize unreliable messaging and latest-data-guaranteed messaging for
ECSync attribute replication.
- Implement client-side irrelevant entity forgetting.
- Investigate separating scene objects into static and dynamic objects,
and transfer the static ones more efficiently using a compressed scene
entity file.
RexLogicModule/OpenSim support: It is possible to support OpenSim/Rex
scenes in a better way on top of the Tundra scene structure. Lots of the
code is there, but it contains several OpenSim-assumptions, which need to
be restructured.
- Rewrite the ECOpenSimPrim as a more generic ECPrimitive, which doesn't
contain any SLUDP-specifics (but can support OpenSim if desired). Remove
the "Prim is a light/mesh/something else" -features in ECOpenSimPrim.
- Refactor the internals so that RexLogicModule can really be used as
optional module. Tundra should build and run without projects RexCommon,
EC_OpenSimPrim, EC_OpenSimPresence, EC_NetworkPosition, RexLogicModule,
AvatarModule, AssetModule, ProtocolUtilities, RpcUtilities, HttpUtilities,
ProtocolModuleOpenSim, ProtocolModuleTaiga, UiServiceModule,
LoginScreenModule, UiModule.
- Merge ProtocolModuleOpenSim, ProtocolModuleTaiga and ProtocolUtilities
to a single coherent RexProtocolModule or OpenSimProtocolModule. The
current fragmentation is not useful. Move what's still needed of
ECOpenSimPrim/ECOpenSimPresence into RexProtocolModule so that it forms a
clean all-or-nothing package you can't misconfigure.
- Implement a login screen script which supports all OpenSim&Rex&Tundra
logins.
- Reimplement support for loading and showing OpenSim/Rex terrains, now
using the new generic ECTerrain component.
- Reimplement support for OpenSim terrain height paint editor.
- Replace the use of custom J2K decoder plugin in favor of the Ogre
built-in J2K plugin.
- Remove the RexCommon project. Move all required code from there to
RexProtocolModule.
Event API:
- Remove EventManager-templates.h and all associated logic.
- Remove the passing of events to components.
Dead code elimination/refactor/cleanup:
- Replace server.exe and viewer.exe projects with a single tundra.exe
which uses command-line flags/config options to decide which modules to
load. (There's no real difference with server.exe and viewer.exe, except a
hardcoded decision of whether to load the server module). By default when
starting up Tundra, we could load the server module, but not start the
server. In the stock Tundra, implement in the menu a "Start server"
option, which can ask for a port/protocol in a dialog. The --headless mode
could automatically start the server.
- Merge UiService into the Ui API. Write the UiModule to operate on top of
the Ui API (as an optional module). Delete UiServiceModule.
- Remove the Resource:: namespace and all identifiers there
(Resource::Events::ResourceReady). These are not used anymore.
- Remove the Asset:: namespace and all identifiers there
(Asset::Events::AssetReady). These are not used anymore.
- Remove all references of request_tag_t . It's been replaced with the
Asset API.
- Remove the ECControllable component altogether. It is replaced with
Input API InputContexts.
- Remove the ECMovable component altogether. It is replaced with custom
js/py scripts and the SyncManager.
- Remove PostProcessWidget.h/.cpp and associated logic. Use the
ECPostProcess component instead.
- Remove ThreadTask.h and ThreadTaskManager.h
- Remove service types ST_Physics, ST_Gui, ST_WorldLogic,
ST_PythonScripting, ST_JavascriptScripting, ST_Console, ST_ConsoleCommand,
ST_Asset, ST_Texture, ST_Sound, ST_Input, ST_Communications,
ST_UiSettings, ST_Player, ST_WorldBuilding, ST_Login, ST_OpensimScene.
Those are/shouldn't be used anywhere anymore.
- Clean up all unused/unneeded objects in the Interfaces project. The goal
is to be able to remove the whole Interfaces project, it is a
misstructuring. (We cannot have a centric place to dump any/all unrelated
interfaces, since it's not extensible. And if there's an interface for X,
it should be in project/library/module X instead.)
- Delete LoginScreenModule. Replace it with a script-version.
- Merge Core, Foundation and what's still remaining of Interfaces after
cleanup together into a single Core project.
Environmental features:
- The current EC_EnvironmentLight, EC_SkyBox/EC_SkyDome/EC_SkyPlane
components need a rewrite, they are structured poorly, due to some of
their logic being separated in EnvironmentModule, which contain
SLUDP-specifics. Not all of the attributes in these components work at
all, and they depend on whether Caelum is enabled/disabled. Merge
EC_SkyBox/EC_SkyDome/EC_SkyPlane into a single ECSky.
- Remove Sky.h/.cpp, Sky-templates.h and the associated logic in
EnvironmentModule. Remove the sky editors from Environment Editor (instead
use the generic ECEditor for editing sky and other environment components
always).
Networking security:
- Implement a mechanism for allowing script-based ECSync permission
control on server side.
- Implement a mechanism for script-based specifying whether to accept or
reject incoming connections.
WebCam support. Implement webcam streaming in 2D/3D.
Documentation: Looking at the documentation pages, these seem to be the
most lacking.
- Write documentation and samples for the Frame API.
- Write documentation and samples for the Audio API.
- Write documentation and samples for the Server API.
- Write documentation and samples for the Connection API.
- Write documentation and samples for the UI API.
Flash asset support:
- Implement support for rendering and interacting with Flash asset files
in 2D or 3D on a texture.
Video assets.
- Implement support for referring to media asset types.
- Implement streaming playback support in 3D textures and in 2D dialog.
Input API:
- Add a sample scene and documentation that describes how to use the
InputContext objects in script (polling and event-based)
- Add a debugger window that shows the currently registered input contexts
and shows which contexts received which events.
- Extend the HandleMouseEvent function in SceneInteract to pass the mouse
button which was pressed in MousePressed event.
ScenePersistenceModule.
- Implement a module which can continuously persist the incremental
changes in the current scene to a database storage, instead of having to
manually (or periodically scripted) re-save the whole scene.
- Passive functionality, driven by a user/owner script that chooses when
to start persisting and where. Support multiple databases, instead of
assuming a singleton target db/table.
Scene attributes.
- Implement a 'EntityRef' attribute type. This attribute refers to an
existing entity in the scene (by using the EntityID or name). Design a
referencing mechanism that is safe when data is transferred between
different locations (serialized/deserialized from a file).
- Implement a 'ComponentRef' attribute type. This attribute not only
refers to an entity, but also to a component inside it.
- Implement a 'ClientRef' attribute type. This attribute refers to a
client that is currently connected, or was previously connected.
Scene Hierarchy.
- Implement support to ECPlabeable component to be able to refer to a
parent node in a transform hierarchy. Uses ComponentRef or EntityRef
attribute.
- Implement a debugging display of these hierarchies and their associated
scene nodes.
- Implement support for grouping and layering object. Perhaps the most
straightforward method is to extend the ECName to contain a GroupName and
a Layer attribute.
ECGrass.
- Implement procedural support for rendering patches of grass,
wind-animated.
ECParticleSystem.
- Improve on the existing ECParticleSystem to be script-friendly.
- Implement a test scene that shows how to use it to create different
kinds of particle effects.
NPAPI/ActiveX plugin.
- Run Tundra client as a web browser plugin. (both offline-enabled with
static scenes, and server-connected)
Notification-enabled asset server.
- Implement an asset server that supports sending notifications to
subscribers when an asset it contains has changed.
ECCamera:
- Rewrite the current ECOgreCamera to be more feature-rich.
- Projection types, sizes. Support custom RTT targets.
- Script-based activation.
- Easy script-based camera drives.
- Automatic obstacle avoidance.
ECRainfall. ECVolumeFog. etc.
ECDecal:
- Implement a decaling system that allows texturing objects with detail
textures.
- Also decaling for terrain.
ECPlaceable:
- Rewrite/cleanup redundant code.
- Remove all hardcoded y/z flipping code.
- Remove the use of a separate adjustment scene node. (bad Opensim-ism)
ECScript:
- Optimize memory usage so that this component can support hundreds of
scripts in a single scene.
- Create a scene of 1000s of scripts-containing (non-visible) entities as
a test case.
Animation system:
- Implement support for animation meta-channels, to allow authoring
properly cued footstep sounds, door-close snap sounds etc.
- Implement support for animated character root pivots to allow removing
footstep skating.
- Render the bone hierarchies for debugging purposes.
Car Physics demo. Implement a test scene that shows that Tundra physics
system can do car-driving games.
User profile server.
- Implement a separate user account server that can store
password-protected user accounts with custom application-specific
information.
Optimizations:
- Remove the number of dependencies to reduce the startup loading times.
(Poco could almost be removed) There's a documentation of the dependencies
at http://www.realxtend.org/doxygen/naaliarchitectureoverview.html , which
should be very up-to-date.
Bugfixes and other tweaks: These are random issues, which might also be
suitable to be marked into the bug tracker.
- Entity actions are treated case sensitive. This is not good, since
script-friendliness and ease-of-use is one of the primary reasons for
their implementation. Replace these to use case-insensitive comparisons
everywhere.
- FunctionDialog.h: Test that it operates gracefully even if an QObject(s)
it is acting on is deleted underneath it.
- Fix issues with QWebView memory leaks. Replace QWebView with some other
web engine? Replace QWebView with the latest webkit version past Qt?
- ProcessExecutableDirectory vs CurrentWorkingDirectory. Allow starting up
Tundra from arbitrary working directory.
- Investigate how to support scaling on the number of simultaneous script
instances.
-- Create a test scene of hundreds of scripts (500 should more than
suffice).
-- Investigate using a single QScriptEngine with shared contexts for
each script.
- Fix Profiler's RemoveThreadRootBlock crash. If the system quits
abnormally, e.g. due to a thrown exception, a thread might clean up a
wrong path, which causes already freed Profiler thread blocks to be
accessed.
- Replace Qt's FileSystemWatcher with a better solution that doesn't lose
the track of a tracked file when it is deleted and later on replaced with
a completely new file.
- Enhance KeyEvent class to convert better between QKeySequences.
- Add a test scene for scene 3D sound playback to test that it works on
all systems.
- When renaming entity names in SceneStructure window, the name sometimes
adds to contain the entity ID as well.
- Do not actively sort columns when editing them in Add Content.. dialog
(and in others too).
- Implement a SceneStressTest sample scene which randomly instantiates and
deletes content in a loop to detect memory leaks and other crashes.
- Rename all signals "OnXXX" to just "XXX". The usual convention is to
have a signal: TextChanged, and slot: OnTextChanged. Some of the code is
currently backwards.
- Break down the IComponent::AttributeUpdated(IAttribute*) signal to a
better attribute-specific signal system.
- Implement auto-normalization, or a normalization utility for terrain
weight blend textures.
- Implement customizable UV scale factors per texture for the terrain
shader.
Phew.
Hope that sparks ideas for the fellow co-developers.
jj
Peter Steinlechner
thanks for this comprehensive status. This will be getting truly awsome.
My bigger question is: Would it be possible and make sense to integrate/use Shibboleth and Grouper for authentication and collaboration?
http://shibboleth.internet2.edu/
http://www.internet2.edu/grouper/
Cheers
Pedro
Antti Ilomäki
continuation of the project. Ideally your post should also act as a
discussion opener for future production on this mailing list, perhaps
even with some people jumping in ready to accept some of the tasks.
PS. I was actually tempted to simply reply tl:dr just to see if I
could actually hear Jukka explode several kilometers away.
2011/3/2 Jukka Jylänki <jukka....@ludocraft.com>:
Toni Alatalo
> my list a task to write about the status of Tundra, and I continually kept postponing, but now, here we go.
Thanks! Let's see how we can get a hold of these, I still hope we get to use use some sane ticketing + roadmapping thing later, but am not going to copy-paste these into google code enchancement tickets now.
> There's already been a few releases of Tundra (get v1.0.2 from http://code.google.com/p/realxtend-naali/downloads/detail?name=realXtend-Tundra-1.0.2.msi ). For those who haven't yet gotten familiar with it, here's a
I added some info to the wiki now too, because that's where the links point and where Naali and Taiga info is etc. So the frontpage in http://wiki.realxtend.org/index.php/Main_Page mentions Tundra now next to Naali, and there is a drafty own 'getting started' page for it in http://wiki.realxtend.org/index.php/Getting_Started_with_Tundra . I didn't duplicate the Naali info there but put a link and some Tundra specific info -- actually pasted this summary of yours as is, and added a different explanation from a private email that just wrote. Will do some cleaning later on and we need to make a clear tutorial/manual for how the basics like drag&drop of meshes + scenestruct editor etc work, I at least can work on those later during spring. The 'working with document files' page on the doxygen side is already good and I agree that can be a good idea to write manual there instead of wiki (is nicer to work on local docs in git instead of edit wiki with browser).
There is kind of bad bug in 1.0.2 btw which is fixed in current now, so we've been talking with Jonne to make 1.0.3 and that is perhaps a pretty solid version now. The bug in 1.0.2 (and all previous versions since the new asset api I think) is that every new load/assignment of an asset 'causes all the previous users of that asset unload&reload it. This was what prevented Heikki's Osprey game working correctly for multiple players: when a new player joined and got an own osprey with the osprey_controller script attached (similar to the avatar_controller in av app), that 'caused also a restart of the code for all the existing players. So they lost state like the flight speed info, whether had captured fish, and got the HUD loaded twice (which lead Heikki to call this bug 'a strange UI bug' :)
I verified that this happens with all assets: drove for a while in Infinite/drive.txml so had ~10 instances of both the road piece & the tree in the scene, then set a breakpoint in mesh loading code, and drove more to get new instances added. That triggered ~20 mesh loads, so indeed all the previous instances in the scene were unloaded&reloaded to Ogre I think. Apparently that is very fast though, it was totally unnoticeable (no glitch while driving).
I first hacked this to work by making ec_script loading work so that it only reloaded a script if the asset ref to it had changed. That fixed osprey game, and is pushed to noscriptreload branch. But this would break the nice devving feat that the asset system automagically reloads changed scripts, so not nice.
You had apparently already put ContentHash to IAsset, and that seemed like a possible solution to only reload the existing instances when the asset had actually changed.
Another possibility would have been to disconnect the Loaded signal from asset handlers once they already have what they need, so subsequent Loaded signals when other entities that also use the same asset would not trigger those anymore. And then e.g. add some new Reloaded signal, or a parameter to the Loaded signal, for the case when the asset has changed .. which now only happens when devving/building using local files.
Jonne chose the ContentHash way so current code now hashes all assets and uses that to not reload unnecessarily -- I haven't read how exactly yet. I hope that hashing all the textures etc always is not too heavy, as it does seem like a simple and robust solution to this (unless something even simpler could be done just with logic in code).
But 1.0.2 is still fine for testing Tundra out and starting to dev, this basically only bites when you have scripts that have internal states and an app where add new instances of those while some are already running.
More comments in a later post, but this now as a quick status update.
~Toni
Jonne Nauha
- EC_Script wont report materials, images (aka textures), particle scripts or ui files as dependencies if viewer/server is headless. This in practically means saving RAM on a headless server as i does not even try to fetch those into memory as it wont need them in rendering (about 1 meg less memory usage for a headless LVM server). This also "demands" the logic in scripts that if we are headless you dont even try to load/show ui assets and show them in healess mode. This is simply handled so that the asset ptr will be null from uiservices.LoadFromFile(uiAssetRef). This fix was partly also done so that you wont get error prints on a server for every material/texture and leaving the user to think that something is actually wrong when everything is running fine.
- AssetRefenrenceList now works as assumed in javascript.
- No LogDebug()s anymore in Release or RelWithDebInfo mode. Affects also the release that was spamming debug stuff to console.
- Drag and drop tooltips. Will show information of source and destination of the drop. Source: filename or filenames (multiple file drop). Destination: the x,y,z position of where you are dropping and the entity name you are dropping to if one is available. For .material drops destination will tell you the entity name if availbale and the submesh index you are dropping to. The tooltip logic works both for web url and local file drops.
- Make the Texture type support also .jpeg (i think earlier was just .jpg).
- Make AssetAPI default to "Binary" type if no hits were found from the known Tundra asset types. This means if you ask asset.RequestAsset(my.zip); wont fail anymore, previously you had to spesify RequestAsset(my.zip, "Binary") for it to go properly.
- The main graphics view will emit also DragLeaveEvent() that is quite important for some logic (needed it with the above tooltips)
- Fixed asset cache bug where the base cache folder was not created if did not exists (resulted in crash during login).
- Several minor fixes in AssetAPI.
Jonne Nauha
Adminotech developer
jukka....@ludocraft.com
thanks for the questions. Integrating different login & asset
authentication methods is possible. By default the built-in Tundra does
not require any kind of authentication, because we cannot expect a single
mechanism to be suitable in all cases. Tundra uses a generic
script-exposed login request->reply messaging structure, and scripts/C++
modules can add in any custom data they want to pass through in the login
mechanism, and have a server-side script/C++ module to examine the custom
data and decide whether to accept the connection or not.
The Tundra core itself doesn't enforce any kind of user account system,
which is great for situations like "connect, fetch the scene, disconnect &
browse the world offline". It would be silly to require the user to
register an account just to view a public "3D web page". Currently when
you use a Tundra client to connect to a server, you can pass in whatever
you like as the username, and the server doesn't check that name against
some registry, but takes that name as the username for the client.
Tundra comes with an example script that shows how to use the login
properties. It applies a single password to a server, so that anyone who
knows the password can connect, and the password isn't tied to a user
account registry. That example could be used as a starting point for
working on integrating a different authentication mechanisms.
So, in summary, yes, it's possible to use other authentication systems.
Whether it makes sense depends on the application at hand, but it wouldn't
make sense to define a certain model to be *the* authentication system
used in Tundra core, as people's needs vary.
Best Regards,
jj
Toni Alatalo
> account registry. That example could be used as a starting point for
> working on integrating a different authentication mechanisms.
I made an experimental little integration of SimianGrid (the current
project where cable beach used in Taiga migrated, basically robust
+cablebeach rewritten in php on of apache and mysql) for authentication
some weeks ago, works, and is included in the releases.
Code (<50 lines is in:
https://github.com/realXtend/naali/blob/tundra/bin/pymodules/simiangrid/auth.py
Usage: point that code to a simiangrid installation somewhere, by
changing this line: url = "http://localhost/Grid/" . Create a user
account if you don't already have one, using the web frontend. It is
easy to install locally, just apache + mysql + clicking thru their web
ui install wizard.
Enable that naali/tundra python module by uncommenting this line in
pymodules/default.ini:
[simiangrid.auth.SimiangridAuthentication]
Login with the current normal Tundra login, use the credentials you
entered for your simiangrid account. Note: due to how the current Tundra
login ui works, you must use _ for whitespace .. e.g. Toni_Alatalo.
Respectively, due to how simiangrid web frontend ui now works, your
account must be in form: Firstname Lastname (that's just a ui
limitation, not in api or db) .. can't do just 'myaccount'.
When running with this auth module enabled, you can not login without
valid SG creds. Probably that other auth system Pedro mentioned is as
easy to integrate.
After that, I started to look if could easily get OpenID authentication
to Tundra using the same SG backend, but got lost in it because didn't
know how to deal with the loginurl that get after the succesfull
authentication .. as we don't use loginurls with tundra :o The client
side of that basically works already, shows a webview at tundra viewer
startup that shows the simiangrid login page where can do openid, and
parses the resulting VWRAP calmj world launch document succesfully:
https://github.com/realXtend/naali/blob/tundra/bin/pymodules/login/webui.py
> jj
~Toni
Peter Steinlechner
Also thanks to Jonne - i just downloaded 1.03
I start to like Tundra more and more every time I try it. Hopefully it will change the web a bit :-)
I assume it also scales nicely regarding the concurrent users count with the available hardware resources?
Cheers and a great Weekend to all of you.
Pedro
jukka....@ludocraft.com
> the
> available hardware resources?
I hope you don't expect too much and disappoint afterwards. Most work in
Tundra has gone into building a structure that is flexible enough that
it's possible and even easy to add new functionality without breaking
things, i.e. "scaling in features".
The task backlog I posted also contained some items for network
optimization. There are acknowledged to-dos in the Tundra SyncManager,
which can limit in ramping up the number of concurrent users. The
SyncManager is good for general use, but for specific replication of rigid
bodies, there is still room for improvement.
I put up a comparison of the current Tundra network profile to
http://wiki.realxtend.org/index.php/A_comparative_analysis_of_Tundra_network_usage
, so you can know what to expect from the current version of Tundra.
jj
Peter Steinlechner
Thanks for the explanation and its beyond my expectations. According to Moore's Law there won't be any performance issues anyhow in a few years, so flexibilty is of a bigger value.
Toni thought it should be able to connect with 30-50 users, which should be ok for a lot of uses for starters.
btw - i noticed some strange behaviour in the Tundra Chesapeak Bay Demo:
If i connect with a second client, I can see both avatars. But from the first client I just get the messages but can't see the second avatar. Same appens when I connect with a third one.
Cheers
Pedro
--
Toni Alatalo
btw - i noticed some strange behaviour in the Tundra Chesapeak Bay Demo:
If i connect with a second client, I can see both avatars. But from the first client I just get the messages but can't see the second avatar. Same appens when I connect with a third one.
Pedro
Jonne Nauha
Best regards,
Jonne Nauha
Adminotech developer