Moving Towards Tundra 2.0
jukka....@ludocraft.com
during this past half, work at LudoCraft has continued at an accelerated
pace on the Tundra platform. LudoCraft designed and developed the original
Tundra 1.0 platform, but instead of stopping there, we've been very busy
working towards polishing what we call the Tundra version 2.0.
The main motivation of this work has been on simplifying the architecture
and optimizing it to work with fewer dependencies and a smaller memory
footprint. I started the work some months ago on a separate Tundrajj
branch which is available on github, and now LudoCraft has taken two or
three big steps further to scale down on the size of the framework. For
comparison, the old codebase has 229807 LOC, 774 classes and 59 projects,
whereas Tundra 2.0 sits at 122979 LOC, 309 classes and 26 projects, and
re-builds in well under 10 minutes on my system.
And the best part: it's half the code - but twice the features! We've
taken the time to unify and utilize the core APIs, asset types and
components in the system to be more flexible than before, and extended the
script bindings considerably by implementing an automatic QtScript
bindings generator, which enables the scripts to be more aware of the
scene and perform more complicated geometric processing which wasn't
possible before. Because the math data structures are now symmetric in
terms of features in both C++ and QtScript, it is easier to implement full
applications using scripting only. Also, because the data structures and
APIs are symmetric, we don't need to develop two different reference
documentation bases, but can do with only one!
As for the documentation, see an example of the new maths library
documentation here: http://clb.demon.fi/dump/DocGenerator/ (still alpha).
We are moving away from using doxygen as the primary documentation system,
and instead are generating our own formatted documentation pages, as shown
above (though using doxygen as the parser backend). This documentation
serves for both C++ and QtScript (and hopefully for other script languages
as well) reference.
The codebase is not currently hosted public, and I don't know of the exact
details of what will be released in which time frame, and into which
branch it all will go to. I have however assembled a general changelog of
modifications, so you can get a grasp of all the things that we have
changed. A word of warning: these are not small things, but a result of
some 15-20 man-months of our development effort, so there are a lot of
changes.
One thing you may notice is that there are a lot of code that has been
removed. There are various reasons:
- the code was superceded or obsoleted by a better construct.
- the code (which we don't develop here at LudoCraft) did not compile
after some related change, at which point I deleted the code instead of
trying to make it build without testing, pending a refresh from the
original author or maintainer.
- the code does not work and has not had an identifiable maintainer in a
long time, in which case I simply consider the feature rotten.
- the code was outright badly written or designed and sits too deep in
the core for me to allow it to exist. (some rare individual cases)
Best Regards,
jj
LudoCraft Tundra 2.0 Changelog
---------------------------------------
Deleted code and deprecations.
- Deleted EventManager and the internal event system queue mechanism
(EventManager.h, IEventData.h). Direct signal-slots are used instead, as
they are more convenient.
- Rewrote all uses of EventManager to provide a proper signalling API,
e.g. in TundraLogicModule Server.cpp and Client.cpp.
- Deleted obsolete ConfigurationManager.
- Deleted ServiceManager and the concept of services.
- Deleted UiServiceInterface.
- Deleted rotten EC_BillboardWidget, EC_MovableTextOverlay.
- Deleted dependency to the PoCo library altogether, and other smaller
3rd party libraries like curl.
- Removed the use of GetModuleManager wherever applicable. Modules are
queried directly through Framework::GetModule, not via a separate
manager.
- Deleted EC_ChatBubble, EC_Clone, EC_Gizmo, EC_Ruler, EC_Selected,
EC_VideoSource, EC_3DCanvas, EC_3DCanvasSource, EC_Videosource. These
features can be reimplemented as plugins proper outside of core.
- Deleted the folder EC_3DGizmo, which has gone unused.
- Deleted PhononPlayerModule, ScenePersistenceModule, UpdateModule,
pending a refresh to the new plugin mechanism.
- Deleted CAVEStereoModule, pending a refresh to the new plugin mechanism.
- Deleted IntervalTimer and LockFreeList core objects (unused).
- Deprecated OgreRendererService (currently under the name IRenderer),
replaced with the mechanism below.
- Deprecated the concept of 'current active scene' and 'the default
scene'. Instead, use a single 'current active camera on the main UI'
query.
- Removed hash-based asset reload checks and booleans for isLoaded.
- Deleted badly placed Renderer::RenderAvatar.
- Deleted QGraphicsEffects from UiProxyWidget.
- Deleted Platform.h, PlatformNix.h, PlatformWin.h as unneeded.
- Deleted unused Root logging functionality and Framework thread task
mechanism.
- Removal of several **Raw workarounds. (more still exist, which should
be rid of asap)
- Removed all legacy code that was unmaintained, unused or otherwise
badly implemented to be harmful to use. Core code should either implement
properly or not at all - temporary hacks will not be accepted without a
very good reason.
Solution reorganization
- Deleted the obsolete projects Foundation, Interfaces, RexCommon.
- Deleted the obsolete projects InWorldChatModule, InventoryModule,
RpcUtilities, RexLogicModule, ProtocolUtilities, ProtocolModuleOpenSim,
ProtocolModuleTaiga, HttpUtilities and Core.
- Deleted the obsolete components EC_HoveringWidget, EC_NetworkPosition,
EC_OpenSimPresence, EC_OpenSimPrim.
- Removed SceneStructureModule and moved all its contents to
ECEditorModule (these two always come together or not at all).
- Moved EC_InputMapper to Input project.
- Moved EC_DynamicComponent to Scene project.
- Grouped EC_SoundListener and EC_Sound to same project to improve
compilation times.
- Moved EC_Light to OgreRenderingModule.
- Deleted the use of PCH in all EntityComponents projects in which it
doesn't help compilation times.
- Replaced server.exe and viewer.exe with a single tundra.exe, uses
--server parameter to decide whether to start a server, and --config
viewer.xml if to start a client viewer UI.
New Plugin API (mostly never used directly)
- New plugin mechanism for a simpler and more flexible plugin
implementation.
- Implemented a new plugins.xml to configure the set of modules and
startup .js files to load. Use --config startup parameter to specify the
set of modules to load.
- Removed the module description XML files and startup-time sorting based
on dependencies.
Finished the work on Asset API
- Fixed several bugs where storages, providers and disk sources were not
remembered for assets when taking them from the web cache.
- Unify the handling different asset ref forms into Asset API.
- Implemented an Asset Ref resolving step, where a relative ref is
resolved (possibly inside some context) to an absolute ref.
- Reimplemented the signalling flow of asset downloads to guarantee a
coherent signal flow independent of how an asset is loaded.
- some changes to signals to remove ambiguity, e.g. renamed
IAssetTransfer::Loaded() to IAssetTransfer::Succeeded() to avoid
confusion with IAsset::Loaded().
- Implemented context-based relative asset handling to allow relative
dependencies between assets to work properly
- Implemented asset refs to named asset storages which allows scenes to
be transferred from one server to another without having to manually
search&replace script refs.
- Implemented subassetname specifications to assetrefs to allow pointing
to a part of an asset inside an asset.
- Implemented asset storage synchronization on login to be able to refer
to assets using relative names or named storage specifiers on the server.
- Implemented a push-based asset update notification mechanism (e.g.
alongside HTTP storages) so clients automatically know to fetch the
newest version of an asset as soon as it is available.
- Implemented a text serialization mechanism ("serialized storage
strings") for systematically handling configuration for different types
of asset storages
- Implemented an asset discovery mechanism that synchronizes folders of
assets from a networked storage (e.g. using PROPFIND when over HTTP).
- Formalized the concept of an Empty asset: URL to fetch is known,
neither loaded or locally cached data is available.
- Formalized the concept of an Unloaded asset: URL to fetch is known,
data is available locally, but the contents are offloaded from (GPU)
memory.
- Added a null asset factory (e.g. for headless mode), and removed
several IsHeadless() checks sprinkled in hardcoded places.
- Deleted the decoding step of an asset transfer (decoding occurs on
asset load)
New math library (float2, float3, float4, float3x3, float3x4, float4x4, Quat)
- Automatically generated bindings instead of manual ones.
- Removed manually written Transform, Quaternion and Vector QScriptValue
bindings.
- Remove use of previous Irrlicht math code which lacked in features and
used inconsistent conventions
- Math objects for geometric computations, symmetric in C++ and QtScript.
(AABB, OBB, Sphere, Line, LineSegment, Ray, Triangle, Plane)
- Example preliminary documentation here
http://clb.demon.fi/dump/DocGenerator/
Reimplemented environment-related functionality.
- SkyX and HydraX integration.
- Deleted the use of Caelum altogether, replaced with SkyX (optionally).
- Deleted all badly structured EC_Skybox, EC_SkyDome, EC_SkyPlane,
Sky-templates.h.
- Deleted rotten EC_OgreEnvironment and EC_OgreSky.
- Reimplemented EC_Sky as a more generic attribute-based sky component
when SkyX is not used.
Avatar
- Implemented AvatarDescAsset and EC_Avatar to use the Asset API.
- Reimplemented avatar script as a more extensible way to allow extending
own animation playback and controlling which built-in behavior is used
and when.
- Simplified the mechanism how avatar description files refer to other
assets.
- Partially reimplemented the avatar editor against the mechanism.
- Removed object camera/tripod camera controls from the avatar script,
they need to be separate applications (or optionally removable).
- Deleted old EC_AvatarAppearance, EC_Controllable, EC_OpenSimAvatar
components and other obsolete avatar uploading functionality.
Reimplemented Console API
- Deleted the old ConsoleCommandStruct registration mechanism.
- Implemented console commands to operate directly on Qt slots.
- Moved FunctionInvoker to core to enable the above. Will be used for
entity actions as well in the future.
- Autocompletion suggestions.
- Enable browsing previously typed commands.
- Deleted MODULE_LOGGING_FUNCTIONS #define macros and reimplemented
logging via much simpler means.
- Deleted DEFINE_LOGGING_FUNCTIONS.
More robust asset support
- Implemented script-accessible modification functions for TextureAsset
(SetContents, SetContentsDrawText, SetContentsFill, QImage interop)
- Exposed OgreMaterialAsset material specifications for C++ and QtScripts.
- Reimplemented asset editors for textures, meshes, materials and
particle scripts, with syntax highlighting support.
- Added support for the handling of relative refs to ParticleAsset,
QtUiAsset, Script and OgreMaterialAssets.
- Implemented QtUiAsset::Instantiate() to be used over
UiApi::LoadFromFile wherever applicable.
New component factory mechanism
- Removed ComponentManager, DECLARE_EC and DECLARE_MODULE_EC.
- Deleted Foundation.h, IComponentRegistrar.h,
- Implemented a new simpler registrable component factory similar to
asset factories.
More robust component types
- Implemented EC_Material which allows replicated modification of
materials e.g. for animation purposes.
- Reimplemented EC_HoveringText with more features and control over how
text is generated. Removed hardcoded animation logic in the component
that is more flexibly implemented in scripts.
- Reimplemented EC_Highlight with attribute-based features.
- Improved EC_ParticleSystem to work with AssetRefListener.
- Renamed EC_OgreCamera to EC_Camera and filled it with utility functions
(GetMouseRay(), IsObjectVisible()) and attributes (up vector). Refactored
it to use the new math classes.
- Implemented isKinematic property for EC_Rigidbody.
- Switched convex hull generation code due to critical bugs in the
previous algorithm.
- Deleted PostProcessWidget. Improved EC_OgreCompositor to enable
asset-based compositors.
- Added a sample demo UI to bin/scenes/Compositor/ to showcase the
different compositors.
- Remove hashing from component type names to prepare for new optimized
sync protocol implementation.
- Reimplemented EC_Billboard to support attributes and AssetRefListener.
- Removed IComponent::IsSerializable() - all components are serializable.
Likely breaking changes
- Reimplemented and simplified EC_Placeable which had became a
inconsistent mess of random conveniency functions. Now math code is
properly handled in the new math classes instead of built-in into
EC_Placeable functions.
- Removed the use of hardcoded conversion scenenodes in the scene graph
to flip y<->z. Now all input data is used as-is without runtime
conversions. Reimplemented asset importers to not perform swap hacks.
- Core API function naming refactored to be more consistent in all classes.
- Deleted Entity::HasComponent. (just call Entity::GetComponent and test
for null).
- Deleted the excessive use of namespaces throughout the core API - they
aren't needed, and make code much more verbose than it needs to be.
- Deleted SceneAPI::GetSceneRaw.
- Deleted Entity::GetComponentRaw().
Refactored the core scene structure to support additional augmented scene
views of the core scene:
- OgreWorld contains the Ogre-specific data of the core scene
- PhysicsWorld contains the Bullet-specific data of the scene
- Users can register their own scenes, in case the
entity-component-attribute mechanism is limiting.
- Exposed to scripts as "scene.ogre." and "scene.physics." objects.
Operates as the primary interface of
both visibility- and physics -related scene queries for scripts.
- Implemented frustum-based visibility queries for scripts along the
raycast queries.
- Implemented OnEnterView and OnLeaveView -mechanisms to allow scripts
and plugins to implement visibility-based optimization on their script
logic.
Editor improvements
- Enabled changing entities' temporary status from Scene Structure window.
- Enabled changing entities' local/replicated status from Scene Structure
window.
- Enabled a dialog-based "..."-selection for assets in the EC editor
asset refs.
- Added "E"-button to ECEditor to open an asset in an asset ref to an
editor.
- Enable cloning assets from the editor.
- Show asset status in Assets window.
- Added "Do not alter asset references" option to scene import dialog,
which leaves relative asset refs intact.
Other API and misc. changes
- Added Entity::Clone()
- Deleted Renderer::RepaintUi().
- Added FunctionDialog for the Assets menu for accessing functions from
individual assets, providers and storages from the UI.
- Added IAssetStorage::Type()
- Removed Framework::GetModuleQObj with Framework::GetModuleByName,
exposed IModule to scripts.
- Added script access for specifying exact EC_AnimationController
animation timing parameters.
- Fixed EC_VolumeTrigger to actually work.
- Optimized OgreMaterialAsset clones to do a direct in-memory copy
instead of going through disk.
- Implemented a separate RedirectedPaintWidget for optimized Qt UI
blitting in certain cases.
- Fixed Profiler crashes and issues where it lost some profiling blocks.
- Deleted UiApi::AddWidgetToSceneRaw.
- Reimplemented the managing of important directory paths to centralized
functions:
- Application::SetCurrentWorkingDirectory().
- Application::CurrentWorkingDirectory()
- Application::InstallationDirectory()
- Application::UserDataDirectory()
- Application::UserDocumentsDirectory().
- Implemented and tested unicode support for path handling.
- Reimplemented all code to use the abovementioned folders instead of
assuming cwd is some specific directory, to prepare to run Tundra
in-process in a web browser plugin.
- Deleted AssetModule::AddHttpStorage and replaced it with a more generic
AssetModule::AddAssetStorage which uses serialized storage strings.
- Deleted poorly structured IAssetProvider::AddStorage, replaced by
serialized storage strings.
- Reimplemented frame-time limiter to sleep inside Qt's event loop
instead of the Ogre renderer.
- Implemented IAssetStorage::GetAllAssetRefs() to enable querying storage
contents.
- Fixed SceneManager entity ID allocation to return guaranteedly unique
IDs. Added runtime checks for known critical (but never yet seen) bugs in
SyncManager.
Input API improvements.
- Tied the setting of mouse cursor to InputContexts so that they get
cleaned up after contexts are deleted or overridden by priority, to avoid
commonly seen user bugs where an override is never cleared.
- Implemented InputAPI::MousePos.
- Reimplemented mouse double click handling in InputAPI, deleted related
test booleans.
- Treat input as being over Qt when mouse cursor is hidden
Entity parenting support.
- Implemented support for parenting entities to other entities or bones of
meshes.
- Implemented EntityReference attribute type.
- Added EC_Mesh::GetAvailableBones().
- Implemented script-accessible support for querying mesh bone transforms.
Peter Steinlechner
Cheers
Pedro
Hi,
during this past half, work at LudoCraft has continued at an accelerated
pace on the Tundra platform. LudoCraft designed and developed the original
Tundra 1.0 platform, but instead of stopping there, we've been very busy
Working towards polishing what we call the Tundra version 2.0.
Ali Kämäräinen
Jonne Nauha
Best regards,
Jonne Nauha
Adminotech developer
Jukka Jylänki
wrote:
> from your new Tundra. I'd like to ask you is this code now removed as in
> "git rm" or just disabled from the build via cmake and waiting for
> porting
> to be done? At what point have you last rebased/merged the tundra branch,
> was it when tunrajj was created?
The code has been deleted from the repository, but of course git history
has it all. I'm investigating moving into a model where the core would be
in one branch, and the applications themselves are in another, or a git
submodule perhaps. We feel that the core in the current public codebase is
getting polluted by features that should be strictly on the application
level. Having an explicit separation of these might help people to avoid
accidentally mixing the two.
Naturally I will not dump our codebase to erase any existing branch in the
public repository, and I will not delete any of the history from the
public branches.
> I think
> unloading this 15-20 man months amount of work/code on the rex project in
> one big splash is a little inconvenient for many of the companies and
> parties involved.
Bitten once, twice shy, past work in the current public codebase
introduced unwanted features and issues for us. I don't get paid from
trying to sort out all the code that gets committed, and I will not
maintain and hand out checklists of things our developers need to remember
to remove from the public codebase to configure it to work for us. Our
vision is that Tundra is a generic network-based application development
platform, and not an application itself, but I see the public codebase has
been drifting away from that goal. Additionally, some of the features we
have been developing cannot be made public at this time due to contract
agreements.
> It's like the core was rewritten in the total darkness without any
> discussion and now we should all just move to it.
You are naturally free to work on any codebase, nobody can stop you. My
message is simply that we don't do our internal development on top of the
public Tundra branch or the Tundrajj branch, which we consider both to be
old versions.
--
------------------------
Jukka Jylänki
Programmer
LudoCraft Ltd.
http://www.ludocraft.com
jukka....@ludocraft.com
------------------------
Jonne Nauha
On Tue, 14 Jun 2011 12:51:14 +0300, Jonne Nauha <jo...@adminotech.com> wrote:The code has been deleted from the repository, but of course git history has it all. I'm investigating moving into a model where the core would be in one branch, and the applications themselves are in another, or a git submodule perhaps. We feel that the core in the current public codebase is getting polluted by features that should be strictly on the application level. Having an explicit separation of these might help people to avoid accidentally mixing the two.
from your new Tundra. I'd like to ask you is this code now removed as in
"git rm" or just disabled from the build via cmake and waiting for porting
to be done? At what point have you last rebased/merged the tundra branch,
was it when tunrajj was created?
Naturally I will not dump our codebase to erase any existing branch in the public repository, and I will not delete any of the history from the public branches.
Bitten once, twice shy, past work in the current public codebase introduced unwanted features and issues for us. I don't get paid from trying to sort out all the code that gets committed, and I will not maintain and hand out checklists of things our developers need to remember to remove from the public codebase to configure it to work for us. Our vision is that Tundra is a generic network-based application development platform, and not an application itself, but I see the public codebase has been drifting away from that goal. Additionally, some of the features we have been developing cannot be made public at this time due to contract agreements.I think
unloading this 15-20 man months amount of work/code on the rex project in
one big splash is a little inconvenient for many of the companies and
parties involved.
You are naturally free to work on any codebase, nobody can stop you. My message is simply that we don't do our internal development on top of the public Tundra branch or the Tundrajj branch, which we consider both to be old versions.
It's like the core was rewritten in the total darkness without any discussion and now we should all just move to it.
Jarkko Vatjus-Anttila
We see that there has been great development done in Ludocraft and I
bet most of the features are really something which would be very
beneficial for us as well, even though we are running in slightly
different scope than the companies around here which rely on
commercialization. We do research on certain topics and one of the
corner stones for our development has been that we will ride with
semi-ready and stable platform where we can focus on developing things
assisting the research targets directly. We consider any work in the
core to be away from the research targets, but having said that we are
glad to bring our improvement there as well, where it seems suitable.
In open source projects one of the greatest benefits is the visibility
over the community to see where the development is going. It gives an
edge and an opportunity to prepare upcoming development and be
prepared with our own code to match those changes. Using a big lump of
fundamental changes to bombard common codebase is not a very nice way
of development and will cause a stir in the project. Even though the
code was not released, it still would have been nice to know the way
of development already earlier. I'm not saying that it is illegal to
work in favor of business cases, but information about the way towards
the future would have been nice. On the other hand one can consider
that this announcement was exactly that information, but IMHO it might
prove too much as one single lump of code.
My fear is that once the code is available, it will prove too big of a
leap (even though changes are good) for us to jump into the same
direction. It would simply cause too much of an integration and
porting work which we are having hard time to prove to belong to the
project anyways. Of course I'm saying this without seeing the code,
and I hope the leap is not that big and hence I hope the integration
effort would not be that large. This will remain to be seen. Looking
forward for those changes.
- j
Toni Alatalo
> I'm unsure what you mean by polluted features, you have mentioned this
> more than once lately. I think the current tundra branch is quite
> clean of these things. Application level might pop up in example
> scenes or at most js startup folder (the camera code you destroyed
That remark seemed nonsensical to me too -- afaik the core is basically
the same in the different branches, and doesn't have app specific
things. Everyone devving now wants a generic app platform. I think Jukka
had actually misunderstood something of the work done during spring,
e.g. how the core ui has been extended.
But now for example this commit from today makes me wonder .. granted,
it's just an EC, and not really core in that sense, but still something
that is proposed (by this commit) to be in the default distro always and
certainly is very application specific:
https://github.com/realXtend/naali/commit/a6f47a0dba1c63779bf0fbd82054376f08168a63
EntityComponents/EC_Sittable/EC_Sittable.cpp
void EC_Sittable::GoTowardsTarget()
Scene::Entity *avatar = GetOwnAvatar();
.. that seems misplaced to me, on first sight at least.
Sitting is a feature of AVs, wouldn't it be better to have the logic to
move AVs etc. be in the AV application .. where also the other av
movement codes are? This EC that grabs control of AVs seems a bit
strange, especially as many apps don't even have AVs. The EC could
perhaps better be just data (even a DC type defined by AV app perhaps,
if sit targets only make sense when also AVs are used?)
Sure, many ECs are some quite specific functionality (like WebView) and
make sense as such, possibly this is ok this way too, but does seem a
bit strange and warrants some thinking.
It is quite a challenge to both allow quick dev and delivery of new
features, but also controlled dev with long term design so that the tech
stays clean. One idea I've been thinking for that would be separating a
more stable long term core, and dev sandboxes with experimental things
and quick hacks needed for feats etc. At least the separation of core +
extra mentioned in earlier posts is good indeed, was discussed also in
the live meets.
> Jonne Nauha
~Toni
Morgaine
-1 million for closed development, unilateral deletion, and springing a dramatically revised core 2.0 as a fait accompli on what was an open project.
Toni gives a good example of open discussion about why a particular feature may not be appropriate for core. That's the way to handle it in an open project, not swinging an axe behind closed doors at code you don't like for 15-20 man months, and then throwing a new core 2.0 over the wall. Such unilateral decision making creates an "us and them" divide, and leads to poor relationships and uncertainty about the future.
It sets a very bad precedent for project openness. I hope that this was just a mistake, and that core redesigns will be handled openly in future. Openness really is for the good health of the project in many social and technical ways.
Morgaine.
=========================
Jonne Nauha
Best regards,
Jonne Nauha
Adminotech developer
Jukka Jylänki
To clarify:
- We are not forcing anyone to move towards the codebase "we wrote in
hiding".
- The development has not exactly been a secret, I've talked about e.g.
coordinate system conversions and asset api implementations before over
IRC, but my time doing general advertisement emails about what we are
doing is rather limited.
- We have had, and are having meetings with related parties, to query if
any things in the backlog cause problems for them.
- We are not taking existing code away from anyone, and strategically, we
are ok, although sad, if "the public" decides to continue on the current
github-hosted codebase.
- Some of the code was developed over "commercial funding", and there
hasn't been a decision made yet to whether the repository will be released
into public. Jarkko expressed grief that this e-mail is coming very late,
but it's on the opposite, I'm letting people know of what kind of work has
been done, well before any moves are made.
- Don't be afraid if some files have been deleted from the codebase. Git
doesn't lose anything, and it's easier to restore files using git than to
manually go restoring commented out code. In the rex project, we have had
too good a track of retaining old code in the repository with the thoughts
"someone's going to fix it Real Soon, and it almost works now." (for
example, OgreAssetEditorModule, EC_HoveringWidget, ScenePersistenceModule,
OpenAssetImport, ...), but even after half a year, it's still not there
yet.
- I would like to powwow every week with everyone in harmony, but this is
not a hobby project I'm working at home, but real deadlines with real
deliverables. In several cases, it's a do-or-die for us, and it makes no
sense whatsoever for me to hold general meetings over a feature X I know
exactly how I need to implement in order to ship our product Y. When/If
the time for release comes, anyone is welcome to reject that
implementation, which is only natural. Making a proposal in terms of
functional working and tested code almost always trumps a design document,
and I find it odd to hear comments that people would have felt more
welcoming towards to getting only design documents instead.
- Again, we are not forcing anyone to move towards the codebase we are
developing.
Perhaps parties external to Oulu don't know the details here - LudoCraft
is not currently running its Tundra development on open-source funding or
similar, but we have customers who are foremost interested in getting
working products. I don't get paid to attend day-to-day scrum meetings
over IRC anymore *sob*. We are making an unpaid effort to bring our
development to the public, but if it causes bad blood, it will just be a
lose-lose for everyone.
We do not want to unilaterally force the direction the public codebase is
developing towards, that is why we have been meeting with the other
developers to hear their thoughts. We have to have a consensus from all
parties, or otherwise we are better off abandoning the thought completely.
We are ok if in the end, the LudoCraft Tundra codebase diverges from the
public codebase, but naturally we hope to reach a convergence here.
Since we have the manpower and experience, I will dictate the direction
the technology LudoCraft uses internally will develop towards. I am good
at making amendments in the case it brings us more actual working
features, valuable bug fixes, better design and outside expertise, but
I've also worked on some open source projects to see bad participants who
only bring loud mouths to the table, and nothing ever progresses.
I understand a sense of "why wasn't I involved!" grief, but the previous
email is exactly that - we are involving you now. So now that that
reaction is out of the way, I am happy to answer to any concerns you might
have related to the actual changelog itself.
And remember, we are not forcing anyone to suddenly "hop on" to anything.
Best Regards,
jj
Toni Alatalo
Thanks for the clarifications, Jukka, I was just about to start writing
a (much briefer!) reply with the same points.
Also other companies sometimes have the issue that e.g. contracts with
customers demand that some code is made public only after a certain
period of time. Actually we are lucky when the permission is gotten at
all -- open source practices are still not too well known everywhere.
Typically it is actually most beneficial for the customers too when the
development they need is done quite directly in the mainline repository,
so also the version they use is the one that gets most testing and
bugfixes etc. But this is a far reach for an organization with a lawyer
who needs an explanation and study of the licenses to see why they can't
get exclusive rights to the *whole* codebase :)
BTW this kind of delays happen also with e.g. Opensim dev, we are not
alone with the problem. For example right now Intel has been developing
their scalability system (server side proxies, or 'client managers') for
1,5 years and none of it is public yet. One reason I heard in January or
so (IIRC from them directly) is that they are making it for some
customer who wants to use it first (which btw sounded strange to me, I
didn't know Intel research labs does custom software dev for customers).
> not a hobby project I'm working at home, but real deadlines with real
> deliverables. In several cases, it's a do-or-die for us, and it makes no
> sense whatsoever for me to hold general meetings over a feature X I know
> exactly how I need to implement in order to ship our product Y. When/If
> the time for release comes, anyone is welcome to reject that
> implementation, which is only natural. Making a proposal in terms of
> functional working and tested code almost always trumps a design document,
> and I find it odd to hear comments that people would have felt more
> welcoming towards to getting only design documents instead.
Yes, these real life demands, business pressures etc. and it all being
about making a useful platform that can be used for some sort of
benefit, is what I nowadays call "normal open source" :) . It is the
same for all the companies involved, and largely the case in other
successful open source projects out there. Sure in good cases it is also
mixed with volunteer efforts etc., and luckily we have that too.
Basically we need to refine the process so that it's most cost effective
-- and I believe that is well in line with the usual open source
practices that Morgaine rightly called for. Because knowing about
developments beforehand, getting to share results as early as possible
etc. minimizes overlapping efforts, more complex merges and heavy
communication. I trust this to be the case because see open source being
mostly about practicality and cost effectiveness, and the common
practices have been refined in many projects by many companies for long
now.
And to be fair, I'm now the only one who gets actually paid (by
realXtend association, thanks for the sponsorship of CIE.fi) part time
to work on the project in general, so I have the luxury time to
concentrate on reviewing and commenting other peoples and companies code
etc. Is my job basically to improve the processes so that companies can
better rely on the shared code, contribute to it etc. without too much
hassle. But there is no way around the need of a fair amount of
communication -- I guess that's what we pay instead of EPIC or so
license fees, or just making an inhouse product.
+1 for running code as design specs, that's why I call many commits
'proposals' :) "Rough consensus and running code" is even the way to
*standardize*, the IETF says, and has succeeded. But is still good to
also post a heads-up beforehand so ppl know what's coming, and share the
w.i.p. since the beginning and continuously during the work, when
contracts with customers etc. permit.
I think enough has been said now, we can happily wait for the code
(should come next week) and go from there then .. it may take a while to
adopt it, but it's ok 'cause 1.x series works fine and these proposed
changes don't cause much a difference up to the application level (apart
from faster startup, lower mem use etc), so we can keep developing apps
and games and modules etc. in the meantime.
> jj
Cheers,
~Toni
Antti Ilomäki
First of all, I understand and know from experience how difficult the
concept of open source can be to corporata lawyers / bosses. I also
know how important such projects are for the developer companies and
thus also to the project - if you just take a look at Jukka's feature
list you can see what a dedicated team of programmers can achieve in a
relatively short time. This is one way for a platform to grow, being
driven by the application projects built on top of it.
The challenge is in fitting together the deadline and secrecy needs of
customer projects and the general platform development work. One thing
I believe could help would be to create possibilities for
communication between developers. For that we have a pretty good tool:
the realXtend Association that was founded for exactly that kind of
work. We already had one general planning session, where people could
talk about their ideas and needs for the future, perhaps this
tradition can continue? I have also heard talks about roadmaps etc.,
which will likely be very useful indeed. At least it would be useful
to discuss ideas, especially related to core development, before they
turn up as (lots sof) code for Toni or others to review.
Morgaine
The challenge is in fitting together the deadline and secrecy needs of
customer projects and the general platform development work.
The consequences of putting commercially sensitive, deadlined or secret customer functions in core are terrible: core releases get deferred because product management wants to spring a surprise for commercial advantage, and everyone else suffers. It creates an atmosphere of secrecy too, and uncertainty for everyone else.
The core really needs to be maximally open and non-secret, with customer projects external to it. That's good modularization policy too! :-)
Morgaine.
==================
Jonne Nauha
Best regards,
Jonne Nauha
Adminotech developer
Morgaine
Morgaine your statements are fine in principle and I can concur with them, but you are badly mistaken if you think any of that happens in our code base
The challenge is in fitting together the deadline and secrecy needs of
customer projects and the general platform development work.
Should it ever be needed though, adding open API hooks to core which can then be used to hook in non-open customer projects is a good way to factor the codebase and keep everybody happy. (And keep the companies' money rolling in. :P)
Morgaine.
===================