A Scala apprentice looking for opportunities to become better
Jussi Mononen
I am very interested in becoming better in Scala by doing something
useful with it and I thought I could try to contrubte to DimDwarf.
I have cloned the git repo but browsing through the 225 files is a quite
a large task. I've read the email list introduction and Esko's blog
article about Dimdwarf's new architecture.
All pointers to the source code and details are warmly welcome and maybe
some explanation about test strategies. I have almost read the GOOS
books so the principles are familiar :-)
Also, if there are any rules about coding style (indentation etc.) it
would be nice to know or preferred editors or development environments.
And a little bit of my background. I've been programming for living over
10 years now. I have programmed with C, Perl, Java, Shell and little
with Scala. For the 10 years I have been building near real-time systems
for large teleoperators. We have used in-mem db's, HA env's etc. so they
are familiar though I can't say that I'm a DB/HA wizard.
So, what should I do next:-)
Br,
--
- Agile Poodle
- http://www.jussimononen.info/
- http://www.twitter.com/agilepoodle
Esko Luontola
> I am very interested in becoming better in Scala by doing something
> useful with it and I thought I could try to contrubte to DimDwarf.
It's nice to hear about you. :)
Having a deep knowledge of Scala should not be needed in this project,
because anyways I prefer to avoid using some of the more advanced
language features, because they may make the code harder to read. So no
need to be afraid of the language. ;)
Pattern matching and closures are the main features which made me choose
Scala for this project. This project's architecture is orgranized into
"controller" and "worker" (the divide is not yet explicit - I'll split
the dimdwarf-core module later). The worker runs application code in
transactions ("tasks"), one worker thread per transaction. The
controller coordinates between all participants - transactions,
database, clients, other servers in the cluster etc.
Scala will be used in the controller side (and the interface between
controller and worker), because pattern matching is useful for the
message-passing which is used extensively in the controller. The worker
is mostly Java code, because that's closest to the application code;
using Java there makes it easier to know what the code does (less syntax
sugar) and also easier for application programmers to read the source
while debugging their applications.
> I have cloned the git repo but browsing through the 225 files is a quite
> a large task. I've read the email list introduction and Esko's blog
> article about Dimdwarf's new architecture.
>
> All pointers to the source code and details are warmly welcome
It might be best to start with reading the documents "RedDwarf
ServerAppTutorial.odt" and "RedDwarf ClientTutorial.odt" at the bottom
of page https://sourceforge.net/apps/trac/reddwarf/wiki/Documentation to
get an understanding of the application programming model. Also my
presentation about transparent references might help in understanding
how the system works under the hood:
http://www.orfjackal.net/temp/GDC%202009%20-%20Transparent%20References%20(Esko%20Luontola).pdf
(I originally implemented transparent references for Darkstar and then
copied them to Dimdwarf)
Dimdwarf has the same application programming model as
Darkstar/RedDwarf, although there are some differences in terminology
and features:
- RedDwarf's ManagedObject interface == Dimdwarf's Entity annotation
(and the internal EntityObject interface)
- RedDwarf requires the application programmer to use ManagedReferences.
Dimdwarf has also the equivalent concept (EntityReference), but hides it
by default using transparent references (the entities are wrapped inside
a proxy, which will load the actual entity from database when a method
on the entity is called).
- RedDwarf requires the application programmer to manually delete
managed objects. Dimdwarf will rely on a garbage collector for that (not
yet implemented).
To begin reading Dimdwarf's code, maybe have a look at the contents of
the packages in this order. They are in module dimdwarf-core unless
mentioned otherwise:
- application API
net.orfjackal.dimdwarf.api (module dimdwarf-api)
net.orfjackal.dimdwarf.api.internal (module dimdwarf-api-internal)
net.orfjackal.dimdwarf.aop (module dimdwarf-aop; needed because of
transparent references)
- tasks/transactions
net.orfjackal.dimdwarf.context (implements scopes for Google Guice)
net.orfjackal.dimdwarf.tasks (except maybe the classes which have FIXME
comments that they will be removed in the new architecture)
net.orfjackal.dimdwarf.tx (just to get the basic idea - the transactions
will be simplified in the new architecture)
- entities and transparent references
net.orfjackal.dimdwarf.entities
net.orfjackal.dimdwarf.serial
net.orfjackal.dimdwarf.entities.tref
These packages will be almost completely rewritten in the new
architecture, so it might be enough to have just a quick look at them:
net.orfjackal.dimdwarf.db
net.orfjackal.dimdwarf.scheduler
(There used to be also a garbage collector implementation, but I already
removed that, because it needs to be rewritten. The Git repository has a
tag old-gc-impl right before it was removed, so I can have a look at it
later as a reminder.)
I'm right now working on the code in the following packages. It might
take a couple of weeks for the to settle down:
net.orfjackal.dimdwarf.server
net.orfjackal.dimdwarf.controller
net.orfjackal.dimdwarf.services
net.orfjackal.dimdwarf.auth
net.orfjackal.dimdwarf.net
> and maybe
> some explanation about test strategies. I have almost read the GOOS
> books so the principles are familiar :-)
All code is written with TDD. Using the terminology in
http://blog.orfjackal.net/2010/02/three-styles-of-naming-tests.html the
tests are written mostly in specification-style, except for end-to-end
tests and some others which are better with example-style.
You can use the script at /src/main/script/verified_push.rb to make sure
that all tests pass before pushing your changes to the Git repository.
To run the end-to-end tests, use the command "mvn verify" (or the
existing run configurations in IDEA), because the end-to-end tests
require that Maven has built the distribution package (module
/dist/dimdwarf-dist) and it has been unpackged into
/end-to-end-tests/target/test-deploy (this is done in Maven's
pre-integration-test phase).
I used to use JDave as the testing framework for Java code, but now I'm
using Specsy (http://github.com/orfjackal/specsy) for testing Scala code
and JUnit for the end-to-end tests. Specsy 1.0.0 should be soon in Maven
Central Repository, but before that you can build it yourself (see
http://groups.google.com/group/specsy/browse_thread/thread/9a4439f02d8bed04).
Tests written in Scala require a workaround so that they can be run with
IDEA: http://youtrack.jetbrains.net/issue/IDEA-57696 I'm right now using
the convention to create a ScalaSpecs.java file for each package which
contains Scala tests.
I haven't yet decided whether I will use in Specsy tests the assertions
from JUnit, Specs or ScalaTest (Specsy doesn't in itself have an
assertion syntax). The assertions from Specs conflict with Mockito's
verify() method, so probably I will not use them in the long run.
JUnit's assertion methods, on the other hand, have some problems with
Scala's generics, so maybe I'll write a Scala wrapper for JUnit's
assertions. ScalaTest I haven't yet evaluated properly.
I have now started following the approach in the GOOS book to start with
end-to-end tests. The current work-in-progress is
net.orfjackal.dimdwarf.test.ClientConnectionTest When that is done, the
architecture for running transactions and communicating with clients
should be ready. After that I'll implement the (in-memory) database, and
then the main components of the new architecture should be in place.
Then there will be many directions which to develop next (for example:
communication over multicast channels, dependency injection support for
applications, on-disk persistence, failover, load balancing, data
migration, server cluster management), so then it will be easier to have
multiple developers working on different parts of the system.
> Also, if there are any rules about coding style (indentation etc.) it
> would be nice to know or preferred editors or development environments.
I'm using IntelliJ IDEA 9 and its Scala plugin. The open source
Community Edition of IDEA should work just fine. It might complain about
missing the Ruby plugin (which is only for IDEA Ultimate Edition), but
Dimdwarf has just a couple of helper scripts written in Ruby so it's
nothing serious (if possible, try to avoid committing the project files
if they change because of missing that plugin).
For indentation etc., use IDEA's per-project code style settings (do a
"git pull" because I just now configured the per-project settings). They
are almost the default settings, except that I always write curly braces
for if/while/for statements even when they could be omitted. Always do a
code reformat (Ctrl+Alt+L) before committing.
For Scala there is a style guide at
http://davetron5000.github.com/scala-style/ which is a good starting
point. For example
http://davetron5000.github.com/scala-style/method_invocation/arity0/index.html
and
http://davetron5000.github.com/scala-style/types/inference/void_methods.html
are good to remember always. On the other hand, with
http://davetron5000.github.com/scala-style/naming_conventions/special_note_on_brevity.html
and
http://davetron5000.github.com/scala-style/naming_conventions/type_parameters/index.html
I slightly disagree, because I prefer readability.
> So, what should I do next:-)
It might be best to meet some day, for example in some cafe in Helsinki
city center, and discuss more details face-to-face. That way it's easier
to answer questions and explain my vision about the system. I'll send
you a private mail so we can make an appointment.
--
Esko Luontola
www.orfjackal.net