[coldsync-hackers] Titrax sync questions...
Izzy Blacklock
to have to build a sync function. What has me worried, is how this will work
with multiple files. I guess the root of my confusion is that I don't really
know how the sync process works. I have to admit, I haven't looked at any
sync conduits yet, so perhaps I'm worrying over nothing.
My biggest concern is on the timing and order of things. Does a sync occur
after each file is read from the palm? ie, Does the palm expect to receive
the sync'd version of a pdb right after it sends the pdb, or does it send all
the pdbs then wait for attempts to write data back? Does coldsync run the
various conduits as it reads the pdb from the palm, or does it wait until it
has them all then run the conduits then write them back to the palm?
I fear that the sync process is being run after each file is received. If
that is the case, building a sync process for titrax would be problematic. I
would need to modify at 4 pdbs; TitrixNamesDB, TitraxDataDB, TitraxNoteDB,
and datebookDB. If I trigger on the Titrix pdb, how do I insure that the
latest version of the rest of the files has already been fetched? How do I
get the conduit to write out the other pdbs after the TitrixNames conduit has
finished?
I hope I'm worrying over nothing, but if I'm not, are there solutions to these
problems?
...Izzy
--
This message was sent through the coldsync-hackers mailing list. To remove
yourself from this mailing list, send a message to majo...@thedotin.net
with the words "unsubscribe coldsync-hackers" in the message body. For more
information on Coldsync, send mail to coldsync-ha...@thedotin.net.
IwantToKeepAnon
> My biggest concern is on the timing and order of
> things. Does a sync occur
> after each file is read from the palm? ie, Does the
> palm expect to receive
> the sync'd version of a pdb right after it sends the
> pdb, or does it send all
> the pdbs then wait for attempts to write data back?
> Does coldsync run the
> various conduits as it reads the pdb from the palm,
> or does it wait until it
> has them all then run the conduits then write them
> back to the palm?
Actually, I don't think it _really_ matters if
it downloads them 1 at a time or not ... or even
the order for that matter. To be *most* flexible,
and thus more likely to succeed, you need to be
able to handle the files in any order.
What I'd suggest is this, have a conduit that runs
for each of the files 1 at a time and retain just
which of your files have loaded; then when you get
to the last one (the 4th one?) ... do your sync being
assured all data is downloaded. Then clear your
downloaded file list to reset for the next sync.
Since the conduit is going to live and die for each
file, you cannot keep this list in memory. Perhaps
you could create a "semaphore" file in /tmp/ named
just like the palm pdb. Once you perform the sync,
delete all the semaphore files. There maybe other
palm pdbs that coldsync downloads but none will be
named like your 4 so you will be set for the next
coldsync.
BTW, I think you were wondering how to add the
database filenames and paths to the config files
(right?). You could simply store the path and
filename in your semaphore file.
Did I explain that clearly?
Have fun ...
--
Rodney S.
=====
--
"A smile is the shortest distance between two people."
Victor Borge
__________________________________
Do you Yahoo!?
Yahoo! Calendar - Free online calendar with sync to Outlook(TM).
http://calendar.yahoo.com
Izzy Blacklock
This makes sense, but still leaves one unanswered question. If I build a
conduit that simply creates "semaphore" files for each file as it comes down,
until the last one has been read, how to I push the changes back for the
other files once I've synced them with the local data?
I took some time to read up on the sync process. I wasn't able to find any
examples of an actual sync. They all seem to be fetch/dump conduits. I'm
guessing what I need is to use the SPC protocol somehow. There doesn't seem
to be a lot of detail on how it works. Is there an SPC opcode to send a pdb
back? I'm guessing I'd need to use the DLPC opcode in some way. Are there
any examples of this in action? Is there more documentation someplace? I
found all the Dlp* man pages, but they all seem to C library calls. Is there
a way to use them from Perl?
>
> BTW, I think you were wondering how to add the
> database filenames and paths to the config files
> (right?). You could simply store the path and
> filename in your semaphore file.
>
> Did I explain that clearly?
That would be another way to do it. The way I've done it for my dump conduit
makes sense, but a sync conduit using SPC, may not need any of this at all.
From what I've read, I can simply read and write the files I need from where
ever I want them. I probably wouldn't even need to use the semaphore files.
But thanks for the idea anyway.
Building a sync conduit sounds like a lot of work. It isn't likely to get
done any time soon. At least not a full one. I'm thinking I may play with
the idea of building an archive sync conduit for the databook which would
simply remove records older then x days and put them into an archive pdb file
(if something like this already exists, I'd love to hear about it! ;). This
is where the majority of data will end up anyway (at least for my use) and
would be the most difficult to manually purge old records for. The Titrax
records are easy to purge.
Eventually, I'm thinking I'll look at the Titrax code itself and remove the
need for three database files to store all it's data. I think it should be
possible to do everything with a single database, which would be a much
cleaner approach for both the app and the conduit. Another project for
another day....
...Izzy
Andrew Arensburger
> I've been thinking about a future need for my conduit. Eventually I'm going
> to have to build a sync function. What has me worried, is how this will work
> with multiple files. I guess the root of my confusion is that I don't really
> know how the sync process works. I have to admit, I haven't looked at any
> sync conduits yet, so perhaps I'm worrying over nothing.
>
> My biggest concern is on the timing and order of things. Does a sync occur
> after each file is read from the palm? ie, Does the palm expect to receive
> the sync'd version of a pdb right after it sends the pdb, or does it send all
> the pdbs then wait for attempts to write data back? Does coldsync run the
> various conduits as it reads the pdb from the palm, or does it wait until it
> has them all then run the conduits then write them back to the palm?
That's not quite how conduits work, or at least not sync conduits.
Fetch and Dump conduits get complete PDB files to work with. Sync
conduits don't. The purpose of a Sync conduit is to talk to the Palm and
ensure that the database on the Palm is the same as the one on disk.
For instance, the (default) generic conduit, which is a sync
conduit, downloads each modified record from the Palm in turn, compares it
to what's on disk, and then decides what to do: if the on-disk record
hasn't changed, it replaces the on-disk record with the on-Palm record; if
both have changed, and aren't identical (i.e., they've been modified in
different ways), it creates new records so that both the on-Palm and
on-disk databases have both versions of the record.
Yes, this can get complex, but that's what a Sync conduit is
expected to do.
> I fear that the sync process is being run after each file is received.
Not quite: part of the job of a Sync conduit is to read the
on-disk file and the on-Palm database.
> If
> that is the case, building a sync process for titrax would be problematic. I
> would need to modify at 4 pdbs; TitrixNamesDB, TitraxDataDB, TitraxNoteDB,
> and datebookDB. If I trigger on the Titrix pdb, how do I insure that the
> latest version of the rest of the files has already been fetched? How do I
> get the conduit to write out the other pdbs after the TitrixNames conduit has
> finished?
The only thing that leaps to mind is to have the conduit handle
all four databases. As each one runs, it creates or checks a temporary
file, and writes to it the current time and the database that it's
currently syncing. If all four databases aren't listed in the file, it
proceeds no further.
Either that, or the first time it runs, it syncs all four
databases and writes a timestamp somewhere. The subsequent three times, if
the timestamp exists and is more recent than a few minutes, it doesn't
bother.
You could probably play other tricks along the same lines. For
instance, if you sync all four databases the first time, then the next
three times, neither the on-Palm nor on-disk databases will have any dirty
records. You could use that as a flag.
--
Andrew Arensburger Actually, these _do_ represent the
are...@ooblick.com opinions of ooblick.com!
Generic Tagline V 6.01
Izzy Blacklock
questions.
...Izzy