I have one. If there are enough people who'd like it I will try to
polish up a little nicer and release it later this week.
TW
What's the difference with the build-in Filer ?
JY
If you mean on the SD card, then yes. That would be pretty cool. I'd
be happy to help with testing.
-Jonathan
SD?
I'm waiting for the polished product holding my breath...
<mmmppphhh>
Hello Tim.
A filer that does the above on the SD card too? I'd find such features
very useful, but it'd be great if you would make stand-alone SD
commands available in the lib also - commands such as the existing
CRDIR and MEM, just for the SD card, so I can use the features from my
own programs too ;-).
Regards
Steen
Probably creation/deletion of dirs on the SD card, move/copy of files
larger than available free MEM, move/copy files to/from subdirs on the
SD-card etc. None of this is possible with the built-in filer.
Regards
Steen
Oops. I guess this is another example of why you shouldn't post things
at 12:10 local time. Yes it is for the SD card exclusively. It is
based on a backend of some excellent work by Claudio. I just threw
together an interface that could use a lot of work. It is an excellent
example however of why sysRPL is still relevant even though HPGCC is
incredible. The entire interface is <1K I think.
I used the 48 browser for simpilcity, but that does mean there are
those limitations. I haven't yet enabled multiple selection, and don't
know if I have time since I am busy on other things. I'd like to
release this because it will be useful, and hopefully others will have
interest in making it nicer/faster/smaller etc. =)
Features:
-full long filename name support
-create/delete directories
-move, copy, rename, delete files
-sorting by date, size, name
Here are some PNGs with a few screenshots. The first shows folder
deletion. The second shows renaming, copying, and moving.
http://www.timandkatie.com/filer.png
http://www.timandkatie.com/filer2.png
TW
Just release it directly right now with full source
"Mr. Smith" will surely make it better
Thank you, Tim!
Please, please, please
That would be verrrrry nice
Thnks
Rui
So, what are you waiting for ? Here it is a bunch of entusiastic fans
waiting for it !
I agree, and has to say I was expecting something like this, to
properly manage the bunch of files I do have in my SD.
Thanks in advanced Tim, because I don´t have the "art" to program
something like that.
Daniel
Will it defrag the SD card, ie move files to contiguous blocks after having
deleted several files? I like my "more permanent" files at the top of the
list and after deleting files, new files show up where the deleted ones
were. I could copy the card to a computer, format it and copy the files
back, but that's a pain.
The directory creation in and of itself would be very useful to me though.
Scott
> Will [a new Filer] defrag the SD card?
Isn't an SD card entirely virtual anyway?
"Contiguous" virtual blocks may not really be so physically,
but what does it matter, since it is electronic and not mechanical?
(sure, there is an extra directory entry per extent to read or write,
but that's pretty low overhead).
One thing an SD card has but the standard Filer doesn't recognize
is time-stamping; this could enable sorting by date and time,
which has no analog in original user or other port memory
(the calc does time-stamp newly stored SD files).
Long file name support is currently present for storing a file to SD,
but not for displaying file names or copying files (copying a "long"
variable name to an SD card attaches the "long" name to the new card file,
as seen by Windows, but displaying in Filer or copying from SD
back to user memory uses only the "8.3" form of the file name).
[r->] [OFF]
Well it is mainly that I am extreemly busy, and so is Claudio who has
done the backend with his excellent filesystem in HPGCC.
I would like to relese this and then hopefully some people with
interest can make it nicer. It is also a good example of how to work
between HPGCC and sysRPL including things like calling different
functions, returning error codes in HP land and returning true/false
values to the stack. I would like to document it well so others can
learn these techniques.
I currently am using the 48 browser, which you find in the flags screen
in MODE. This has a few limits however. For example, there is no way
to use key definitions, so using the arrow keys for navigation doesn't
work.
The 49 browser, like most of the choose boxes in your calculator, don't
display full screen, so the size of the items are limited. Since it
has full filename support, limiting it in this way seems kind of silly.
So the alternative is to write my own complete choose box application
using a Parameterized Outer Loop or something, but that takes more work
and would be slower than the built in stuff as I don't do much ASM (ok,
not really at ALL ;-) and would take more time. . .
It will come out, if in beta only, sometime soon I hope.
TW
I don't think defrag is the term you want, as that means phsically
moving it to a new location. Flash memory access time is constant
everywhere AFAIK, so this has no purpose.
The files are sorted in the order they appear in a directory, so the
oldest files modified least will appear at the top of the list while
the most frequently modified will appear at the bottom.
TW
I think it is the term I want, because physically moving them would solve
the problem. Presently the listing seems to be in the order that the files
are in by cluster. Let's say all files and clusters are the same size for
ease of explanation and you have files ABCDEF in that order in your listing.
When you erase C and store G you have ABGDEF, becuase G takes the first
available cluster....right?
I would want ABDEFG instead, so that periodic backups don't get scrambled
with the important files that I access frequently. Although, if your program
can create directories, I could store backups in their own directory. That
would alleviate the nuisance.
Scott Chapin
> I could store backups in their own directory.
> That would alleviate the nuisance.
I have consecutively numbered my own backups,
also prefixing them with the same constant character,
which groups them all together when alpha sorted.
Using characters at end (or beginning) of the sort sequence
might produce the arrangement you desire.
It seems useful for me, however,
to file backups away in a subdirectory,
because the time taken by the current Filer to list SD files
seems somewhat proportional to the number of root entries
(or entries in the directory being currently listed).
There is already a format of the level-1 argument for
STO and RCL which can directly specify paths in SD subdirectories,
and the Filer seems already to be able to browse SD subdirectories
and to recall any object found therein -- all this together
means that it's prety easy to backup into port 0, 1 or 2, say,
and from there copy via RCL...STO to an SD subdirectory
(the directory path could be built into a little permanent program,
lest the manual composition of the folder path name cause concern);
it's also very easy to recall later or directly copy back via Filer
from any SD subdirectory to any port (even to "3" for root of SD).
Does this offer any further promise for SD organizing ability,
even with what we already have in our hands?
The one additional Filer function which I have always missed
is an extra menu function to recall not the contents of the object,
but rather its PATH (ending with the simple name), which
you could then use with subsequent commands of your own.
When you recall objects from *menus* for either user memory or ports,
even via the right-shift key before the menu key,
then LASTARG always recovers its [port-tagged] name as well;
this is very often just what I need,
but can't get in any similar fashion from the Filer.
[r->] [OFF]
:3:"folder/file" STO
(similarly with RCL and PURGE, altho the standard filer
can also navigate to folders and do the latter things directly).
A small simple program can create the full path and do a STO,
given just the simple object name (or a port-tagged name).
[r->] [OFF]
John H Meyers wrote:
<...>
> Long file name support is currently present for storing a file to SD,
> but not for displaying file names or copying files (copying a "long"
> variable name to an SD card attaches the "long" name to the new card file,
> as seen by Windows, but displaying in Filer or copying from SD
> back to user memory uses only the "8.3" form of the file name).
HPGCC new filesystem (in which Tim's filer will be based) goes one step
further. It supports both long filenames AND case sensitivity, so you
can store (and RCL) all your variables with their original calculator
names. That's a big advantage vs. the original filer: the ability to
copy a group of variables to a SD directory and then RCL them all at
once with their original names.
Claudio
> HPGCC new filesystem (in which Tim's filer will be based) goes one step
> further. It supports both long filenames AND case sensitivity, so you
> can store (and RCL) all your variables with their original calculator
> names. That's a big advantage vs. the original filer: the ability to
> copy a group of variables to a SD directory and then RCL them all at
> once with their original names.
An "HP Dir" object (or list)
containing a bunch of variables (and names)
will also preserve their exact original names
(though currently the Filer won't "browse into" it on SD,
as it was happy to do in ports 0-2).
I think you mentioned some renaming convention
for allowing names like "AAA" and "aaa"
to coexist in the same folder on an SD card,
but still the card names can't always exactly match, right?
(also because of illegal "external OS" characters etc.),
whereas HP Dir objects can encapsulate any set of HP variables.
That's the "musical chairs" nature of almost everything --
"gain this, lose that... there's more than one way to skin a cat" :)
One can see that HPGCC is a fine project and product;
too bad I can't take every road yet be one traveler :(
http://www.bartleby.com/119/1.html
http://www.bartleby.com/104/67.html
Currently implemented is move, rename, copy, delete, create dir, delete
dir. Sort by name, date, size. Ability to store without am HP header
is a command.
TW
> So what functions do people want included?
I've nominated (even for the built-in filer)
a function that would return only the *location*
of an object (i.e. that which you could put on level 1
and then do RCL to perform the already-provided RCL function
of the Filer itself, so perhaps it might be implemented
just by omitting the final RCL from the RCL function :)
I.E.:
For variables:
{ HOME dir1 ... dirN name }
this can be produced by appending the variable name
to the PATH [PATHDIR], and is a valid argument for RCL.
For port [0-2] objects:
:N: name [general objects]
:N: { dir1 dir2 ... dirN name } [objects within stored HP DIRs]
The latter case occurs only when a stored HP DIR
is "entered" via the Filer, and is a valid argument for RCL.
For SD card objects:
:3: name [root folder files]
:3: { dir1 dir2 ... dirN name } [subfolder files]
These all seem to work for RCL or PURGE,
and in fact is what the Filer itself displays
when it asks to confirm a PURGE request,
so the Filer always has at hand anyway
the value which I am asking it to explicitly return.
By the way, :3:"XX/YY/ZZ" STO creates file system folders XX and YY
if necessary, and then stores file ZZ in folder "XX/YY";
the same argument can also be used for RCL or PURGE,
as can the alternate list form :3: { XX YY ZZ }
but only the "string" form can be used for STO.
I have found no way to delete a file system folder
(you can delete all the objects in a folder, however,
leaving the folder empty).
[r->] [OFF]
John H Meyers wrote:
> An "HP Dir" object (or list)
> containing a bunch of variables (and names)
> will also preserve their exact original names
> (though currently the Filer won't "browse into" it on SD,
> as it was happy to do in ports 0-2).
The idea is to have the variables all separated in the SD card, so you
can recall one, a subset or all of them at your will. It happened to me
many times that I want one particular routine that was inside a
directory and I have to store the directory in the calc first, then get
what I want.
>
> I think you mentioned some renaming convention
> for allowing names like "AAA" and "aaa"
> to coexist in the same folder on an SD card,
> but still the card names can't always exactly match, right?
Right but not quite.
You will only see the difference from a PC. A filer running in the calc
will only see the names as you wanted them. The trick is completely
transparent to the user.
> (also because of illegal "external OS" characters etc.),
> whereas HP Dir objects can encapsulate any set of HP variables.
There's only one character that I know is allowed on the calc but not
on the SD card, the '?' char. Al other chars can be stored and rcl'd
with no problems back and forth (you obviously can't use chars that are
illegal in the calculator, but that's not a problem if you stored
actual variables to the SD card).
Claudio
Scott Chapin wrote:
<...>
> I think it is the term I want, because physically moving them would solve
> the problem. Presently the listing seems to be in the order that the files
> are in by cluster. Let's say all files and clusters are the same size for
> ease of explanation and you have files ABCDEF in that order in your listing.
> When you erase C and store G you have ABGDEF, becuase G takes the first
> available cluster....right?
>
> I would want ABDEFG instead, so that periodic backups don't get scrambled
> with the important files that I access frequently. Although, if your program
> can create directories, I could store backups in their own directory. That
> would alleviate the nuisance.
I think you are confusing the physical storage of files and the
directory listing. The fact that you see the new name between old names
doesn't mean the file was stored in the middle of your disk, just that
the directory entry was reused for the new file. However, this is not
the behavior in Windows or hpgcc file system. Deleted entries are
preserved and new entries are added at the end of the directory like in
your example.
Claudio
> The idea is to have the variables all separated in the SD card,
> so you can recall one
Of course, and it will be more convenient to have that,
as well as more flexibility in managing and navigating thru the SD,
preserving exact names on copied objects, etc.
But anyone with a calc "right out of the box"
might still like to know that an HP DIR (with original object names)
can first be copied either to user memory or to any port (0-2)
and then "looked into" by existing Filer,
then any individual object(s)
can be recalled or copied from there;
the cost is one extra copying step up front.
> You will only see the difference [in file names] from a PC.
> A filer running in the calc will only see the names
> as you wanted them.
> The trick is completely transparent to the user.
Neat!
[r->] [OFF]
Perhaps a back-up / restore
that saves ENTIRE calculator contents to/from the SD
that includes flags and internal Flash from port 2
perhaps port 1, too!
One could select via checks what to include
used BOTH in backup AND restore commands
[V] MAIN RAM [ ] Port 0
[V] Flags [ ] Port 1
[V] Port 2
Use Date as Name [V]
Add Time to Name [V]
Name:
[20060615.140216 ]
> It is also a good example of how to work
> between HPGCC and sysRPL including things like calling different
> functions, returning error codes in HP land and returning true/false
> values to the stack.
So... you mean... you can MIX your programing in SYSRPL and C+ ?
You CAN manage your data files in RPN and then analize them in a C
program ?
I should like to learn how to do this !
> I would like to document it well so others can learn these techniques.
So .... please take your time ... I am VERY interest on your program,
but I am MUCH MORE INTERESTED on learning how to do it ! Best of
two worlds !
> Well it is mainly that I am extreemly busy, and so is Claudio who has
> done the backend with his excellent filesystem in HPGCC.
> It will come out, if in beta only, sometime soon I hope.
>
> TW
Thanks in advance for all your efforts. Just remember that here we are
a lot of enthusiastic fans, NOT computing profesionals, so we need
"easy" clean and clear examples to properly learn how to program.
Daniel
> So what functions do people want included?
> Ability to store without am HP header is a command.
>
> TW
Can you do that ? and run it within RPN programing variables /
directories?
Wonderful !
I need it to properly communicate my HP49g+ with the Magellan GPS.
Now I can read the GPS data and process it in the HP, but after
analysis, sometimes I wish to send data back to the GPS, but can´t in
a strait line because of the header.
So, I made a routine in C+ that help cleaning the header but its a
pain in the ..... and don´t know how to mix my RPN analysis programs
with C+ file manager.
So... you are getting all my atention !
Daniel
Yes, RCL and STO do appear to be the needed items for storing a backup to a
subdirectory. However, they also seem to maliciously change the object type
(as reported by OT49's DType) from type 17: Backup (Dispatch #9Fh) to type
15: Directory (Dispatch #2Fh).
This causes hidden issues. I believe the biggest issue is that the hidden
directory goes wacky, becoming a directory with a " for its name, and
causing memory errors/reboots if any attempt is made to mess with it. This
means you lose (at the very least) your alarms/user keys.
Here is the code I used to call from an alarm every morning at 2am until I
figured out this hidden trap:
WARNING to googlers: don't use this to actually make backups...it's broken,
as I explained above.
Second note: I'm having trouble adding the \nnn codes, so I added them by
hand. I probably made mistakes, so entering by hand is highly recommended
if you want to reproduce my issue. It is highly probable I missed or
misplaced a \-> or two.
\<< PUSH -76 SF
IFERR HOME RCLF 'FLAGS' STO ""
DATE 2 TRNC 100 * \->Q + TIME 0
TRNC \->Q + ".BAK" + DUP ":1:B"
SWAP + OBJ DUPDUP ARCHIVE RCL
UNROT PURGE ":3:" "\"BKUP/" ROT
"\"" + + + OBJ\-> STO
THEN "CRBKUP ERROR"
END 'FLAGS' PURGE POP \>>
It was only by careful combining of an older, proper type 17 backup with
this strange HPDIR that I was able to avoid data loss.
So be careful when using RCL and STO on backup objs!
> [r->] [OFF]
Greg M.
> So... you mean... you can MIX your programing in SYSRPL and C+ ?
> You CAN manage your data files in RPN and then analize them in a C
> program ?
>
> I should like to learn how to do this !
That's what we do most of the time with C programs - and it's quite
easy to do so too, thanks to the power of HP-GCC.
Regards
Steen
<SNIP>
>
> I think you are confusing the physical storage of files and the
> directory listing. The fact that you see the new name between old names
> doesn't mean the file was stored in the middle of your disk, just that
> the directory entry was reused for the new file. However, this is not
> the behavior in Windows or hpgcc file system. Deleted entries are
> preserved and new entries are added at the end of the directory like in
> your example.
>
Thanks Claudio,
I wouldn't know, so I trust you here. I assumed that the newly stored object
occupied the same pysical location of the one deleted, and that was why they
appear in the same position of the directory listing.
One reason I was thinking this way is an observation I made. When several
contiguously listed small objects were deleted and a much larger one stored,
(larger than the total size of the deleted files) the much larger one
appeared at the end of the directory listing, instead of replacing the
smaller deleted objects which were in the middle of the listings. Maybe that
was just a fluke.
In any event, it would be nice to have new files added at the end of the
directory listing, so older more important objects "float" to the top.
Scott Chapin
> In any event, it would be nice to have new files
> added at the end of the directory listing,
> so older more important objects "float" to the top.
Since SD items are time-stamped, even when stored by the calc
(if the date and clock are right :)
this might be possible by modification-time sorting.
Would you like to make a date/time stamp optionally visible
in this new filer (currently the "J" key toggles on/off
the file types and lengths, perhaps the "I" key
might do the same for time stamps?)
[r->] [OFF]
> Yes, RCL and STO do appear to be the needed items
> for storing a backup to a subdirectory [of the SD card]
> However, they also seem to maliciously change the object type
> (as reported by OT49's DType) from type 17: Backup (Dispatch #9Fh)
> to type 15: Directory (Dispatch #2Fh).
Only if you copy it from a *port* (0-2) to the SD card (3);
*not* if you copy it from the root of the SD card itself
to a subdirectory within the SD card (but it really
doesn't matter which way you do it, as we'll shortly see).
There is complete absence of malice (or problem)
with this fact, and it's simply the natural result
of the storage system used in port memory.
The *normal* result of :port:name ARCHIVE (ports 0-2 in 49 series)
is in fact a "Directory" object (try it and see), which is simply
a faithful "unrooted" copy of your entire HOME directory.
*Only* arguments such as :IO:name ARCHIVE or :3:name ARCHIVE
first enclose the original directory object within a "wrapper"
which inserts a length, a name ("HOMEDIR" for the I/O version),
and a checksum, which then becomes what is called a "Backup object."
ALL objects (except libraries) that are stored into ports
are also *automatically* enclosed within a Backup object,
basically to provide a single consistent structure
within the contiguous memory of a port,
giving every stored object a length and a name;
conversely, that "wrapper" is *automatically* removed
whenever objects are recalled from ports,
leaving us with exactly what was originally stored.
Since we ordinarily never catch sight of a real,
live "Backup object" outside of its usual lurking place
(buried inside a port), this well-hidden internal OS behavior
is generally completely transparent and invisible to us.
However, when a home directory backup is *externally* stored
(sent either to a computer or to an SD card), then it is
in the form of a "Directory" object within a "Backup" object
(perhaps some confusion arises because we mentally
blur the idea of a "Backup" of our HOME directory
with the HP "Backup" object type, but the latter
has its own specific meaning, unrelated to the former).
At any rate, if we at any time store such an external backup
into a port, since it is *already* a "Backup" object type anyway,
it just gets stored directly, without the usual automatic
extra "wrapping" that ports require to standardize
their internal filing system.
When we next *recall* that same object from its port,
however, its internal "wrapper" is automatically discarded,
just as for all other objects that have been invisibly
"wrapped" before storage -- and now it is seen to be
what it really has been all along -- a Directory!
When we use a Filer to inspect objects in ports (0-2),
the Filer is also well aware that port objects
are by default encased in "Backup" objects,
so it always likewise "uncloaks" them,
and tells us what they really are inside;
that's why any HOME backup made by ARCHIVE,
when stored in a port (0-2),
is always also a "Directory" object!
> This causes hidden issues.
One issue that you didn't mention is
how to restore your HOME directory from a backup.
A computer or SD card backup that is recalled to the stack
(displaying "Backup xxxxx") can be used directly
from the stack, as an argument to RESTORE.
If your HOME backup is currently in a port (0-2),
then :port:name RESTORE will restore it (after
completely purging the current contents of HOME,
of course, so that you might as well have done
ON+A+F, "No" to wipe it out first and save the time :)
What isn't always realized is that this operation
can restore *any* saved directory at all to HOME
from a port (0-2), so be sure you know what you are doing!
By the way, in such cases, an empty "hidden directory"
is automatically created, in which you will have
no alarms and no "user key" definitions.
If your HOME backup is currently on the SD card
*and* in the form of a "Backup" object
(which it will be if you originally created it
directly into the SD card via :3:name ARCHIVE,
even if you later copied that to an SD subdirectory),
then you can do :3:name RESTORE
However, if you copied the backup to the SD card from a port
(in which case it is now stored as a Directory object),
then you will have to first copy it from the SD card
to a port and then do :port:name RESTORE
(port 0 will do fine for this purpose,
so you can always do ON+A+F, "No" to clear out
the HOME directory you are about to replace,
then there is sure to be enough RAM available).
> I believe the biggest issue is that the hidden directory
> goes wacky, becoming a directory with a '' for its name
That is the actual (correct) name of the "hidden directory,"
which is always present as the last object in HOME --
but the OS declines to display that (empty) name in the VAR menu,
and the Filer (ever since being fixed) also doesn't display it
when actually viewing the current HOME (though you *can*
display and "visit" it in backup copies of HOME),
thus keeping the current HD well hidden
and protected from being "messed with" :)
> and causing memory errors/reboots
> if any attempt is made to mess with it.
Did you attempt to *copy* things from a backed-up HOME
directory into the *current* hidden directory?
(or to use an unfixed Filer to mess with it?)
That's not the way to restore things, and is likely,
as you noted, to warmstart and/or corrupt or lose that data,
because the OS uses its own "direct access" into that directory,
and can get very upset if anything tampers with it.
> This means you lose (at the very least) your alarms/user keys.
When you RESTORE from a backup (or directory object)
which contains the hidden (null-named) directory at its end,
the saved alarms and user key definitions are properly restored.
If you want to be able to restore *only* your alarms and/or
user key definitions, without having to restore the entire
HOME directory at the same time, then you should back up
those items by other means -- RCLKEYS/STOKEYS for the user keys,
and short programs for similarly recalling and restoring
the alarm list, such as:
\<< RCLF -55. CF IFERR 1. MAXR FOR i i RCLALARM NEXT
THEN 1. - \->LIST SWAP STOF END \>> 'ALRCL' STO
\<< IF DUP SIZE THEN REVLIST \<< STOALARM \>>
DOLIST END DROP \>> 'ALSTO' STO
Deja vu all over again:
http://groups.google.com/group/comp.sys.hp48/browse_thread/thread/edf35a08d4f80511
May all your problems fall into a hidden directory,
and only good get published :)
With best wishes from:
http://www.mum.edu
http://www.maharishischooliowa.org
> If your HOME backup is currently on the SD card
> *and* in the form of a "Backup" object
> (which it will be if you originally created it
> directly into the SD card via :3:name ARCHIVE,
> even if you later copied that to an SD subdirectory),
> then you can do :3:name RESTORE
You could also just recall it directly to the stack
(which even the present built-in Filer can do
from an SD subdirectory, using the filer's RCL menu key)
and then perform RESTORE (from the stack object).
If you recall it to the stack from the SD card
and see a directory (rather than "Backup xxxxx")
then you would first need to store it into a port,
for example :0:xxx SWAP OVER STO RESTORE
[r->] [OFF]
> On Fri, 09 Jun 2006 14:11:38 -0500, Greg M. wrote:
>
>> Yes, RCL and STO do appear to be the needed items
>> for storing a backup to a subdirectory [of the SD card]
>
>> However, they also seem to maliciously change the object type
>> (as reported by OT49's DType) from type 17: Backup (Dispatch #9Fh)
>> to type 15: Directory (Dispatch #2Fh).
>
<SNIP>
> The *normal* result of :port:name ARCHIVE (ports 0-2 in 49 series)
> is in fact a "Directory" object (try it and see), which is simply
> a faithful "unrooted" copy of your entire HOME directory.
>
Aha, I wrote my program first, which PURGE'd the backup object from port 1
before it finished, so I never noticed this. Interesting!
<SNIP, SNIP> (Wonderful info, just trying to make sure you can find my
replies ;)
> At any rate, if we at any time store such an external backup
> into a port, since it is *already* a "Backup" object type anyway,
> it just gets stored directly, without the usual automatic
> extra "wrapping" that ports require to standardize
> their internal filing system.
>
What is wrong with a "Backup" object inside of a "Backup" object? Can they
not be nested like other composite types?
> A computer or SD card backup that is recalled to the stack
> (displaying "Backup xxxxx") can be used directly
> from the stack, as an argument to RESTORE.
>
> If your HOME backup is currently in a port (0-2),
> then :port:name RESTORE will restore it (after
> completely purging the current contents of HOME,
> of course, so that you might as well have done
> ON+A+F, "No" to wipe it out first and save the time :)
>
OK, I'll keep this in mind if this ever comes up again. Copy to port 0-2
first, then use RESTORE on it, correct?
>> and causing memory errors/reboots
>> if any attempt is made to mess with it.
>
> Did you attempt to *copy* things from a backed-up HOME
> directory into the *current* hidden directory?
> (or to use an unfixed Filer to mess with it?)
>
Yes, after RESTORE bailed out on me after I attempted to run it on my HPDIR
which I EVALed to get on the stack, I used the filer to copy the backup
object to my home directory. Which left it as a subdirectory of my home
directory. Using the filer, I attempted to copy all items inside this
folder (including the hidden directory) to the HOME directory. That is when
the warmstart happened.
What I did to work around this was to RESTORE a previous backup which I had
ARCHIVEd directly to :3: to grab the hidden directory, and then used the
filer to copy all but the hidden dir to HOME.
Thanks for all the help! (and my apologies for jumping the gun and pointing
to RCL & STO as the culprits, I was unaware of this strange method of
storing items in ports)
Greg
> my program PURGE'd the backup object from port 1
> before it finished, so I never noticed this.
:3:name ARCHIVE can be used directly, if you like,
which will create a real Backup object (containing the Directory),
or:
:3: name @ you supply any port (0-2) or card (3) initial location
@ then this program makes an archive and transfers to SD subdir
@ (purging the original after the transfer)
\<< DUP DUPDUP ARCHIVE RCL SWAP OBJ\-> DROP
"dirname/" SWAP + "3" \->TAG STO PURGE \>>
Of course, you end up with either a "Directory" or "Backup" type,
depending on whether initial arg is Port (0-2) or SD card (3).
Runs out of memory if HOME is 128K or larger.
> What is wrong with a "Backup" object inside of a "Backup" object?
> Can they not be nested like other composite types?
Sure, but <backup object> STO needs only a port number as argument
(if you supply :N:name then only N is used for ports 0-2),
using the name already embedded in the backup, and doesn't
create a second backup (which would also need its own new name);
however, <backup object> :3:newname *does* add a new "wrapper,"
so I guess you can do this if you want, on your ARM-based calc,
using your SD card!
The following [SysRPL] program embeds an object in a backup:
( with object and 'name' on stack )
:: CK2&Dispatch SIX
:: ID>$ SPACE$ &$ $>ID SWAP TRUE THREE {}N OB>BAKcode ; ;
This is highly simplified from #21674h in 48GX ROM "R"
which can be found identically in 49G+ ROM
at next PTR after DUPTYPEIDNT? in FPTR 2 11,
which is in turn called directly from xSTO
And to extract the original object from a backup:
:: CK1&Dispatch #9F :: BAK>OB ; ;
Now it's very interesting that the following thread
says that no one can find this BAK>OB function,
even though it has always existed since the HP48.
It also says "Backup object had to be created
in order to be stored in port (this is not needed any
more, as it is done on the fly on the HP49), but then,
the RCL was doing the recall of the content of the object
directly, so BAK>OB was not needed."
For further interpretation of this, ask Cyrille (he wrote it :)
http://groups.google.com/group/comp.sys.hp48/browse_frm/thread/634b07c1f265cd13
(DON'T do what the first original post suggests; it's NG)
> Copy [a DIR] to port 0-2 first, then use RESTORE on it, correct?
Yep, though in a preceding follow-up, I suggested just
recalling to stack and doing :0:xxx SWAP OVER STO RESTORE
(after ON+A+F, "No" if insufficient memory).
You can keep little "boot helper" programs like this in some port :)
> I used the filer to copy the backup object to my home directory.
> Which left it as a subdirectory of my home directory.
> Using the filer, I attempted to copy all items inside this folder
> (including the hidden directory) to the HOME directory.
As was suspected :)
You *can* use this approach to selectively restore all or part of
any backup -- *except* for user keys (not sure about alarms;
if you know how to store directly into HD, you could
try that and see whether it crashes or not :)
Or you can "visit" parts of the backup using the Filer,
and just recall or copy the parts you want.
Of course, if the backup you want to selectively restore
is actually a Backup object to begin with,
then you *can't* do this -- until you've "unwrapped"
the Directory object hiding within the backup
(which can be done, as you now know,
simply by storing the backup into any port 0-2,
or via the BAK>OB system function mentioned above).
"Port Outbound, Starboard Homebound" [a POSH stateroom?]
Naaah:
http://web.cn.edu/kwheeler/lit_terms_F.html
http://dictionary.reference.com/search?q=posh
New on the block:
http://www.google.com/finance?q=POSH
[r->] [OFF]
<Snip>
> Would you like to make a date/time stamp optionally visible
> in this new filer (currently the "J" key toggles on/off
> the file types and lengths, perhaps the "I" key
> might do the same for time stamps?)
That would work, especially if there were a default sort.
Scott Chapin
> So what functions do people want included?
>> Ability to store without an HP header as a command.
I hope this was meant to suggest literally having it
as a separate command, callable from all programs
(perhaps even built into a future ROM?),
rather than only as a menu option in a special add-on filer.
I'm thinking also along the lines of an even more generalized
command, to cover more than this one additional request:
How about an "Export" command, which could perhaps
use a level-1 option to control output file formats,
like the way that the level-1 option used by \->GROB
formats different image fonts and styles (text vs. EQW)
for different purposes, but in this case the options might be:
o Ascii mode:
Decompile and insert "%%HP" first line,
as when a file is externally transferred
via Kermit "in Ascii mode"
o "EDITB" mode:
Like the above for all except strings,
which would be stored verbatim -- this would
correspond with how EDITB dispenses with the
decompiler and compiler when editing strings.
o "STOB" ("STOre in Best format") mode:
Like "EDITB" mode, except reverting to the usual
"internal binary" object format whenever the object
can't be properly decompiled (i.e. has SysRPL etc.),
which means whenever the decompiler has to insert "External..."
If some such built-in "Export" command existed
(or was available in a separate library),
then any Filer could also use it,
or anyone at all could embed it in his/her own programs,
which is more powerful and general than functions only available
through a particular manual GUI interface (which all filers are).
If building it into ROM is out of the question,
then perhaps having it as a separate library
(or at least as a "visible" command in a new filer library)
might be equally helpful to other programmers.
[r->] [OFF]
[my post deleted]
I suppose the existence of an "Export" command
might also prompt a desire to have a corresponding
"Import" command -- a command which could deal with
all types of external files that might be found on an SD card.
Various programs have already been offered
just for the export/import of UserRPL computer files
like those made by "Ascii mode" file transfers,
as you may try to transfer on SD cards -- there's such a program
in the 49G+ "connection kit," although that program
gets transferred into the calc temporarily while connected,
and doesn't remain afterwards (it's also overly liberal
when it comes to executing any text at all as commands);
here's another such system (mostly UserRPL) which I've posted:
Without "%%HP" headers (not execution-protected, either):
http://groups.google.com/group/comp.sys.hp48/msg/78ccdda2a7e971a6
With "%%HP" headers (and as fully protected as Kermit's own import):
(scroll thru entire post, it starts halfway down)
http://groups.google.com/group/comp.sys.hp48/msg/46b8bc3f34d9dfe0?dmode=source
But wait -- if we're going to create new commands for Ascii export/import,
why not fill in a detail which never yet found its way into the "%%HP"
header line, because the original HP48 didn't have a special object type
for Integers, hence didn't have a choice of Exact vs. Approximate input modes.
All the built-in ROM implementations that I'm aware of,
both in HP48 and HP49/50 series, will completely ignore any characters
inserted *before* the standard "%%HP" mode-setting header;
therefore, if we insert something like "I(E)" or "I(A)"
(representing "Exact" vs. "Approximate" mode
for interpreting integers without decimal points) just before "%%HP"
then any real HP48/49/50 will ignore it, but *our* "Import" function
could use this to set the appropriate mode automatically
before interpreting the text.
Since no previously created "exported" files ever inserted this option,
it would basically serve as something we could insert now,
into previously exported files or future exports,
to automate the proper setting
and guarantee correct future import whenever it is present,
at least when we import using our own new "Import" command.
Any UserRPL files *created* on HP48 should have "I(A)" manually inserted,
to guarantee proper import to an HP49/50, whereas all files
exported from an HP49/50 should have "I(E)" automatically inserted
(regardless of the actual internal flag setting while exporting!),
to equally guarantee correct import back into any HP49/50,
without depending upon the user to inspect the files
and carefully prepare the correct mode before each individual import.
[r->] [OFF]
When he mentioned a separate command he meant a simple STO routine I
wrote for SD cards. This routine can store objects with or without the
HPHP49-... header, it can also store raw data if you provide a string,
hxs string, etc. The companion RCL routine will open any file with or
without HPHP header, and if it doesn't find a valid object, it returns
the file as a string (raw data).
>
> If building it into ROM is out of the question,
> then perhaps having it as a separate library
> (or at least as a "visible" command in a new filer library)
> might be equally helpful to other programmers.
Tim was talking about a filer, but other people like you want a library
for advanced SD card access. I guess some commands could be left
visible for programmers, or even put on a separate library.
Claudio
Hi Tim.
It'd be useful with a set of functions to handle files larger than
available free memory. I was thinking about R/W of large image files or
a simple pdf-viewer for the calc for instance. To do that, we need
something like this;
To read a file chunk by chunk:
------------------------------------------
1: Path
->
OPENFILE (doesn't really "open" it, but merely fetches some useful data)
->
2: File size in bytes
1: Start address
2: Start address
1: Bytes to read
->
READFILE
->
2: Next start address
1: Data
------------------------------------------
To write a new file, chunk by chunk:
------------------------------------------
1: Path
->
NEWFILE
->
3: File handle
2: Max file size in bytes
1: Start address
2: Start address
1: Data
->
WRITEFILE
->
1: Next start address
1: File handle
->
ENDFILE
->
------------------------------------------
Also, some type of PVARS and TYPE commands working on the SD would be
useful. I don't know if Claudios C-code will be relaesed as a lib, to
let me do the above myself with HP-GCC? Because it's quite more than
what a "simple" SD file browser ought to support...
Regards
Steen
The HP-41 style line-by-line read/write of simple text file
or comma-delimited files would be quite OK
Inserting several records "just" needs that CR (CR-LF) sequence
reading could have a parameter to read from, liker
5 READ @ read 5th record
{5 1} READ @ same as above
{5 2} READ @ read two records
result could be
"Nousiainen, Veli-Pekka
Schmidt, Steen"
WRITE could then do the opposite
2: "Nousiainen, Veli-Pekka
Schmidt, Steen"
1: 5
WRITE
Notice: there is no need for several record writing
eg. {5 2} WRITE is not needed
because of the CR (CR-LF) convention used
> a simple STO routine I wrote for SD cards can store objects
> with or without the HPHP49-... header
Does "store" mean the same as "ascii vs. binary" modes
of sending data over cable to a computer,
plus an additional "verbatim" mode for storing strings?
In binary mode, which means exactly
as the object appears in RAM when used inside the calculator,
which *always* includes some hard-coded ROM addresses,
there must *always* be an HPHP49-x header,
so that only the calc series on which the object was made
will later recognize and accept it (one can override this
safety feature by using a FIXOB or OBJFIX program).
Since the calc can always recall such objects properly,
directly from the SD card, there is no reason to omit
the HPHP49-x header from binary objects,
as the next thing following that 8-byte string
is *always* a 2.5-byte binary calc ROM address,
which never means anything to other computers anyway.
Ascii mode means the displaying of the object
as it would appear when edited in the command line
(and then with some 8-bit or other characters possibly expanded
for representation in pure ascii),
prefixed by a "%%HP" line to indicate how most modes
that affect proper interpretation are currently set
(character translation mode, angle mode, and decimal point mode,
but unfortunately nothing about Exact/Approximate mode).
The only other style not provided for during cable transfers
would be the transmission of character (or hex) strings
representing exactly what one would want an external file to contain;
this is what seems to be desired for GPS data or other external formats
that other computers or programs might need.
Now, since the SD card cannot realy be used as a calculator "port"
(to host libraries, for example), and since the card actually represents
a form of disk storage for other computers and devices,
the SD card should really always have been treated exactly the same way
that we handle external computers connected via a cable,
and we should have been able to transmit files in both directions
in at least the two usual ways (binary transfer and ascii transfer)
that we have always had available for those transfers.
However, what actually has been provided thus far, built into the calc
while using STO and RCL, is the exact equivalent of *binary* transfer alone,
producing the same results for STO as binary cable transfer
to an external computer, and the same results for RCL
as binary transfer from an external computer to the calc
(this includes the ability to *receive* any file as a literal string,
which is what happens by default when HPHP49-x is missing,
but there is no such equivalent ability for sending literal strings).
What I am thus proposing is the expansion of STO as you perhaps
have already offered, with a command permitting output
of any literal string (character or hex),
plus the currently missing ability to transfer in both directions
in the "ascii mode" available over cable.
I was at first suggesting to incorporate all possible modes
into a common command, but now I'm thinking that it might be better
if each mode had its own command, so perhaps a command set like this:
STO - existing (binary) STO
STOC - store literal character (or hex) string (valid only for these types).
STOA - store "in ascii mode," exactly like a cable transfer in ascii mode.
RCL - existing (binary) RCL (this already includes "RCLC" by default,
but you could implement an extra RCLC to deliberately prevent the
automatic attempt to convert strings beginning HPHP49-x to internal objects)
RCLA - recall "in ascii mode," exactly like a cable transfer
If file already starts with HPHP49-x and is a string,
then perhaps RCLA might interpret that string in the same way.
RCLA could optionally default to binary import for other object types,
or for strings which have compile errors, which would in effect
make RCLA a nearly "universal" import function,
or perhaps it might be best to let a short "user" program
embed a strategy of trying RCLA on a file, then proceeding
to try RCL if RCLA errors, allowing the user to decide
whether or not to "overload" the RCLA command.
> [Claudio's STO]
> can store objects with or without the HPHP49-... header
> and can also store raw data if you provide a string,
> hxs string, etc. The companion RCL routine will open any file
> with or without HPHP header, and if it doesn't find a valid object,
> it returns the file as a string (raw data).
I think it very dangerous to do a straight binary output,
simply removing the 8-byte HPHP49-x header, and even more dangerous
to do import binary objects without that "series identifying" indicator,
because this entirely removes the "safety interlock" wisely designed in
by the original HP design team, anticipating that binary objects
made using different ROMs can be incompatible, and could
cause crashes if transmitted between incompatible series,
as is very much the case now that the HP49 series
has a completely different set of even the "supported" ROM addresses.
Even though the initial ROM address
which is always present immediately following HPHP49-x
remains the same for those object types common to the HP48S/G series,
note that any program, any list, any directory, any unit object,
any object which didn't exist in the prior series, and even
some real numbers are likely to have incompatible ROM addresses,
and thus are dangerous to exchange between series
(possibly even including any future models like the HP60 series :)
Thus I would suggest re-considering, to never allow discarding of the
series-identifying HPHP49-x headers on all binary object files
(note again that FIXOB and OBJFIX are available to override
header discrepancies, at the user's own risk).
By the way, do all current ARM-based products (48Gii, 49G+, 50G)
employ the same binary object prefix "HPHP49-x" ?
[examine the internal VERSTRING function]
> Tim was talking about a filer, but other people like you want a library
> for advanced SD card access. I guess some commands could be left
> visible for programmers, or even put on a separate library.
A separate library would be extremely handy!
Thanks for everything.
[r->] [OFF]
My opinion only.
TW
Filer beta is here--> http//www.timandkatie.com/SDfiler.zip
> some type of PVARS and TYPE commands working on the SD would be useful.
Note that the SD is a bit more complex, because:
o It isn't really a "port" ("P"vars)
o It isn't a "flat" structure containing just simple names
(and some file "names" may be illegal calc IDs)
o It can store external files which aren't HP objects at all,
including both ordinary disk files and disk subfolders
(so what TYPEs are all those?)
o What to return if it is readonly? (available memory, or "ROM")?
So this might need more thorough design to spec out.
OTOH, you might rather want a DIR (or "ls") command
(and possibly a Unix-like "file" command for types),
to reproduce what an external computer could make
of the files, because the SD card is really the same thing
as an external computer disk, quite unlike any internal port.
There actually already exist PKT commands (intended for Kermit)
to get *external* file properties and directory listings --
could/should these be adapted to work with the SD card as well?
[r->] [OFF]
> I personally don't ever use userRPL, or transfer ASC text programs,
> so I don't have any interest in making anything like what you propose.
> If someone needs to translate characters, requiring the use of a wire
> doesn't seem like too much to ask.
I must have obfuscated what I meant to convey, sorry.
I don't myself need the "ascii conversion" part, because I have posted
(and use) programs (in UserRPL with syseval only for K[INV]IS[LF])
which do all that, but it's easy and could be internalized
within other libraries (or even built-in commands) if anyone wished.
Anyone who wants to save UserRPL programs in externally editable form
might well appreciate this, because otherwise you must use an editor
(and email/posting software) with complete ISO-8859-1 character set
(and even then, the 49 series now has a couple of exceptions).
> Yet I have seen many people ask for a method to save objects
> (mainly just text strings) to the card without a header
> so they are editable in text editors.
Of course -- that was always assumed! (as well as being the
necessary basis for any "decompiled and ascii translated"
additional option or command).
However, what I was suggesting is that "mainly just text strings"
should be "ONLY text (and hex) strings" (in which case you probably
actually omit the entire first 13 bytes of the default binary file,
which is the "HPHP49-x" plus the prolog word plus the length word,
the remainder of the object being the literal string itself).
For any other object types, what would you propose storing instead?
A function to do that kind of STO for strings alone
can be coupled with separate functions such as have been posted
to first decompile and ascii-out translate before writing strings,
or to ascii-in translate and compile strings after RCL,
thus providing the other capabilities I suggested;
if one library would like to incorporate all that together, however,
then it could provide convenient "one-stop" shopping,
just as does the cable connection kit itself, which includes translators
that it sends into the calc first, during its own setup.
Technical note:
Strings and Hex Strings (the latter is the same type as "User Binary")
specify their length in nibbles, and it's technically possible for that
number to be odd (i.e. an extra half-byte); in such cases, it's better
to round up, to make sure that the "odd nibble" is included
(necessarily with an extra nibble of padding to fill up the last byte).
A real-life example where this matters a lot is in BZ-compressed objects,
which often result in an odd-length strings, and where the loss
of the last nibble would be likely to crash the calc on re-expansion;
there are also various internally-generated "user binary" values
which happen to have an odd length!
> Filer beta is here--> http//www.timandkatie.com/SDfiler.zip
Will try, thanks.
[r->] [OFF]
The SD card does not act anything like a "port" in the usual sense,
but rather is equivalent to a removable disk drive,
with files intended to be transferrable to/from other computers,
so its use is basically equivalent to cable transfer to another computer
(this is even how its interface has been designed, to store binary objects
in exactly that same format, as if transferred in binary mode via cable,
with no other choice currently offered).
Because of that limitation in writing files from within the calculator,
we keep encountering this basic problem:
> I have seen many people ask for a method to save text strings
> to the card without a header, so they are editable in text editors.
This is so basic a function for so many necessary and useful purposes,
including providing a basis for easily adding "ascii transfer"
even via UserRPL commands, that I would make it a feature request
to build into ROM as an additional *UserRPL* command
(accepting both character strings and hex string object types,
which have identical internal structure).
If this were done, then it would not be necessary to install
the ARM toolbox, nor any library, nor to do the other tricky things
to avoid having the library interfere with normal calc operations
(the current README indicates that you have to subsequently "reset"
something in the calculator, or else built-in STO/RCL will now fail).
There does not need to be any other command added for corresponding
RCL of a text string, because "verbatim string recall to stack"
is already the default, for anything which is not already a valid
internal binary object (also having an "HPHP49-x" header prefix),
so the RCL side of the matter is already taken care of!
Comments are invited and welcomed.
[r->] [OFF]
This will leave you with the "headerless" string by itself,
ready for further text editing or any other use.
Some text editors might even be capable of deleting those
leading bytes by themselves, though this is often impossible
if the editor reserves "null" characters for its own use.
This is a big kluge, however, and does not allow you
to pop your SD card right into other devices
(I wonder how this impacts the surveying market?)
Text files that you write on computers or other devices
are already directly importable to the calculator as strings;
however, this "one way only" transfer capability
is currently a severe restriction for some applications.
[r->] [OFF]
Note that Xmodem transfers are also binary only. It seems to me
that it shouldn't be that difficult to implement "ASCII", that is,
decompiled and optionally translated object transfers, for Xmodem,
just as with Kermit transfers. After all, the decompiler, compiler
and KVISLF, KINVISLF, and KVIS "translators" are already
implemented as supported SysRPL commands, and the binary/ASCII
transfer and via IR/Wire system flags are available. With the 48G
series, "ROM is full and life is short" may well have been a good
reason for not implementing this, but surely it could've been
added with the 49 series.
Much the same applies to "transfers" to and from the MMC/SD card
starting with the 49g+. Perhaps another system flag (or two) could
be used, making the transfer via IR/USB/RS-232/card, assuming that
the 50g will indeed have both RS-232 compatible and USB I/O
available.
In any case, it seems to me that the user should have a choice of
Binary or "ASCII" transfers to the card.
Perhaps KVISLF could even be fixed so that a CRLF pair would be
correctly translated to CRCRLF instead of being left as is.
--
Regards,
James
On Mon, 12 Jun 2006 21:38:35 -0500, James M. Prange wrote:
> Note that Xmodem transfers are also binary only. It seems to me
> that it shouldn't be that difficult to implement "ASCII", that is,
> decompiled and optionally translated object transfers, for Xmodem,
> just as with Kermit transfers. After all, the decompiler, compiler
> and KVISLF, KINVISLF, and KVIS "translators" are already
> implemented as supported SysRPL commands, and the binary/ASCII
> transfer and via IR/Wire system flags are available.
I believe that the connectivity kit (Conn4x) already does exactly this,
sending the conversion programs to the calc upon connection
(a version of the "Xmodem server" is even provided for older HP48S/G),
then simulating "ascii transfer" using Xmodem alone.
> Much the same applies to "transfers" to and from the MMC/SD card
> starting with the 49g+. Perhaps another system flag (or two) could
> be used, making the transfer via IR/USB/RS-232/card, assuming that
> the 50g will indeed have both RS-232 compatible and USB I/O
> available.
For Kermit, I believe that the existing functionality
takes care of all such matters; for USB, the connectivity kit
should also still take care of it -- except for its being somewhat
cavalier in *evaluating* almost any transferred file whatsosever,
rather than just *storing*without*execution* (HP Kermit properly
guards against inappropriate execution, as do the more advanced
versions of my posted UserRPL text IN/OUT commands for independent
in-calc processing for Emu48 and other equivalent I/O purposes).
> In any case, it seems to me that the user should have a choice
> of Binary or "ASCII" transfers to the card.
Well, although I started out trying to suggest biting off that whole
thing at once, I am now trying to settle for something more basic
and more fundamentally vital, which is just to be able to store
a string (or hex string) directly to the SD card, verbatim,
without 13 bytes of prefixes ("HPHP49-x" + prolog + length),
and to be able to do this without an extra Filer library
and the entire ARM toolkit, if at all possible;
ideally via a built-in new *user* command, if that could be inspired,
whose function is simply "store this literal string to that file"
(support from the Kinpo ARM OS side might also be involved, though
I don't know exactly how the entire responsibility is divided up).
That alone takes care of the entire "hard part,"
on top of which we can use our own rather simple programming
(even UserRPL with syseval to invoke K[IN]VIS[LF])
to supply everything else that we'd also like to add
for storing our UserRPL programs in the popular Ascii (editable) form,
which is just one of the many uses to which we could put
a more basic and universal "store this literal string as a file" command.
No extra flags would be necessary in this case, because I am
not now proposing "overloading" any existing STO command;
the existing flags control K[IN]VIS[LF] as needed,
and we'll supply the very simple additional programming
to incorporate that part ourselves -- all we need
from anyone else is the basic "store this text as a file" commmand,
and with that alone, "all else can be added unto us" :)
> Perhaps KVISLF could even be fixed so that a CRLF pair would be
> correctly translated to CRCRLF instead of being left as is.
Have you filed a bug report at bugs.hpcalc.org?
(is there any possibility it might have been a deliberate decision,
for some purpose we haven't thought through yet?)
By the way, I've filed an enhancement request for just the basic
"store this bare string as a file without adding anything" command:
http://bugs.hpcalc.org/long_list.cgi?buglist=216
with the following "pitch":
"Bare" text files from SD can be RCL'ed to stack,
but can not be STO'd back without 13-byte prefix.
Since SD card is an external storage medium,
rather than a port (which is the very reason for the
extra 8 bytes "HPHP49-x" in the prefix), "bare text" literal
string storage (for editors, GPS, surveyors, and all similar
file exchange purposes) would be highly desirable,
and would increase the potential for marketable
external applications using the calculator for SD output.
For longer discussion please see
http://groups.google.com/group/comp.sys.hp48/msg/1d55d1a371887294
(or the entire thread), thanks.
Do you think you could get Joe to pray for it, too? :)
Okay, but I think that it is related.
> On Mon, 12 Jun 2006 21:38:35 -0500, James M. Prange wrote:
<snip, about lack of Xmodem "ASCII" transfers>
> I believe that the connectivity kit (Conn4x) already does exactly this,
> sending the conversion programs to the calc upon connection
> (a version of the "Xmodem server" is even provided for older HP48S/G),
> then simulating "ascii transfer" using Xmodem alone.
The Xmodem server library isn't compatible with the 48SX/S, which
isn't too surprising as they don't have Xmodem built-in.
A Conn4x "Text" transfer does something very similar to a Kermit
"ASCII" transfer, but not exactly the same. I believe that the
"ASCII translations" are done on the PC, instead of using the
K[IN]VISLF SysRPL commands built in to the calculators. That would
be just fine, except that, if I recall correctly, the translations
for NUL, literal ", and \ (as decompiled to \00, \", and \\ on the
49 series) differ from the Kermit translations, and what's more,
they depend on whether they're inside of a quote-delimited string,
and sometimes even on whether they're within a composite. No doubt
this was intended to "simplify" the text (avoiding the \\00, \\",
and \\\\ from Kermit translations), but I think that Conn4x
complicates translations of these characters to the point that
mistranslations can occur, even without any Kermit transfer being
involved. Of course, when both Kermit and Conn4x transfers of a
file that contains these "special" characters are involved, things
get a bit more complicated.
Outside of a quote-delimited string (for example, within a counted
form string from a 48 series or edited on a PC, or within a name),
when transferring to the calculator, Conn4x mistranslates a \
followed by two or three characters that together would make a
valid translation code sequence. A work-around for this is to
manually edit the \ to \092, which Conn4x correctly translates
back to just \.
I believe that the whole mess could've been avoided by simply
translating any \ to \\, no matter what follows it, and regardless
of whether it's within a quote-delimited string, just as Kermit
does.
Conn4x does have the advantages of correctly translating <CR><LF>
pairs to <CR><CR><LF> or optionally leaving <LF> untranslated, and
optionally translating ASCII control codes (except <LF>) to \nnn
sequences, on transfers from the calculator.
Anyway, Conn4x isn't the only PC application capable of Xmodem
transfers, and in my opinion, it would be nice to be able to use
"ASCII" transfers with any Xmodem application.
> > Much the same applies to "transfers" to and from the MMC/SD card
> > starting with the 49g+. Perhaps another system flag (or two) could
> > be used, making the transfer via IR/USB/RS-232/card, assuming that
> > the 50g will indeed have both RS-232 compatible and USB I/O
> > available.
>
> For Kermit, I believe that the existing functionality
> takes care of all such matters; for USB, the connectivity kit
> should also still take care of it -- except for its being somewhat
> cavalier in *evaluating* almost any transferred file whatsosever,
> rather than just *storing*without*execution* (HP Kermit properly
> guards against inappropriate execution, as do the more advanced
> versions of my posted UserRPL text IN/OUT commands for independent
> in-calc processing for Emu48 and other equivalent I/O purposes).
Are you saying that files without a Binary transfer header valid
for the particular model or an ASCII transfer header are executed
or stored other than within a character string?
Indeed, just having a built-in UserRPL (or even SysRPL) command to
store text without a transfer header, character string prologue,
and length field would take care of the hardest part, as far as
I'm concerned. I wouldn't want to require an entire library with
various other capabilities for the sake of gaining the use of just
this one command, and I especially wouldn't want to require an
entire ARM toolkit for it. That said, even a (relatively small)
stand-alone program for this would suffice; it wouldn't
necessarily have to be built-in.
> > Perhaps KVISLF could even be fixed so that a CRLF pair would be
> > correctly translated to CRCRLF instead of being left as is.
>
> Have you filed a bug report at bugs.hpcalc.org?
Good question! I have now.
> (is there any possibility it might have been a deliberate decision,
> for some purpose we haven't thought through yet?)
I expect that it was indeed a deliberate decision, but that the
48SX developers didn't think it through correctly. <LF> is the
calculator's "line terminator", which KVISLF correctly translates
to <CR><LF>, Kermit's canonical line terminator for "text"
transfers, *except* when the <LF> is immediately preceded by <CR>,
in which case it's incorrectly left as is. As a result, both <LF>
and <CR><LF> are transferred from the calculator as the canonical
<CR><LF> line terminator, and when transferred back to the
calculator, KINVISLF correctly translates the <CR><LF> to <LF>,
whether it was originally <LF> or <CR><LF>.
As the line terminator on the calculator is just <LF>, any <LF>
should be translated to the Kermit's canonical <CR><LF> line
terminator before sending, including translating <CR><LF> to
<CR><CR><LF>, so that it would be correctly translated back to
<CR><LF> when transferred back to the calculator.
Note that it's up to the other Kermit application to translate the
canonical <CR><LF> pair to (and from) whatever's appropriate for
it.
Of course, since both <CR> and <LF> are control codes, they're
actually encoded to #M and #J at the data link layer for
transmission, assuming that # is used as Kermit's "control prefix"
character. That's mandatory for all Kermit transfers, whether
"text" or "binary".
<snip>
> Do you think you could get Joe to pray for it, too? :)
I don't know that I have much influence with Joe, but I hope that
he will.
Hey, Veli-Pekka, you could pray for it too.
--
Regards,
James
ARM Toolkit should be in the ROM
STOTXT UserRpl command sounds nice
>> Do you think you could get Joe to pray for it, too? :)
>
> I don't know that I have much influence with Joe, but I hope that
> he will.
>
> Hey, Veli-Pekka, you could pray for it too.
I just did
It has been a good while since I did the Conn4x text handing.
However, I am pretty sure it is a lot worse than just using \\ for \.
I recoded the whole mess a couple of times to get to where we are now.
As you noted, it still has bugs!
There are just a lot of funny situations Kermit handles in ways that still
seem somewhat "ad hoc" and I did not know of quite a few of these. So for
whatever its worth (not very much) I'm sorry about the incompatibility.
Unfortunately I am not in control and could not update the product if I
wanted too. [now, of course, there is the dilemma of a re-code introducing
yet a third way of handling things!!]
-- - - - - - - - - - - - - - - - -
Bill Graves RKBA!
bgr...@ix.netcom.com
> Note that the SD is a bit more complex, because:
> o It isn't really a "port" ("P"vars)
I know, I also wrote a 'PVARS type of command'. A way to get a
directory listing given a path on the SD card. There's currently no way
to do that built into the ROM (even though PVARS intuitively ought to
work on Port 3 as well).
> o It isn't a "flat" structure containing just simple names
> (and some file "names" may be illegal calc IDs)
Illegal IDs have never stopped us before. There are numerous ways to
create those anyway built into the calc. Doesn't matter in my eyes.
> o It can store external files which aren't HP objects at all,
> including both ordinary disk files and disk subfolders
> (so what TYPEs are all those?)
External?
> o What to return if it is readonly? (available memory, or "ROM")?
"ROM" just like a read only RAM card on the 48 series.
> So this might need more thorough design to spec out.
Of course. It was merely a suggestion.
Regards
Steen
<snip>
> All the built-in ROM implementations that I'm aware of,
> both in HP48 and HP49/50 series, will completely ignore any characters
> inserted *before* the standard "%%HP" mode-setting header;
> therefore, if we insert something like "I(E)" or "I(A)"
> (representing "Exact" vs. "Approximate" mode
> for interpreting integers without decimal points) just before "%%HP"
> then any real HP48/49/50 will ignore it, but *our* "Import" function
> could use this to set the appropriate mode automatically
> before interpreting the text.
>
> Since no previously created "exported" files ever inserted this option,
> it would basically serve as something we could insert now,
> into previously exported files or future exports,
> to automate the proper setting
> and guarantee correct future import whenever it is present,
> at least when we import using our own new "Import" command.
>
> Any UserRPL files *created* on HP48 should have "I(A)" manually inserted,
> to guarantee proper import to an HP49/50, whereas all files
> exported from an HP49/50 should have "I(E)" automatically inserted
> (regardless of the actual internal flag setting while exporting!),
> to equally guarantee correct import back into any HP49/50,
> without depending upon the user to inspect the files
> and carefully prepare the correct mode before each individual import.
Yes, this does seem to be true. Is anyone currently using, or
planning to soon implement, such a scheme? I'm rather interested
as I might (or might not) use something like this, if I ever get
around to starting on a revised "ASCII" storage scheme for MMC /
SD cards, and it would be best if everyone who used it did it
exactly the same.
I rather like "I(E)%%HP..." and "I(A)%%HP..." for "Exact" and
"Approximate". Of course really only one character would be
required for a marker, for example, "E%%HP..." and "I%%HP...", but
then a single character of random "garbage" could be misleading.
Having only a distinct pair of 4-character markers seems to me to
make the risk negligible.
But perhaps it would be better to wait a little while and see if
the 50g addresses the issue at all before implementing this.
--
Regards,
James
JHM> [SD] can store external files which aren't HP objects at all,
JHM> including both ordinary disk files and disk subfolders
JHM> (so what TYPEs are all those?)
SS> External?
True "ports" on HP48/49/50 are only available for
internal calculator use, and nothing but calculator objects
can be stored into them, only in their binary internal form,
exactly as used within calculator memory.
The SD card is equivalent to an external hard drive, however,
and can be used directly on computers;
the files stored on the SD card
can thus be arbitrary files which do not have binary "prolog" words,
and are not calculator-compatible binary files.
This is an entirely new situation, starting with the HP49G+,
and any such "external" files (lacking a binary "prolog" word,
which is what the "TYPE" command inspects)
have no "type number" that corresponds
to any calculator internal object type,
so what "type" would you then assign to those files?
(And also to file system folders?)
If the calculator stores anything onto the SD card,
it currently always stores a special header,
followed by an internal binary calculator object
that begins with the usual "prolog" word,
but when a computer creates files on the SD card,
there is no such header and no such prolog word,
and the file itself can not be directly used
within calculator memory, except by inserting it
into some existing internal calculator object type,
such as a string object.
It has always been the custom, when transferring files
from external computers (over cable or IR) to an HP48/49,
that any file not internally compatible with the calculator
is transferred in a string object, containing the entire
literal external file content, and the same is done by RCL for SD files
(so the Filer, for example, seems to think of such files as "string" type,
although they aren't yet even converted to that internal type);
this still leaves "subfolders" on the SD card as a type
which can't be recalled via RCL and can't be directly handled at all
by the calc, except that the Filer itself can explore into them.
No built-in provision was ever made for storing an arbitrary string
back onto the SD card *without* a header and prolog+length;
that of course is what I've been trying to survey the need for,
to see whether it's felt important enough to add built-in support for it,
whether via a user command or other readily accessible "supported" function
(ROMPTR or FPTR, which in turn could be invoked even in UserRPL
via LIBEVAL or FLASHEVAL).
So "external" means the entire universe of files that could exist
on an SD card, but which could not exist in that same form
in calculator memory -- files which never before could exist
in what has been called a "calculator port" --
so I would not even call the SD card a "port" at all;
the use of the numeric tag :3: is simply a syntax convention
for designating the SD card as an external storage device,
just as :IO:filename has always been a syntax convention
to tell the calculator to send a backup out thru the cable,
to a file on a computer, rather than to a "port" named "IO"
[r->] [OFF]
[skipping Ascii translation stuff, already answered by William Graves]
Of course, all "line endings" are just whitespace to the compiler,
and even LF-delimited lines can be viewed by Windows write.exe,
so I've never encountered any problems w/r/t my program files.
> Indeed, just having a built-in UserRPL (or even SysRPL) command to
> store text without a transfer header, character string prologue,
> and length field would take care of the hardest part, as far as
> I'm concerned. I wouldn't want to require an entire library with
> various other capabilities for the sake of gaining the use of just
> this one command, and I especially wouldn't want to require an
> entire ARM toolkit for it. That said, even a (relatively small)
> stand-alone program for this would suffice;
> it wouldn't necessarily have to be built-in.
A "stand-alone" can use named "supported" ROM entry points,
along with numbered ROMPTR and FPTR function calls,
and all of these are at the "Saturn OS" level;
is this enough, or do we need also to get into "ARM OS" level?
If the latter, do we have to use HPGCC tools?
What exactly are the ARM\-> and POKEARM and PEEKARM commands?
Are there any "supported entry points" in the ARM OS
that one can use?
Well, I'm armed with no information in my own head,
so I hope that someone else will supply the answers,
and if it turns out to be too much bother for this small function,
it would be a great boon if HP built it in, whether through UserRPL command
or ROMPTR (LIBEVAL) or FPTR (FLASHEVAL), because right now, we have only
one direction in which we can transfer arbitrary external data files,
which is *from* the SD card (which is like an external disk drive,
not a port) *to* a string in calc memory, but we can't do the reverse,
which is really necessary for generalized use of the external medium,
which should be a lot more than merely an extension of purely internal storage.
[r->] [OFF]
> Are you saying that files without a Binary transfer header valid
> for the particular model or an ASCII transfer header are executed
> or stored other than within a character string?
Try Conn4x on various Ascii transfers
(with or without "%%HP" line),
from computer to the calculator,
with the following (single line) file contents
(you could name your computer file "Z.hp"
to be transferred into a calc variable named "Z"):
VARS
This should be transferred so that the receiving variable (Z)
contains only the VARS command itself (object type 19);
it should not *execute* the command.
123 'X' STO 456 'Y' STO
This should be transferred into a receiving variable (Z)
which should then contain this entire line ("wrapped" in a
"progam" without visible delimiters, which is object type 8),
and there should also be no error occurring on the receiving calc
(and no stuff left on its stack); the original line
(containing various objects and commands) should not be *executed*
I believe that "ascii" transfers using built-in Kermit by itself
(which might be difficult on the 49G+ due to lack of a serial port,
but can be performed using 49G and 48G with Hyperterminal
or even original MS-DOS Kermit on the PC) will behave as suggested,
because the original (48 series) internal programming
took all this into careful consideration.
I've posted both "quick & dirty" ascii importers that evaluate everything
(which I use in my emulator to import/export what I post)
and a complete system that behaves just like the original
Kermit ascii transfers (including generating an "%%HP" line
upon export and obeying any such line upon import).
[r->] [OFF]
> so what "type" would you then assign to those files?
> (And also to file system folders?)
My suggestion was "External", aka 'external object 4' which is user
TYPE 27 and currently unused. I wasn't asking what you meant by
external :-) It's kind of fake though, as the dispatch code doesn't
exist in the object for real.
The real problem is that these files can start with anything, also a
string of bits identical to a true prolog, making the system mistake a
PC file for an HP object. So thorough inspection of the entire object
is necessary, before the system can/may return it's verdict on the
object type of files on the SD card.
The system could otoh simply return '?' for non-HP objects.
Regards
Steen
> My suggestion was "External"... TYPE 27
Oh, *that* external! Might be okay.
> It's kind of fake though, as the dispatch code
> doesn't exist for the real object.
Non-calc objects are not "real" objects
(*not* meaning numeric real :)
It's a very "closed society" inside the calc,
and "foreign" piles of bits are unwelcome,
much like other immigrants crossing certain borders ;)
> The real problem is that these files can start with anything,
> also a string of bits identical to a true prolog,
> making the system mistake a PC file for an HP object.
Only if it starts with "HPHPnn-x" (of the right calc series)
and *then* a valid prolog... [except with this last-announced
HPGCC-aided Filer, which says it's willing to strip the "HPHPnn-x"
(but leave the prolog), and also willing to load anything,
without the "HPHPnn-x" compatible series check,
which IMHO is defeating an important "safety interlock"
for no reason, and thus inviting trouble].
> So thorough inspection of the entire object is necessary
FIXOB (and hopefully OBJFIX) also tests something further,
that SAFESKIPOB can be performed (with correct length?)
(I wonder whether various Filers make the same tests?)
Even then, one could contrive a dangerous object
(a program or algebraic or unit object containing invalid addresses,
say), but that's no more dangerous than any real but buggy program,
so at this point it hardly pays to try to sniff any further.
> The system could OTOH simply return '?' for non-HP objects.
Could, as long as you don't "overload" an existing internal function,
with which existing programs might depend on a real-valued result
within certain known bounds, to avoid crashing on subsequent actions.
[r->] [OFF]
Hi,
<..>
> The real problem is that these files can start with anything, also a
> string of bits identical to a true prolog, making the system mistake a
> PC file for an HP object.
Should a file "coincidentally" start with a sequence used as an Object's
prologue, it will inevitably be identified as an instance of that
object, since this problem is formally undecidable.
Even if you invent a tricky "AI" charged heuristic, it will (sometimes)
fail!
Simply because the prologue, looked at as an input symbol in a formal
grammar G, by definition can't be element of the recognized input
stream, looked at as a formal language L(G).
So thorough inspection of the entire object
> is necessary, before the system can/may return it's verdict on the
> object type of files on the SD card.
Won't work ...
The point is, that you must always make an assumption on what to expect.
Hence your "binary" stream must be interpreted somehow.
See the CR/LF binary vs. text problem, or Postscript vs. ASCII as
analogous dilemma situations...
Cheers,
Ingo
The overall rule is that any string which can (after KINVISLF)
be compiled without error by "palparse"
must be stored as that single object returned by "palparse,"
with just two exceptions:
"\<< ... \>>" ---palparse--> :: xSILENT' :: x<< ... x>> ; ;
"'name'" ---palparse--> :: x' ID name xENDTIC ;
The first case can be identified by being a program object
containing exactly two elements, of which the first is xSILENT'
The second case can be identified by being a program object
containing exactly three elements, of which the first is x'
In these two special cases, you should either EVAL or extract
the second element from what "palparse" returned
(either method gives the same result).
Did on-board HP48 Kermit even bother with the second case?
http://groups.google.com/group/comp.sys.hp48/msg/f685d20e738317aa
[r->] [OFF]
Me thinks that names ending with .txt or .TXT
could be stored and recalled to/from SD without changes
or rather according to ASCII translation flags
Lets' say .txt is translated and .TXT is RAW
That way the current ROM/Filer could very easily adapt to
these new SD-compatible store modes
Naturally only with the :3: before the name
so used memory and internal ports would only have valid strings
bur externally on SD they would be without the header & prolog
I think that Cyrille could quite easily bring this feature to 2.10 C
Veli-Pekka
> [file] names ending with .txt or .TXT
Name-based "associations," as in good old MS Windows?
Proverbial "can of worms," IMO
(where will your "Registry" be stored?)
OTOH, *content* beginning with model-matching "HPHPnn-x"
is already recognized by default on import (as a binary object),
as is "anything else" (which imports as an unprocessed string);
perhaps "%%HP:" might likewise be specially recognized on import
as inviting ascii translation and compiling.
This still doesn't provide optional "pure text" *export*
for arbitrary string/hex objects, which still needs
an independent STO-like command (everything else can be handled
as any individual desires, using that command plus built-in RCL,
plus optional Ascii[Kermit-like] IN/OUT programs which already exist).
[r->] [OFF]
John H Meyers wrote:
<...>
> > The real problem is that these files can start with anything,
> > also a string of bits identical to a true prolog,
> > making the system mistake a PC file for an HP object.
>
> Only if it starts with "HPHPnn-x" (of the right calc series)
> and *then* a valid prolog... [except with this last-announced
> HPGCC-aided Filer, which says it's willing to strip the "HPHPnn-x"
> (but leave the prolog), and also willing to load anything,
> without the "HPHPnn-x" compatible series check,
> which IMHO is defeating an important "safety interlock"
> for no reason, and thus inviting trouble].
Not really, for several reasons:
1) There's no inter-calculator compatibility problem with the prologs.
All RPN calcs since 48SX have the exact same prologs for the same
objects. The only problem could be to execute a sysrpl program written
for other calculator, but since other calculators can't create files
without the HPHP signature, that can't happen.
2) The ONLY calculator with SD card is the 49G+ (so far), and it
supports all known types of objects.
3) 20 bits is more than what's commonly used for type checking. Old DOS
had no header on .COM executables (and I never heard anyone complaining
that it was unsafe), .EXE files have only a 16-bit marker (MZ for
16-bit, PE for 32-bit), bitmaps have also a 16-bit signature (BM), the
rest is done by validating the internal fields of the structure that
follows. Same thing is done in SDLIB. The library attempts a SKIPOB,
and will return FALSE on invalid objects. Of course there will be
certain combinations that can fool the routine, but the same can happen
with any file, even the ones with the HPHP49-C signature, there's no
guarantee that data afterwards is a valid object.
Besides that, the idea is not to skip safety checks, but to give more
freedom to developers. Originally, the "enhanced" STO routine I had in
mind would be able to append saturn objects to a file, therefore the
need to sto without the header. The companion enhanced RCL was going to
be able to RCL any number of objects from a file, starting at a
particular index, so you could for exampl store a large matrix by rows
(in a single file), and RCL only the row/rows of interest. Sort of a
GET/PUT command but with objects stored in a file.
Of course, I don't have time to do that for now, but maybe a future
release.
Claudio
> There's no inter-calculator compatibility problem
Past and present series have the same first 20 bits
for the same object type (except for new 49 types
which didn't exist in 48),
but that's where all general compatibility ends.
The "HPHPnn-x" prefix is the "safety feature" which can be used
to make sure that one knows which model is required for what object,
without which you've lost something important, just as if you'd
removed the "salt" vs. sugar" labels on kitchen containers
of plain white powders.
You are also gambling that no future series or ROM version
will ever *become* incompatible with the current 49G ROM --
actually it's already a fact that it is possible to create
binary code that works only on a 49G+ and not on a 49G,
because of "ARM-assisted" coding, and who knows
what developments will come in the future?
What about taking stored objects from an SD card
and transferring them to EMU49, or to another calculator
(a calculator which doesn't also use your filer)?
Without the "HPHPnn-x" header, such transfers will be difficult
("that's going to be hard without your space helmet, Dave" :)
> the idea is not to skip safety checks,
> but to give more freedom to developers.
Once you have the ability to store a "bare" string, any developer
can use that to strip off the "HPHPnn-x" if (s)he wants,
just as anyone can drive away without fastening a seat belt,
but I wouldn't *encourage* such unsafe behavior
by making it a "standard" function.
> Originally, the "enhanced" STO routine I had in
> mind would be able to append saturn objects to a file,
> therefore the need to sto without the header.
Perhaps it might best be reserved for an "append" function,
rather than for an "individual object STO" function.
Thanks for considering the matter.
[r->] [OFF]