Connectivity Kit

[Scott Chapin]

Is the connectivity kit only available for Windows? Maybe it is the key to getting binary strings with null characters into an emulator? I would prefer to do it on my MAC

[Egan Ford]

On 12/7/12 11:37 AM, Scott Chapin wrote:
> Is the connectivity kit only available for Windows? Maybe it is the key to getting binary strings with null characters into an emulator? I would prefer to do it on my MAC
>

For Mac you can try http://hpconnect.sourceforge.net/ or Kermit. I've
used both, but not in a long time. I usually just shuttle with the SD
card (50g). However 48GX, I still use Kermit. IIRC, to use Kermit on
the 50g you may need to order the serial cable from hpcalc.org.

[Scott Chapin]

Thanks Egan. My emulator is on my iPhone (m48+). When I transfer my HP50 home directory backup from my SD card OTA, it opens up as a binary string with null characters.

I was hoping that EMU48 might allow me to create a useable file. Don't know. I'm not a programmer and a bit lost.

[John H Meyers]

On 12/7/2012 2:00 PM, Scott Chapin wrote:

> My emulator is on my iPhone (m48+).
> When I transfer my HP50 home directory backup from my SD card OTA,
> it opens up as a binary string with null characters.
>
> I was hoping that EMU48 might allow me to create a useable file.

A calculator "directory" is a binary object.

Binary objects, when stored as computer (or SD card) files,
are prefixed with an 8-byte "header" followed by the binary image
of the object as it resides within calculator internal memory.

The 8-byte "header" is "HPHP49-x" for HP49/50 series objects,
and "HPHP48-x" for HP48[S/G] series objects, which are generally
not mutually compatible, hence are normally rejected
(by remaining as a string containing all bytes of the computer file,
including the first 8 bytes "HPHP4?-x")
when loaded into the internal memory of
a different series calculator than they were stored from.

Some transfer software also even tests objects having
a matching "binary header" for the same series,
to see whether the object type is valid,
and whether an internal function for "skipping over"
the object agrees with the prefix-removed file length;
objects failing either test are corrupted, hence are best
left encapsulated in the string, where they can't cause a crash.

On the HP49/50 series, you can even use a simple UserRPL program
to clip off the "binary prefix" and force any string to be interpreted
as an internal binary object, but if it turns out to be
an invalid or corrupt object, you might soon have
either a crash or a complete "Memory Lost" wipe to show for it.

Are you trying to transfer binary objects between different series?

Is your "OTA transfer" being performed in a "binary" mode,
rather than assuming that what's being transferred is
a human-readable plain text UserRPL "source" file, as is done
when using HP's built-in Kermit (or other file transfer software)
"in ascii mode"? "Ascii mode" means that the computer side file
is the human-readable version of the binary file on the calculator side,
involving "compiling" of plain text files going from computer into calculator,
before final storing on calculator side, or first de-compiling (to plain text)
of files going from calculator to computer, prefixed in this case
by a "mode header line" such as: "%%HP: T(3)A(D)F(.);"

The "mode header line" helps to assure that re-compiling of UserRPL text
is done using modes matching when they were de-compiled and "translated,"
because proper UserRPL interpretation may vary with current settings of
character [T]ranslation mode, [A]ngle mode, and [F]raction mark mode.

In the computer -> calculator direction, file transfer software
usually recognizes either "binary" or "ascii" file headers,
automatically adjusting all transfer modes accordingly;
in the calculator -> computer direction,
the user-chosen file transfer mode is selected,
and an appropriate mode header is prefixed.

One HP49/50 series mode affecting proper UserRPL interpretation
is not reflected in "ascii" headers, which is "Exact" vs. "Approximate"
mode -- in the HP49/50 series, "Real" numbers always display with a
"fraction mark" ("decimal point") but a different "Integer" object type
displays with no such character; this same display distinction is used
to compile each object to its separate type in "Exact" mode,
whereas they are both compiled to "Real" type in "Approximate" mode.

The HP48 series also compiles all numbers to "Real" type,
because it has no "Integer" type, and while de-compiling,
it omits a trailing "fraction mark" ("decimal point")
for numbers that appear to have an integer value.

Therefore, when using HP49/50 series calculators,
one should therefore normally compile HP49/50 series UserRPL text
in "Exact" mode, but compile HP48 series UserRPL in "Approximate" mode,
to avoid introducing new object types into HP48 series programs, and
HP49/50 UserRPL programs might not work in HP48 series calculators
if they depend on any new object types.

Although "binary" objects (other than strings) are not human-readable,
nor are they transferable in general between calculator series,
they do avoid all the potential ambiguities associated with UserRPL program text,
and are best for "guaranteed correct" backup/restore and calculator <-> emulator transfer.

[r->] [OFF]

[John H Meyers]

Things I forgot to ask after first losing my "mental notes":

On 12/7/2012 2:00 PM, Scott Chapin wrote:

> My emulator is on my iPhone (m48+).
> When I transfer my HP50 home directory backup from my SD card OTA,
> it opens up as a binary string with null characters.
>
> I was hoping that EMU48 might allow me to create a useable file.

Where does the transferred data go?
(E.g. on the stack, in a variable, in a port?)
what does "opens up as" mean,
what's seen in the calculator's display (stack),
and what are the first 8 characters of the string?

What does "Load object" (of your SD backup to EMU48) produce
on the display? Anything similar?

What's the result of DUP TYPE after having the object on the stack?

A backup should have type 15 (directory), should be storable
into port 1 or port 2, then e.g. :2:HOMEDIR RESTORE should restore it
(completely replacing the entire HOME directory,
thus don't restore a different backup if you first need
to back up the current contents, which will be "wiped" out
by restoring a different backup).

RESTORE will accept any directory at all, even one which was
not produced by an ARCHIVE command, but a directory not produced
by ARCHIVE will not contain a "hidden system directory"
as its last object, hence the result of a RESTORE of any
"non-HOME" directory object will generate a new
hidden system directory, initialized to a default state,
and possibly some new CAS variables as well.

[r->] [OFF]

[Scott Chapin]

John....Thank you!

> A calculator "directory" is a binary object.
>
>
>
> Binary objects, when stored as computer (or SD card) files,
>
> are prefixed with an 8-byte "header" followed by the binary image
>
> of the object as it resides within calculator internal memory.
>
>
>
> The 8-byte "header" is "HPHP49-x" for HP49/50 series objects,
>
> and "HPHP48-x" for HP48[S/G] series objects, which are generally
>
> not mutually compatible, hence are normally rejected
>
> (by remaining as a string containing all bytes of the computer file,
>
> including the first 8 bytes "HPHP4?-x")
>
> when loaded into the internal memory of
>
> a different series calculator than they were stored from.
>
>
>
> Some transfer software also even tests objects having
>
> a matching "binary header" for the same series,
>
> to see whether the object type is valid,
>
> and whether an internal function for "skipping over"
>
> the object agrees with the prefix-removed file length;
>
> objects failing either test are corrupted, hence are best
>
> left encapsulated in the string, where they can't cause a crash.
>
>
>
> On the HP49/50 series, you can even use a simple UserRPL program
>
> to clip off the "binary prefix" and force any string to be interpreted
>
> as an internal binary object, but if it turns out to be
>
> an invalid or corrupt object, you might soon have
>
> either a crash or a complete "Memory Lost" wipe to show for it.
>
>
>
> Are you trying to transfer binary objects between different series?

The m48+ ROM is a 49G+ ROM which should be compatible with a 50G that the backup is from. The backup was copied from an SD card to my computer and transferred to my iPhone via the m48+ server. The emulator's load object command places it on the emulator's stack.

In the stack it appears as a string starting with "HPHP49-Xb+@ü. It is full of null characters and cannot be edited. Unfortunately, from reading your next post, TYPE returns a 2 which is a string and not 15 which is a backup object.

The reason I was excited about the m48+ upgrade was getting port 2 support and a skin that is like the 50G.

I do use my 50G in approximate mode. Does that mean I should create the home directory backup in exact mode?

[John H Meyers]

On 12/8/2012 6:37 AM, Scott Chapin wrote:

> The m48+ ROM is a 49G+ ROM
> which should be compatible with a 50G that the backup is from.
>
> The backup was copied from an SD card to my computer
> and transferred to my iPhone via the m48+ server.
> The emulator's load object command places it on the emulator's stack.
>
> In the stack it appears as a string starting with "HPHP49-Xb+@ü"

> It is full of null characters and cannot be edited.

> TYPE returns 2, which is a string

The string contains the entire content of the file from the SD card;
that content is the binary memory image of your backup,
prefixed by "HPHP49-X"; a binary memory image is likely
to contain some bytes equal to #00h (null characters),
and we wouldn't expect it to be editable anyway as text,
even if it didn't happen to contain any null-valued single bytes.

When loading an object, EMU48
(when emulating an HP49/50 series calculator)
automatically extracts the embedded binary object from such a string
if the computer file begins with "HPHP49-x" and if
the binary remainder of the file represents a valid object.

I don't know, however, whether m48+ normally does the same.

I forgot to ask the size of your backup file
(which should equal the size of the string on the stack),
to learn whether it might be so large that the coding used
to extract the object might not have enough memory to run.

Another thing I forgot to ask is whether you can load a much
smaller binary object, and see whether or not m48+
automatically extracts it.

We can easily create a small binary object to test this,
e.g. you could simply type and run the following,
either on a real calculator, on EMU48, or on m48+

"A" 2 \->GROB

This should produce an 18-byte binary object,
a 6x8 pixel image of the letter "A"
displaying on the stack as "Graphic 6 x 8"
(the image itself may also appear on the HP49/50 series stack).

This object has one more useful property for experimenting,
which is that it's an identical binary object
on either HP48 series or HP49/50 series calculator,
although when you store or transfer it to a computer
(or SD card), one series will insert an HPHP48-x
file prefix while the other series inserts HPHP49-x,
which is a convention used to thwart the potential
extraction of incompatible binary objects,
if loaded onto a different series calculator.

Can you store this object on your SD card,
transfer it to m48+ and see what happens?
(does it arrive on the stack as a string or as a graphic object?)

If the small graphic object self-extracts but the large memory backup
doesn't, then perhaps the large file is either corrupt or just too big.

If even the small object doesn't self-extract, then you could use
a program such as FIXOB for HP49/50 (or FIXIT for HP48), which
forcefully extracts binary objects from prefixed computer file strings,
even if the prefix is from the wrong calculator series,
although it does some minimal testing, I believe,
to avoid storing truncated objects or invalid object types.

The following UserRPL object extractor for HP49/50 series
is adapted from something once posted by JYA;
aside from expecting a string starting with "HPHP4"
this program does no validity or safety checking at all,
but as long as you are willing to accept the risk of
possibly corrupting all data already stored in your calculator
(or emulator), you can throw all caution to the winds:

\<< "" ADD IF DUP "HPHP4" POS THEN
#100002h LIBEVAL 26 + #100003h LIBEVAL NEWOB END \>> 'fixhp' STO

Good luck!

[r->] [OFF]

[Scott Chapin]

Thanks bunched John!!

I'm getting closer. The modified "JYA"
routine compiled to libraries into objects. One, L226 returned External, and I realize it is part of the calculator's ROM.

My home directory is huge, 94305 bytes, and I get an insufficient memory error when running your routine on it. I guess I need to make directory objects and load them one at a time.

The 49G+/50 will be 10 years old next autumn, wow. I'll get this all figured out and then have to transfer to a new calculator? It was hard enough getting 48GX programs into the 50...LOL. I'm assuming a new series will be out next year?

[John H Meyers]

On 12/10/2012 7:06 AM, Scott Chapin wrote:

> I'm getting closer. The modified "JYA"
> routine compiled to libraries into objects.

Are you saying "compiled two libraries into objects"?

> One, L226 returned External,
> and I realize it is part of the calculator's ROM.

Library 226 is data for the HP Equation Library;
it may come with a calculator ROM file,
but it is not part of the calculator's ROM
(some old ROM files appended the equation libraries
and overwrote data in port 2 while installing ROM,
but this became obsolete at some point, after the
equation libraries were made independent of ROM versions).

I don't understand what steps you took
that arrived at a point where the "object extractor"
would have been useful -- did you load Library 226
from a computer file to the stack of m48+
and see it appear as a string beginning with "HPHP49-x"?

If so, and if the library was transferred without error
and then extracted correctly, the extracted object
should be a single object, and should say
"Library 226: EQLIB:Equation Data" on the stack,
rather than say "External" -- also, there is no
"compilation" involved in extracting the object.

What you have written above thus makes me wonder
whether something is going wrong -- perhaps
the original data transfer? Did you try transferring
the small image file, as was suggested earlier,
and see in what form it appeared when transferred,
and see whether the correct image (of the letter "A")
can be extracted from that?

> My home directory is huge, 94305 bytes,
> and I get an insufficient memory error when running your routine on it.
> I guess I need to make directory objects and load them one at a time.

If too large a string is the only problem, then perhaps so,
but if data corruption during data transfer is the problem,
then perhaps the data transfer is occurring improperly.

Sorry I have no info about m48+
and can't figure exactly what's going on there.

[r->] [OFF]

[Scott Chapin]

On Tuesday, December 11, 2012 6:25:43 PM UTC-5, John H Meyers wrote:
> On 12/10/2012 7:06 AM, Scott Chapin wrote:
>
>
>
> > I'm getting closer. The modified "JYA"
>
> > routine compiled to libraries into objects.
>
>
>
> Are you saying "compiled two libraries into objects"?

Yes, I am, I was typing fast at work, sorry. They loaded as strings, and fixhp left a library in the stack.

> Library 226 is data for the HP Equation Library;
>
> it may come with a calculator ROM file,
>
> but it is not part of the calculator's ROM
>
> (some old ROM files appended the equation libraries
>
> and overwrote data in port 2 while installing ROM,
>
> but this became obsolete at some point, after the
>
> equation libraries were made independent of ROM versions).

Ah, that was transparent to me. When the string converted, it said external, so I went lurking around and found that library already installed in port 2.

> I don't understand what steps you took
>
> that arrived at a point where the "object extractor"
>
> would have been useful -- did you load Library 226
>
> from a computer file to the stack of m48+

Yes. I believe once upon a time that I used the connectivity kit to back everything up on my computer. When the m48+ server received it, it loaded as a binary string with the null characters and could not be edited. fixhp nicely converted my libraries to library objects except Library 226 which loaded with the emulator's ROM.

>
> and see it appear as a string beginning with "HPHP49-x"?

Indeed, all my objects from the computer appear as binary strings starting with "HPHP49-X".
fixhp seems to handle them very nicely.

> If so, and if the library was transferred without error
>
> and then extracted correctly, the extracted object
>
> should be a single object, and should say
>
> "Library 226: EQLIB:Equation Data" on the stack,
>
> rather than say "External" -- also, there is no
>
> "compilation" involved in extracting the object.

No compilation does not surprise me, I am not a programmer and know just enough to be dangerous and get by. My libraries so far do (extract?) as you say leaving the form "Library NNN: EQLIB:Equation Data" , except for Library 226 which when extracted let "External" on the stack. I assumed that was because it was already installed in the emulator, but maybe not?

> What you have written above thus makes me wonder
>
> whether something is going wrong -- perhaps
>
> the original data transfer? Did you try transferring
>
> the small image file, as was suggested earlier,

Yes I did and it worked a treat. It appeared properly on the 50G and in the emulator.

> If too large a string is the only problem, then perhaps so,
>
> but if data corruption during data transfer is the problem,
>
> then perhaps the data transfer is occurring improperly.

I offloaded 3 large data bases from my calculator and made another archive of my home directory without them. I ran \<< :3:SMALLBACK ARCHIVE \>>. Then I copied the object from the SD card to my computer (windows based) and used the m48+ server to tranfer it.

It copied to the stack as a binary string headed by HPHP49-X complete with null characters. fixhp extracted it as a backup object and it restored nicely, or apparently so, Then I transferred the three data bases back and it was all good.

> Sorry I have no info about m48+
>
> and can't figure exactly what's going on there.

It's a nice emulator for the iPhone and I enjoy having my 50G going with me wherever I go, but it probably has more bugs than I am aware of. It is EMU48 rewritten for the iPhone. The authors of EMU48 are credited. Somehow I think the wireless WiFi transfers is where things get dicey.

Thank you!!!

Scott

[John H Meyers]

On 12/11/2012 6:46 PM, Scott Chapin wrote:

> My libraries so far do (extract?) as you say

> [from computer file strings beginning with "HPHP49-x"]

> leaving the form "Library NNN: EQLIB:Equation Data"

> except for Library 226 which when extracted left "External" on the stack.

>
> I assumed that was because it was already installed in the emulator, but maybe not?

I think it's because that file must be corrupt.

The only point where "already installed" should come into play
is when trying to store an already extracted library
into a port where it's already stored.

By the way, here's a third version of the (unsafe) UserRPL object extractor
(this time inserting a forgotten MEM DROP):

\<< \->STR IF DUP "HPHP4" POS 1 == MEM DROP THEN

#100002h LIBEVAL 26 + #100003h LIBEVAL NEWOB END \>> 'fixhp' STO

One of the things that you can do with the above
is to download the real (safer) FIXOB.49
and save it to the calculator (real and emulated):
<http://www.hpcalc.org/search.php?query=fixob>

FIXOB should have no problem extracting even your complete memory backup,
as noted in author Werner Huysegoms' comments (below) -- FIXOB returns
only the embedded object, but also cleverly arranges that if you do
DUP FIXOB on your original string, the original "duplicate string"
changes into a list in which the extracted object is also the first element,
and the second object is just a short "filler":

* This program will change an object of the form:
*
* 5 5 16 x y
* DOSTR lllll HPHP4.-. ob slack
* to
* 5 x 5 5 y+6 5
* DOLIST ob DOHSTR length ... SEMI
*
* and will do so in-place, so that the string is not limited to half the available memory.
* This means that all references to the original object will point to a list.
*
* The actual tests performed:
* .string dispatch
* .string starts with "HPHP4"?
* .is there an embedded object (prologue check)
* .can it be skipped?
* .is it not too large ie is it contained in the original string?

The above includes both HP48 and HP49/50 versions of FIXOB.

An older program by "H&H" (Horn & Heiskanen),
for HP48 series objects only, apparently handling
only up to 1/2 of free memory, can be found via
<http://www.hpcalc.org/search.php?query=fixit>

[r->] [OFF]

[Scott Chapin]

Thank you John.

I downloaded FIXOB.49 to my desktop and transferred it to m49+. It loaded to the screen as "HPHP49-X" and I extracted it using your "fixhp". Hopefully it's not corrupt. It appears to work, so it cannot be corrupt?

[John H Meyers]

On 12/12/2012 6:12 AM, Scott Chapin wrote:

> I downloaded FIXOB.49 to my desktop and transferred it to m49+.
> It loaded to the screen as "HPHP49-X"

That's the computer file's content,
the first 8 bytes being HP's way of "labeling" computer files
that are stored directly from internal memory of HP49/50 calculators.

m49+ evidently does not automatically recognize these files
when "loaded" to the emulated stack, as does EMU48,
apparently leaving it to you to manually "open the can"
and scoop out the object that was otherwise sealed inside,
and FIXOB is now your "can opener" for that task.

> and I extracted it using your "fixhp"

> Hopefully it's not corrupt. It appears to work.

It may appear on your stack like this
(where one special character is instead posted
as its numeric value \164):

External External
\164 3h Code \164 0h
External

That's how programs using internal SysRPL functions
(and machine language "Code" objects) may appear,
because the normal built-in "decompiler"
handles UserRPL only.

"External" is the UserRPL decompiler's way of saying
"I can't translate this into anything you can read."

Any attempt to "Edit" this program as if it were UserRPL
will produce either a syntax error
or a bunch of useless things on the stack.

If you have installed the "extable" library
(which contains names for a few thousand more locations in ROM)
and also "attached" the internal "Development" library that's in ROM,
you can then use that libary's SysRPL/ML decompiler and compiler
to view and/or edit that program, using its "lower level" language,
which looks like what you find in "source" file FIXOB.s
in the zip file which you downloaded from www.hpcalc.org:

::
CK1NoBlame
CK&DISPATCH0
str
CODE (this is executed for string input)
GOSBVL =SAVPTR
[...big snip here...]
GOVLNG =GETPTRLOOP
ENDCODE
ZERO (anything other than string input is left as-is)
NOP
;

To merely verify that the FIXOB program is not corrupted,
you can verify that applying commands BYTES HEX to it
results in a checksum of #3F75h and a size of 124.5 bytes.

Now that you have obtained and transferred FIXOB to your emulator,
and stored it into a variable of that name, you can purge the
UserRPL program 'fixhp' whose value was only so that you could
get FIXOB itself, which replaces 'fixhp' from now on.

The use of something very light and small,
just to pull something heavier and sturdier across,
is a common technique for starting to build a bridge.

On the evening of 6 August 1974, following six years of dreaming
and planning of what he called "le coup," a Frenchman named Philippe Petit
shot an arrow attached to a fishing line to begin such a process,
and then passed larger and larger ropes across the space
until he was able to pull a 450-pound steel cable across.

And then he did this:
<http://www.youtube.com/watch?v=mAZppPSbxxs> [Skip to 5:00 if you don't have time]
<http://www.youtube.com/watch?v=k3zZVQPaKKQ>
<http://en.wikipedia.org/wiki/Philippe_Petit>
<http://www.youtube.com/watch?v=8jov-HMaOPQ>

[r->] [OFF]

[Scott Chapin]

Thank you so much for everything John! I appreciate it. The video was amazing. I wonder why I don't recollect that happening at the time.