Page 1 of 1

Text from disk to XMEM and vv using V41 and ILPer

PostPosted: Tue Dec 24, 2019 1:17 pm
by mike-stgt
I was looking for this subject but did not find more than the paragraph "Transferring Data Between Extended Memory and Mass Storage Devices" in 82180-90001 "XF/XM Owner's Mnl". Alas neither GETASA nor SAVEAS create a file on the target system, it remains user's task to create destination files with the appropriate size.
No problem, I did two routines for that:
Code: Select all
 05 RTN
 10 END

'RDFLAS' works as designed, but the complement 'WRFFLAS' does not, SAVEAS ends in a first test with 'END OF FL' and copied only part of the test file to disk.

The test I did virtually using V41 V9C and ILPer V2.31 (DosLink disabled). The "disk" I am reading is a LIF file dumped 15 years ago from an ancient disk. On it my file has 59 Regs, copied to XM to a file with 59 Regs it contains all I expect. The attempt to copy it to a new file with 59 Regs on a virtual disk fails.
So the problem is, I can't remember how I filed the data to disk with the correct file size decades ago. Should I create a larger file and resize it after SAVEAS?


Re: Text from disk to XMEM and vv using V41 and ILPer

PostPosted: Tue Dec 24, 2019 6:54 pm
by Garth
I can't believe I never ran into that, since I have used the ASCII files a lot. Apparently neither the 41cx nor the XIO module handle it directly, according to a quick look in the manuals for both. FLSIZE gives the number of registers not including the header. How much of the file in XMEM is getting left out when you copy the file to disc? Does it look like just incrementing FLSIZE's result before using CREATE will fix it? I have two tape drives but have not used them in many years. (I did use one a few years ago for my HP-71, and even after 25+ years, tape after tape, file after file, it all worked perfectly.)

Re: Text from disk to XMEM and vv using V41 and ILPer

PostPosted: Tue Dec 24, 2019 11:14 pm
by mike-stgt
First, my idea to use an oversized disk file and resize it after SAVEAS will not work, there is no resize command for disk files.

To your questions: The file in question has 59 Regs on XF/XMem. ASROOM shows 5 bytes left, so I do need the 59 registers for it. Saving it to a disk file of 59 Regs fails with 'END OF FL'. Reading it back to a XF/XMem file of 59 registers shows with ASROOM 29 bytes not occupied. The original file has 68 records, the crippled only 64.

The original file on a disk dump of a LIF disk shows 59 Regs. So I assume a discrepancy in using real hardware and its virtual remake.

It can't be my file's content. If I use an oversized disk file it is saved completely. Reading it back to XF/XMem I may resize it to the original 59 Regs. and it still contains all I expected. When I look with a hex-editor to the disk dump of the original and the new LIF file, the data structure is exactly the same -- at first glance. My file consists of 67 records of 5 bytes and one of 4 bytes. On disk the 5-byte records are padded to be even length so they take 8 bytes each (2 bytes for the record length), the single 4-byte record occupies 6 bytes on disk. This sums up to 542 bytes, padded with two FF (end markers?) gives 544 what is a multiple of 32, 17 times -- and the 'END OF FL'-crippled version is only 16 x 32 bytes, so exactly four 5-bytes entries are missing, 4 x 8 = 32. Aha!

No Aha! -- it still does not explain why the real machinery worked correctly. Currently I have only access to the virtual one, V41 and ILPer. (Yes, I know, I could also try Emu41 under DOSBox.)

Test: a one-byte record would take 4 bytes on disk, 128 of them would take 512 bytes (but only 37 register in XF/XMem). How much will be missed after SAVEAS? Result: one record. The FFFF end marker is inserted four bytes too early. And on real hardware?

A more blatant test. Create a new text file in XMem of 73 registers with CRFLAS. Fill it up with one-byte records. Wenn done RCLPT should return 254.001 indicating there are 255 records. Save this file to disk, set its designation in alpha and execute FLSIZE, CREATE, SAVEAS. Alas the last command fails when I use V41 and ILPer V2.31. Discard the file from XMem with PURFL, read it back from disk (designation may still be in alpha), execute FLLENG, CRFLAS, GETAS. Use ASROOM to see how much space is unused. Or read all records and get by RCLPT the count: 190.001 or 64 less than before.

Where are the LIF format experts? AFAIK, the noted "size" of text files does not specify the used space on the LIF disk. Other controlers just set the size entry to 0 and the text is as long as found in place. The text format on disk differs from the format in XMem, on disk it takes some bytes more. I wonder how the SAVEAS command manages the increased size when CREATE (what is also used for data files) uses a free slot which turns out to be too small for a text file. And the question is, how does ILPer manage the enlargement?

2nd Edit:
Looking into the traffic of virtual IL I have no evidence to point fingers, neihter towards V41 nor ILPer. Requesting a file of 73 registers CREATE clears 3 "swath" of 256 bytes what is about 50% more than 73 x 7 = 511 bytes. Alas the 255 ony-byte records require 255 x 4 = 1020 bytes on disk what is a bit more than the 768 CREATE 'prepared'.
The question remains, why I found in an old diskdump my text file with correctly noted "register size" containing all records. How comes I can't repeat this virtually?


Re: Text from disk to XMEM and vv using V41 and ILPer

PostPosted: Mon Jan 06, 2020 2:26 pm
by jeffcalc
Hi Mike,

(long answer, at least as long as Mike's question)

I'm not sure to have the right answer to your question, but may give some indications.
First, when you create a DATA file with CREATE, you specify a number of "registers".
Filling the file with HP41 registers is consistent with this size, you can fill up to the declared size.
However, the physical size of the DATA file is 8 bytes times the declared size, because each HP41 register is using 8 bytes in the DATA file, there is one unused byte (HP41 registers are 7-byte long, as you know:-).
For instance, a 59-register DATA file is 59*8=472 bytes long (using two 256-byte records on disc)

Now when you use a DATA file to store ASCII data, things are more complicate and there is no more a direct correspondence between the ASCII file size expressed in data registers and the ASCII data you can store in the file.
The size of the ASCII XMEM file (in HP41 reg.) is only *approximatively* the size of the DATA file you have to CREATE. The exact size needed depends of the length of the ASCII records.
BTW, the X-I/O manual states that FLLENG returns "undefined" values for ASCII files...

So back to your question: how did you create the 59-reg file long ago?
Since you can reload it in XMEM without loss of data, you should be able to rewrite it again, no?
You wrote:" My file consists of 67 records of 5 bytes and one of 4 bytes."
So it uses in XMEM: 67*6+5=407 bytes that is 407/7 = 59 XMEM reg. Ok this is what you reported.
But how much space does it use on the LIF file? As you noted, it uses more because of the 2-byte line length and the even length padding.
So it uses 67*8+6=542 bytes =542/8 = 68 DATA file "reg" (remember, DATA file "registers" are 8 byte long)
IT NEEDS a 68-reg DATA file ! It can't fit in a 59-reg DATA file on disc.
And so, the question is (again): how did you do years ago??

To answer, we must go to the LIF file system details:
First, LIF files are not expandable, once they are created they can't be made longer. Or in other words, a LIF file is always made of one fixed piece of contiguous sectors, that I will call "slot" below.
Second, when you create a LIF file, the catalog entry is build in the first free slot available (of size equal or larger than the requested size). If there is no such free slot, the file is just created in a new slot at the end of the catalog, with exactly the requested size. But if a free slot exists coming from a previously deleted file (and larger that your requested size), all the slot will be used and the unused area is no more available for other files, until the file is deleted and the slot is available again.
Third, the physical file size is recorded in the catalog entry in number of 256-byte sectors. The size in "registers" is for information only and is not used by the LIF system.

So my hypothesis is:
At the time, you created a 59-reg data file using by chance a larger slot (from a deleted file). The physical file size was then large enough for the ASCII data, even if the size in data register was recorded as 59 registers.

To prove it, please follow these few steps:
- start with an new LIF volume: NEWM 100
- create a 59-reg DATA file: "FILE1" 59 CREATE
- create a 68-reg DATA file: "FILE2" 68 CREATE
- create a 59-reg DATA file: "FILE3" 59 CREATE
- delete FILE2: "FILE2" PURGE
- create a 59-reg DATA file: "FILE4" 59 CREATE
Now, DIR says:
Now try to copy your XMEM ASCII file in FILE1, FILE3, FILE4 with SAVEAS.
I predict that it will fail with FILE1 and FILE3, but pass with FILE4.
FILE4 is using the 2nd slot, previously used by FILE2. Even if the size in "reg" is recorded as 59, it physically uses 3 sectors, large enough to hold the ASCII data.

Let me know if I'm right.


Re: Text from disk to XMEM and vv using V41 and ILPer

PostPosted: Tue Jan 07, 2020 1:08 am
by mike-stgt
Hello JF,
most respected Esquire!

Let me know if I'm right.

After boiling down your recipe: you are right, 5 stars for it.

It is a lengthy recipe with a detailed explanation but that helps a lot to understand what's up. Thank you for your time and effort.
Quite possible I once managed to write the file in the manner you describe, it is now about 35 years ago, alas I can't remember how I done it. At least I now have one working way, even it requests some well thought out strategy -- quite possible at that time.

Thank you