Omnimaga

Calculator Community => Major Community Projects => The Axe Parser Project => Topic started by: DrDnar on December 19, 2012, 12:31:20 pm

Title: [Axiom] Detokenize routine for Axe input command
Post by: DrDnar on December 19, 2012, 12:31:20 pm
As far as I know, nobody has yet written a routine to detokenize the output of the Axe input routine. I'm a bit surprised, because the routine is pretty easy to write. Below I have posted my implementation. It is rather large, because I just threw it together tonight and haven't optimized it at all. Maybe I will work on that tomorrow, maybe I won't. You could definitely shrink it by at least 10%---probably more like 30%---if you removed the local stack variables and did something more fancy.

UPDATE 14 January 2013: Fixed a bug in the Axiom, specifically a bug in the buffer overrun check. It was accidentally hard-coded for a fixed buffer size of 256 bytes because an arithmetic error. Also shaved two bytes off the routine.

Here's a sample program that just redisplays whatever you type in:
Code: [Select]
:.AA
:#Axiom(DETOKENI)
:input->A
:Detokenize(A,L1,128)
:Disp L1

Following is the readme.

====== Detokenize( =============================================================

    The Axe input command is small, but it returns data in the OS's tokenized
form.  As far as I know, there is no built-in system for limiting what the user
can enter.  The input command allocates a temporary variable in user memory to
hold the user's input.  This may or may not be an advantage for you.  The
temporary variable is a legitimate system variable, inasmuch as it has a VAT
entry and is length-prefixed.  However, the OS will automatically delete the
temporary variable for you when your program ends, so you do not need to worry
about leaking memory.

    The Detokenize( command uses the String>Eqn( token as its key.  The token is
located in the Catalog and is not in any other OS menu.  This routine is about
115 bytes.

    Detokenize( takes three arguments: The pointer returned by input, a pointer
to where you want to store your output, and the size of the output buffer.
Detokenize( does not return any status information.  If the detokenized string
cannot fit into the output buffer, the last full token that fit will be the last
token in the string.  The output is zero-terminated, which is what the Axe Disp
command expects.

    It is not safe to point make the output overwrite the input.  Many tokens
expand to more than one character, so the detokenized text could overwrite the
input token stream.

    Detokenize( can also be used to detokenize equations, strings, BASIC
programs, your Axe program's own source code, &c.


====== CleanAll() ==============================================================

    The input command creates a temporary variable in RAM.  This command uses
the CleanAll B_CALL to erase all temporary variables, including the one created
by input.  This is not necessary, but may be useful if you want to make sure the
RAM is free so you can do other things.

    The key for CleanAll() is ClearAllLists.  This routine is 3 bytes.


====== IsA2ByteTok( ============================================================

    This command is useful for parsing tokens yourself.  It returns 0 (false) if
the input byte is a one-byte token.  It returns 1 (true) if the input byte is
the first byte of a two-byte token.  The high byte of the input argument is
ignored.  This wraps the IsA2ByteTok B_CALL.

    The key for ISA2ByteTok( is Fpdf(, under DISTR.  This routine is 9 bytes.


====== Get_Tok_Strng( ==========================================================

    This command is useful for generating detokenized strings yourself.  It
calls the Get_Tok_Strng B_CALL to get the string for a particular token.  The
input is a POINTER to the token to get the string for.  Thus, if A points to the
token you are processing, you do Get_Tok_Strng(A) to get the string for A, not
Get_Tok_Strng({A}).  The string is stored into OP3 (address 848E in hex).  The
string is not zero-terminated.  Instead, this routine outputs the length of the
string.

    The key for Get_Tok_Strng( is Fcdf(, under DISTR.  This routine is 13 bytes.
Title: Re: [Axiom] Detokenize routine for Axe input command
Post by: stevon8ter on December 19, 2012, 12:37:20 pm
Nice. Is the source also in the .zip? (Can't view on ipod) for people who would want to optimise it themself...?
Title: Re: [Axiom] Detokenize routine for Axe input command
Post by: Streetwalrus on December 19, 2012, 12:46:55 pm
Pretty neat. I wonder if it is possible to detokenize with the Axe strings instead of the default, though.
Title: Re: [Axiom] Detokenize routine for Axe input command
Post by: DJ Omnimaga on December 19, 2012, 01:43:25 pm
I thought the Input command was removed over 2 years ago? O.O
Title: Re: [Axiom] Detokenize routine for Axe input command
Post by: Streetwalrus on December 19, 2012, 01:44:16 pm
Nope, it is still in there. ;)
Title: Re: [Axiom] Detokenize routine for Axe input command
Post by: Sorunome on December 19, 2012, 08:33:49 pm
I thought the Input command was removed over 2 years ago? O.O
I never used it, i tried it once but didn't figure it out, lol
Title: Re: [Axiom] Detokenize routine for Axe input command
Post by: DJ Omnimaga on December 20, 2012, 12:01:31 am
Apparently it was quite buggy and never fixed, so most people made their own. I think it was finally never removed in case some people decide to use it anyway.
Title: Re: [Axiom] Detokenize routine for Axe input command
Post by: DrDnar on December 20, 2012, 08:11:02 pm
Nice. Is the source also in the .zip? (Can't view on ipod) for people who would want to optimise it themself...?
Yes, it is. SPASM is the recommended assembler, but you could trivially substitute Brass, or with a little work use Mimas. You could probably even use TASM because there are no absolute jumps.

By the way, Runer pointed out to me that the input command isn't totally useless: it's actually just fine for getting the name of a program. You see, the tokens for the A-Z, theta, and the numbers 0-9 are the same as their regular ASCII values. So to validate the name, all you have to do is check that the length (which is stored before the token data itself, just like regular files) is less than 8 and that the only bytes in there are in the right range.

Also, here's a bonus trick:
The program-to-appvar converter I wrote for the runprgm Axiom takes its input from the Ans variable. Here's how to do it in Axe:
Code: [Select]
:Asm(EF524BD77EEE04EB2802ED62)->A
:!If A
:Disp "Error in input!"
:Return
:End
; More code here
; Assembly: b_call(_AnsName) \ rst 10h \ ld a, (hl) \ xor StrngObj \ ex de, hl \ jr z, $+4 \ sbc hl, hl
The above Asm( code returns 0 (false) if Ans is not a string. Otherwise, it returns a pointer to the length bytes of Ans, not the start of the data like Axe normally does. You can then detokenize the string by doing something like this (note the +2 to account for the length bytes):
Code: [Select]
:Detokenize(A+2,L1,256)Consider the following:
Code: [Select]
:.ANSDISP
:#Axiom(DETOKENI)
:Asm(EF524BD77EEE04EB2802ED62)->A
:!If A
:Disp "Error in input!"
:Return
:End
:Detokenize(A+2,L1,256)
:Disp L1
Code: [Select]
"Test string to
display!":Asm(pr
gmANSDISP)
Test string to d
isplay!     Done

EDIT 14 Jan 2013: Fixed a bug in the Asm( code. It would sometimes return 0xFFFF if Ans was not a string. Also added 8xps for super lazy people.
Title: Re: [Axiom] Detokenize routine for Axe input command
Post by: DrDnar on January 14, 2013, 10:40:52 pm
Fixed bugs in the sample program above and in the Axiom itself.
Title: Re: [Axiom] Detokenize routine for Axe input command
Post by: ClrDraw on October 22, 2013, 11:53:33 am
Nice. Does 2nd mode quit the program like the input command in basic?
Title: Re: [Axiom] Detokenize routine for Axe input command
Post by: DrDnar on October 25, 2013, 08:27:08 pm
I don't believe so, but honestly, it's been so long that I can't recall. You'll just have to try it.
Title: Re: [Axiom] Detokenize routine for Axe input command
Post by: nikitouzz on November 07, 2013, 08:38:01 am
If a few people are interessed by Input routine, you can see this : http://ourl.ca/19966 ;)