Omnimaga

Calculator Community => Other Calc-Related Projects and Ideas => TI Z80 => Topic started by: the_mad_joob on September 04, 2014, 09:49:26 pm

Title: HYBRID (8X+)
Post by: the_mad_joob on September 04, 2014, 09:49:26 pm
Hello everyone...
 
Alright, i'm trying to dedicate as much time as possible on that new project of mine, and decided to create this topic on the way (not to mention some of you were curious about it).
It will be something between an OS and a shell, which explains the name.
I intend to make it some kind of dev environement too, but that part is still a bit obscure.
The main idea is to be able to run large programs (at least 32K, but probably more), while still being able to use the TI-OS for what it is supposed to be, a calculator.
The hybrid environement will be independant from the TI-OS, so rom calls won't be available at all.
I'm creating a few basic routines, but maybe libraries will be handled, not sure yet.
When using hybrid, RAM will only be used as a temporary storage.
Basically, what it does is backing up the TI-OS RAM data in the archive, making pages 0 & 1 free to use.
It will be an old-school command line interface (yeah, we're in 2014, plz stop crying).
It will only recognize archived appvars, as executable or non-executable data, but i intend to use specific headers for file types.
When loaded, everything is unlocked (flash and RAM/ROM execution).
Will be compatible with all OS and models (basic 83+ excepted, due to too much hardware differences).
 
memory layout project (subject to changes) :
$0000>$3FFF : reserved for TI-OS (ROM page 0)
$4000>$7FFF : free space (initially RAM page 1) (programs loaded at $4000)
$8000>$BFFF : free space (initially RAM page 2)
$C000>$FFFF : free space , hybrid routines , hybrid data , hardware stack (initially RAM page 0)
 
What's done so far (without using any rom call, except jforcecmdnochar ofc) :
- starting loading interface
- recognizes, saves and displays model, total RAM, and total ROM
- overall archive check for corruption (missing swap sector, invalid sector status bytes, invalid variable status bytes, variables crossing sectors, zeros in unallocated space).
- detects corrupted stack & VAT.
- check if enough archive for TI-OS RAM data (the amount should vary but never exceed 32158 bytes).
- large system font, matching ASCII, ISO-8859-1, and CP-1252, with some dedicated chars for title bars.
- a few basic optimized standalone routines, using only registers as input, which will be callable from any program :
-- lcdres (lcd clearer)
-- lcdreswin (clears all but title bar)
-- lcdset (lcd filler)
-- lcdinv (lcd inverter)
-- lcdsave (printscreen, saving to custom location)
-- lcddisp (grbufcpy, restoring from custom location)
-- lcddispchr (putmap)
-- lcddispchri (putmap inverted)
-- lcddispstr (puts, with size to be put in a register, automatic wrapping)
-- lcddispstri (same, but inverted)
-- lcdinvchr (inverts an already displayed char)
-- lcdinvstr (inverts an already displayed string, automatic wrapping)
-- arcchk (archive space check for corruption, also returns the first page of apps area)
-- arcenough (check if enough contiguous free space in archive)
current used size in the app : 4833 bytes (font itself takes 2048 bytes).
 
Next step : some kind of smart garbage collector / defragmenter, which will only make space for the inputted amount, without defragmenting the whole archive to save the chip's life.
 
More to come...
Title: Re: HYBRID (8X+)
Post by: DJ Omnimaga on September 04, 2014, 11:07:02 pm
Seems pretty interesting. I am impressed by what is done so far. If you decided, however, to go for a full OS, then make sure it offers educational features that could convince someone to use this outside calculator gaming too, since people might need the calc in class. If it's a shell, just keep in mind it would be competing head-on against Doors CS7, Ion and MirageOS.

Good luck!
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 05, 2014, 12:45:55 am
Hello DJ, good to see the veteran =]

I'm definitely not ready for a full OS, and that's one of the reason i came with that idea.
Also, i think it's quite important to still be able to use the TI-OS, since many have lots of stuff running from there.
And about doors/ion/mirage, i don't intend to compete them, but if things go well, you should be able to run very fat programs with hybrid (maybe 40K+).
If it sees the light, it could be quite an interesting platform for running huge RPGs, for example.
Initially, i wanted to make it handle a custom page swapping system, allowing to run up to 65K programs.
But i think i'll see how things go for now, cause there's so much to code...
Title: Re: HYBRID (8X+)
Post by: DJ Omnimaga on September 05, 2014, 12:50:29 am
Lol actually it feels kinda weird to be called a veteran in an ASM project thread, considering I have never got into ASM and you did :P, but I guess it's more the fact I founded Omni or that I have been around for ages.

And thanks for the clarification. As for large programs, I believe that the max file size on a 83+ is 64 KB and it's possible that using files this large might be possible via Celtic III on the 83+, but you have to copy chunks of code into a separate program in order to do so, you can't run the entire program. Running programs this large from the shell, including maybe basic programs, would definitively be a big plus.
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 05, 2014, 05:13:58 pm
Alright, i'm gonna re-write my arcenough routine.

My goal is to make it work the following way :
- first pass : check for space avaiability, considering deleted variables as unavailable space
- if success : return page and location to store data (no defrag + Garbage Collect required)
- if failure :
- second pass : same, but this time, consider deleted variables as available space
- if success : defrag+GC only the concerned sector, then return page & location to store data
- if failure : return error

As you can see, there are 2 things i won't handle :
- defrag+GC in apps area : It is not always the case, but i will assume all apps are deleted from the TI-OS, which already does that job.
- overall defrag+GC in vars area : Doing it for all sectors will 1) abuse flash 2) be hard/long/boring to code/debug, and i need to move on.

Let's try that and see how it goes...
Title: Re: HYBRID (8X+)
Post by: Geekboy1011 on September 05, 2014, 05:14:51 pm
This sounds a lot like what calc84 does in tiboy and I think its an awesome idea :D

I probably wont use it my self but it will still be pretty cool to see what people can do with it. That is for certain.
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 05, 2014, 05:17:11 pm
He he, thx for sharing your opinions guyz =]
And yeah, i'm pretty sure many already thought of something like that...

EDIT : Just coded pass 2 of the new arcenough routine at work. Now time to code the sector defrag...
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 13, 2014, 01:39:40 pm
UPDATE
> Totally rewritten arcchk & arcenough routines (bug fixes, great speed/size ratio optim)
> arcenough pass 2 coded (can now find if a sector could handle the data after a defrag, and returns the sector 1st page in that case).
> Still working on the sector defrag algorithm. If i manage to make the routine light enough, i could run it in the hardware stack area, allowing to have 2 available banks (would remove the need to swap between read/write, speeding things quite a lot).

In the mean time, and for the curious, hybrid system font will look like this :

(http://i.imgur.com/JWHD5vd.png)
Title: Re: HYBRID (8X+)
Post by: DJ Omnimaga on September 13, 2014, 11:20:52 pm
They actually look pretty nice, especially the top ones which should fit well for menu titles and stuff. :)
Title: Re: HYBRID (8X+)
Post by: 123outerme on September 13, 2014, 11:24:51 pm
I hate to ask the obvious question, but is it compatible with the CSE?
Title: Re: HYBRID (8X+)
Post by: DJ Omnimaga on September 14, 2014, 12:10:42 am
Since this uses ASM, I doubt it (unless he decides to make a separate version)
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 14, 2014, 01:22:50 am
They actually look pretty nice, especially the top ones which should fit well for menu titles and stuff. :)

Thx for the feedback =]
Yes, i used the first two lines especially for that, since those chars were initially old ASCII control characters (non-displayable), not handled by modern text editors anyway.
I also added the black arrows and smileys, in the last unused slots.
But everything else is following ASCII/ISO-8859-1/CP-1252.
That means text data will match what you can see in a regular computer text editor.
Here is an example of a title bar usage (hybrid loading screen actually) :

(http://i.imgur.com/UwfVThI.gif)

I hate to ask the obvious question, but is it compatible with the CSE?

Well, hybrid works almost like an OS, in the way that it is totally independant from the TI-OS.
That basically means it will only run hybrid-compatible programs (you can forget about basic, TI-asm, ion, mirage, DCS, etc...).
On a side note, it doesn't destroy anything already existing on your calc.
In other words, you could for example have DCS and hybrid on the same calc without any problems.
Then, you would use DCS to run DCS-compatible programs, and hybrid for hybrid-compatible programs.
Hybrid will share the archive space with the TI-OS, but will only be able to recognize archived appvars as files.
Those appvars will first include the TI-OS standard header (for you to be able to eventually manage the files in the TI-OS, but also send/receive them with TI-connect), followed by a hybrid header (defining if the file is compatible with hybrid, if the data is executable or not, and stuff like that).
Title: Re: HYBRID (8X+)
Post by: DJ Omnimaga on September 14, 2014, 01:45:24 am
Yup, this definitively looks pretty good IMHO, from that screenshot. :)


Also looking at all this RAM, I wish we could use all of it for our daily calc uses and perhaps have the programs stored in archive all the time like on newer calcs. :P
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 14, 2014, 02:35:35 am
Also looking at all this RAM, I wish we could use all of it for our daily calc uses and perhaps have the programs stored in archive all the time like on newer calcs. :P

Had this project idea exactly for that =]
Title: Re: HYBRID (8X+)
Post by: DJ Omnimaga on September 14, 2014, 10:25:27 am
Ah ok but if it's a shell rather than a full OS, how will it be possible to access the entire 128 KB of RAM even from the TI-OS? Or will it just be restricted to games/programs running from inside the shell? In both cases you would have to make sure that no compatibility issues occur with programs that uses extra RAM, like TI-Boy SE.
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 15, 2014, 05:35:31 am
Ah ok but if it's a shell rather than a full OS, how will it be possible to access the entire 128 KB of RAM even from the TI-OS? Or will it just be restricted to games/programs running from inside the shell? In both cases you would have to make sure that no compatibility issues occur with programs that uses extra RAM, like TI-Boy SE.
When you run the app, all the TI-OS RAM data is backed up in the archive (stuff in pages 0, 1, and 2|3).
Then, page 1 in mapped in bank A, page 2 in B, and page 0 in C).
As far as i know, those pages are not used for permanent storage, except by the TI-OS.
By default, hybrid will not touch the other pages.
But, hybrid-compatible programs will still be free to use them if they need to (for temporary or permanent storage).
So bascially, like on the regular TI-OS environement, the program creator is responsible for compatibility conflicts (but also the user if he hasn't checked the program readme).
I initially had the idea to be able to use some other pages for program execution using a custom page swapping method, in order to be able to run 64K programs.
But i don't know if that will be possible (depending on how the project goes), but again, the user is free to decide to run such programs or not.

UPDATE
> re-coded a few routines : flasherase / flashwrite / flashcopy
> still working on sector defragmenting (once done, i can finally code the RAM backup part)
Title: Re: HYBRID (8X+)
Post by: chickendude on September 15, 2014, 07:54:24 am
This sounds really great! I wrote a reply a week ago but i guess it didn't get sent (China's GFW). Having all that RAM available would be great, i really don't like writing apps so having a huge space available for execution (and not having to mess with page swapping) sounds amazing :D

What's left to be done before it's ready to actually run/load programs?
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 15, 2014, 08:14:51 am
Yeah, looks like that reply got lost somewhere in the wordwide flow.
Anyway, thx a lot for your interest =]

What's left to do ? Well, let me see...
For the system to be operational, i need :
> finish the RAM backup section
> define which system RAM locations will be available to the user, move the data in the system RAM, and add the equates in hybrid.inc
> same thing for system rountines (a little reformating required for some of them, cause they are currently executed from the app)
> define the location and available size of the hardware stack
> code the kernel (should include the command line interface, program loading/return code, and hybrid exiting)
> code the RAM restore section (should be way easier than the backup, but it will include a VAT update which should be a bit hardcore)
I probably forget a few things...
Title: Re: HYBRID (8X+)
Post by: chickendude on September 15, 2014, 08:50:44 am
Ah, so that's quite a bit left to do then :P I thought the main bulk of it had already been done. What sort of things do you have planned for the CLI? I figured it was essentially a shell so all you'd really need to do would be to run programs. A CLI sounds a lot more complicated to write than a GUI :P
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 15, 2014, 09:34:48 am
Quite a lot to do, indeed...

Well, the CLI will be quite minimalist.
There won't be many features, just the essentials.
Mainly because i think the most important things should be done by the programs themselves (at least on a calculator).
Every command that you enter will cause the kernel to search the archive space for an appvar with that name, check if it contains hybrid-compatible executable data, verify the size, load the stuff @ $4000, and jump to it.
The commands history will scroll down, but there won't be any way to scroll it up (i want to avoid wasting some precious RAM, yeah, that's an excuse XD ).
But still, i intend to reserve 112 bytes for some kind of txtshadow, for when you exit a program.
You will be able to enter lower and uppercase letters, numbers, and that's all (following the appvars name formatting).
The input line length will probably be restricted to 1 line.
As you can see in one of the previous screenshots, the char @ (15,0) will be reserved for busy state, as well as current type mode.
The only available special key combinations will be [ENTER], [CLEAR] (clears the current line, or the whole window if the line is empty), and [2ND]+[MODE] (back to TI-OS).
After that, and due to how it works, special commands could simply be added by making tiny programs that you normally run (a bit like in .COM files in our good old DOS).
I had some projects about handling special programs, a bit like batchs, but i need to code a proper text editor first (not for tomorow).
Title: Re: HYBRID (8X+)
Post by: chickendude on September 16, 2014, 08:27:02 am
So basically typing "ls" will just run an AppVar named ls? I honestly think a GUI that scans all hybrid-compatible AppVars/programs ion-style would be easier, but i dunno. The CLI sounds nice though and is probably what a lot of people do anyway with so many programs on their calculators. I'm curious what sort of "special commands" you could have or that it'd be worth having. Can you pass arguments to them?
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 16, 2014, 01:09:13 pm
Don't worry, a CLI won't represent much trouble (i easily "see" the algorithm in my mind).
About arguments, that was just an old idea, in the middle of a many.
That would imply handling specific headers, and such new formatting could be annoying for the developper, which will already have to adapt to hybrid basic rules.
Instead, i could, later on, create a program that automatically handles data input/output such as parameters, file names, etc...
I will probably go for that solution, cause i'm sure parameters will complicate the CLI way too much for me XD
But i must say a batch system (without params, just to automate the run of multiple programs) sounds really appealing to me =P
Title: Re: HYBRID (8X+)
Post by: chickendude on September 17, 2014, 08:26:10 am
The batch system sounds essentially like NoShell, where you could just run programs straight from the homescreen. That would be nice, as would having autocompletion (perhaps pressing right would autocomplete the program) :D

I can't wait to see what this turns out to be :)
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 20, 2014, 01:39:18 pm
2 new routines, used by the kernel, but which will also be available to the developper who wishes to use the system CLI for input/output, for simple interactions with the user :
hybdispchr : displays a character at the current cursor location (scrolling handled). That one will actually be called every time you type something.
hybdispstr : same thing but for strings (wrapping+scrolling handled).
I will code hybnextline and hybclr tonight (easy), and the output part of the CLI will pretty much be ready.
More to come...

EDIT :
Alright, the output part is now done (so far : hybclr, hybclrin, hybnextline, hybdispchr, hybdispstr).
Despite those routines are initially for output, some are also designed to make the main input code easier (especially hybclrin & hybdispchr).
During the coding, i also had to decide which pointers i'll use.
Only 3 will be needed : the cursor (only one value), the start of the current input, and the end of it.
I made things so that the programmer should not need to adjust them.
The system is currently using a 144 bytes text shadow (128 + 16 spaces only used for scrolling).
Title: Re: HYBRID (8X+)
Post by: DJ Omnimaga on September 22, 2014, 01:54:27 pm
Will the utility have routines to make it easier to use sound inside games and perhaps Calcnet stuff?
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 23, 2014, 04:53:49 am
Will the utility have routines to make it easier to use sound inside games and perhaps Calcnet stuff?

Well, i don't intend to implement such routines in the early release, but that would sound great in the future, if it's possible =]
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 24, 2014, 07:11:53 am
I'm currently reformating some routines, again, but that definitely needs to be done now, while i still understand my own code XD
It's a bit of a headache, since i have to make them compatible both for the app and for the user.
I've decided to follow some kind of a scheme.
Reading ROM will always be performed in bank A, and writing in bank B.
That means i will have to run my defrag code from bank C during the app execution, which promises to be kinda tricky, since i don't want to touch any RAM not used by the system.
Indeed, that code will be too large to fit in the stack area, so i will probably do the following :
1) Backup the 1536 bytes used by mathprint (page 2 or 3 depending on calc) into savesscreen & appbackupscreen (appears to fit just right).
2) Map page 2|3 into bank C and use those 1536 bytes for my code, a bit of data, and a reallocated stack.
3) Restore stuff when done.
That will save many complicated page swaps and should definitely speed up the defrag job.
Let's try that...
Title: Re: HYBRID (8X+)
Post by: DJ Omnimaga on September 26, 2014, 12:33:32 am
Just make sure to not accidentally overwrite part of the Flash, especially the certificate or OS :P
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 26, 2014, 11:57:18 am
He he, of course.
My flash routines can only write to archive space =P
My LCD routines are safe too, in case the user inputs bad values potentially leading to test mode (lcd bleading).
Title: Re: HYBRID (8X+)
Post by: DJ Omnimaga on September 30, 2014, 11:55:47 am
Ah right, although I would say test mode is safer than erasing both the certificate and OS simultaneously. :P At least with test mode you just need to remove a battery fast enough and not trigger test mode too often in a short period of time.
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on September 30, 2014, 12:19:57 pm
Well, erasing the OS itself is no big deal, but the boot code would definitely be annoying.
Fortunately, it cannot directly be erased, but the certificate, however, can.

Anyway, such things shouldn't happen cause my flash write routines automatically return if you attempt to write anywhere else than bank A or B, as well as if the page swapped in is not included in the archive space reserved for variables (making it impossible to damage applications too).

But, since it will still be possible to jump after the implemented security, i'm thinking of putting those routines in a dedicated 1024 bytes area with execution restriction, that the system unlocks when needed.
That way, accidental jumps would be handled, except if you manually unlock that region first of course.
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on October 21, 2014, 06:07:10 pm
Small update...

Hybrid will basically use 2 types of routines :

1) routines located in bank C : code only executable from ram, but also code reading/writing huge amount of data (that could potentially be located on the 3 banks).

2) other routines, that will directly be executed from the app page (basically, routines using only registers as input/output, or dealing only with ports).
I just implemented a dedicated code to call those, that handles automatic page restoration.
It will be easily usable with a macro, defined in hybrid include file : hcall(xxxx).
Was a bit tricky, but i managed to make it alter nothing except the accumulator (had to modify a few routines actually using it).

On a side note, i can confirm that hybrid will allow to run code bigger than 32768 bytes (very probably more than 40K).
It should be (i hope) especially useful for low ram model users.

The way it currently goes, the app itself shouldn't exceed 1 page.

Also, i'm thinking of leaving the possibility to chose to backup the TI-OS RAM or not, at startup.
That way, users not using it at all could add extra life to their flash chip.
Chosing "no" would cause a RAM clear upon exiting hybrid, ofc.
It should particularly be appropriate for testing and debugging =]
Title: Re: HYBRID (8X+)
Post by: DJ Omnimaga on October 21, 2014, 06:56:20 pm
I'm glad that larger code can be ran. On 8x calcs it was annoying to be limited to 8 KB of code, although it has since been resolved (with TI making Flash APP development free then almost a decade later the many programs to remove the 8xp 8 KB of code restriction)
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on October 22, 2014, 05:21:55 am
Yep, already heard about such interesting solutions, like crabcake or similar.
However, hybrid will allow to run contiguous code on the 3 available banks.

Another good point is the usage of RAM for temporary storage.
Indeed, you will be able to use A LOT (not only those tiny ram locations provided by the TI-OS).
It will be as easy as just putting a label at the very end of your program, and refer to it to read/write data.
Such space will be as high as what you have left after the program has been loaded, without having to swap anything (128K RAM model users will still be able to use the additional 5 pages if necessary, of course).
Title: Re: HYBRID (8X+)
Post by: Matrefeytontias on October 22, 2014, 05:23:04 am
How is that done ? Do you use some sort of interrupt or writing hook to swap pages when necessary ?
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on October 22, 2014, 07:13:13 am
Well, just remember hybrid is 100% free from the TI-OS code and RAM.
That means TI-OS hooks aren't available at all.

$0000>$3FFF : unused (shame)
$4000>$7FFF : RAM page 1 - free to use for both execution & storage
$8000>$BFFF : RAM page 2 - free to use for both execution & storage
$C000>$? : RAM page 0 - free to use for both execution & storage
$?>$FFFF : RAM page 0 - some hybrid routines + hybrid data + stack

Inputting a command in the CLI causes the system to find the matching appvar in the archive space.
Then, its content is copied at $4000, and executed.
It can then freely cross banks a/b/c without having to worry about swapping at all (since it's contiguous RAM).
That basically means hybrid-compatible programs will start with a ".org $4000" directive, just like apps do.

About the system routines located in the app page :
I didn't want to mess up with IM2, to allow the user to use it without restrictions.
I will use the following code :
Code: [Select]
#define hcall(xxxx) call hybcall \ .dw xxxx ; available in hybrid.inc

hybcall ; This routine will be available in ram page 0, in order to be accessed from anywhere.
    ex (sp),hl
    ld a,(hl)
    ld (hybcall_id),a
    inc hl
    ld a,(hl)
    ld (hybcall_id+1),a
    inc hl
    ex (sp),hl
    in a,($06)
    ld (hybcall_backup),a
.db $3E ; ld a,
hybcall_page
.db 0 ; replaced by the app page during system initialization
    out ($06),a
.db $CD ; call
hybcall_id
.dw 0 ; updated by the code above
.db $3E ; ld a,
hybcall_backup
.db 0 ; updated by the code above
    out ($06),a
    ret
I know what you're thinking.
That sucks many cycles, but remember i don't want to use interrupts and must avoid destroying too many registers.
Don't hesitate to let me know if you see a faster way (only A must be altered).
Title: Re: HYBRID (8X+)
Post by: Matrefeytontias on October 22, 2014, 07:20:26 am
Well, just remember hybrid is 100% free from the TI-OS code and RAM.
Not quite true. At that point the TI-OS still clutters $0000-$3FFF.

Well, just remember hybrid is 100% free from the TI-OS code and RAM.
That means TI-OS hooks aren't available at all.
Of course I wasn't talking about TI-OS's hooks. I know the RAM chip has execution hooks, so I thought maybe it had read/write hooks too.

$0000>$3FFF : unused (shame)
$4000>$7FFF : RAM page 1 - free to use for both execution & storage
$8000>$BFFF : RAM page 2 - free to use for both execution & storage
$C000>$? : RAM page 0 - free to use for both execution & storage
$?>$FFFF : RAM page 0 - some hybrid routines + hybrid data + stack
You can't write to $4000-$7FFF, as it is not RAM, unless you map a RAM page to it, but in this case it won't be compatible with TI-83+ BE which only have 2 RAM pages for $8000-$FFFF.
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on October 22, 2014, 07:42:48 am
Not quite true. At that point the TI-OS still clutters $0000-$3FFF.
By "free", i was of course meaning that i don't use it, not that i removed it (i would have builded an OS in that case).

Of course I wasn't talking about TI-OS's hooks. I know the RAM chip has execution hooks, so I thought maybe it had read/write hooks too.
Oh, my bad.
I don't intend to implement any hooks for the moment (so much to do yet).
On a side note, some of the routines used by the CLI have already been coded so that the user can call them =]

You can't write to $4000-$7FFF, as it is not RAM, unless you map a RAM page to it, but in this case it won't be compatible with TI-83+ BE which only have 2 RAM pages for $8000-$FFFF.
Well, you can actually write to $4000>$7FFF, even if it's not RAM, but anyway, like i said, RAM page 1 will be mapped in this region by default.
And like i stated in my very first post, i don't intend to make hybrid compatible with 83+BE hardware at all (not enough time to handle those annoying hardware differences =P).
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on October 26, 2014, 12:01:38 am
Alright, finished the hard part of my sector defrag routine (the one that copies valid entries to the swap, and updates DAL/DAH/page on the way).
That was kinda hardcore, but it's finally done =]

Now, i need to :
- erase sector to defrag
- copy defragmented content from swap
- erase swap

Can't wait to see if my algo actually works, and how fast it is...
Title: Re: HYBRID (8X+)
Post by: TIfanx1999 on October 26, 2014, 09:08:57 pm
Just wanted to stop in and say that this looks pretty neat. Also, I'm rather curious why this won't work with the 83+BE. I didn't think the hardware was that much different between the 83+BE and the 83+SE/84+/84+SE. The only thing that really comes to mind is the slower processor in the 83+BE and the lack of crystal timers (and USB of course, but that shouldn't matter). *edit* Forgot to mention the lesser archive space of the 83+BE, but I don't think that matters either in regards to this project. However, IIRC the 83+BE also has less of the extra RAM pages, and I guess that could be an issue?
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on October 27, 2014, 05:47:38 am
Hello Art_of_camelot.

First, thx for your support =]

About the 83+BE :
The list of reasons is quite huge, in fact.
Most of all, the idea of this project was to take the maximum advantage of RAM for execution and storage, and with the 83+BE, the gain would be quite unsignificant compared to the TI-OS environement (a few K's more is no big deal).
But additionally :
- small archive space : Hybrid uses that area to save files (appvars), and too many defragmentations could potentially end up being annoying for the user.
- ports : Many are not present, especially port $23, which appears to be really useful to quickly get the start of the apps area (requires quite a heavy routine with the 83+BE, and implies saving the page number to RAM to save speed, which is way more dangerous if you think of it).
- Only 2 RAM pages changes things quite a lot in my code (which is using the first 3 by default).
- different keyboard delays (6MHZ CPU)
- no possibility to change the RAM page in the last bank without changing the memory map mode
- ...

All those changes would basically represent the need to create a separate app just for the 83+BE, because handling those in one app for all models would represent a significant "unoptimization", mainly due to many hardware checks.

Last reason : laziness, of course XD

EDIT :
Just finished the routine =]
Hybrid will officially be able to defragment a whole sector !
Considering the complexity of that routine, that's already a great personal achievement.
Now, let the debugging begin, which promises to be a pain if i f***ed up too much...
 
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on April 01, 2015, 08:01:51 am
After a total 1-and-a-half years break, looks like i decided to continue the project XD
Started to re-read what i had written like, a weak ago.
Hopefully, not much troubles about understanding myself.
Currently coding some keyboard routines.
I also intend to create some kind of custom APD feature.
More to come.

Peace bros.
Title: Re: HYBRID (8X+)
Post by: Sorunome on April 01, 2015, 02:12:04 pm
Wha, welcome back!
Also good luck on picking up your project again, it can be quite tough to pick up old projects.
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on April 09, 2015, 11:20:14 am
Basic keyboard input routines coded.
I'm currently working on an effective way to detect multiple keypresses.
I found out that it would be faster to actually "ask" for a specific combination, instead of scanning the whole keyboard each time.

I'm also trying to code "softpoweroff" & "softpoweron".
Using the HALT z80 instruction works like a charm, except that i need to have interrupts enabled to make it work.
That is a problem, cause i cannot use IM 1, since bank B won't hold what it is supposed to.
Currently, the only workaround i see is to use IM 2 instead, but that implies wasting 257 bytes just for that.

More to come...
Title: Re: HYBRID (8X+)
Post by: chickendude on April 11, 2015, 10:03:16 am
I was just wondering what was going on with this project, glad to see you working on it again!
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on April 15, 2015, 10:49:03 am
Wha, welcome back!
Also good luck on picking up your project again, it can be quite tough to pick up old projects.
Thx !
Especially with asm.
Fortunately, i did my best to document my own code enough =]

I was just wondering what was going on with this project, glad to see you working on it again!
Glad to see some interest =]
These days, i currently have my head straight on the CLI routines.
Also, i decided to let the user decide if he wants to backup the TI-OS RAM at startup, rather than making it automatic.
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on April 20, 2015, 06:40:44 am
CLI PROGRESS

After some long thoughts, i decided to make it ins/del only (no overstrike input).
It is gonna use a 128 chars buffer, similar to TI's textshadow, which will basically be used for refresh.
Vertical scrolling will definitely be handled.
However, the entries will only scroll up, not down (basically, no history access apart from what you can see on the screen).
Currently, about 50% of it has been coded.
Wasn't easy, as like most of what i write, everything must be usable from both the kernel and user programs.

done :

lcddisphyb (shadow display)
lcdinvcur (cursor blinking display, subroutine)
hyboutclr (clears everything except title bar)
hyboutnl (newline, automatic scrolling)
hyboutchr (putc, automatic scrolling)
hyboutstr (puts, automatic scrolling)
hybinclr (clears current input field or screen if empty, subroutine)
hybinchr (character insertion at current cursor, subroutine)
hybindel (character deletion at current cursor, subroutine)

in progress | to do :

poweroff (manual & automatic power off, subroutine)
hybpause (waits for ENTER)
hybchoice (custom arrow-based choice input)
hybprompt (effect very similar to when using the input token in a basic program)



ROM USAGE

The low-level part is already coded.
I will have the following main routines to code once i'm done with the CLI :

fileinfo (checks if an appvar exists, and returns the start of the actual data)
filenew
filedel



RAM USAGE

Definitely have some things left to figure out there.
About program execution space, i'm still hesitating between only 32768 bytes, or more.
About the TI-OS RAM backup, i am currently considering to allow the usage of extra RAM on 128K models, to reduce flash abuse.
Also, i'm already thinking of potential compatibility problems between versions...
Title: Re: HYBRID (8X+)
Post by: chickendude on April 21, 2015, 09:22:24 am
Nice work, i can't wait for the first betas to come out and play with them.
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on April 22, 2015, 03:36:13 pm
Thx chicky !

Successfully coded poweroff, hypause, and hybchoice.
The next one, hybprompt, definitely promises to give me some nice headaches XD
After that, the CLI input/output system will be complete.
I promise a few screenies at that point =]
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on April 27, 2015, 12:26:51 pm
(http://i.imgur.com/yBuxb0N.gif)
Title: Re: HYBRID (8X+)
Post by: TheMachine02 on April 28, 2015, 06:37:25 am
This screenshot does look really cool ! I definitly will play with this when there will be a beta out.
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on April 29, 2015, 02:33:07 am
Thx TheMachine02 =]
To tell the truth, i'm kinda proud to have a working CLI without having used a single byte from TI.
If everything goes well with my flash routines, i'm gonna code some little higher level archive interaction stuff soon.
More to come...
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on April 30, 2015, 11:41:11 pm
Well, it seems that i didn't have that much bugs in my flash routines.
I was expecting some hard debugging, since i barely tested them, 1 and a half year ago.
To give you an idea of the speed, here is a quick screenshot.
What the routine does is copying 65536 bytes from ROM to ROM, directly (no RAM buffer at all).
The busy cursor (upper right corner) is used as an indicator of the duration.
That is at 15MHZ minus the tiny slowdown from wabbit & imgur.
There you go :
(http://i.imgur.com/WVN0Z3T.gif)
Title: Re: HYBRID (8X+)
Post by: chickendude on May 06, 2015, 05:26:17 am
Wow, that looks great! It's cool to see some screens of it finally. I think the last screen we saw was of the character set :P
Title: Re: HYBRID (8X+)
Post by: c4ooo on May 07, 2015, 07:44:50 pm
I don't get it, so is  his a shell or os? ??? How can an os be "hybrid" and be in a the shape of an app for a different os? ???
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on May 07, 2015, 10:33:45 pm
I don't get it, so is  his a shell or os? ??? How can an os be "hybrid" and be in a the shape of an app for a different os? ???
Well, i decided to use that name because it shares the properties of both a shell & an OS.
Technically, it is executed from the original OS, like a shell, but once it's loaded, 100% of both executable & non-executable data is standalone.
That means it uses its own low level interactions with the hardware, and is free to use every single byte of RAM, just like an OS does.
Hybrid starts from the TI-OS, whereas an OS starts from the boot code. Apart from that, there is not so many differences.
Title: Re: HYBRID (8X+)
Post by: c4ooo on May 08, 2015, 07:03:05 am
I don't get it, so is  his a shell or os? ??? How can an os be "hybrid" and be in a the shape of an app for a different os? ???
Well, i decided to use that name because it shares the properties of both a shell & an OS.
Technically, it is executed from the original OS, like a shell, but once it's loaded, 100% of both executable & non-executable data is standalone.
That means it uses its own low level interactions with the hardware, and is free to use every single byte of RAM, just like an OS does.
Hybrid starts from the TI-OS, whereas an OS starts from the boot code. Apart from that, there is not so many differences.
Ok, the fact that it boots from the os explains it.
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on May 09, 2015, 12:39:18 am
A few things to keep those interested updated :

I'm still working on the high level archive routines.
Things are going pretty well so far, but i can say that 100% of my coding concentration is definitely required XD
The idea is to make things simple, like handling page swapping automatically, as well as using memory offsets as inputs (for example, 0 would mean the very first byte of the actual data of the file).

About the memory layout :
Since things are currently being more & more precise, i finally had to make the choices i needed to make.
RAM pages 0 & 1 will be reserved for program execution in banks 1 & 2 ($4000>$BFFF).
RAM page 2 will be reserved for the system in bank 3 ($C000>$FFFF).
1024 bytes will be reserved for the stack, which will be located at $FC00>$FFFF (SP initialized at $0000).
The other RAM pages will be usable for temp or permanent storage, but will require to be manually swapped.
So, basically, hybrid will handle 32768 bytes programs, which is still way better than under the TI-OS.

About the initial setup :
CPU at 15MHZ
Entire RAM executable
Flash unlocked by default, but don't be scared, hybrid routines cannot write anything outside of the archive area reserved for variables, and the probability to actually code a flash write sequence by mistake is so damn close to zero.

About limitations :
The hybrid environement will pretty much be a great solution on many aspects, but recommended for experienced coders.
No interaction with the TI-OS whatsoever.
No TI-OS ROM calls, only hybrid RAM calls (regular CALL instruction).
No RST instructions.
No interrupt mode 1, only mode 2.

About file formats :
As already mentioned, hybrid will be able to read/write appvars.
Executable code will require an additional header.
Things are not done yet, but i intend to be able to have multiple programs inside one appvar.
Haven't decided yet if a batch system will be handled or not.
I'm also thinking of making writeback possible.

What won't be available in the first release (for sure) :
Basically, there won't be any link/USB RAM calls, but who knows in the future...
Title: Re: HYBRID (8X+)
Post by: chickendude on May 10, 2015, 11:34:17 pm
It sounds like things are coming along really well. The only thing i really use bcalls for is to mess with the VAT (find files, appvars, etc.) and for hooks, i don't think most games will miss TI's routines.
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on May 12, 2015, 12:56:58 pm
It sounds like things are coming along really well. The only thing i really use bcalls for is to mess with the VAT (find files, appvars, etc.) and for hooks, i don't think most games will miss TI's routines.
It's good that you talk about the VAT, cause there are things i didn't mention.
Hybrid won't use any form of alloc table, only realtime archive space read/write.
It may be slower than using a table in RAM, but i really hate the idea to be restricted by RAM just for that.
BUT, things are not so easy, since i also want files created/deleted under hybrid to be handled properly under the TI-OS.
That means i have to keep track of the available RAM for VAT entries.
Also, before returning to the TI-OS, i have to actually rebuild a part of the VAT when restoring RAM, which will definitely represent some boring coding sessions.

PROGRESS :

I just finished coding/debugging "fileinfo" & "filedel".
"fileinfo" returns the following informations : file size, starting page & adress, and the same things for the actual data.
"filedel" handles the following error situations : missing/corrupted files, low batteries, and flash errors.
On my way to code "filenew", which promises to be quite tricky due to possible VAT interactions.

EDIT :

"filenew" coded successfully =]
It handles everything for the user, including sector defragmenting if necessary.
Time to code probably the last 2 user-available routines, "filewrite1" & "filewritex".
"filewritex" should definitely be hot, since source data can potentially start in a bank and end in another.
Hopefully, the low level code i'm gonna use automatically detects bank overflows =P
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on June 07, 2015, 04:13:56 pm
Alright, so, looks like wabbitemu is more a liar than i expected (had some funky surprises on real hardware).
I am currently recoding lots of stuff.
The good point is that i already updated around 75% of the source (mainly everything lcd/keyboard/GUI related).
Was a great opportunity to optimize here & there.

In the mean time, a quick overview of "hybchoice", a high level routine that will be available to coders :

(http://i.imgur.com/fKClR5S.gif)

What you see is easily performed with the following code :
Code: [Select]
...

ld hl,hybchoice_example
call hybchoice

...

hybchoice_example
.db 7 ; (7 lines total)
.db "choice ?        " ; (line 1 used for title)
.db " one            " ; (line 2 used for choice 1)
.db " two            " ; (line 3 used for choice 2)
.db " three          " ; (line 4 used for choice 3)
.db " four           " ; (line 5 used for choice 4)
.db " six            " ; (line 6 used for choice 6)
.db " five           " ; (line 7 used for choice 5)
.db 0 ; (separator)
.db 2 ; (choice 1 located line 2)
.db 3 ; (choice 2 located line 3)
.db 4 ; (choice 3 located line 4)
.db 5 ; (choice 4 located line 5)
.db 7 ; (choice 5 located line 7)
.db 6 ; (choice 6 located line 6)
.db 0 ; (separator)

What i actually do is pressing down 5 times, then up 5 times.
The routine supports 1-7 lines, but scrolls if necessary (the whole block is handled as a home entry).
Also, like "hybpause" & "hybprompt", it can be exited with ENTER or QUIT (a bit like Ctrl+C in DOS), and automatically handles key repeat and a 3 minutes APD.
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on April 23, 2019, 09:49:05 pm
Hello there, good old omni-folks =]
First of all, sorry for the lack of news.
Long story short, 4 years ago, the whole source (and something like 15 years of personal data) got properly nuked by a ransomware, all different backups i had.
Pretty recently, i found an unexpected clean backup on my WW1 laptop, not the most recent one though, but pretty close to it.
I'm currently in the process of crossing the data with some cheese i was able to recover from the crypted files.
I'll keep you updated on how far the project has a chance to resurrect, but it looks like there may be decent hope.
In the mean time, kisses !
Title: Re: HYBRID (8X+)
Post by: Xeda112358 on April 24, 2019, 08:39:44 am
Oh, great news! I've had a few big projects like that (lost and then years later found a place that had an earlier backup). Good luck; it's a lot of work trying to relearn your old code.
Title: Re: HYBRID (8X+)
Post by: the_mad_joob on April 25, 2019, 12:13:59 am
Oh, great news! I've had a few big projects like that (lost and then years later found a place that had an earlier backup). Good luck; it's a lot of work trying to relearn your old code.
I'm not there yet, but that part will come soon enough, i hope.

#####

Aight, just finished reconstructing everything i could.
The result is pretty encouraging, i have almost everything back, including the full 120KB latest source.
The only thing absolutely unrecoverable is a tiny text file that holded some data about the ti-os ram backup structure, but it looks possible to rebuild it from scratch without too much trouble.
More to come...
Title: Re: HYBRID (8X+)
Post by: TIfanx1999 on May 03, 2019, 08:27:23 pm
Hey there buddy, that really sucks about the data loss. I feel your pain on that. It's lucky that you had a fairly recent backup float around though. It's always nice to see old faces around, so welcome back!
/me waves