Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - Kristaba

Pages: [1] 2
1
Other Casio Calculators / Re: FiXos, a UNIX-like kernel for fx9860
« on: July 27, 2014, 05:50:41 pm »
Once again, some news of the kernel advancement ;)
Of course, the summer is not the better time to code everyday, but since my laptop has a great backlight, I can even enjoy the sun and write code at the same time!  [-.-]~

Main new features are :
  • Dynamic memory allocation, through sbrk() system call. This is not as powerfull as mmap() using anonymous area, but it's enough for malloc() implementation!
  • New device, /dev/keyboard, for low-level keyboard handling. It's an event-based device, which deal with raw key codes, allowing efficient way to implement IsKeyDown()/GetKey() in userspace. Of course, its more designed for graphical interface and games, console applications should deal with terminal input!
    It is possible to open this device with O_NONBLOCK flag to avoid waiting if no new keyboard event are present.
  • BSD-like sysctl() syscalls, to deal with kernel information from userspace.
    For now, available sysctls are mainly kernel/architecture information (kern.ostype, kern.osrelease, kern.osbuilddate, hw.machine, hw.model), and, more interesting, process information (kern.proc.pid.<pid> and kern.proc.all).
    Process related sysctls are the BSD way to look for process state from userspace (for ps/top and other utilities), instead of the /proc special filesystem present in Linux. For now, some data are still missing (process name and arguments...) but it is still possible to have a minimal top-like application ;)
  • Average process CPU usage is now calculated on a short period (around 1 second), in addition to total kernel ticks spent in kernel and user mode. It is never used internally, only designed to be accessed from userspace (using sysctl()).
  • A special kernel process (idle process) is now scheduled when no other process are running (it is using SuperH 'sleep' assembly instruction, so the power consumption is reduced when CPU is not needed 100% of the time).
  • Wait queues are now implemented, to allow a blocking syscall to put the corresponding process in a sleep state (only uninteruptible sleep is supported for now) until a specific event. For example, a process reading keyboard input will sleep until some characters are ready for it, instead of actively waiting for a character.
  • nanosleep() syscall implementation, to force a process to sleep some time. For now, this sleep is not interruptible, so a signal can't be handled until timeout.
  • stat() and fstat() syscalls!
  • getdents() syscall, used for listing directory entries (files and sub-directories). I think this will be the only way to list a directory content, and, as Linux and every BSD, the POSIX dirent.h functions will be part of the libc.
    The getdents() syscall is a bit tricky, and not really pleasant to use in a relatively high-level application. In the same time, the latest C standard do not define anything related to directory, so I can't rely on Newlib to have a nice interface.

Newlib should now be more or less functionnal now, I did some tests recently (mainly for malloc() and date/time related functions) and its working fine. I will try to make a new repository for the Newlib port, but in the same time I think Newlib is a good temporary solution (probably the fastest way to have a working libc/libm on any new system/architecture!), but it should be more interesting in the medium/long run to use Newlib sources but with a lot of custom changes in a lot of files.
As Newlib is designed to separate platform-specific code, and to never deal directly with the generic implementation, its probably better to don't keep the Newlib build system if changes are too much importants...


Some week ago, I tried to port a game from PierrotLL on FiXos (gravity duck for fx9860), mainly to see how difficult it is and to discover potential issues in the case of a "real-time" interactive application.
So I ported MonochromeLib in a first time (using the /dev/display device, for direct rendering), which work as well as a static library and as a dynamic library with acceptable performances (around 500 FPS for overwriting the whole screen and display it).
In a second time, some changes were needed in GravityDuck code itself for replacing Casio OS functions used (GetKey(), IsKeyDown() and some functions related to ticks ellapsed to limitate the frame rate). An other function (PrintMini()) was simply removed (a similar function should be added to MonochromeLib), but it is not a big deal as Gravity Duck do not use a lot of text... As the game makes use of malloc(), I also linked the Newlib libc, but I think it is the only libc function used.

The final result is really encouraging, the game run at normal speed, and everything seems to work as the original version!  :P


I didn't released the ported Gravity Duck for now because I did it for testing purpose, and everything is not really clean, but I will try to do that in order to show than it is still possible and relatively easy to write real applications for FiXos (less than a day for porting MonochromeLib and Gravity Duck, and less issue than I feared).


Is there any functions done to start a shell? I could do one if you want!
I think more or less everything needed is working, if you want to do that, but you will really need a libc if you want to have some helper functions, else you will have to deal only with syscalls.
Two things missing for a such project are the current/working directory (for now processes are not associated to a working directory, and every path need to be absolute for open(), stat() and similar syscalls), and maybe some ioctls and VT100-like escape codes for the TTYs. I will also need to implement process groups, sessions and controling terminal for a better UNIX concept support.

2
Other Casio Calculators / Re: FiXos, a UNIX-like kernel for fx9860
« on: July 03, 2014, 08:41:31 am »
I spend several days to write many proof-of-concepts around FiXos, and I finally added some intersting things to the kernel  ;)

First of all, a lot of work was done on dynamic libraries!
After some reflexion and studying existing implementations on several platforms and OS, I decided to implement shared object stuff in kernel. In most modern OS (linux for example), dynamic linker is a userland tool, and the kernel is agnostic about any dynamic linking.
As FiXos kernel already have some ELF facilities, it seemed easier and lighter to have a kernel-part implementation (so userland is simpler to understand and to maintain). I also implemented it in a way it's still possible to write a full userland dynamic linker, without breaking the kernel implementation (and it's even possible to disable any kernel dynamic linker related stuff at build time with a config option ^^).

The main problem I faced was than dynamic linking is deeply dependant of the toolchain target arch/OS, and the one I used is configured to target sh3 arch without operating system (and it makes no sense to support dynamic linking for unexisting OS).
So, the two choices I had were to do some changes in GCC's sources to target my OS, or to cheat a bit in order to make a part of the normal linker job "by hand". As changes in GCC is not a good idea for now (a lot of time waste to maintain and distribute it...), I did the second choice.
Mainly, to build a shared library for FiXos, the following is done (and automatized by some shell scripts and Makefile itself) :
  • Mark each "public API" function with special GCC's visibility attribute "default".
  • Compile each file with -fpic or -fPIC option (which make Position Independant Code, allowing to load the library at any arbitrary virtual address), and -fvisibility=hidden (which "hide" every symbol not explicitely marked as part of the public API in C code).
  • Link all resulting object files, the -shared option is used but seems to have no effect for sh*-elf toolchains (so the resulting file is an ELF executable file, but this make no real difference with an ELF shared object).
  • Extract every symbol marked as "GLOBAL" with visibility set to "DEFAULT" from the linked file. This should be the list of public API symbols, and it is used to achieve to distinct things :
    • strip every non-public symbol information to make the dynamic library lighter!
    • generate a "stub" assembly file for each public function
  • The stubs are designed to be staticaly linked to client executable (its a set of some instructions for dynamic symbol resolution, and the symbol name as a C string), and go to the .plt section of the executable.
    They are compiled and placed together in a single static library named "lib<name>-shared.a".
When a program should use a dynamic library, you just have to link it with the corresponding stubs library, and to call the functions exactly as there were normal, local ones.
The first time a dynamic-linked function is called (which, in fact, call the corresponding stub code), some complex job is done, including a specific system call to find the corresponding symbol by its name.
Then, all subsequent calls will be faster by using the cached value of this symbol (in fact the overhead is only 4 instructions, so it should not be a problem for performances).

The kernel ELF loader will only care about a special segment of the ELF (the one of type PT_SHLIB, defined by ELF specification but with no predefined semantics). It expects to find a list of zero-terminated strings, each one corresponding to the shared object name (the library name, with no path). For each name, it will try to find and load the content of the shared object. Once loaded to an arbitrary not used address range, the kernel try to "relocate" it (a vector of absolute-based addresses is located, and each element is incremented by the base address where the code was loaded).
Please note this segment is populated automaticaly (when at least one of the stubs for a given library is used, the library name is added to this section).

I'll have to write some documentation about all that in the project source tree, because it involves many complex things, and it's possible that my implementation fail for complex patterns.
In addition, it's only an early implementation (the shared memory is not implemented, only one library may be linked to a process...), but the more difficult part is done and seems to work fine for now!


An other cool thing added is a more powerful post-mortem debug information display.
A new function, kdebug_oops(), may be used when the kernel reaches an abnormal state (to not say "crash"  [-.-]~ ), which should display every potentially useful information (some CPU registers, current process related stuff like a memory map, and a call trace of the kernel stack).
   Unfortunately, SH3 calling convention does not allow a very good call trace without a lot of additional information (more or less all the "unwind" data needed for C++ exception handling...), so the current code only display the "guessed" calls, and may repport some false-positive :/

Even if it's usefull to have addresses of caller functions, having their names allow to don't spend much time to find it by looking at the disassembly dump of the kernel. So a bit tricky mechanism may be triggered by the config option CONFIG_DEBUG_SYMBOL_NAMES to extract all symbols located in ".text" section, and to include a table for "begin address" <-> "symbol name" conversion (it is possible to keep symbol names in ELF, but it takes more memory and its more difficult to read...).
Finally, this configuration option takes many space (currently around 30% of the kernel size...), but leads to a more understandable call traces, and so should be used for kernel development.

To make this more visual, here is an example of displayed message when an unaligned data is accessed in kernel mode :
Code: [Select]
> Address : 0x3FFFFFF3
> SPC Value = 0x880025F6
Fatal kernel oops!
(CPU Access Violation (R/W))
Following informations may help :
Running pid 1 (asid=0)
Memory Map :
DIR(0x10000000~0x10008000):
  vvuuuuuuuuuuuuuuuuuuuuuuuuuuuuuu
DIR(0x3FFF8000~0x40000000):
  uuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuv
(End of Memory Map)
Stack=0x880126F0
Call trace:
@0x880126F4: <_exception_handler + 272> (0x88008AC0)
@0x880126F8: <_printk + 0> (0x88006E78)
@0x88012708: <infloop + 0> (0x8800896A)
@0x88012710: <_sys_ioctl + 0> (0x88005DD8)
@0x88012714: <_interrupt_inhibit_all + 0> (0x88008680)
@0x8801271C: <_exception_handler + 0> (0x880089B0)
(End of call trace)

---- Previous Context ----
       PC: <_display_ioctl + 58> (0x880025F6)
       PR: <_exception_handler + 192> (0x88008A70)
Stack=0x8801277C
Call trace:
@0x88012780: <_exception_handler + 192> (0x88008A70)
@0x8801278C: <infloop + 0> (0x8800896A)
@0x880127A0: <_exception_handler + 0> (0x880089B0)
(End of call trace)

Of course, without knowing the kernel internals it's difficult to interpret, but in this case, the error was located in _display_ioctl(), itself called by _exception_handler(), which is displayed in the second context info.


Now, I try to add the needed syscalls to allow a fully functionnal port of Newlib.
Some work is done, and it is still possible to build Newlib for sh3-unknown-fixos target, but without a working sbrk() syscall, malloc() can't work and a large part of Newlib depends on it.
The last syscalls really needed are sbrk() and stat(), the others are either not essential for the most part of the library (symlink(), unlink(), chown()...) or implemented by using other syscalls (isatty() may be replaced by some ioctl() calls or fstat()).

3
Other Casio Calculators / Re: FiXos, a UNIX-like kernel for fx9860
« on: June 22, 2014, 01:49:50 pm »
Sorry I tl;dr'd the thread but I guess this is like KnightOS for the (more powerful) Casio calcs ?

In fact, for now, the name of the project is a lie (FiXos is only a kernel currently, but it should become more or less a "full" operating system someday). So it should look like KnightOS, but more focused on a "powerful" kernel, because the targeted CPU is far more complex (MMU, USB controler, SD Card, and dozen of other on-chip peripherals) and hardware is enough powerful to support a bit heavier kernel.

Just make sure that the OS plays nice with the calculator hardware, because with certain PRIZM add-ins, if you run OS 2.0 and do certain things with your calc, you can permanently brick it. (there's a topic about this on Cemetech with more details)

I read this topic, and unfortunately with the Casio boot design, with everything on the same EEPROM, I imagine there is small chance to brick the calculator when rebooting at the (very) worst time of an EEPROM write/erase operation.
For now I try to avoid playing too much with EEPROM, it will be necessary, but I prefer to have a stable kernel before to do that.
Of course, I'll do the best to avoid any problems, especially on other calcs than my ones (in fact I'm surprised to never broke my calc as I ran thousand times my kernel to debug it, with really not stables things sometimes), but I can't be 100% sure that will never happen (as Casio themselves apparently failed...).
Only thing I can say is that a buggy free software have a chance to be fixed if a such a thing ever happen, and a protected-mode kernel which run user applications in user-mode limits the possibles location of a such issue ;)

4
Other Casio Calculators / Re: FiXos, a UNIX-like kernel for fx9860
« on: June 21, 2014, 02:32:14 pm »
might be out of topic, but can anyone tell me where I can find the said GCC toolchain to biuld .g1a executables?

It's late to reply, but if anybody would like to have a sh3 GCC toolchain, as the SuperH architecture is not so common, the better way is to compile yourself GCC from sources. To avoid having to understand every details on GCC configuration, you can compile it through crosstool-NG (http://crosstool-ng.org/).
The only other tool than GCC binaries themself you need is a G1A wrapper tool, and you will be able to create G1A from a GCC toolchain ;)


About the FiXos, as usual with me, I stopped a time, and I'm working again on it from one or two months.
The kernel like more and more to a usable UNIX kernel, the main changes from the last time I posted here are :
  • Time management is now really better, kernel soft timers allow to defer kernel function execution, the 'real' time is maintained as time ellapsed since Epoch (read init time from RTC), and process kernel/user mode time ellapsed is counted. All this allow implementation of times() and gettimeofday() syscalls for user process.
  • Functional UNIX Signals (only a small subset are implemented, but the more difficult part is done and fully working).
    kill(), sigaction(), sigprocmask() syscall are usable, so it is possible to send signal to a user process from another process (or from kernel itself of course), to set signal handlers...
    Some interesting unhandleable signals are also usable (SIGSTOP and SIGCONT, SIGKILL... ^^)
  • UNIX Pipes support, this is the first pseudo-IPC in FiXos ;)
    It is now possible to create a pipe with pipe2() system call, and to use file descriptors to read/write data into the pipe.
  • Virtual Terminals! You may have several TTYs multiplexed on your screen (for now switching is done with F1~F6 keys, and only two TTYs are created with default settings, to minimize the need of memory).
    For now, sessions, process groups and TTY owning are not implemented, but user process can use any TTY by opening /dev/tty<n> device  ;) .
  • A proof-of-concept of direct screen rendering for userland is implemented as the "/dev/display device". This device implement only some ioctls :
    • one to map dedicated VRAM into user space (mmap() is not implemented for now in FiXos  :-[ )
    • an other to get display information such as size in pixel, VRAM size, color format used for the VRAM (this provides a platform-independant way to manage a screen)
    • a third for activating/deactivating the direct display rendering
    • a last to display the content of the VRAM on the real screen
    Using these ioctls allow to have graphical applications in userland, which is a first step to enable user friendly interface (but keyboard handling is only implemented in TTY, so its only the first step ^^).
  • Support for boot-time parameters (command line arguments for the kernel), as well in the bootloader and in the kernel itself. The first usable parameter is 'console={tty<virtual_terminal>|ttyUSB0}', to set the device to use for kernel console (a virtual terminal or the USB emulating a serial port).
    The USB serial output allow to use minicom-like (gtkterm for example), running on a computer host, as the kernel debug console, which is really more powerful than the tiny calc screen ;D ).
  • Better kernel critical sections, with possibility to disable kernel preemption, or all interruptions (hard critical section).
    A basic mutexes implementation was added, but not used for now.
  • Some important changes on the projet structure :
    • Now, all definitions which are used by both kernel and userland are separated from the source tree (that includes definition of types and constants  used in system calls, ioctl related stuff...)
    • Some work done to avoid useless recursive makefiles without making a unique unmaintenable one (if you are interested about why, you can find some interesting article like this one http://aegis.sourceforge.net/auug97.pdf on the internet).
    • There is now a way to set configuration options both for compile-time and build-time (inside makefiles), using a "config.h" file, in which you can enable/disable/customize feature by changing some macros. This file is then transformed into a Makefile include file, so build system and compilation share the same configuration values.
      I think I will try to make configuration easier with tools like Linux kernel's kconfig in the future (ncurse or GUI based tool to change configuration).
  • A "generic pool allocation" implementation, to make easy to have some dynamic allocation inside the kernel (this is done for fixed-size objects, by allocating physical memory pages as needed, and using them like memory pools http://en.wikipedia.org/wiki/Memory_pool ).

I think it's more or less all the important changes. If you were patient enough to read all of them, you can figure the kernel is now close to be mature to support a consequent user space, but some crutial things still missing.
The memory management is not enough powerful and will need big improvements (to support Copy-On-Write, shared area of memory, memory mapped files with mmap() syscall...).
The other really missing feature remains the shared library support. I know on some platforms it is done in user space (using mmap() and oter stuff), but I still plan to implement it in kernel space, maybe with a couple of non-POSIX syscalls.


Finaly, userland need a working libc, even before dynamic libraries implementation, in order to be really usable. Playing with syscalls is fun, but a libc is more efficient to make real programs...  [-.-]~
I'm considering to port Newlib for FiXos (Newlib is already working fine on SuperH bare-metal target, and is relatively easy to port on a new platform). I tried some month ago, but there was a problem with Newlib usage of automake, which is fixed now I hope...


I don't know if I will be active on this project during the summer, but I'll let you know if any progress is done.
And of course, if some experimented programmers are interested for working on FiXos, there is probably tons of things to do or to improve. I'm still interested to work on a port of FiXos for any other platform (Prizm or other SH4-based casio calculators, maybe m68k TI calc like TI-89, or even a x86/PC port could be fun ^^), because it will force a good kernel design to be able to separate arch/platform specific code from generic kernel code, and in the same time a portage often highlights hidden bug in a software.  ;)

5
Other Casio Calculators / Re: FiXos, a UNIX-like kernel for fx9860
« on: February 06, 2014, 03:20:15 pm »
DO you have any explanations for using GCC on Windows? I'm trying to compile GCC for sh3-elf target, but it's not the domain I'm at my best.
Sorry for the delay, but in my case I am not familiar with MS Windows, so I can't help you a lot.
I think the best idea is to cross-build GCC from Linux (even using a VM only for that), or with a full Cygwin install. If I have some time I'll try to build latest GCC toolchain (binutils and GCC binaries, libraries are not needed, even libc), but I think there are some binaries available somewhere, used for some Prizm toolchain, and should be able to build for SH3 big endian target.

Nice to see you still active in the Casio community. This project seems interesting,although I would probably not use it much, since I am not familiar with Linux-like stuff and mainly use my calc for games, but I hope this eventually come to fruition. Linux is now available on the TI-Nspire (although not installable as main OS) and some people managed to browse internet from it (while the calc is connected to an USB dongle)
Nice to see you still active in both communities too ;)

To make things clear, this project is far far away to be considered as a Linux-like. Even if I finish everything I would like to do, it will probably be closed to the 1990's System V than to a modern Linux or BSD ;D
TI-Nspire has a lot of ressources availables compared to fx9860 (Prizm is more interesting, but still realy light to imagine a Linux running on it).
For example, in TI89-titanium and TI-Nspire, there is a USB "on-the-go" controler, allowing to act as a usual computer with USB devices (Keyboard, WiFi dongle, USB pen or HDD...). On Casio's calculators, it's only the device part of the USB which is available (so the calculator may act as a USB device, but there is no way for it to deal with other devices, and for exemple it's not possible to use WiFi dongle to give internet capabilities).

In the other hand, as everything is written from scratch, it's not impossible to design some part of the kernel to allow games to run at a correct speed.
It's not my main goal, of course, but basicaly what is needed is a near-realtime scheduler (or a "single process" mode to avoid useless context-switches), and some realy specific device drivers to have a fast access to display rendering and keyboard input.
And a good kernel can allow game developpers to have better (and more reliable...) bases to code their software. Imagine at least a good filesystem access, memory swaping to allow to use more memory than available RAM (even if in theory, swaping in EEPROM is either fast nor a good idea for EEPROM lifetime  :ninja:).
In addition, something I realy want to do in a near future is shared library support, which can be also realy useful for games (lighter executables and possibility to update/fix libraries without compiling again).

Do you have some tasks I could do Kris?
It's not easy to say you a precise thing to do, mainly because I have no precise plan myself x.x
If you want to do some kernel functionalities, first try to read some piece of existing code to check a bit how the kernel is designed. After that, you can contact me using mail/pm, to find a better way to discuss a bit (irc, skype or something else).
A possible task should be to implement a serial device driver (using the real USART, not the USB emulating a serial device ^^).
It will be useful to both kernel (serial debugging) and user space (calc-to-calc comunication), and is a task which need to use the CPU's documentation, some plateform-specific code, and a device driver. So it's probably a good exercice to begin kernel programming  :D



Now, about the project advancement, I had not a lot of free time, but I did some important things last days.
In a first time, a realy better keyboard handling. Before there were only some functions inherited from my old projects, like is_key_down(), and was only used for debuging purpose.
Now there is a real implementation of keyboard, with 3 abstraction layers. The first is the 'keymatrix" level, to check the state of physical keys, and interpret state changes as keyboard pressed/released. These informations are used by the logical-keyboard layer, able to handle key modifiers (SHIFT/ALHPA and later a character case modifier), and when I will have the time to implement it, the key repetition parameters. Finaly, the higher layer is implemented in the /dev/console driver, to act as standard input stream (with DEL used as backspace for corrections).
The keyboard state is updated with a 64Hz interrupt, and later will use some pin state detection interrupt to minimize overhead when no key are pressed, but because of hardware configuration there is no "perfect way" to handle each key press/release whithout some timer interrupt usage.
Everything is not finished, but now there is a correct stdin for interractive user process. I will have to add ioctls to be able to change the input properties (at least ICANON and a raw mode, which are realy needed for ncurses-like usage).

In a second time, the first partialy working implementation of exec-like syscall (execve), one of the most important syscall on a UNIX-like system.
So it's now possible to begin to write some useful userland software (a bash-like use mainly fork/exec, as well as a userspace init program).

6
Other Casio Calculators / Re: FiXos, a UNIX-like kernel for fx9860
« on: January 20, 2014, 01:05:16 pm »
Do you think it could be easily ported to PRIZM? (Especially since it appears there are hardly any graphics functions yet)

I added some informations about that in README of the project, but in some words, it seems not so difficult to port it on Prizm (excepted for not documented at all parts, like filesystem, SD card...). I have no Prizm to work on that, and I don't remember exactly what is known or not about it's CPU.
If the documentation of the closest generic processor is correct for MMU, exception and interrupt handling, and some other registers, I think 80% of the job will be to write the CPU register definition header, and to change some constants values.
But if even for low-level things we have to deal with reverse engineering because of undocumented informations, it will be, of course, more difficult...

The arch-specific code is sometime used directly in generic code, so some changes have to be done before any portage, but if its targeting an other SuperH CPU it will not be a heavy task, I think/hope.

7
Other Casio Calculators / FiXos, a UNIX-like kernel for fx9860
« on: January 20, 2014, 12:42:40 pm »
Welcome back, Omnimaga!

I'm working on a litle project from a long time ago (but as I have to deal with IRL and my personal motivation, the project progress a bit randomly ;)), but I only released informations about it on Planète Casio forum.
Now, it's enought mature to begin to introduce this project on other communities, I think.

So I will not say here all I said on french Planète Casio topic, but I will try to summarize main informations, and of course to reply to any question if something is not clear.


FiXos is a kernel, writen from scratch to work on fx9860 and similar calculators. I would like to support all the SH3/SH4 models, but for now I prefer to fully support the only test model I have (fx9860G SD).
It's a UNIX-like kernel, which will try to support most important UNIX paradigms (device management, VFS, main syscalls...), but the goal is not to be 100% compatible with any POSIX version, only to be enought close for main usages.
Once the kernel is running, no more part of the Casio's OS is executed, and the kernel is fully independant of any other software.

For now, the kernel is never executed from the EEPROM directly!
It will not replace Casio's OS definitively, and it is copied on RAM before to be executed. So a simple reset of the calculator will allow to return to Casio's OS.
Later, it may be interesting to write it on a part of the EEPROM (to avoid heavy RAM usage before any user code is running), and maybe to redirect the Calculator reset code to the bootloader, to choose on which one to boot after a reset. But for now it's too much dangerous for nothing, so RAM is our best friend! ;D


From few days ago, the project includes an independant bootloader, which take the form of a G1A file. It has its own code to access efficiently to the SMEM filesystem, and use Casio's OS "syscalls" to display a menu. The whole bootloader is configurable using a config file, to change global properties (default selected entry, quiet mode or interactive mode), and individual entries content (kernel file, kernel arguments, type).
This booloader is able to prepare and run any ELF file, so the kernel is an ELF binary (with minimum informations to keep a small binary).

All the kernel, userspace tools and the bootloader are designed to be compiled by GCC toolchain for SuperH processors. It's also why the ELF file format was chosen : it's a very powerful binary format, and is now used on almost all UNIX-like and other OS. Any modern compiler (so, not the Renesas one...) is able to output ELF binary.


The kernel itself is still young, but begins to be interesting and partialy usable.
It takes advantage of the modernity of the SuperH processor, with virtual memory, separation of adress spaces between processes, and all embeded modules like SDHI for controling SD Card, USB device controler...

Main features for now are :
  • Virtual File System (VFS), which allow to use a uniform interface for any filesystem, and to mount any existing filesystem on any directory.
  • Casio's SMEM filesystem implementation, in read-only (based on reverse engineering, but hundred times faster than the Casio's OS implementation...).
  • ProtoFS, a minimal filesystem in RAM used as root filesystem, supporting creation of directory and device nodes.
  • Virtual Memory translation, depending to the current process address space.
  • Dynamic device management, using UNIX-like concepts (a device is registered dynamicaly to the kernel using a unique "major" number, and is free to implement any number of "minor", each one containing a functionality), which can be accessed in userland using filesystem device node with corresponding combination or major/minor numbers.
  • Early log function (wrapped on a printf-like function), targeting the T6K11 display driver, allowing to print some informations during first step of kernel boot, before any VFS and high-level devices are initialized.
  • More advanced terminal access with corresponding device, usable through /dev/console file.
  • USB device protocol abstraction, enought flexible to allow writing plateform-independant USB class implementation (and of course the implementation for 7705's USB controler)
  • Implementation of USB class "CDC/ACM", to use USB as a serial communication (accessed as a COM/ttyS on the host), which is used as a device, linked by /dev/serial. It's so possible to communicate in both way with a computer, without any additionnal driver (as CDC/ACM are reconized by main OSs), and for example to use hyperterminal-like (gtkterm works fine) to have a real VT100 screen and keyboard as calculator's terminal. :love:
  • Ability to run userland process (with their own independant address space and kernel stack), directly from any ELF file accessible from VFS.
  • Some syscalls already implemented for user->kernel context switch. open(), read(), write(), fork(), exit(), getpid(), getppid() and wait() are working, others are pending ;)
  • As fork() syscall indicate, a beginning of multi-process system, allowing many process to run, and a (temporary) basic implementation of a scheduler to select active process and give him some CPU time. The multi process code have only some days, and will change quickly to have a cleaner implementation (and a real scheduler, with timers to change active process)...
  • Some work was done on the SD Host Interface , to control directly SD Card, but it's a realy obscure part of the CPU (a 7720 controler used on the 7705 modifed version, and not documented at all even in 7720 documentation because of commercial secret and SD Association stupid rules). So it's a lot of reverse engineering on the Casio's OS code, and it's long to have something working. For now, it's possible to send/receive SD commands, initialize SD card, get meta-informations like size, volatages supported, manufacturer... but I probably missed something for data read/write, which is not working in my implementation. If I have some hour to spend to check everything, I plane to implement SD abstract block device, and ext3/fat32 filesystems.

If you want just to take a look on a user program code, you can read user/first_test_elf.c. Of course, it's a file used for testing purpose, so do not try to look for a concrete goal in this program. ;D


The project is hosted on Gitorious (https://www.gitorious.org/fixos for the project and https://www.gitorious.org/fixos/fixos for the main repository).
Of course, it's an open source project, anybody who would like to take a look to the code, to modify it for educationnal and curiosity purpose, or to contribute to the project is welcome.
(about it, sory for two things :
- my english mistakes in my messages and in my code or comments, I probably do a lot of stupid grammar mistakes, especialy in old comments)
- the quality of the kernel code is not always as clean as I would like to, sometimes I do some black magic usage, or quick implementation of something, before to clean the code later when I have a better idea or I am motivated again. So I try to maintain beautiful and clean code, but it's possible for you to burn your eyes when reading some functions and tricks used :ninja: )

I probably forgot some things, so if you have any question, I will modify this post if the reply is relevant to take place in this introduction.

8
Casio PRIZM / Re: Casio Prizm documentation
« on: February 20, 2011, 06:46:26 pm »
I wrote a little G3A to test the performances of the Prizm's screen :
http://www.2shared.com/file/AMshOlW5/fps.html
(@DJ_Omnimaga : this version works now ;D)

In a loop, my application fill the VRAM with a color, print the FPS, and call PutDisp_DD().
According to Light_Spirit, the FPS is about 21, I guess PutDisp_DD() is FPS-killer, but I don't know if we could write our own VRAMtoDD really faster.
Oh, and important detail : you need to reset your calc (soft reset, whithout data lost) to quit fps.g3a

9
Casio PRIZM / Re: Casio Prizm documentation
« on: February 20, 2011, 12:32:09 pm »
F. i. the RTC base-address h'FFFFFEC0 has been changed to h'A413FEDC.

Ok, so it look like the 7720/7721 architecture, probably without the DSP capability : http://am.renesas.com/products/mpumcu/superh/sh7700/sh7721/sh7721_root.jsp
However, I know the keyboard is connected to :
"input" (column selection I guess) from PTP0 to PTP6 (7 column, similar to the fx9860 keyboard)
"output" (row?) from PTM0 to PTM5 and from PTN0 to PTN5 (12 row, similar to the fx9860 keyboard too)
But the 7720 haven't port N, so it's probably customized by Casio.

See if you find something in the OS code using the informations in the 7720/7721 datasheet :
Ports are A, B, C, D, E, F, G, H, J, K, L, M, P, R, S, T, U, V (*no* port I and N)
Ports Control Registers start at H’A405 0100 (2 bytes per port, PMCR is H'A405 0116 and PPCR is H'A405 0118)
Ports Data Registers start at H'A405 0140 (2 bytes per port, PMDR is H'A405 0156 and PPCR is H'A405 0158)

Maybe in the Prizm CPU the port N is simply between M and P using the same starting addresses, something like :
PMDR : H'A405 0156
PNDR is H'A405 0158
PPDR is H'A405 015A

I'll try to test a code using that on the Prizm emulator if I have some time, and if I have a working IsKeyDown I'll post a new message here.

10
Casio PRIZM / Re: Casio Prizm documentation
« on: February 19, 2011, 08:06:49 am »
Impresive, you're a pretty good hacker Simon :)
I still have some questions, if you have time to answer them.
Firstly, what do you know about the G3A header checksum at the offset 0x16? Some guys say it's a CRC, but I tried a lot of CRC polynomials algorithms with different state of the file (only binaries data, only the header, with and without the other checksum, etc...) and I have no matches. So, if as I guess you know anything about it, please explain how it's computed (it's the last information I need for my G3A wrapper, I think).

Then, do you progressed about the keyboard mapping comprehension?
I wrote a FastGetKey for fx9860 (beacause the Casio's IsKeyDown was shitty and the Kucalc one was bugged), and I think it's something needed to do any good add-in (even if the GetKeyWait work well now :D).

And finally, I don't understand what the syscall you call "GetVRAMBackgroundAddress" is. There are two VRAM? So what is copied to DD when "VRAMtoDD" is called?

Thank you in advance for taking time to answer my questions, and, once more, good job!

11
Casio PRIZM / Re: Casio Prizm documentation
« on: February 16, 2011, 01:29:42 pm »
By the way, does anyone have a quick bit of code to disable virtual memory for add-ins?

I never played with the MMU yet, but I guess you have just to disable the MMU (through MMURC.AT bit, the last bit of the 32bit register at 0xFFFFFFE0). Set it to 0 and the MMU is disabled.
However, the Renesas documentation specify "Any program that modifies MMUCR should reside in the P1 or P2
area." (P1 is 0x80000000 to 0x9FFFFFFF address range and P2 is 0xA0000000 to 0xBFFFFFFF), so I don't know if we can easily do that...

Also, I don't understand why you want to disable the MMU. Without it, you can't know where the OS mapped your program or which range of the RAM you can safely use. And I don't see a lot of disadvantages in it

12
Casio PRIZM / Re: Casio Prizm documentation
« on: February 16, 2011, 12:56:22 pm »
I have the notion, that they expanded the customized part of the MPU. T. i. some ports they use, are not longer documented in Renesas documentations. This delays the meals. The first thing I want to know is how the keyboard is read. PORT A of the good old 7705 is not detectable any more.
Seriously?
What happened if you try to access to 0xA4000120? MMU error or you simply always read 0?
It would be annoying the Prizm processor isn't like a 7705, and I don't understand why Casio would do that :(
About the USB, I guess you speak about the USB protocol, but you probably can use the 7705 built-in USB features to send a ROM dump for instance, isn't it?

This is one of the projects I'd like to finish quick. At present I use the CASIO SDK assembler to program testcode. But C/C++ objects should work as well. I'd prefer C++. The result should be a G3A, of course.
Why did you use the old, buggy, and proprietary Renesas toolchain instead of GCC?
I think this is the moment to start an open SDK for the Prizm, as you wanted to do few years ago with fxsdk for fx9860...

13
Casio PRIZM / Re: Compiling stuff for Prizm
« on: February 16, 2011, 09:10:52 am »
Great! I can't really check this out now, since I don't have a Prizm yet... But it certainly does look nice. Is this mostly for assembly, or is it for C?

* In addin.ld change "rom  : o = 0x00300200" by "rom  : o = 0x00300000"
Shouldn't this be 0x00307000 on the Prizm?

Yeah, I know the problem because I haven't my one yet too :/
It's a fully GCC so you can compile a lot of languages, mainly asm and C, but also C++ (with some change to remove the runtime), Ada, Fortan, etc...

And no, it seems Prizm set the MMU to 0x300000 as the first code address (Andreas wrote this in Casio Kingdom same days ago)

Edit 02/16/11 :
First post updated

14
Casio PRIZM / Compiling stuff for Prizm
« on: February 15, 2011, 08:43:56 am »
Hey guys :)
I saw nobody here posted a real tutorial about how to build stuff for SH3 (and later for Prizm itself) :(

Simon Lothar and Andreas Bertheussen, two casio hacker, made some years ago a good reverse engineering of fx9860 calculator, and they wrote some tools
for, and some informations about how to use GCC to make G1A Addin :
http://sourceforge.net/apps/trac/fxsdk/wiki/UsingGCC

I advise you to subscribe and download the Kpit GNU tools (most recent and up-to-date) at http://www.kpitgnutools.com/index.php
They have a strange policy : they allow to download freely the code (because GCC is under GNU GPL license of course ::) ) but you need to subscribe to download the binaries, so I think it could be a good thing to distribute ourselves the binaries from their source in the future...

The link I give is about fx9860, so they are some thing to do before use this for the Prizm :
* In addin.ld change "rom  : o = 0x00300200" by "rom  : o = 0x00300000"
* crt.s need to be modified (we need to to do a reverse-engineering on the Prizm initilization code)
* a g3a wrapper

Edit :
Okay, so I disassembled Converter addin this morning (downloaded on casio.edu) to try to isolate the init stuff.
The complete disassembled sources (all conv.g3a), with only some labels to understand more the code (because 200 lines of assembly without any label to indicate constant values, loops and functions is really hard to read) is here http://pastebin.com/xT8XNAhH

I rewrote the code that seems to be the init stuff, and as I expected it's very similar to the fx9860 crt0.s written by SimLo and/or Andreas, but it seems call only _GLibAddinAplExecutionCheck after the ram is initialized, and immediately jump to _main...
So it's either the Prizm OS does a part of the initialization that was did by the addin code in fx9860 OS, or a part of the Converter code I thought it's the "_main" code is in fact init code.
Maybe Simon Lothar already isolate the init stuff, so, if you see this, please say what you think about that...
crt0.s : http://pastebin.com/ccE3NPhZ
my addin.ld : http://pastebin.com/YMrdDHWS

(and really sorry for my poor english ;D)

15
Casio PRIZM / Re: PRIZM Emu
« on: February 14, 2011, 03:53:41 pm »
You're true, but I don't know if you really bypass a lot of SH3 functionalities (for example, the Casio OS use the MMU, the interrupts (through the interrupt vector), a lot of system registers, and probably more than 90% of the opcodes, the only think not really used on it is the protected mode, but normally when an interrupt is triggered the CPU switch in privileged mode).
So, except for advanced built-in functionalities (as the UART or the USB internal driver), I don't see a lot of things you can avoid to do :(

Have you a general idea of your implementation that I could see? :P

Pages: [1] 2