Author Topic: Claw, the open embedded development system  (Read 14812 times)

0 Members and 1 Guest are viewing this topic.

Offline benedikt.muessig

  • LV2 Member (Next: 40)
  • **
  • Posts: 35
  • Rating: +2/-0
    • View Profile
    • My homepage
Claw, the open embedded development system
« on: July 24, 2016, 07:27:47 am »

Introduction
For the last year, I have been working on Claw, a development system for embedded devices.
It has a very small RAM footprint and can run on systems with as few as 2 KB of RAM.
The VM's own memory footprint is tiny and most of the RAM remains to be used by the applications.
Claw will probably support multitasking in it's operating system releases and also support CEFS (Claw embedded read-only file system) and FAT32 for loading (and storing) files from and to.
The files are streamed and therefore can be as large as 4 GB (on 32 bit VM's) or 64 KB (per executable) on 16 bit systems. 64 bit is supported as well, though 4 and 8 bit is not.
Claw features and assembly language and an high-level language that will compile to CXE executables that Claw can execute. On any platform, as Claw executables are binary compatible!
So you focus on your code and write and compile it once. Claw will deal with libraries and device specific hardware! But this freedom is of course still limited by hardware factors (e.g. it needs a screen to display graphics, or processor speed and RAM size).

Memory
Claw is entirely stack-based and uses three main stacks. There is a call-stack which stores the source address every time you call another function. As Claw has file system support built-in you can of course call functions from libraries dynamically. Dynamic linking is performed by Claw automatically, so you don't need to worry about library versions. The next stack is the work stack. It stores the function arguments, is used for arithmetic and other instructions and is arbitrarily accessible. The third stack is the array pool which programs use to allocate parts of memory to store data to. New memory can be allocated at any time but as in a stack, only the last array can be deallocated at a time. There cannot be holes in the memory map.

Supported platforms
The architectures and systems that I will port Claw to include but are not limited to 32- and 64-bit Intel and AMD computers running Windows, Linux or BSD; AVR (especially Arduino Uno and Mega); MIPS (routers); ARM and XTensa (ESP8266). Somebody might want to do a port the TI-84+CE and I might do a port to the TI-84+ if I find time.

Online IDE
There is also a planned online IDE that includes an online compiler as well as a standalone, offline compiler and assembler that runs on Windows, Linux, and BSD. The assembler also works for Mac OSX.

Open source
Claw is an open source project and is on GitHub: https://github.com/muessigb
The chosen license is as permissive as it gets: The "new" / "revised" 3-clause BSD license.
This means Claw can be used free of charge, can be modified and used in closed source and commercial projects, as long as the name is not miss-used and I get credited appropriately.

Development status
I have completed most of the assembler so far and a good part of the VM. The compiler is just a draft so far, but will be started soon.  Most parts of Claw that are not yet made are already documented and planned but just not yet written.

Have fun!
« Last Edit: September 28, 2016, 11:13:04 am by benedikt.muessig »

Offline Sorunome

  • Fox Fox Fox Fox Fox Fox Fox!
  • Support Staff
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 7920
  • Rating: +374/-13
  • Derpy Hooves
    • View Profile
    • My website! (You might lose the game)
Re: Claw, the open embedded development system
« Reply #1 on: July 24, 2016, 07:53:38 am »
So this is like a custom language? How does it work? Does it compile to bytecode which gets interpreted?

THE GAME
Also, check out my website
If OmnomIRC is screwed up, blame me!
Click here to give me an internet!

Offline SpiroH

  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 729
  • Rating: +153/-23
    • View Profile
Re: Claw, the open embedded development system
« Reply #2 on: July 24, 2016, 09:24:38 am »
Not very clear to me either.

A  couple few questions:
1. How do/can you interface with the embedded system? GUI (eg, xml), High level language (eg. Java, C++), Assembly language, other?
2. What's the Claw timeline? When do you think you'll have a prototype ready that runs, say, on Windows or Linux or on a real embedded system OS.

I'm afraid this presentation is a bit difficult too general, abeit it seems interesting.

Offline benedikt.muessig

  • LV2 Member (Next: 40)
  • **
  • Posts: 35
  • Rating: +2/-0
    • View Profile
    • My homepage
Re: Claw, the open embedded development system
« Reply #3 on: July 24, 2016, 11:56:58 am »
So this is like a custom language? How does it work? Does it compile to bytecode which gets interpreted?
This is a three part system. There is an assembler or a planned compiler that will be used to generate bytecode. These are .cxe files that can be run in the Claw VM. It's similar to Java, where you write something once and run it on several systems without (much) modification. But Claw is targeted at embedded electronics such as Calculators, Arduinos, and similar electronics.
Code stays binary compatible between platforms, as Claw uses (kernel-) modules and libraries to implement native functionality and therefore does not rely on platform-specific code.
Claw is designed to completely try not having to worry about platform-specific things (too much).
E.g. graphics are plotted to a virtual canvas that translates to the actual screen. Vector graphics are preferred over normal bitmaps, as these will simply be blown up in squares.

It does get compiled to bytecode, yes, which is then run in a platform-specific virtual machine.

Not very clear to me either.

A  couple few questions:
1. How do/can you interface with the embedded system? GUI (eg, xml), High level language (eg. Java, C++), Assembly language, other?
2. What's the Claw timeline? When do you think you'll have a prototype ready that runs, say, on Windows or Linux or on a real embedded system OS.

I'm afraid this presentation is a bit difficult too general, abeit it seems interesting.
1. There are two custom languages. One being Claw HL which is a Lua like language and then there is Clawsemble which is just assembly instructions for the virtual machine.
Claw has some custom audio and image formats that allow writing graphics code once and actually running it everywhere. More about that later.
2. The Claw Assembler is quite far already and I plan finishing it that week. The VM is halfway done; arithmetic, stacks, array pool and call stack are mostly or fully implemented. Missing are still the jumps and some basic file loading. Also there are no modules yet (e.g. string, graphics, io, network, ...).

Offline SpiroH

  • LV8 Addict (Next: 1000)
  • ********
  • Posts: 729
  • Rating: +153/-23
    • View Profile
Re: Claw, the open embedded development system
« Reply #4 on: July 24, 2016, 05:40:13 pm »
Thank you and congrats. Now is a lot more clear! Keep it up an good luck 'cause it looks rather versatile, so to speak.
I'll be looking forward to give it a go, when it becomes available for testing.

Offline benedikt.muessig

  • LV2 Member (Next: 40)
  • **
  • Posts: 35
  • Rating: +2/-0
    • View Profile
    • My homepage
Re: Claw, the open embedded development system
« Reply #5 on: August 07, 2016, 05:02:56 am »
Thank you and congrats. Now is a lot more clear! Keep it up an good luck 'cause it looks rather versatile, so to speak.
I'll be looking forward to give it a go, when it becomes available for testing.
Thanks :)
Awesome to have you as a tester!

I am making progress. Fixed some bugs in the assembler.

Offline benedikt.muessig

  • LV2 Member (Next: 40)
  • **
  • Posts: 35
  • Rating: +2/-0
    • View Profile
    • My homepage
Re: Claw, the open embedded development system
« Reply #6 on: August 07, 2016, 06:52:20 pm »
Thank you and congrats. Now is a lot more clear! Keep it up an good luck 'cause it looks rather versatile, so to speak.
I'll be looking forward to give it a go, when it becomes available for testing.
I would love if you could test the mathematical preprocessor of Claw. Details follow:

So, I fixed a ton of bugs since the last post and added quite a few new features.
Also I finished the mathematical part of the preprocessor.

So I need your help now, by testing the robustness of the preprocessor.

The latest snapshot can be downloaded here:
http://claw.bmuessig.eu/dl/

It can be run on Linux by:
mono ./csm.exe ./Sample01.csm
or on Windows:
csm .\Sample01.csm

Please try breaking it in a mathematically and syntactically valid fashion. I would really appreciate that.

Have fun!
« Last Edit: September 24, 2016, 01:35:05 pm by benedikt.muessig »

Offline benedikt.muessig

  • LV2 Member (Next: 40)
  • **
  • Posts: 35
  • Rating: +2/-0
    • View Profile
    • My homepage
Re: Claw, the open embedded development system
« Reply #7 on: August 08, 2016, 06:14:45 pm »
I FINISHED THE PREPROCESSOR! ;D
It can do, expressions, hexadecimal escapes, defines, undefines, nested if's, ifdefs, elseifs, custom errors, file includes and more.
Also the error handling is totally finished and displays nicely formatted errors.
Feel free to test and download the version from here:
http://claw.bmuessig.eu/dl/clawsemble-16_aug_09-a.zip
« Last Edit: September 24, 2016, 01:35:23 pm by benedikt.muessig »

Offline benedikt.muessig

  • LV2 Member (Next: 40)
  • **
  • Posts: 35
  • Rating: +2/-0
    • View Profile
    • My homepage
Re: Claw, the open embedded development system
« Reply #8 on: August 10, 2016, 07:58:18 am »
I have finished the instruction list. A lot was already there but I have finally completed it.
Can you PLEASE have a look if I am missing anything and maybe suggest some improvements?

Here is the full instruction list:
https://docs.google.com/spreadsheets/d/1Tfi8z7maQLM3RUHfQpnS50gAkGyfiizkQ54TTZGl9e4/edit?usp=sharing


Also, here's a little Clawsemble preprocessor overview for all of you:

Let's start with the conditionals.
They can be nested as much as desired and each if can have an unlimited amount of elseif's/elseifdef's and elseifndef's.
There can only be one else which is optional and has to be the last element in the if block.
#if (expression)    is used to only process the enclosed code if the condition is met, or the number or string inside the parantheses is non-zero
#ifdef defininition    is similar to the above but the condition is met, if a variable with the name is defined
#ifndef definition    as above, but condition is met, if the variable is not defined
#elseif (expression)    is used to provide an alternative, if the previous condition(s) is/are not met
#elseifdef definition    as above, but the condition is if a variable with that name is defined
#elseifndef definition    as above, but the if the variable is not defined
#else    if the condition is not met and the elseif's don't met their conditions either
#endif    is used to close the if-block

Expressions are simple.
All C arithmetic operators are supported and are using the same symbol, except for modulo, which is // now.
Numbers can be written with +/- prefix to show whether they are positive or negative. Negative numbers do
not need to be enclosed in parantheses. You can easily write something like: (-1 + -2 -3 - -4) and you can
even omit the spaces and Clawsemble will still be able to deal with it correctly.
Using the negative sign to negate an expression is NOT supported though. Use *-1 to do so instead.
Expressions should always be enclosed in one set of parantheses ()
Hexadecimal numbers are prefixed with an $ABAB, characters are written like this %d.

Multiple files work too.
Just use #include "filename.csm", to include other source files. They are just inserted into your main program at the position
of the #include statement and have access to the same #defines as your main program.

You can define variables.
Use #define MY_VAR 1234, #define YOUR_VAR %d, #define HEXVAR $DEADBEEF or #define ANOTHER_VAR (123 + $1242 * 123) to define your own variables
that you can check later on with the #if statement and it's companions. #undefine is used to delete a variable from memory again.
Variables can also be redefined at any point by simply calling #define again.
Be careful with the names, as they are case-sensitive. Also, all exact word-occurences of the variable name in the program are
automatically replaced by the value that the variable has at that point of time. If it changes later on in the assemblation,
the value won't change again.

Custom errors are easy to use.
Let's say, you have some conditions that have to be met for a program to be assembled correctly and you would like to prevent
the user from such accidents then you can simply use the #error statement to abort the further assemblation of the program.
You can even display an error message, the result of an expression or the contents of a variable by passing it after the #error statement:
e.g.: #error ("The architecture " + ARCH + " is unfortunately not supported at the moment!")

Make it short.
Many shorthand forms are available to save you typing:
#error, #err
#define, #def
#include, #inc
#elseif, #elif
#elseifdef, #elifdef
#elseifndef, #elifndef

Comment your code.
You can use the ; character to comment out lines if you don't want them to be parsed.
Everything after the start of a comment until a newline character is ignored and will not be processed.
You can put the comment at the beginning or the end of a line. Multiline comments are not supported.
Comments can be disabled by putting a \ in front of the comment. Like this:    \;
The \ character can also be used to break single line codes into multiple ones:
e.g.: #define MY_COMPLEX_EXPRESSION \
(123 + 124 + 23230 + 12040 + ONE_OF_MY_VARS + 12332 * 124224 - 20 + \
1402 + $BEEF)


This tutorial will be finished later on, when more of the assembler is done and ready for primetime.

Offline benedikt.muessig

  • LV2 Member (Next: 40)
  • **
  • Posts: 35
  • Rating: +2/-0
    • View Profile
    • My homepage
Re: Claw, the open embedded development system
« Reply #9 on: August 21, 2016, 09:17:09 pm »
I have finally finished Claw's instruction set :)

You can find a list with all the instructions and their descriptions and parameters here:
https://github.com/bmuessig/Clawsemble/wiki/Instruction-set
« Last Edit: September 24, 2016, 01:35:50 pm by benedikt.muessig »

Offline Runer112

  • Moderator
  • LV11 Super Veteran (Next: 3000)
  • ***********
  • Posts: 2289
  • Rating: +639/-31
    • View Profile
Re: Claw, the open embedded development system
« Reply #10 on: August 22, 2016, 06:47:40 pm »
Looks pretty neat. However, I should point out that you've somehow lost all the plus signs (not just here, on git as well).

Are floats not part of the plan? I'd be slightly disappointed if that's the case. I recognize the pain of supporting them on hardware without an FPU (or even a good ALU), but any programs involving serious math or graphics really need them.

Offline benedikt.muessig

  • LV2 Member (Next: 40)
  • **
  • Posts: 35
  • Rating: +2/-0
    • View Profile
    • My homepage
Re: Claw, the open embedded development system
« Reply #11 on: August 22, 2016, 07:47:05 pm »
Looks pretty neat. However, I should point out that you've somehow lost all the plus signs (not just here, on git as well).

Are floats not part of the plan? I'd be slightly disappointed if that's the case. I recognize the pain of supporting them on hardware without an FPU (or even a good ALU), but any programs involving serious math or graphics really need them.
Thanks :)
Oh, and thank you for pointing that out. I have really not noticed that :)
EDIT: FIXED.

Floats are not going to be part of the default instruction set. Floats are not going to be part of the default extended instruction set. They might be something added to the extended instruction set at a later point, but they will be one thing, incredibly slow. Fixed-point however, will be supported by the default extended instruction set and it will certainly be a lot faster :)
This design limit was set early on, as really none of the platforms I am targeting have a dedicated float processor or float instructions. Most platforms are 16 or 32-bit microcontrollers or the 84+ which all use soft float. Using float for the main data type would be totally silly and adding more than one data type is making everything way too complex and RAM-consuming.
« Last Edit: August 23, 2016, 09:10:54 am by benedikt.muessig »

Offline Sorunome

  • Fox Fox Fox Fox Fox Fox Fox!
  • Support Staff
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 7920
  • Rating: +374/-13
  • Derpy Hooves
    • View Profile
    • My website! (You might lose the game)
Re: Claw, the open embedded development system
« Reply #12 on: August 26, 2016, 06:18:58 am »
Looks pretty neat. However, I should point out that you've somehow lost all the plus signs (not just here, on git as well).

Are floats not part of the plan? I'd be slightly disappointed if that's the case. I recognize the pain of supporting them on hardware without an FPU (or even a good ALU), but any programs involving serious math or graphics really need them.
Thanks :)
Oh, and thank you for pointing that out. I have really not noticed that :)
EDIT: FIXED.

Floats are not going to be part of the default instruction set. Floats are not going to be part of the default extended instruction set. They might be something added to the extended instruction set at a later point, but they will be one thing, incredibly slow. Fixed-point however, will be supported by the default extended instruction set and it will certainly be a lot faster :)
What kind of fixed point? 8.8? I'd definitally recommend adding floating point or something similar, even if it is slow, it's still way simpler than having to write the routines yourself.
Quote
This design limit was set early on, as really none of the platforms I am targeting have a dedicated float processor or float instructions. Most platforms are 16 or 32-bit microcontrollers or the 84+ which all use soft float. Using float for the main data type would be totally silly and adding more than one data type is making everything way too complex and RAM-consuming.
Yet people will probably still want to program things with floating point variables, and having slower ones in an extendended instruction set or something, like you mentioned, would allow that while trying to encourage the user to rather use like 8.8 fixed point.

THE GAME
Also, check out my website
If OmnomIRC is screwed up, blame me!
Click here to give me an internet!

Offline benedikt.muessig

  • LV2 Member (Next: 40)
  • **
  • Posts: 35
  • Rating: +2/-0
    • View Profile
    • My homepage
Re: Claw, the open embedded development system
« Reply #13 on: August 26, 2016, 09:01:19 am »
Wel I agree that it can be useful but it would be the most hacky thing imaginable.
Basically floats would be arrays within Claw (8 bit array that is 8 elements wide) and instructions that deal with them would require working on these arrays.
Yes, it can work, but it would be hacky as hell :P

Offline benedikt.muessig

  • LV2 Member (Next: 40)
  • **
  • Posts: 35
  • Rating: +2/-0
    • View Profile
    • My homepage
Re: Claw, the open embedded development system
« Reply #14 on: September 25, 2016, 06:12:59 am »
So I finally, after months of work, completed the Assembler  ;D
I would be glad if you would please give it a try ;)

The universal binary can be downloaded from here: http://data.bmuessig.eu/Projects/Clawsemble/alpha-160923a/
Despite being a .exe, it runs fine on Linux and Mac. Actually, I am developing this on Linux.
It requires Mono (or MS .NET FW) to run on all of them.