Omnimaga
Omnimaga => Our Projects => Ndless => Topic started by: tangrs on April 02, 2012, 01:33:00 am
-
This topic will be for technical discussion for the bFLT loader.
Source code and API documentation (https://github.com/tangrs/ndless-bflt-loader)
Let me start off this discussion.
I propose that, once bFLT becomes stable enough, it be integrated into the core Ndless code and have the SDK updated to only produce bFLT executables. However it seems that if bFLT were to be integrated into Ndless, we'll need to change some parts of the Ndless SDK itself to work in it's favor. I hope we can quickly deprecate the current way of loading binaries.
First, because bFLT stores sections in the order of .text, .data, .bss, we need to update the Ndless ldscript to reflect that. Otherwise, we run into overlapping problems. More information can be found at the bottom of the README (https://github.com/tangrs/ndless-bflt-loader/blob/master/README.md). This is the first and easiest step.
Next: Because we do all the relocating, the startup files become redundant - even a hindrance when we deal with shared libraries. The only thing the startup files should do is branch to main since the bFLT loader will handle everything else.
Also, we need some sort of way to work out the assignment of library IDs. bFLT shared libraries work by IDs. Each library is given a unique ID and it is up to the user to ensure it is unique. Unfortunately, the number of IDs are limited and only 254 are actually available for use (255 and 0 are reserved).
I'll leave it at that for now.
@edit: is there a way of implementing _syscallvar_savedlr without relying on a linker script?
@edit2: I'm also having trouble producing a bFLT executable that doesn't rely on nspire-gcc or nspire-ld. Anyone have luck with this?
I'm having trouble writing a ldscript that will force the GOT to be at the beginning of the data section. Got it.
Update:
The bFLT loader has a working SDK ready for developing and testing! You can now produce bFLT binaries without affecting your normal Ndless SDK!
-
This is great! :D I was thinking about porting bFLT a while ago, but lacked skills and time to do it. This will definitely advance Nspire development quite a bit! :D
Also, do you think there's a way to change that limit?
Reading this (http://www.ucdot.org/article.pl?sid=03/11/25/1126257), it says that the limit is "due to the way [he] encoded library addresses...can be changed but it requires quite a bit more effort".
I think the ID system should be changed if possible... it would be kinda weird to start assigning IDs, or having some kind of system manage IDs and such. I think having some kind of Java-style naming system, like "com.alberthrocks.myawesomelibrary", would be better. (And then call this bFLT2, Nspire edition :P) Just my 2 cents. ;)
Again, great work! :D This will open the platform to C++ code, and make porting ten-fold easier! :D
EDIT: Actually, bFLT has a version of 4.0 - I'm assuming you're using that spec, right? Also, might want to call the version 5.0 instead... :P
-
This is great! :D I was thinking about porting bFLT a while ago, but lacked skills and time to do it. This will definitely advance Nspire development quite a bit! :D
Also, do you think there's a way to change that limit?
Reading this (http://www.ucdot.org/article.pl?sid=03/11/25/1126257), it says that the limit is "due to the way [he] encoded library addresses...can be changed but it requires quite a bit more effort".
I think the ID system should be changed if possible... it would be kinda weird to start assigning IDs, or having some kind of system manage IDs and such. I think having some kind of Java-style naming system, like "com.alberthrocks.myawesomelibrary", would be better. (And then call this bFLT2, Nspire edition :P) Just my 2 cents. ;)
Again, great work! :D This will open the platform to C++ code, and make porting ten-fold easier! :D
There may be a way to change the ID limit but it requires modifying the linking tools and potentially breaking application support with other bFLT loaders (is this really an issue though?).
Shared libraries are implemented by having the 'ID' of the library in the high byte of the offset record in the relocations.
For example, if a offset record read 0x030000b4, it means "look for and load a library with ID of 3 and replace this offset with (the absolute address of that library image + 0xb4). That means the maximum ID number is 255 and the lowest 0 - giving us a total ID space of 256. The first and last are reserved so that only gives us 254 to work with. When you look at it this way, it becomes difficult to modify it to have more IDs because it's more of a fundamental issue.
You're right, I like the reverse domain notation too but it seems unlikely. I'm sure we can work out something though XD.
@edit: Yep, referring to version 4 XD
-
Great work tangrs!
I think this (and the other loader stuff), deserve a news here :)
-
Excellent! I'll test this as soon as possible.
-
Great to see more development tools. Sadly I cannot understand the use of this, since I'm pretty much illiterate regarding TI-Nspire ASM and C programming, but hopefully this helps a lot.
And yeah I think this and the Elf loader or whatever it is both deserve a news, although somebody else with more knowledge on the subject might have to write it so I can move it afterward.
-
At least when it comes to nSDL, having a working and stable loader would be absolutely great. It would right away open a lot of doors and remove the burden of tedious debugging. Newlib would work, giving access to a working libc and making porting a lot easier (math library and whatnot). SDL_image and other libs would work with quasi no changes and nSDL code would be considerably cleaner. Good job tangrs, this'll be a great plus.
-
Update:
Houston, we have a problem!
I've been working on getting shared libraries working and I've modified the toolchain slightly to produce shared libraries and have them link (almost!) correctly to binaries. I can also grab the address of the function of the shared library at runtime. Basically, the bFLT loader support for shared libraries is done.
But there's a problem. I noticed that the address the binary actually branches to doesn't match the address of the function. Whatever it's branching to, it's also crashing the calculator.
Turns out GCC inserts some code verneers (or bridge, if you like) that is supposed to "glue" the main binary to the shared library. This is bad for us because the address the code verneer directs us to is not relocated.
Here's a nice colorful diagram:
(http://i40.tinypic.com/1904s5.jpg)
Basically, calling a shared library function looks like this right now:
library_call() -> gcc code verneer -> unrelocated address = crash
We want GCC to skip making a code verneer so it looks more like this:
library_call() -> library function = win
Theoretically, we could work around this by defining and using a macro like this:
#define DIRECT_LIB_CALL(x, args...) do { \
typeof(x) * volatile tmp = x; \
tmp(#args); \
} while (0)
DIRECT_LIB_CALL(library_call);
That should bypass the code veneer but of course, this isn't very elegant nor portable.
All the code is in the shared-library-support branch of the main project.
If anyone can help me look for an option to disable the making of a code veneer (also known as code glue, code bridge etc...) when linking with shared libraries, that would be great.
Never mind, discovered a workaround (although quite annoying)
-
Looks like the PLT code ( http://stackoverflow.com/questions/5469274/what-does-plt-mean-here and others)...
Indeed, either we don't want that code at all, or the Ndless loader needs to be able to resolve this kind of stuff :(
EDIT: pages such as http://www.acsu.buffalo.edu/~charngda/elf.html or http://fossies.org/unix/misc/glibc-2.15.tar.gz:a/glibc-2.15/FAQ might contain some useful information.
-
Update!
Surprise!
Shared library support (https://github.com/tangrs/ndless-bflt-loader/tree/shared-library-support) is now stable!
Now, we need to work out a solution with the library ID problem. How do we allocate 254 IDs to the many people wishing to develop a library? Perhaps reserve IDs 3-10 for Ndless, IDs 10-250 for allocating to projects that are big enough and IDs 250-254 for private use? What do you guys think?
Apart from that, all that the bFLT loader needs now is a little polishing and integration into Ndless :)
(Don't you just love the look of all those "PASS"es? :P)
(http://i44.tinypic.com/dbm10.jpg)
-
Very nice :D
-
At least when it comes to nSDL, having a working and stable loader would be absolutely great. It would right away open a lot of doors and remove the burden of tedious debugging. Newlib would work, giving access to a working libc and making porting a lot easier (math library and whatnot). SDL_image and other libs would work with quasi no changes and nSDL code would be considerably cleaner. Good job tangrs, this'll be a great plus.
I assume nSDL is kinda like shells on the 84+ (MirageOS, Ion, etc), coming with routines to make development easier?
-
Update: Committed some experimental Ndless patches (https://github.com/tangrs/ndless-bflt-loader/tree/master/ndless-patch) for bFLT to the repo. The patches integrate bFLT loading directly into Ndless itself.
-
Great, this will speed up testing and integration further :)
BTW: you should update your local copy, and the patch, with Ndless 3.1 beta SVN r568 (or whatever the latest current version is). fixprint, for instance, requires r568 or later.
-
I think I'm on latest revision (r573 was it?). I think there was a typo in the file name. Let me fix it.
-
Ah, OK, false alarm then :)
-
I'm hoping to get this bFLT loader integrated into Ndless. Is there anything else that needs to be done for this to happen?
Update:
C++ toolchain is on its way! Very basic support is already working (new and delete works, function overriding works).
Source code is in a different branch at the moment. https://github.com/tangrs/ndless-bflt-loader/tree/c++-support
-
Any progress on this? :)
-
Any progress on this? :)
Been working on it on and off.
I'm trying to work out a efficient system for shared and dynamic libraries. Ater that, I'm also trying to get the C++ standard library and exceptions working.
-
For a second I thought you were replying to your own post somewhere but then I realized you deleted your double post and the above was by someone else.
Hopefully this continues being developed :)
-
I managed to build the toolchain on Windows with MSYS (after a lot of errors and hours). I've also patched Ndless (r632 instead of r573, because I have not found the r573 here (http://www.unsads.com/projects/nsptools/downloader/download/release/1)) and I successfully make(d) it.
But when I tried to run the "test" (here (https://github.com/tangrs/ndless-bflt-loader/tree/master/test)), the nspire_emu console give me :
bFLT: Begin loading
bFLT: Version number does not match
bFLT: Bad header
bFLT: Caught error - exiting
<EDIT>: The true error message is :
bFLT: Begin loading
bFLT: Magic number does not match
bFLT: Bad header
bFLT: Caught error - exiting
But I tried to change the magic number ("ELF" to "bFLT") of the .tns and I obtained the error message above.
</EDIT>
I don't know what to do, because the test is supposed to work... So if you have an explanation :)
totorigolo
Sorry for my English, I'm French :)
-
I managed to build the toolchain on Windows with MSYS (after a lot of errors and hours). I've also patched Ndless (r632 instead of r573, because I have not found the r573 here (http://www.unsads.com/projects/nsptools/downloader/download/release/1)) and I successfully make(d) it.
But when I tried to run the "test" (here (https://github.com/tangrs/ndless-bflt-loader/tree/master/test)), the nspire_emu console give me :
bFLT: Begin loading
bFLT: Version number does not match
bFLT: Bad header
bFLT: Caught error - exiting
<EDIT>: The true error message is :
bFLT: Begin loading
bFLT: Magic number does not match
bFLT: Bad header
bFLT: Caught error - exiting
But I tried to change the magic number ("ELF" to "bFLT") of the .tns and I obtained the error message above.
</EDIT>
I don't know what to do, because the test is supposed to work... So if you have an explanation :)
totorigolo
Sorry for my English, I'm French :)
It seems that you're just renaming the ELF file to bFLT format and changing the magic number. That won't work :)
You also need to download the elf2flt program and use that to convert your ELF to a bFLT file. Look in the Makefile of the test program (https://github.com/tangrs/ndless-bflt-loader/blob/master/test/Makefile) to see an example.
Information on how to set up a proper bFLT toolchain can be found at https://github.com/tangrs/ndless-bflt-toolchain (https://github.com/tangrs/ndless-bflt-toolchain). The toolchain contains scripts that handles all the linking commands for you so you can use it to produce a bFLT straight away.
-
Hi,
I am happy to say I got all of the example code working!
But I did run into an issue; and I'll see what I can find out but I figured I'd still post it in case others already seen the same thing.
So with the shared_library_test after I already got everything building and working with no problems I then tried to expand on it.
I wanted to make example_executable a nSDL executable. So I added another include at the top of main.c:
#include <SDL/SDL.h>
Then somewhere in the main function I added some code to utilize something in nSDL; I added this line:
SDL_Init(SDL_INIT_VIDEO);
The nspire-bflt-ld command was adjusted to have -lSDL and the error I am getting is:
ajorians@ajlaptop:~/Downloads/bflt/ndless-bflt-toolchain-master/shared_library_test/example_executable> makenspire-bflt-gcc -Wl,-R,../example_shared_lib/bin/lib3.so.tns.gdb -r -o test.bflt.tns.o
nspire-bflt-gcc -Wall -W -c main.c
main.c: In function ‘main’:
main.c:24:14: warning: unused parameter ‘argc’ [-Wunused-parameter]
main.c:24:26: warning: unused parameter ‘argv’ [-Wunused-parameter]
nspire-bflt-ld test.bflt.tns.o main.o ../example_shared_lib/bin/lib3_wraps.o -o test.bflt.tns -Wl,--wrap,library_call
/home/ajorians/bin/arm-none-eabi-gcc -Wl,-elf2flt -nostartfiles -T /home/ajorians/Downloads/ndless-v3.1-beta-r695-sdk/ndless/bin/../system/ldscript_bflt -L /home/ajorians/Downloads/ndless-v3.1-beta-r695-sdk/ndless/bin/../lib -static /home/ajorians/Downloads/ndless-v3.1-beta-r695-sdk/ndless/bin/../system/crt0_bflt.o /home/ajorians/Downloads/ndless-v3.1-beta-r695-sdk/ndless/bin/../system/osstub.o test.bflt.tns.o main.o ../example_shared_lib/bin/lib3_wraps.o -o test.bflt.tns -Wl,--wrap,library_call -lSDL -lnspireio -lndls
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_NONE unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_NONE unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_NONE unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_NONE unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_NONE unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_NONE unsupported in this context
ERROR: reloc type R_ARM_PREL31 unsupported in this context
ERROR: reloc type R_ARM_NONE unsupported in this context
48 bad relocs
collect2: ld returned 1 exit status
make: *** [test.bflt.tns] Error 1
I can change example_executable to be a nRGBLib or nspireio executable but nSDL executable gave the above errors.
I'll keep looking for other ways to do things and I'll let you guys know what I find! Let me know if you have any ideas or hints! :)
Thanks for your time!
-
The loader has actually been integrated to Ndless's trunk a while ago but I'm getting the same error when linking to nSDL. Without any solution I can't release the update.
-
Yes, during development I also got a lot of these errors. Unfortunately, I'm not exactly certain of why these errors occur. I suspect the elf2flt program is too old to handle the new kinds of relocations in gcc. I understand this is a bit of a pain for a lot of people so I'll probably end up either writing a custom program to do the conversions from ELF to bFLT when I have time.
Technically speaking, elf2flt is trying to handle most of the linking which is why it's giving all these errors. In reality, we can tell the linker to do it for us and add the --emit-relocs flag. This creates a section in the ELF file that has a list of relocations that the linker did. All we have to do is search for the R_ARM_ABS32 relocations and somehow add the base address to it at run time.
The program I am thinking of writing will call the linker to link it for us while adding the --emit-relocs flag. It will then dump the text and data section into a bFLT file and search through the list of relocs for R_ARM_ABS32 types and add them to the relocations list in the bFLT file.
Sorry for all this mess. The runtime linker might be working but the toolchain doesn't appear to :(
-
Thank you so much for your replies! Your answers were very helpful!
Have a great day!