Omnimaga
Calculator Community => Other Calc-Related Projects and Ideas => HP Prime => Topic started by: Lionel Debroux on November 09, 2013, 04:20:26 pm
-
See the end of the post for download links.
Over the past few weeks, as mentioned at http://tiplanet.org/forum/viewtopic.php?f=55&t=13240 and http://ourl.ca/19769 , I've been working on a toolkit for communicating with HP Prime calculators, strongly inspired by libti* for TI calculators, and unsurprisingly dubbed libhpcalcs. It can be downloaded from https://github.com/debrouxl/hplp .
Thanks to critor's USB dumps and tests (EDIT: and later, other persons' tests), we believe the code now works well enough for being beta-tested used by a wider public (on Windows, Linux and MacOS X) :)
While neither critor, nor I, believe that anything can make a significant dent into TI's market share because TI is too entrenched and has lobbying power, we're nevertheless trying to do something with the Prime... as it's clear that if nobody bothers, the Prime won't stand a chance to get widely used (chicken-and-egg problem).
"Toolkit for communicating with HP calculators" means that it's a library backend for people to build GUIs on top of it, but libhpcalcs itself is a GUI-less library, and its terminal-based test program is currently the only UI for using libhpcalcs...
Excerpt from the README about the library's current capabilities:
The code base does:
* support communication with a single Prime calculator connected to the computer. The communication is known to work with the "SDKV0.30" firmware version from mid-August 2013, but might not work with other older or newer versions, if HP performs backwards-incompatible changes in the protocol (like TI does);
* implement six operations (without _known_ bugs): ready check, get calculator information, receive screenshot, send file, receive file, receive backup, set date and time;
* provide a terminal-based UI: the test program "test_hpcalcs".
The code base doesn't:
* perform color conversion on PNG screenshots (libhpcalcs should depend on libpng to uncompress the files, and recompress them after mangling);
* provide a way to select one of multiple HP calculators (probing USB devices);
* provide a user-friendly GUI for the above, but a GUI can definitely build upon the library easily enough;
* implement _proper_ conversion from UTF-16 LE to other charsets on its own: that's a task for libicu, Qt or other libraries;
What's next ?
* first of all, making sure that people are interested in the work in the first place. I'm clearly not going to keep spending most of my free time (as I did over the past week) on something people do not find useful ! [EDIT a couple weeks later: seems that some people are interested :)]
* fixing bugs which are bound to be reported, none are known right now but there certainly are some :)
* implementing several extra features from the TODO list embedded into the README, starting with probing devices;
* implementing a GUI for the program (with Qt, which has become the main choice for a portable, good-looking, fast UI toolkit and software abstraction layer)... but I don't plan on doing it by myself, although I'll obviously gladly support anyone trying to do so.
* implementing 39gII support ?
* certainly more... again, iff people are interested.
Go forth and test, thanks in advance ;)
Development source code: https://github.com/debrouxl/hplp (master branch is stable, master2 contains changes for testing, which may be rewritten before integration to master).
Ready-made install script: https://raw.github.com/debrouxl/hplp/master/install_hplp.sh . Unless your distro packages hidapi (Debian and derivatives do not), you'll have to compile it yourself (autotools-based program, so configure && make && sudo make install).
Latest Windows binaries + source tarball: http://tiplanet.org/beta/libhpcalcs-0.0.1-package.zip .
Also, I'm looking for 39gII users who can tolerate command-line tools, for providing me USB dumps (usbpcap) from their Windows computers when running the HP Connectivity Kit, the way critor did for the Prime.
[latest EDIT: 2013/11/16]
-
While neither critor, nor I, believe that anything can make a significant dent into TI's market share because TI is too entrenched and has lobbying power
Not to go too much off-topic nor to offend anybody, but in Critor's case, I always wondered if his negative opinion about the Prime was influenced more by the presence of a certain troublemaker in the HP community (although that person did really bad things such as real life threats towards TI-Planet admins, I feel it would be unfair if HP and the TI community that tries to expands its horizons were punished for the actions of one person). A lot of his comments about the Prime were not very positive, despite its potential, and I felt it wasn't only due to a certain bias towards TI.
Now back on-topic, I'm glad this project is coming along and that some people supports the calc. I will probably try to test it when I have some time, providing that it isn't command-line based, isn't a major PITA to install and doesn't require 2000 dependencies (as I am not very tech-savy, I don't want to spend two weeks trying to install a software like I did many times before, halting my progress on other projects x.x). I wouldn't mind giving you an USB dump of my HP 39gII at one point, though, so that you can add 39gII support.
By the way, the official 39gII connectivity kit or OS doesn't even support file transfers properly, because any program you download from the computer to your calculator will freeze the calc. <_<.
-
I will probably try to test it when I have some time, providing that it isn't command-line based,
test_hpcalcs is terminal-based, though interactive.
There's mild point in spending time developing a GUI for some code backend nobody cares about in the first place - and so far, hardly anybody cares about libhpcalcs: no reply on TI-Planet (where I posted a week ago), no reply on Cemetech, only your reply here, and a single beta-tester (besides critor) so far.
isn't a major PITA to install and doesn't require 2000 dependencies
Well, I provided ready-made Windows binaries, just unzip the ZIP file attached to my previous post ;)
libpng needs to be added as a dependency at some point... but only if the program has actual users. For now, that's not a given.
-
Yeah true, and just take a look at my 3D graphics topic, it was pretty much ignored since it was posted, with very few people checking and not bothering to reply. I don't know if it's because of site navigation issues (for example, there are no links to those sub-forums on the front page), but that is kinda worrying, especially considering that some of the people who checked the thread without replying already have an HP Prime.
-
Sounds pretty good to me. Since I'm on Linux and I plan on getting a Prime, it's a no brainer for me. I'm willing to help develop a GUI for it once it's matured, but a set of CLI progs in the titools style would be cool too.
-
Definitely, like libti*, libhpcalcs aims at being a backend for both CLI programs and GUI programs.
http://desowin.org/usbpcap/tour.html gives some information about capturing USB packets.
-
I have fixed several bugs reported on TI-Planet, and performed large changes on error codes, which I had started before the bug reports. The code base doesn't seem to be completely broken, according to critor's tests (upwards propagation of errors was already mostly done).
For now, until further testing, changes were pushed to a new "master2" branch. You need to pull and explicitly checkout the branch to be able to test the new changes.
New Windows binaries + source tarball (hopefully) attached.
Thanks in advance ;)
[EDIT: removing extraneous file, I attached it twice]
-
1: Check ready success
2: Get infos success
3: This one doesn't seem to work: No matter what I type as format choice ("png", "bmp","gif", or fully spelled out or with a dot at the start), It just says "Canceled" ???
4: Haven't tested yet since I don't know the syntax
5: Same
6: After one minute with nothing happening, this is what I got:
hpcalcs_calc_recv_backup failed
0 TODO human-readable error message
7: Not sure how this works, but I typed 1 as ID then it said prime_send_data succeeded
It might be good to include more documentation on how to use this, with examples, as I seemed to have absolutely no clue about what I am doing.
-
Thanks.
3: in fact, the calculator produces only PNG images, and one needs to enter 8, 9, 10 or 11 to get screenshots in two resolutions and color depths (all of which need unimplemented color conversion). But only the source code tells that story...
6: could you send me the "trace.txt" file which was created in the same folder ?
Lack of user feedback about operations which are longer than a second or two is a major utilisability no-no, but I haven't implemented any form of progress callbacks yet. When I do, the callback system will be different from that of upstream (EDIT: I mean libticalcs).
Finally, backuping a calculator is slow, due to HID being slow, I can't do anything about that.
In fact, test_hpcalcs is only a test client (derived from test_ticalcs_2), aimed at making small tests and showing how to use the library's functions. If it were to stay the only user-facing tool, it would badly need some documentation indeed. The library aims at being the backend for higher-level GUI and CLI tools; I just hope someone will take on the task of writing a GUI for a backend which largely works...
Offline, I have started implementing error messages and i18n, but I don't think i18n actually works yet, and I think I know why.
-
Ok thanks for the info. I'll e-mail you a copy of my trace.txt file now.
By the way, how long should it take to backup a calculator that has 400 KB of data? (Only a single program, which shows 63 KB, but I'm sure the program menu displays wrong sizes, since it's over 400 KB on the computer in unicode text form)
EDIT: Screenshots works (they are discolored as we know, but at least I got no error)
-
The trace shows that there have been a number of silent packet losses during transfer. Besides being fundamentally slow due to usage of HID and delays (even if seemingly still too fast for Windows' crappy USB stack on your computer - maybe some driver or anti-virus is interfering, as Tim Wessman doesn't witness slowness to the extent you do ?), the protocol used by HP isn't robust wrt. packet losses...
The computer sends increasing first bytes, but the calculator does not, so packet losses are harder to detect when the calculator is the sender (more precisely, it's basically impossible detect where data was lost, without more intimate knowledge of formats).
This means that I'll have to rewrite recv_backup functionality.
While triggering recv_file in a loop works most of the time, it falls apart if there are any packet losses. Receiving as much data as the calculator sends, then attempting to split it after the fact, would make it possible to salvage more data, though the heuristics might fail in very infrequent cases.
Another usability problem with the current recv_backup implementation is that in case of an error, it destroys any data received from the calculator before returning to the caller. Users would probably rather have partial, corrupted backups than nothing :)
To reduce the risks of your largest programs being lost as a result of a combination of transfer failure (induced by packet losses, statistically more likely on large programs) + calculator-side bugs (some of which require erasing stuff through the maintenance menus), you'll have no choice but to split them somehow. I don't know how.
-
Yeah I need to check if it's possible to use sub-programs on the Prime. I hope a solution is found, though, else if absolutely no large game can be sent, then this calc will be a failure programming-community-wise.
-
On a side note, I might try to get new batteries to put in my 39gII so if you need an USB dump and have good instructions on how to do it, I think I have three days off this weekend so maybe I could do one.
This is in response to http://www.hpmuseum.org/cgi-sys/cgiwrap/hpmuseum/forum.cgi?read=256650#256650 , but since I am unable to register there, I had to reply here.
-
The instructions that I know of are at http://desowin.org/usbpcap/tour.html . Only steps 1 and 2 are required on your side, step 3 is my job :)
Step 4 is an option, but then, you'd have to install Wireshark and produce files from Wireshark, which is more complicated.
Thanks.
-
Due to Lionel being now banned from Omnimaga, further comments about HPLP adressed to the author will have to be posted on TI-Planet in the matching topic:
http://tiplanet.org/forum/viewtopic.php?f=69&p=152761#p152761
Don't worry, all the team speaks english there too ;)
-
http://ourl.ca/19423;msg=307018
I think most people who have an HP Prime are on HP Museum forums it seems. It's hard to attract HP fans on a TI forum when there is an established HP-only board around, same problem we had with Casio people. Certain people might see TI boards as the "enemy" or something.
The two problems, though, with HP Museum, is that
1) Their most active contributor hates TI-Planet, Critor and the rest of the team with a passion, and he's banned from both Omni and TI-Planet, two sites which are affiliated together. You know who I am talking about. If we try to lure HP Museum people here, I am unsure what his reaction (via public posts and private messenging) would be, and I already notice that some people such as CompSystems and Tim Wessman hardly ever post on Omni anymore for unknown reasons.
2) The HP Museum registration system is absolutely restrictive in the way that any common e-mail address such as Gmail, Hotmail, Yahoo, etc, are not allowed to register. Not everyone can get an obscure e-mail in less than 5 minutes, and most people don't want to spend more than 1 minute to register on a website anyway. Why not use Facebook connect or StopForumSpam if spam is such a problem?
As for hacking the calc itself, the issue is that on Omnimaga, there are hardly any ARM assembly programmers anymore, which is required to hack the calc, and most people who are interested in HP Prime programming wants it primarily for the BASIC language (since they find high level languages more user-friendly). There might also be some doubts about the popularity of the calc, since it just missed the school start sales by 2 months.
That said I am absolutely unsure about why the sudden drop in interest towards the HP Prime in the last month or two from the people who used to post in this thread and other ones in this section. Is it due to people being busy with school? Do they just prefer to wait until next August when the calc will meet back-to-school 2014, to ensure that the calc will be popular enough to warrant efforts to hack it? Has anything else have happened behind the scenes that could have caused our HP fanbase to migrate to HP Museum or something?
Also I notice lately that every single Youtube video that TI-Planet or Critor3000 posted got at least one downrating.
I may miss a major part of the whole picture here - the author of the single most useful piece of software created for the HP Prime got banned because of some drama that happened on a different forum?
-
Not for the quoted drama.
It happened yesterday.
-
His ban is not related with HP calcs, but on something entierely different. He got banned here on Omnimaga because the drama has been brought here, that's why.
-
We're out of topic, but I thougt he was banned for being rude towards the admins (I can't be sure as there is no public annoucement). He's not the one who brought the drama here.
-
Then it seems I could figure out most of the drama. I'm surprised you (Critor) didn't get banned. Oh well at least here it's possible to find posts of the past, as opposed to the english-language alternative.
Here's the important part to get this back on topic:
Critor said:
Due to Lionel being now banned from Omnimaga, further comments about HPLP adressed to the author will have to be posted on TI-Planet in the matching topic:
http://tiplanet.org/forum/viewtopic.php?f=69&p=152761#p152761
-
Did I do something which could have deserved a ban?
The last time I did get an unfair restriction of my account (not a ban) was on Yaronet in 2008-2009.
I did leave the site and did never post any work on it again.
Today, the site has totally disappeared from the active TI scene.
Edit: reply received by PM
Ok, so back on topic on TI-Planet or HP-Museum, until better times:
Due to Lionel being now banned from Omnimaga, further comments about HPLP adressed to the author will have to be posted on TI-Planet in the matching topic:
http://tiplanet.org/forum/viewtopic.php?f=69&p=152761#p152761
-
We're out of topic, but I thougt he was banned for being rude towards the admins (I can't be sure as there is no public annoucement). He's not the one who brought the drama here.
You're right. I didn't said he did brought the drama here, but he contributed towards the drama here. (DJ did, actually.) Lionel's been rude, that's all I can say.
Now back on that wonderful topic (although it might be kinda more boring as the principal interested is banned for the week...)
-
Did I do something which could have deserved a ban?
I don't think so. I think the reason why both me and Lionel were banned is because we received multiple warnings in the past for our respective actions (in my case, bringing up the whole issue in public on the forums, which I did before, out of anger, with the Axe vs ASM/Grammer argument, and in Lionel's case, threatening admins to force them to run Omnimaga the way he wants). I don't think you ever received multiple warnings before and even if you did, your posts in the language war topic didn't seem as rude, not to mention you apologised. However, since this is supposed to be solved by now and that the Language Wars topic was locked for a reason, maybe it would be better to continue that discussion with the admins if you have any concerns.
Anyway, back on topic, I finally got batteries for my 39gII, but that calc is so incredibly unstable that creating an USB dump will most likely be much harder than I thought. It took me 40 minutes this weekend to get the connectivity kit to detect it, unplugging/plugging the cable over and over, same with restarting the software. To make matters worse, half of the time you plug or unplug an USB cable into the calc, it freezes, requiring you to pull all 4 batteries. We also even get crashes and freezes when trying to reset the calc with ON+F1 or when deleting programs from time to time.
However, I am unsure if it would even be worth the effort to implement 39gII support in libhpcalcs, as it seems that even HP has given up on this model. The HP 39gII seemed more like a base for the HP Prime OS and the last firmware update was from September 2012 or so. Considering that the HP Prime firmware fixes most HP 39gII bugs (even though a lot still remains, but the 39gII OS is so unstable that it makes the unnoficial HP Prime emulator from May look bug-free), if HP really intended to continue the 39gII, they would have released a new firmware long ago. Also, since the HP Prime was announced, I hardly ever see any post mentionning the 39gII on HP Museum nor Youtube, so it's possible that poor sales might have played a role as well.
So I would say only implement 39gII support if it's not a major PITA to add it besides the existing HP Prime support, else it might be better to wait until there's more demand, if ever.
-
Thank you DJ_Omnimaga. Yes, we're currently discussing this kind of things in order to handle the situation better next time.
-
Just to clarify: as I mentionned in the TI-Planet thread, no matter someone's programming skills, his status in the TI community (for example, admin of an affiliated site) and how long/much he has contributed to the TI community, the Omnimaga rules and punishments remains the same. No one get special treatment when it comes to forum rules here (I was banned too, after all).
I agree that it would have been nice for people to know the true ban reasons, though (including mine). Heck, I'm not even sure if anyone was aware that I myself was banned either. ???
EDIT: I see you got a warning the day I was banned. This confirms my thought about the new ban system (it takes multiple offenses in order to get a ban).
-
Just to clarify: as I mentionned in the TI-Planet thread, no matter someone's programming skills, his status in the TI community (for example, admin of an affiliated site) and how long/much he has contributed to the TI community, the Omnimaga rules and punishments remains the same. No one get special treatment when it comes to forum rules here (I was banned too, after all).
And this is quite problematic from my point of view. But this is not about getting special/better treatment at all - I've been totally misunderstood.
We'll go on discussing things and come back with a PM/email anyway if we don't get anything else before.
Whatever we do, it seems like we'll have to wait for Lionel before going back on topic... :P
-
On a side note, regarding the HP 39gII connectivity, what makes matters worse is that the official HP Prime connectivity kit from August no longer detects the 39gII at all. They totally broke compatibility. While you can have both 39gII and Prime kits installed simultaneously, installing the Prime one causes the HP 39gII firmware updater to stop running altogether. Since the HP Prime connectivity kit doesn't detect the 39gII, this basically means that you can no longer update your HP 39gII firmware until you uninstall the HP Prime connectivity kit then re-install the 39gII one! (and even then, will it work again?)