New Developments in the Text-to-Speech Keyboard

Recently my brother, Nets Hawk Katz, sent me a new keyboard program for Bow to use on his touchscreen computer. He was learning a new programming language called GameMaker and so he used it to create a new program to put on Bow's touchscreen. Nets has made the different versions of the program available for download on his site. The source code is also available in case you are interested in modifying the program yourself, but to do so you would need to get GameMaker.

The first version of the program that Nets sent me was keyboard1.exe. I downloaded the program, installed it on Bow's computer, and then I said to Bow: "Uncle Nets has sent you a new program to use on your computer. Would you like to try it?"

Bow did not answer the question directly. Instead he spelled: "Uncle Nets is good."

I took this as a positive response, and I let Bow play with the program. Here is some footage of that:

This was on July 8. Though Bow was pleased to be able to play with the program, his use of it was not especially productive. I was not able to open it for him in full screen, so it was a small window superimposed on the desktop. Bow could close the window and open other programs, and this disrupted any communicative use of the program. He had fun playing with his computer, but it was not being used as intended. Other issues were that the delay between writing and then hearing what was written pronounced seemed too long.

Nets then sent us the next version of the program: Keyboard2.exe. The delay was changed, noises were made every time a key was pressed and a transcript file was added. Nets wrote: "The keyboard writes a transcript in a file transcript.txt in the same directory where you place the executable. (This transcript contains blank lines every time the timer runs out when nothing has been written.) You don't have to remove the transcript file every time you use the keyboard. The program simply appends additional lines to the end. "


I went and installed the new version of the program as soon as I received it, but Bow did not want to try it because it was late afternoon and nearly dinner time, and after dinner he was too content and exhausted to do anything more than lie around and chew his cud. Here is some footage of me playing with second version of the keyboard, without Bow:

Nets told me I was confusing "maximize" with "full screen." I would normally get full screen by pressing F4, but that was hard to do with a touchscreen and no keyboard. I had a keyboard I could attach, but it was not in the pen. There is also a virtual keyboard as part of the operating system of the touchscreen, but I had trouble finding it. Before I found another way to resolve this issue, Nets had already written another version of the keyboard to take care of it: Keyboard3.exe. It allows for an irrevocable full screen option. Once in full screen, no way back out, other than closing the window.

Here is some footage of our use of Keyboard3.

Under Keyboard3, here are some things that Bow typed:

IJ

EKFEKUP PO

UUU

NNNNNUZ

Arguably, the last one could have been a reference to Nets, as his name is spelled NZ in Hebrew. But that might be stretching things. Bow did not seem to be using the keyboard to communicate. He was just playing. 

The keyboard is for English words. Bow and I do not speak English with each other. Hebrew text to speech is very complicated, so this keyboard is meant to be used by Bow to communicate with English speakers, such as Lawrence. By the time Lawrence came over on Wednesday to spend the day with Bow, Nets had created a new version of the keyboard: Keyboard4.exe. There was no more x for closing the window. 

I was going to leave Bow and Lawrence with the keyboard, and I was hoping that any interactions would be recorded on the transcript, but I also went out and got a cheap camera from Wal*Mart for Lawrence to film Bow using the keyboard. The transcript shows everything that is typed, but it does not show who typed it. So I wanted footage of Bow typing and a transcript to show what he typed.

However, at the end of the day, Lawrence reported that Bow would not type anything when he, Lawrence, was in the room. But when Lawrence went to empty the potty, Bow took the opportunity to use the keyboard.

This morning I looked at the transcript. Here is what I found from yesterday:

B

O

BOW

BO

WHHAT

SAY

BO

DO

U

WA

N

TO

I'm pretty sure most of that was written by Lawrence. Was any part of it by Bow? I have to find a better protocol in order to distinguish what each contributed to the text. There's no video footage, because Lawrence was charging the new camera most of the day. Pretty inconclusive...

Meanwhile, now that I have a new camera, I tried taking some still shots of Bow.

This is Bow's evening face from last night, when he was chewing his cud after dinner. Here is a shot of Bow outside this morning: