Some thoughts on automated testing of ncurses apps

My major issue when making changes to code is that existing dependent program break. Typically the break is a UI related break, or a cursor positioning bug. UI means something may not print in the correct position, or may not print at all (thanks to copywin).

My log files contain details of positioning of copywin, and cursor. I can compare these after making changes to see if anything has changed. However, for that i must execute the same keys. Here is where expect can help.
An expect script with some basic key movements should create the same log file after code changes. The main issues with this are the timestamps and object references which i clean out. Also, log files change when i add a debug statement or change the same. But its still a good clue to help me see what’s gone wrong if suddenly a program is printing wierdly, or I am not sure whether this is how program once behaved or displayed its screen.

These issues are however, not with older versions of rbcurse. It’s only the new version with container such as tabbedpanes, splitpanes and scrollpanes which often contain instances of each other. I am also now going to store screenshots of how test programs look when the screen loads so i know after making changes how it once looked.

expect seems to have an issue with the sleep command. It executes all the sleeps before loading the screen and then executes all the keys with no gap. However, my log file should be created okay.

(Makes mental note never to do any programming that cannot be tested automatically!)

—
Who are you? You have no entry in /etc/utmp! Aborting…

Progress on scrollpanes within tabbedpanes (ncurses)

Thankfully, some progress on the cursor front. These images show a textview. In one case the textview is inside a scrollpane inside a tabbedpane. In the other the textarea is directly inside a tabbedpane.

TabbedPane with ScrollPane

TabbedPane with TextArea directly placed in it

I am reworking and trying to simplify the cursor placement – a major headache until now with such embedded objects.
In some previous screenshots, scrollpanes were starting display from 1,1 instead of 0,0. Correcting that has resulted in breakage, but helped in simplifying some code. I am really hoping to rewrite scrollpanes and splitpanes with a simpler approach if i can think of one. The ncurses method copywin is a major headache and keeps getting me.

I am hoping to write a splitpane like widget which is more like vim’s splits. So all the splits are on one level. Currently, a splitpane takes only 2 objects. You can embed a splitpane inside another, which is what most examples give. This makes it painful for the user to indicate which splitpane he wishes to expand or contract with a keyboard.
—
# If you are one of those sorry ba$tards who has to work with python you’ll know what I mean.
URL
— Micha Niskin

Textareas within tabbedpanes – cursor positioning issue

When placing a textarea (or textview) within a tabbedpane, the cursor is off by a row and a column. I am not toally clear why, and I don’t want to sit over it too long. I am leaving it there for a while. In any case, its not likely that you would want to edit a textarea within a tabbed pane (hopefully).

textarea within tabbedpane

I might have to re-evaluate my cursor positioning method, and let widgets themselves handle cursor display since this has been the major paint point for me of late when putting widgets within widgets. I’ve spent days and days figuring out what’s wrong. Totally saps motivation.

I’ll probably test out other widgets inside tabbedpanes, so any major issues are resolved. Please feel free to use readonly (noneditable) textareas or textviews inside tabbedpanes in the meantime.

I’ll probably work on a scrollable form – i think that should be simple and very useful – it will allow us to place objects on a form larger than the visible area, allowing user to scroll and edit. That obviates the need to create multiple forms, or pages and move between them (saving state and moving data back and forth). We should be able to pop the form inside tabbedpanes and other components.

—
# grep for foo OR bar OR baz
$ cat | ruby -pe 'next unless $_ =~ /(foo|bar|baz)/'

Tables within Scrollpanes (rbcurse)

The following is an image of a rbcurse Table within a Scrollpane. One may scroll the table horizontally or vertically, editing etc is all working. This follows placing of textareas, textviews, listboxes and splitpanes within scrollpanes and splitpanes.

a table within a ScrollPane

It’s now time to start testing out various objects within TabbedPanes. The release several months ago had only radio buttons and checkboxes inside different tabs.

I’ve just started putting various detailed technical notes about rbcurse in NOTES (only if you are creating your own widget or trying to get your head around the code).

—
# git branch wizardry
URL
– Ben Lynn

macports selfupdate crashing

And so I decide to do a macports selfupdate. And here’s what it tells me. (I on 10.5.8 PPC)

sudo port selfupdate
Password:
--->  Updating the ports tree
--->  Updating MacPorts base sources using rsync
MacPorts base version 1.8.1 installed,
MacPorts base version 1.8.2 downloaded.
--->  MacPorts base is outdated, installing new version 1.8.2
Installing new MacPorts release in /opt/local as root:admin permissions 0755 Tcl-Package in /Library/Tcl

Error: /opt/local/bin/port: port selfupdate failed: Error installing new MacPorts base: shell command "cd /opt/local/var/macports/sources/rsync.macports.org/release/base & ./configure --prefix=/opt/local --with-tclpackage=/Library/Tcl --with-install-user=root --with-install-group=admin --with-directory-mode=0755 --enable-readline & make & make install" returned error 2
Command output: gcc -c -DUSE_TCL_STUBS -DTCL_NO_DEPRECATED -g -O2 -W -Wall -pedantic  -I/opt/local/include   -DHAVE_CONFIG_H -I.. -I.  -I"/usr/include" -fno-common sha1cmd.c -o sha1cmd.o
In file included from sha1cmd.c:40:
/opt/local/include/tcl.h:400: warning: ISO C90 does not support 'long long'
/opt/local/include/tcl.h:401: warning: ISO C90 does not support 'long long'
In file included from sha1cmd.c:57:
/opt/local/include/openssl/sha.h:177: warning: ISO C90 does not support 'long long'
/opt/local/include/openssl/sha.h:178: warning: ISO C90 does not support 'long long'
/opt/local/include/openssl/sha.h:180: warning: ISO C90 does not support 'long long'
gcc -c -DUSE_TCL_STUBS -DTCL_NO_DEPRECATED -g -O2 -W -Wall -pedantic  -I/opt/local/include   -DHAVE_CONFIG_H -I.. -I.  -I"/usr/include" -fno-common curl.c -o curl.o
In file included from curl.c:49:
/opt/local/include/tcl.h:400: warning: ISO C90 does not support 'long long'
/opt/local/include/tcl.h:401: warning: ISO C90 does not support 'long long'
gcc -c -DUSE_TCL_STUBS -DTCL_NO_DEPRECATED -g -O2 -W -Wall -pedantic  -I/opt/local/include   -DHAVE_CONFIG_H -I.. -I.  -I"/usr/include" -fno-common rmd160cmd.c -o rmd160cmd.o
In file included from rmd160cmd.c:39:
/opt/local/include/tcl.h:400: warning: ISO C90 does not support 'long long'
/opt/local/include/tcl.h:401: warning: ISO C90 does not support 'long long'
gcc -c -DUSE_TCL_STUBS -DTCL_NO_DEPRECATED -g -O2 -W -Wall -pedantic  -I/opt/local/include   -DHAVE_CONFIG_H -I.. -I.  -I"/usr/include" -fno-common readline.c -o readline.o
In file included from readline.c:30:
/opt/local/include/tcl.h:400: warning: ISO C90 does not support 'long long'
/opt/local/include/tcl.h:401: warning: ISO C90 does not support 'long long'
readline.c: In function 'attempted_completion_function':
readline.c:128: error: 'filename_completion_function' undeclared (first use in this function)
readline.c:128: error: (Each undeclared identifier is reported only once
readline.c:128: error: for each function it appears in.)
readline.c:130: error: 'username_completion_function' undeclared (first use in this function)
readline.c:136: warning: implicit declaration of function 'completion_matches'
readline.c:136: warning: assignment makes pointer from integer without a cast
make[2]: *** [readline.o] Error 1
make[1]: *** [all] Error 1
make: *** [all] Error 1

—
# Rescue to the rescue:

h = { :age => 10 }
h[:name].downcase                         # ERROR
h[:name].downcase rescue "No name"        # => "No name"

— http://www.rubyinside.com/21-ruby-tricks-902.html

Scrollpanes – what do they actually do

Do scrollpanes scroll the object placed inside, or the text within the object (by passing a message to the object).
I assume a scrollpane scrolls the object inside. Thus, the object can be larger than the pane, that’s the point, right?

However, i recall reading about six years back that Java components used to implement scrolling. Then that functionality was removed, and we had to wrap (say) a list inside a JScrollPane. I presume the component had to implement an interface (perhaps Scrollable) and its methods would be called. Thus, JScrollpane would scroll the text within the textarea or listbox. But what would happen if the listbox were larger than the scrollpane?

What if the component inside does not implement Scrollable? Does it have to ?

Currently, text objects and listboxes in rbcurse do implement internal scrolling (C-n C-p). OTOH, rscrollpane scrolls the entire text object or whatever. That does mean i need to define a separate key for scrollpanes (M-p and M-n). That does suck.

So what does one do ? If the object implements some methods, pass a message to the internal object to scroll, otherwise scroll the entire object. Does this mean that an object that implements Scrollable should not be larger than the Scrollpane whereas an object that does not, can ?

Any suggestion on how I should go about this in rbcurses ?

scrollpane within a splitpane

The attached image shows an outer vertical splitpane, with a horizontal splitpane in the left (as its first component). The horizontal splitpane’s first component is a Scrollpane which contains a TextView. The textview is deliberately larger than the display area. Thus, there are 2 kinds of scrolling actions that I understand. C-n will scroll the data within the textview. M-n will scroll the textview itself. If it were not for the scrollpane, scrolling the data would not display the last page or so, since the textview is larger than the display area. That can be seen in the second image, where there’s no scrollpane.

textview directly inside a splitpane

rbcurse Textarea overwrite issue in some samples

I think it’s like this.. (reference test2.rb sample program)

rbcurse’s Textarea uses a pad, which does a prefresh onto screen.
All the other widgets (unbuffered) write to a window which does a
wrefresh. This refresh overwrites the write that prefresh did.

Acco to the manpage, certain optimizations are done with what is
already there on the screen. My guess is that a window wrefresh does not
disturb what is on the right, but it does overwrite what’s on the left.

Now, scrollpane and splitpane use a buffer_to_screen passing their own
graphic object, so the main window is written onto. I need to confirm
this by placing something on the right. Here, since textarea was not a
child or embedded object, a pad.refresh was done.

I think i need to ensure that a copy_pad_to_win is done in these cases,
by passing a graphic object. At least the source of this bug is now
clear to me now.

SOLVED YEAH !!!

The so-called  “buffered” objects write onto their own pad which is blit onto the screen. The “unbuffered” objects write to a window which is then blit using wrefresh. So after the buffered object has written, if the window is refreshed it overwrites what the pad had written. Now here’s the interesting part — if the unbuffered widgets are on the left of the buffered one, the window wrefresh does an optimization and does not write onto the space on the right. So the textarea is untouched. However, if _anything_ lies to the right, the window.wrefresh optimizes and clears out the space on the left, of course not touching any other widgets that it has painted. It does not know that something else had been written.

So now, I simply do a copywin() for the buffered objects onto the forms.window. So now the textarea is part of the normal form. I’ve checked the other cases when the textarea is embedded in a scrollpane, those work fine as before since it goes through the route of a “parent_buffer”.

—
If you are one of those sorry b***ards who has to work with
python you’ll know what I mean.
— Source

Stumped by some strange behavior in ncurses

I have a fully functional widget textarea. This writes directly onto the window. For reasons I won’t get into here, I’ve created a new version that writes to a pad which is then copied onto the window. I call this the buffered version.
I then go to a test program that has a lot of widgets in it (test2.rb) and replace the old textarea withe new one. There is a list box on the right of the textarea. Whenever the list box is painted, it *now* wipes out part of the buffered textarea. It does not overwrite the old textarea.

So when the form loads, i see the textarea flash for second, and then listbox wipes it out. When i tab into the textarea it paints itself. When i tab out of textarea into the listbox, the listbox wipes it out again.

Since the listbox does not wipe out the old textarea, I have to assume that the listbox is fine. Its been functioning fine until now. The list box seems to wipe out the textarea rows that are on the same rows. If i remove the listbox from the program, the textarea displays fine. So is the listbox doing some hanky-panky, or is there some bug or unexpected behaviour (gotcha) hidden deep in the bowels of ncurses relating to pads, the internals of copy_win and the states ncurses maintains.

The sequence seems to be like this:

1. listbox on the right of a textarea (or widget that writes to a pad first)
2. listbox’s repaint method is invoked
3. Only part of the textarea is wiped out (the same rows as those listbox writes to).

Maybe I should put some other widget there and see. That might exonerate listbox.

Okay, listbox is innocent. Even a simple textbox has the same effect of wiping the line off.
Man, I am stumped !

Edit: See next post. Issue solved.

—
XML is like violence – if it doesn’t solve your problems, you are not using enough of it.
– sig of Aaron Peterson (nokogiri)

bash command-line tumblr client

I’ve been looking for a command-line client for tumblr. I could only find a ruby one, and sadly it gave a trail of errors starting with:

F, [2009-12-28T18:34:18.630749 #75034] FATAL — : uninitialized constant Tumblr::Account::API (NameError)
/opt/local/lib/ruby1.9/gems/1.9.1/gems/tumblr-2.2.0/lib/tumblr.rb:43:in `block in ’
/opt/local/lib/ruby1.9/gems/1.9.1/gems/fattr-2.1.0/lib/fattr.rb:90:in `instance_eval’

Could be a version or dependency broken issue. Then I found a bash example, and had a bash script working in minutes that does posts, quotes, links, deleting, reading and editing. If anyone wants it, it’s here:

http://gist.github.com/267130

I’m still not clear what problem tumblr solves, that blogs or live-journal did not.
How is it different from blogging?

Battle of the cursor over

I’ve spent the last 3 days or so battling an issue with getting the cursor position correct on the screen
when the user is editing a textarea that is within a scrollpane. The insertion point in the textare was fine but the cursor display which is handled by the application at a higher level was going out of synch when the user scrolls the pane.

All it took was a couple lines of code, a couple extra variables so the textarea knows its been scrolled vertically or horizontally, but it took 3 painful days to figure that out. Still not perfect but at least the cursor and insertion point are always in synch, even if the app continues to show the cursor outside the scrollpane when the insertion point gets hidden due to scrolling (hehe). I guess that will get taken care of by the time i’ve got all the widgets upgraded.

Now I am back to the opinion that GUI’s really don’t have a cursor, although widgets know how to appear highlighted when focus falls on them. Textareas and textfields *do* however have an insertion point (perhaps caret is the correct term). Thus, the widget (rather than the app) should maintain and display a caret. This will take care of issues like making the caret invisible if the insertion point is overlapped or scrolled off, or moving it around.

—
# Overheard on the net

Also, Python strives for readability. I feel the existence
and popularity of Ruby has been a net positive for Python,
because it keeps programmers who don't greatly value
readability away from our Python codebases.

– Python, Ruby, Power