The Accidental Rubyist

invalid byte sequence in UTF-8

Archive for November 2011

Major restructuring in rbcurse 1.5.0

with 2 comments

I’ve just finished a massive operation of splitting rbcurse into 3 gems. rbcurse-core contains the core infrastructure and basic widgets. These should allow for most ncurses applications. I intend to focus on keeping this as well tested and stable, and backward-compatible as possible. I’ll also work on simplifying the code of core as much as possible. The earliest widgets have also made it into core since a lot of users are using them. Thus, menu and textarea have made it, even though the editable textarea will never be very stable, and its better to use Vim to edit bodies of text. Core does have some new work which is quite experimental but i hope to have it as an integral part of rbcurse and to stabilize soon.

I’ve moved some more complex widgets to rbcurse-extras. These are widgets you should require less often. I’ve moved Table to extras, the simple. non-editable version will be in core. Similarly, the editable version of listbox is in extras, but the readonly version will be in core.

Stuff that is usable but experimental, and not thoroughly tested is in rbcurse-experimental. Essentially, I really am leaving it to users to improve. Some of it could move to extras if it proves useful.

Henceforth, rbcurse will be a meta-gem that installs the other three. You may install only core and develop with that. The directory structures have also changed appreciably. I did this in order to make the movement into 3 separate repositories a bit easier.

BTW, the first part of this operation was separating code into code, extras and experimental directories. Then adjusting the ‘requires’ in each file. Shell and perl (pie) made this quite easy. The second part was to create separate repos. After reading many posts on stackoverflow, I managed to get that done. I hope the history of each file is correct.

I did all this in the 1.5.0 branch, not master, so pushing this to master was another small job. Again found some articles and help on SO. Then to create separate gems for thm repos and tie them together. Someone pointed me to rspec and I’ve used their model.

I still have all the examples lying in the old repo. These still need to be changed so they can run only using core or extras. Most of them use features from all or two gems. I still have to change the names of some widgets that have moved from core to extras or vice-versa (listbox and table), and replace messagebox and tabbedpane with the simpler rewrites. This should take me several days, esp rewriting the examples. I intend standardizing the core widgets in 1.5. I’ve dome work on stacks and flows for placing widgets which needs to be tested further and integrated fully. I’ve done work on color-formatting of text which needs to be integrated and the API firmed up.

1.5.x should mainly be simplifying and cleaning up stuff, removing cruft. Very few feature additions.

Written by totalrecall

November 17, 2011 at 11:58 pm

Posted in ruby

colorizing bash output on terminal (non-curses)

leave a comment »

I’ve found that my ruby command-line scripts (gems) that put information into sqlite tables and then report from there, take time to load. If not loaded for an hour, the command can take as much as 8-10 seconds to load. I’ve tried faster-rubygems but that has made no difference (I am on 1.9.2 on OSX Lion).

The load time is okay, when i wish to add some info, but when I keep checking, its not acceptable. However, my earlier bash scripts that do similar reporting (and addition) respond instantly. Now i am not going back to writing stuff in bash, since the syntax keeps befuddling me after all these years. However, it takes only a few minutes to report data from sqlite, and get instant responses. You can also throw in 256 colors to get a neat presentable output. Here are some things that can help you.

This is the meat of the script. Define a temporary input and output file

echo ".mode column" > $file
echo "select id, title, priority, status from mytable where status != 'closed' order by priority , id;" >> $file
sqlite3 bugzy.sqlite < $file >> $out

Now your output is in the outfile. You can less or most the file.

If you want to color the rows here are some options.
colorize is a neat program that allows you to define regexes that work on a line basis and color the entire line. There’s a switch for "only color the pattern not entire line", but you can’t have both. So I stuck with the default.

cat <<ZZZ > colorize.cfg
/ cancel/ black
/ closed / black
/ started / blue on_yellow
/ P1 / reverse
/ P2 / bold white on_blue
/ P3 / yellow
/ P4 / magenta
/ P5 / bold blue
colorize --config=./colorize.cfg $out | less

So here I create a config file for colorize to use. It does the rest. Notice it has attribute, background and foreground, although it often does not work if you give both attribute and other options. Also, it does not seem to support 256 colors.

If you are willing to write a few lines of sed, you can add a variety of other colors or backgrounds rather than the garish 8 or 16. I like shades of grey.

I’ve put the basic color codes into a file that can be sourced. I’ve added some for the gray scale also. You need 256 color support in your Terminal (OSX’s Terminal has it, so does iTerm2). gnu-screen will have to be recompiled with 256 (see previous post).

I found a function that lets you get any color.

 function EXT_COLOR () { echo -ne "33[38;5;$1m"; }

You can use it as "$(EXT_COLOR 128) hello". Or use backticks.

Now use these combinations in sed (warning: blog has eaten up slashes and maybe more).

text=$(sed "
  s/(.* started .*)/${BLUE}${ON_WHITE}1${DEFAULT}/g;

  s/(.* canceled .*)/${BLACK}1${DEFAULT}/g;



  " $out)
echo  "$text" | less

There, you have instant display of data from an sqlite3 table. No ruby load times.
There's also a project called ansi-color on google-code, however, you would use that if you are generating the output. Let me know if you come across any project that allows for using 256 colors in bash in a cleaner way.

Also, let me know if you know how to get rubygems to load faster. I am sure if wrote the above program as a stand-alone ruby program it would respond immediately.

Here's a sample screenshot output, it's still garish and yucky, but i did it at 2am. Will improve the colors soon.

Written by totalrecall

November 14, 2011 at 1:06 pm

Posted in ruby

OSX Lion compile screen with 256 colors

leave a comment »

Terminal now supports 256 colors. But screen is not compiled with 256, so no matter what you do in the screenrc file, you won’t get 256 colors. homebrew wont install screen since OSX provides it.

This blog gives you the steps to compile, and within a minute you’ll have a 256 color screen.

Written by totalrecall

November 11, 2011 at 7:40 pm

Posted in being_geeky, Mac OS X

Tagged with ,

Displaying colored documents / text in ncurses

leave a comment »

I have wanted to show documents or text with coloring (like syntax highlighting) for some time. Here is a quick take on this. The image below shows 2 textviews. One has text colored using the TMUX format that I liked. The next document is the output of dooby which uses ANSI escape sequences.

ANSI escape sequences don’t allow us to close a "span", all previous colors are disabled whenever a new color is specified. The tmux-format allows us to nest colors, so when a color ends, the previous one can continue.
The tmux format uses syntax such as : "#[fg=green, bg=yellow, bold]". I’ve added a "#[end]" to signify end of a span. I still have to work on substringing text with such markup but the basics are in place.

Any new markup format supplies a parse_format which yields either the text or the color and attributes. The rest is handled by rbcurse which currently converts the text to an intermediate format, an array containing text, color and attribute. This is used for printing colored text. Any further operations, such as finding the length of a line, or slicing the marked-up text happens in this internal format coded once.

Written by totalrecall

November 8, 2011 at 2:41 pm

Posted in ruby