The Accidental Rubyist

invalid byte sequence in UTF-8

Archive for the ‘unix’ Category

zfm : fast file selection and directory navigation (zsh/commandline)

with 2 comments

I’ve just uploaded a zsh script for fast navigation, file selection etc. Try it out and give me feedback.

Command-line junkies should appreciate this.

https://github.com/rkumar/zfm
zfm
Please submit it to Reddit or Hacker News if you like it.

Written by totalrecall

December 25, 2012 at 3:41 pm

Posted in unix, unixscript, zsh

ffmpeg – output buffer too small – solved

with one comment

Lame 3.98 is the problem.

Go to macports, search for lame, go to revision history and select the portfile for 3.97b2 (click on @16761).
Download the portfile clicking on Original Format (bottom of page).

Preferably, put the portfile in a folder. cd to the folder and run “sudo port install

Now port deactivate the 3.98 port and port activate the @3.97b2_0 one.

Now run the ffmpeg command and you should not get the output buffer small error.

(In my case, I was using ffmpeg to extract mp3 from flv. The mp3 was usable from iTunes but *not* from iPod shuffle).

Written by totalrecall

May 17, 2010 at 12:29 pm

Posted in unix

macports selfupdate crashing

leave a comment »

And so I decide to do a macports selfupdate. And here’s what it tells me. (I on 10.5.8 PPC)

sudo port selfupdate
Password:
--->  Updating the ports tree
--->  Updating MacPorts base sources using rsync
MacPorts base version 1.8.1 installed,
MacPorts base version 1.8.2 downloaded.
--->  MacPorts base is outdated, installing new version 1.8.2
Installing new MacPorts release in /opt/local as root:admin permissions 0755 Tcl-Package in /Library/Tcl

Error: /opt/local/bin/port: port selfupdate failed: Error installing new MacPorts base: shell command "cd /opt/local/var/macports/sources/rsync.macports.org/release/base & ./configure --prefix=/opt/local --with-tclpackage=/Library/Tcl --with-install-user=root --with-install-group=admin --with-directory-mode=0755 --enable-readline & make & make install" returned error 2
Command output: gcc -c -DUSE_TCL_STUBS -DTCL_NO_DEPRECATED -g -O2 -W -Wall -pedantic  -I/opt/local/include   -DHAVE_CONFIG_H -I.. -I.  -I"/usr/include" -fno-common sha1cmd.c -o sha1cmd.o
In file included from sha1cmd.c:40:
/opt/local/include/tcl.h:400: warning: ISO C90 does not support 'long long'
/opt/local/include/tcl.h:401: warning: ISO C90 does not support 'long long'
In file included from sha1cmd.c:57:
/opt/local/include/openssl/sha.h:177: warning: ISO C90 does not support 'long long'
/opt/local/include/openssl/sha.h:178: warning: ISO C90 does not support 'long long'
/opt/local/include/openssl/sha.h:180: warning: ISO C90 does not support 'long long'
gcc -c -DUSE_TCL_STUBS -DTCL_NO_DEPRECATED -g -O2 -W -Wall -pedantic  -I/opt/local/include   -DHAVE_CONFIG_H -I.. -I.  -I"/usr/include" -fno-common curl.c -o curl.o
In file included from curl.c:49:
/opt/local/include/tcl.h:400: warning: ISO C90 does not support 'long long'
/opt/local/include/tcl.h:401: warning: ISO C90 does not support 'long long'
gcc -c -DUSE_TCL_STUBS -DTCL_NO_DEPRECATED -g -O2 -W -Wall -pedantic  -I/opt/local/include   -DHAVE_CONFIG_H -I.. -I.  -I"/usr/include" -fno-common rmd160cmd.c -o rmd160cmd.o
In file included from rmd160cmd.c:39:
/opt/local/include/tcl.h:400: warning: ISO C90 does not support 'long long'
/opt/local/include/tcl.h:401: warning: ISO C90 does not support 'long long'
gcc -c -DUSE_TCL_STUBS -DTCL_NO_DEPRECATED -g -O2 -W -Wall -pedantic  -I/opt/local/include   -DHAVE_CONFIG_H -I.. -I.  -I"/usr/include" -fno-common readline.c -o readline.o
In file included from readline.c:30:
/opt/local/include/tcl.h:400: warning: ISO C90 does not support 'long long'
/opt/local/include/tcl.h:401: warning: ISO C90 does not support 'long long'
readline.c: In function 'attempted_completion_function':
readline.c:128: error: 'filename_completion_function' undeclared (first use in this function)
readline.c:128: error: (Each undeclared identifier is reported only once
readline.c:128: error: for each function it appears in.)
readline.c:130: error: 'username_completion_function' undeclared (first use in this function)
readline.c:136: warning: implicit declaration of function 'completion_matches'
readline.c:136: warning: assignment makes pointer from integer without a cast
make[2]: *** [readline.o] Error 1
make[1]: *** [all] Error 1
make: *** [all] Error 1


# Rescue to the rescue:

h = { :age => 10 }
h[:name].downcase                         # ERROR
h[:name].downcase rescue "No name"        # => "No name"


http://www.rubyinside.com/21-ruby-tricks-902.html

Written by totalrecall

January 19, 2010 at 7:03 pm

Posted in unix

bash command-line tumblr client

with 12 comments

I’ve been looking for a command-line client for tumblr. I could only find a ruby one, and sadly it gave a trail of errors starting with:

F, [2009-12-28T18:34:18.630749 #75034] FATAL — : uninitialized constant Tumblr::Account::API (NameError)
/opt/local/lib/ruby1.9/gems/1.9.1/gems/tumblr-2.2.0/lib/tumblr.rb:43:in `block in ’
/opt/local/lib/ruby1.9/gems/1.9.1/gems/fattr-2.1.0/lib/fattr.rb:90:in `instance_eval’

Could be a version or dependency broken issue. Then I found a bash example, and had a bash script working in minutes that does posts, quotes, links, deleting, reading and editing. If anyone wants it, it’s here:

http://gist.github.com/267130

I’m still not clear what problem tumblr solves, that blogs or live-journal did not.
How is it different from blogging?

Written by totalrecall

January 1, 2010 at 8:44 pm

Posted in unix

Rewriting shell-scripts in ruby

leave a comment »

Just a note to myself (before I start some real work), that I’ve gotta rewrite some of my most used shell scripts in ruby, and write future ones in ruby. Reasons for frustration with bash scripting:

– if and loop syntax gets me
– unpredictable behavior of unix commands due to differences between gnu, macport and bsd versions. The options differ (take date for example, or split). The same program could pick up one egrep when running as a cronjob and another when executed by me on the prompt.
– regexp: one is no longer used to escaping + in a regexp as in “[0-9]\+”. Which reminds me that “\d” does not seem to work everywhere.
– File name and cat loops: data in loops gets split on spaces, have to go back and change delimiter to newlines.
– sensitivity to spaces
– updates to os x or macport often broken (one version of curl has stopped doing SSL due to some reason I forget — took ages to debug)

Writing these programs in ruby will help me learn faster. I could use highline for this work


Code comment:
// we are such morons. Wiz97 underwent a redesign between IE4 and IE5
http://www.kuro5hin.org/story/2004/2/15/71552/7795

# reverse order of lines (emulates ‘tac’) IMPROVE
$ cat | ruby -ne ‘BEGIN{@arr=Array.new}; @arr.push $_; END{puts @arr.reverse}’

Written by totalrecall

September 8, 2008 at 12:43 pm

Posted in ruby, unix

Automating export of Firefox 3 bookmarks and history from command-line/cron

with 7 comments

Firefox 2 used a bookmarks.html file for bookmarks. History was in a mork file that was incomprehensible. Firefox 3 uses SQLite for both these. Export manually is simple from the Bookmarks-Organize Bookmarks menu option. This page gives us info on how to automate bookmarks export using about:config, however this export happens:


  1. only for bookmarks, not history

  2. only when Firefox is shutdown


So the only option I had was to access the database via python or ruby. Both have packages for that, the python one comes with python (no install) so i tried that and got “database locked”. Googled but no one had an answer for that.

Finally, I just copied the file (hint courtesy), places.sqlite to places1.sqlite and was able to open (pls note that FF3 is still running).

It seems I already have a program called “sqlite3” on my os x. Loading it on the backup file showed me that I had various output formats such as html, csv, list etc. I could output to a file … same stuff as any other sql client – if you’ve used mysql, you’d be very familiar.

Now automating the output of the history and bookmarks without shutting firefox, and without user intervention in any format is as simple as:


cd /Users/john/Library/Application\ Support/Firefox/Profiles/8j7l8qg8.default
cp places.sqlite places1.sqlite
sqlite3 'places1.sqlite' <<!
.dump html
.output moz_places.html
select url,title from moz_places;
!

This may be run as a cronjob. Be sure to change the path in the cd command to your user, and edit the dump to list or csv as required. The output would go into some work folder for other applications to access easily. The existing location has a space which can cause bash programs to gripe (wish they would make life easier for us command-line junkies).

Bookmarks are stored in the moz_bookmarks file with title but not url. To get the url you would link to moz_places with the fk field as:


select a.url, b.title from moz_places a, moz_bookmarks b where a.id=b.fk and b.title like '%zsh%';

in order to dump all rows, just give:


select a.url, b.title from moz_places a, moz_bookmarks b where a.id=b.fk;

Before processing this text you may wish to filter out pages your other apps do not need such as google searches, your own websites/blogs, pagead urls etc. Also you may want to remove duplicates with the “sort -u” command.

Hope this helps.


# join pairs of lines side-by-side (like ‘paste’)
$ cat | ruby -pe ‘$_ = $_.chomp + ” ” + gets if $. % 2’

Written by totalrecall

September 3, 2008 at 11:33 am

Posted in being_geeky, unix

Getting gmx to work without novalidate-cert

leave a comment »

This was simple (thanks to madboa)


openssl s_client -connect www.gmx.com:443

This gives an output. Copy the begin certification to end certificate part and place in a file named gmx.pem.
It should look like this (lines deleted in between):

-----BEGIN CERTIFICATE-----
MIIDUzCCArygAwIBAgIQIvcebQE7u+QGIM7tD/CuCjANBgkqhkiG9w0BAQUFADCB
zjELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJ
qPqkqsLcZWVrMrGQsc0MXhixYXVns4aHpkeXfUPbFr/Cv9ly5NAa
-----END CERTIFICATE-----

openssl x509 -in gmx.pem -hash -noout

#(this outputs: ca09c2c4)

cp gmx.pem ca09c2c4.0

#(copy the .0 file to the location pointed to by openssl version -d)
sudo cp ca09c2c4.0 /opt/local/etc/openssl/certs

# fire up alpine and send a mail using gmx, and then check .pine.debug1
alpine -d tcp,imap=4,verbose=4

#show me:
# IMAP DEBUG 10:17:32 9/2: AUTH CRAM-MD5


$ man woman

Written by totalrecall

September 2, 2008 at 10:39 am

Posted in being_geeky, unix