Thoughts on photos-management system

Nowadays everybody around me have digital cameras. Photos that I am interested to look at are everywhere - on facebook, flickr, picasa, people's harddisks, some photo management system, etc. Rarely someone is kind enough to send me the photos.

Let's analyze the situation.

The situation

There are mainly two distinct categories of users of a photos management system - producers and consumers. Each of them want different features from the system.

Producers

Producers have many requirements:

• Editability: Producers need to operate (rotate, sort, tag, etc) on the photos. Automatic or manual face recognition would be nice to have.
• Multi-user, networkable: There might be several producers (members of the family) operating on the same photos. They connect from different accounts on the same computer or from different computers (and OSes) in the same network.
  
• Search: photos organizations must be flexible enough to support both folders and "search folders" - based on search string.
  
• Publishing: enable consumers to view the photos over internet. The requires Sync or Upload to a web server.
• Comments: enable consumers to comment on the photos, rate them, tag photos and faces.
• Portability: no lock-in into some proprietary photos management system, OSS or proprietary. Practically this means that all meta data must be stored/duplicated into the photo file itself, using some meta-data format (JFIF, EXIF, IPTC, XMP, etc).
  

Consumers

Consumers value different features than producers:

• easy and fast way to view the pics, preferably with captions (if any).
• they might want to specify filter that sorts out photos with nobody they know.
• ability to become producers (if allowed by host)

Current intermediate solution

Use shotwell program (I am linux user) to manage the photos on a network share. The shotwell maintains a database for quicker lookup, which is located along the photos. All changes are duplicated in metadata area of the photos as well. Publishing is done to picasa web album (it properly renders UTF8-encoded captions and tags in the IPTC header of the photos). Picasa is used as publishing point only, disregarding comments, ratings or faces tagging (blocking these if possible).

Ideal solution

The ideal solution would be in bi-directional synchronization of the photo files with some sharing-enabled cloud-based service, whereas the service embeds all additional meta-data generated by "consumers" into the files in open format. Clouds, are you listening?

How to install fb2pdf

Recently I installed fb2pdf on ubuntu maverick amd64. As there is no installation instructions anywhere, I decided to write down my notes I took during the installation:

cd /var/www
svn checkout FB2PDF
mkdir /var/www/fb2pdf-read-only/logs
chown -R www-data: /var/www/fb2pdf-read-only/logs
# Edit etc/apache.conf, register with /etc/apache2/sites-enabled
apt-get install libapache2-mod-php5
a2enmod rewrite
apt-get install texlive-latex-extra texlive-humanities python-boto \
python-pytils python-imaging libapache2-mod-php5 python2.4
texlive-lang-cyrillic

apt-get install php-pear php5-dev
pear update-channels
pear install Crypt_HMAC
pear install HTTP_Request

# Mysql server
apt-get install mysql-server php5-mysql
cd /var/www/fb2pdf-read-only/src/sql
mysqladmin create fb2pdf -p
cat create_db.sql | mysql fb2pdf -p
cat create_tables.sql | mysql fb2pdf -p
cat bootstrap.sql | mysql fb2pdf -p
#mysql
GRANT ALL ON fb2pdf.* to fb2pdf@"localhost" IDENTIFIED BY "THE_PWD";

cd /var/www/fb2pdf-read-only/www
cp awscfg.php.template awscfg.php

cd /usr/share/doc/texlive-lang-cyrillic/generic/t2/etc/utf-8/
cp utf-8.def /etc/texmf/tex/latex/pict2e/
gzip -dc utfcyr.def.gz > /etc/texmf/tex/latex/pict2e/utfcyr.def
gzip -dc utflat.def.gz > /etc/texmf/tex/latex/pict2e/utflat.def

Buggy Intel RAID

Recently I solved an very nasty problem in the Intel RAID controller. The problem took me and my colleagues many hours and hairs, so I decided to describe it here in hope it will help someone.

The Intel RAID controller was configured from the BIOS to have two disks in RAID1 configuration and all was good.

00:1f.2 RAID bus controller: Intel Corporation 82801 SATA RAID Controller

Some time ago we put in additional harddisk (hot-swappable SATA) that was part of some other RAID in the past (it is important!), the machine recognized it, we formatted it and started using the disk. All was good until the moment we had to reboot the machine.

It just would not boot. The screen looked like this:

The message was (transcribing for keywords):
Gave up waiting for root device.
Check rootdelay=
Check root=
ALERT does not exist. Dropping to a shell!

"dmraid -ay" would sometimes detect the root filesystem and sometimes (with different disk) not.

After a while an idea struck me - could be that the RAID controller in addition to CMOS-kept configuration tries to autodetect the RAID partitions that might exist on the new disks.

This was the answer. After I cleared both start and end of the new disks, the RAID signature on the disks would no longer confuse dmraid and prevent the kernel to find the real root file system.

dd if=/dev/zero bs=1000000 count=200 of=/dev/sdc
# For a 1TB disk. For different disk you need to calculate bs and seek accordingly.
dd if=/dev/zero bs=1000000000 seek=1000 of=/dev/sdc

If this story helped someone, write in the comments.

Fiddler on Linux

A colleague of mine (web developer) recently undertook effort to move from Windows to Ubuntu. One last obstacle was lack of adequate analogs of program "Fiddler". Wireshark is too general to work with and does not allow http-specific operations (for example - change values in a request and replay it). It's cumbersome interface was also hard to work with.

Enter burp proxy suite. I know, I know, it is proprietary, closed source, cpu and memory hogging java application. But hey, if the alternative for the colleague to stay in Windows, anything counts.

As good as the program is, it remains to be proxy application. This posed a problem because the developer had to intercept http requests from a stand-alone application.

While intercepting network-originating requests is not trivial, on Linux there is simple way to redirect local traffic to the burp proxy. The trick is to run the program in "invisible proxy" mode using "root" account and redirect all traffic coming from certain user to port 80 through the proxy. This solves the catch22 problem of "intercepting all outgoing requests".

Here are the commands that accomplish that:

sudo iptables -t nat -I OUTPUT -p tcp --dport 80 -m owner \
--uid-owner evgeny -j REDIRECT --to-port 8080
sudo java -jar burpsuite_v1.3.03.jar
sudo iptables -t nat -D OUTPUT -p tcp --dport 80 -m owner \
--uid-owner evgeny -j REDIRECT --to-port 8080

An nice advantage of the burp proxy over fiddler is that developer can choose to modify the request on-the-fly. Now I have something to brag about to all fiddler-wielding windows losers :) .

mp10 DV003-XXL manual on russian

Инструкция по эксплуатации "шпионского фотоаппарата/камеры" MP10 (DV-003) на русском:
http://skliarie.meshanet.com/skliarie_blog/mp10_russian.odt

O GTalk team, where were thou?

Instant messaging (IM) penetrates our lives deeper and deeper, whether we like it or not. Most people start using whatever their peers so to be able to network with them. For that reason, switching default IM is tremendously hard thing to do, especially if one already accumulated number of contacts in previous network. This is called "Network Effect".

There are several factors that can make one consider the switch - presence of their peers on the alternative network, number of functions the IM client and the network offer, easy to use IM client, availability of the client on their platform and (unfortunately often neglected) - how proprietary the network protocol is.

For past several years GTalk team was consistently failing on almost every point (at least on Linux). There is no native GTalk client for Linux. Interoperability with open-source clients is poor, especially on advanced features (voice and video conferences). No Video in standalone GTalk client. No built-in group chat support. No way to transfer bunch of contacts. File transfer is not reliable. The XMPP servers occasionally refuse to allow login (on ports 5222 and 5223). No way to organize contacts per groups (in the official GTalk client). These are problems that I have struggled with. I am sure there are many others.

Some of the problems are caused by protocol restrictions. But Google already showed that it can extend protocols by adding Jingle to XMPP. Some are caused by open-source clients. Google also showed that it can fund that as part of summer of code projects. Apparently that was not enough. Google missed great opportunity to have an viable IM network by underfunding the GTalk team and most popular open source clients. For Google's finances that would be pocket charge expenses.

With the Network Effect the Skype network has accumulated, it will be impossible to revert the situation, unless something dramatically occurs. That makes me sad as XMPP+Jingle had great potential to become the IM of choice for the world.

I speculate that Google Voice will fail for this very reason, but once Google understand that, it will probably be too late.

Hebrew thesaurus for stardict

Someone asked me to find him stardict-compatible version of hebrew thesaurus. I have found torrent link that has english-hebrew, hebrew-english dictionaries and hebrew thesaurus. All files are in stardict format:
babylon_stardict.torrent

I am still looking for stardict-compatible version of russian-hebrew and hebrew-russian dictionaries. If someone has informative links, drop me an email please.