Hue 1.2.0 upgrade and Beeswax

Hey, its been awhile since I've typed anything out on this blog.

This will be short and sweet. Basically, my configuration is this:

node0 contains all user-based applications:
-pig
-hive
-hue
-sqoop

upgrading everything BUT hue was as easy as can be. I already had the proper nodes added to my hdfs-site.xml, core-site.xml, and mapred-site.xml files, including port numbers, etc. but I would still see the dreaded "localhost:8003" error on the web page whenever I loaded it up.

The secret ended up being in the /var/log/hue/beeswax*.out file, indicating a missing com.mysql.jdbc jar file in the classpath, so I simply copied "mysql-connector-java-5.1.11-bin.jar" to "/usr/share/hue/apps/beeswax/hive/lib/" and restarted the hue service. Beeswax now runs without issue. That was the only thing holding me back for 6 hours :( I feel so dumb, but hopefully this will help others in their quest to get beeswax working properly.

Oh, and the "hive.aux.jars.path" property in hive-site.xml doesn't do a damn thing to help. Don't even bother with that property, it doesn't work. This is as of 1.2.0.

Sqoop, SQLServer, DateTime, and Fun!

Ok, so writing a simple query like the following:

SELECT a.* FROM table a WHERE a.timestamp >= '20101125'

Should be pretty straightforward for importing data into Hive through Sqoop, but alas there are some hurdles.

I'll just get straight to the solution, I know people hate to read, so here is the final command line example:

sqoop import --driver com.microsoft.sqlserver.jdbc.SQLServerDriver --connect 'jdbc:sqlserver://localhost;user=xxxxxx;password=xxxxxxx;database=MyDB' --query "select a.* from tableX a where (a.timestamp >= '2010-11-25 00:00:00') AND \$CONDITIONS" --target-dir /data/feedlog --split-by 'MyIDColumn' --fields-terminated-by '\t'

Basically, enclose the entire --query in double-quotes, and then escape the $ on the $CONDITIONS value with a \ as well so that is escapes the initial bash evaluation.

Happy hunting!