Well... I have found a little free tool for integrating linux operating systems with Windows Active Directory fairly seamlessly. It is called LikeWise Open (http://www.likewise.com/community/index.php/download/). Just using a couple of simple command-line statements, I was able to
a)Add the machine to the active directory
b)Allow users to log in, changing the default shell to /bin/bash
The only thing I needed to do was manually modify the /etc/sudoers file to add our ADs group called "hadoop users" be able to run all commands
DOMAIN\\hadoop^users ALL=(ALL) NOPASSWD: ALL
Done! Now Hive and Sqoop can be executed using 'sudo hive -e "SELECT * FROM table1;"', or "sudo sqoop-import ......."
This is by far the easiest way to get your windows users to get access to the the cluster without having to manage every account individually.
Showing posts with label ldap. Show all posts
Showing posts with label ldap. Show all posts
2010-11-23
Active Directory/LDAP and Pig
So, in order to connect the Staging server to our Active Directory for user authentication, I simply used LikeWise-Open, which can be downloaded, and easily installed from http://www.likewise.com/download/index.php
Running the CLI utility was a breeze and only requires a Domain Admin account and a reboot to succeed.
I'm going to assume that the machine you are attempting to get this going on has Sun's Java 6. I have u22 x64 installed from the rpm I downloaded from sun.com.
Now we can log in with our LDAP accounts. Great, but we have no permissions. What next?
Fortunately for me, Hive worked straight away, and was automatically connecting to the cluster's HDFS. So I'm good there. Created a table and it showed up in Hue under the proper user/group.
BTW, use Hue for all of your HDFS exploration tasks, it is so much easier than typing 'hadoop fs -ls /', and then 'hadoop fs -ls /' to find your way through the tree!
To get Pig to run successfully in such a limited user account on linux, we need to set JAVA_HOME with the following lines, but where?
Adding this line to each user's ~/.bash_profile or ~/.bashrc doesn't take automatically because when an LDAP user logs in, they are not going to be running BASH (assuming you are running CentOS 5x). We need to have this export set globally, so simply add the above two lines to /etc/profile (anywhere). Have an LDAP user log in again and make sure that it works.
You *could* also set up each user to have /bin/bash as its default bash by adding them to the /etc/passwd file, but this does not scale well with the increase in the number of users that *will* be accessing the system.
OINK OINK!!
Running the CLI utility was a breeze and only requires a Domain Admin account and a reboot to succeed.
I'm going to assume that the machine you are attempting to get this going on has Sun's Java 6. I have u22 x64 installed from the rpm I downloaded from sun.com.
Now we can log in with our LDAP accounts. Great, but we have no permissions. What next?
Fortunately for me, Hive worked straight away, and was automatically connecting to the cluster's HDFS. So I'm good there. Created a table and it showed up in Hue under the proper user/group.
BTW, use Hue for all of your HDFS exploration tasks, it is so much easier than typing 'hadoop fs -ls /', and then 'hadoop fs -ls /
To get Pig to run successfully in such a limited user account on linux, we need to set JAVA_HOME with the following lines, but where?
JAVA_HOME=/usr/java/latest
export JAVA_HOME
Adding this line to each user's ~/.bash_profile or ~/.bashrc doesn't take automatically because when an LDAP user logs in, they are not going to be running BASH (assuming you are running CentOS 5x). We need to have this export set globally, so simply add the above two lines to /etc/profile (anywhere). Have an LDAP user log in again and make sure that it works.
You *could* also set up each user to have /bin/bash as its default bash by adding them to the /etc/passwd file, but this does not scale well with the increase in the number of users that *will* be accessing the system.
OINK OINK!!
Subscribe to:
Posts (Atom)