Installing Sun Grid Engine

[muxecoid]

Hello. Maybe Ubuntu is not the best distro to install SGE but I'm already familiar with it. I see gridengine on the repository but I can not configure it properly. Is there a guide to configuring it? If no do you know any newbie-friendly guides to installing it from Sun official binary (other that doc at Sun website which is excessively technical)?

[mert]

I also need to get SGE running on ubuntu. I just discovered that there is some documentation and a quick start guide in /usr/share/gridengine-common (of all places!) . I will try to get it running and let you know how it goes...

[NaitoNeko]

Any news about this post ? I also want to know about the configuration of SGE in ubuntu.

[hwttdz]

I successfully installed pbs, but now I'm thinking of switching to sge. It does seem to have a few advantages.

[hwttdz]

Sorry to wake a long gone thread, but I set up sge successfully (on a few different machines). So if you have any questions let me know and I'll see if I can help any. The big ones I came up with are just making sure you add your machine as both an execution host and a submit host, and making sure the name matches that from your hosts file exactly.

A quick detail of my install, I'm sure missing the useful bits: http://astoryworthtelling.wordpress....local-install/

[HighCommander540]

Not to be a **** about this or anything, but why wouldn't you just use Ubuntu's built-in cloud clutering technology?

[hwttdz]

I use sge because it's easy to use and does what I want, there are also other reasons. For instance to submit the job "echo 'hello world'" to sge the command is

qsub -shell n -b y "echo 'hello world'" (some might prefer qrsh)

what would the command be using ubuntu's cloud system?

Other reasons:
1) because the documentation is better
2) because it (or similar queuing systems) are standard in compute environments
3) because I am familiar with the environment and I can make scripts which run on both my local instance of sge and other queues on which I work
4) sge is designed to manage computationally intensive jobs, whereas the ubuntu cloud management is not (the overhead of running a vm seems silly)

[jefftb82]

hwttdz,

do you have any more information on installing sge?

J

[hwttdz]

Well for a single machine install (which granted is an unusual situation, but for a quad core processor I feel allows me to use the processor quite a bit more effectively). First install everything you're going to need:

sudo apt-get install gridengine-client gridengine-common gridengine-exec gridengine-master gridengine-qmon

I think it also installs a few other things, like lesstif2 and bsd-mailx, choose automatically configure, then for the cell name I just named it sge_cell or somesuch, doesn't seem to come up. The admin host is the hostname of your head machine, i.e. $HOSTNAME and then you get back to the command prompt.

Run "sudo qmon" to get a nice graphical interface to administer your mini-cluster. The first thing to do is add a queue, so go to queue control, then add, you're going to want to change the name, I just went with $HOSTNAME_queue, I changed slots to be the same as the number of processors, and I changed the shell to be the bash shell. Exit queue control.

Go into host configuration and add $HOSTNAME as a submit host (it should already be listed as both an admin host and an execute host). Back to main menu.

I went into the scheduler and changed the scheduler time to something shorter than 15 seconds, I often submit a large number of jobs none of which take terribly long, and I don't think that the sge scheduler uses up much in the way of cpu cycles, so I moved mine all the way down to 3 seconds from the default of 15.

Finally you should be good to go. Try,

qsub -j y -o /dev/null -N test_job -q $HOSTNAME_queue -cwd -V -b y "echo 'hello world' > my_output.txt"

Where
j = join output and error
o = send output to
N = name
q = queue name
cwd = do in the current working directory (no effect in this case)
V = save environment variables, in this case the path, otherwise "echo command not found"
b = binary?
finally a command

Let me know if you have any more issues.

I assume if you wanted to put more machines in the mix as exec hosts, you would need to install at least gridengine-exec on them (possibly common and client as well), and then add them as exec hosts, adjusting the number of slots in the queue as necessary.

[jefftb82]

Hey hwttdz,

Thanks for the response and the extra info. I have done everything you have listed. Iam currently testing on a single machine for master and compute node but will be extended in the near future(if I can ever get it going).

My main problem is that I keep getting this error whenever I submit a job.

"your job is not allowed to run in any queue"

Any idea's?

J