|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Objectorg.apache.hadoop.mapreduce.JobContext
org.apache.hadoop.mapreduce.Job
public class Job
The job submitter's view of the Job. It allows the user to configure the job, submit it, control its execution, and query the state. The set methods only work until the job is submitted, afterwards they will throw an IllegalStateException.
| Nested Class Summary | |
|---|---|
static class |
Job.JobState
|
| Field Summary |
|---|
| Fields inherited from class org.apache.hadoop.mapreduce.JobContext |
|---|
CACHE_ARCHIVES_VISIBILITIES, CACHE_FILE_VISIBILITIES, COMBINE_CLASS_ATTR, conf, credentials, INPUT_FORMAT_CLASS_ATTR, JOB_ACL_MODIFY_JOB, JOB_ACL_VIEW_JOB, JOB_CANCEL_DELEGATION_TOKEN, JOB_NAMENODES, MAP_CLASS_ATTR, OUTPUT_FORMAT_CLASS_ATTR, PARTITIONER_CLASS_ATTR, REDUCE_CLASS_ATTR, ugi, USER_LOG_RETAIN_HOURS |
| Constructor Summary | |
|---|---|
Job()
|
|
Job(Configuration conf)
|
|
Job(Configuration conf,
String jobName)
|
|
| Method Summary | |
|---|---|
void |
failTask(TaskAttemptID taskId)
Fail indicated task attempt. |
Counters |
getCounters()
Gets the counters for this job. |
static Job |
getInstance()
Creates a new Job
A Job will be created with a generic Configuration. |
static Job |
getInstance(Configuration conf)
Creates a new Job with a given Configuration. |
static Job |
getInstance(Configuration conf,
String jobName)
Creates a new Job with a given Configuration
and a given jobName. |
String |
getJar()
Get the pathname of the job's jar. |
TaskCompletionEvent[] |
getTaskCompletionEvents(int startFrom)
Get events indicating completion (success/failure) of component tasks. |
String |
getTrackingURL()
Get the URL where some job progress information will be displayed. |
boolean |
isComplete()
Check if the job is finished or not. |
boolean |
isSuccessful()
Check if the job completed successfully. |
void |
killJob()
Kill the running job. |
void |
killTask(TaskAttemptID taskId)
Kill indicated task attempt. |
float |
mapProgress()
Get the progress of the job's map-tasks, as a float between 0.0 and 1.0. |
float |
reduceProgress()
Get the progress of the job's reduce-tasks, as a float between 0.0 and 1.0. |
void |
setCancelDelegationTokenUponJobCompletion(boolean value)
Sets the flag that will allow the JobTracker to cancel the HDFS delegation tokens upon job completion. |
void |
setCombinerClass(Class<? extends Reducer> cls)
Set the combiner class for the job. |
void |
setGroupingComparatorClass(Class<? extends RawComparator> cls)
Define the comparator that controls which keys are grouped together for a single call to Reducer.reduce(Object, Iterable,
org.apache.hadoop.mapreduce.Reducer.Context) |
void |
setInputFormatClass(Class<? extends InputFormat> cls)
Set the InputFormat for the job. |
void |
setJarByClass(Class<?> cls)
Set the Jar by finding where a given class came from. |
void |
setJobName(String name)
Set the user-specified job name. |
void |
setMapOutputKeyClass(Class<?> theClass)
Set the key class for the map output data. |
void |
setMapOutputValueClass(Class<?> theClass)
Set the value class for the map output data. |
void |
setMapperClass(Class<? extends Mapper> cls)
Set the Mapper for the job. |
void |
setMapSpeculativeExecution(boolean speculativeExecution)
Turn speculative execution on or off for this job for map tasks. |
void |
setNumReduceTasks(int tasks)
Set the number of reduce tasks for the job. |
void |
setOutputFormatClass(Class<? extends OutputFormat> cls)
Set the OutputFormat for the job. |
void |
setOutputKeyClass(Class<?> theClass)
Set the key class for the job output data. |
void |
setOutputValueClass(Class<?> theClass)
Set the value class for job outputs. |
void |
setPartitionerClass(Class<? extends Partitioner> cls)
Set the Partitioner for the job. |
void |
setReducerClass(Class<? extends Reducer> cls)
Set the Reducer for the job. |
void |
setReduceSpeculativeExecution(boolean speculativeExecution)
Turn speculative execution on or off for this job for reduce tasks. |
void |
setSortComparatorClass(Class<? extends RawComparator> cls)
Define the comparator that controls how the keys are sorted before they are passed to the Reducer. |
void |
setSpeculativeExecution(boolean speculativeExecution)
Turn speculative execution on or off for this job. |
float |
setupProgress()
Get the progress of the job's setup, as a float between 0.0 and 1.0. |
void |
setWorkingDirectory(Path dir)
Set the current working directory for the default file system. |
void |
submit()
Submit the job to the cluster and return immediately. |
boolean |
waitForCompletion(boolean verbose)
Submit the job to the cluster and wait for it to finish. |
| Methods inherited from class org.apache.hadoop.mapreduce.JobContext |
|---|
getCombinerClass, getConfiguration, getCredentials, getGroupingComparator, getInputFormatClass, getJobID, getJobName, getMapOutputKeyClass, getMapOutputValueClass, getMapperClass, getNumReduceTasks, getOutputFormatClass, getOutputKeyClass, getOutputValueClass, getPartitionerClass, getReducerClass, getSortComparator, getWorkingDirectory |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
|---|
public Job()
throws IOException
IOException
public Job(Configuration conf)
throws IOException
IOException
public Job(Configuration conf,
String jobName)
throws IOException
IOException| Method Detail |
|---|
public static Job getInstance()
throws IOException
Job
A Job will be created with a generic Configuration.
Job
IOException
public static Job getInstance(Configuration conf)
throws IOException
Job with a given Configuration.
The Job makes a copy of the Configuration so
that any necessary internal modifications do not reflect on the incoming
parameter.
conf - the Configuration
Job
IOException
public static Job getInstance(Configuration conf,
String jobName)
throws IOException
Job with a given Configuration
and a given jobName.
The Job makes a copy of the Configuration so
that any necessary internal modifications do not reflect on the incoming
parameter.
conf - the ConfigurationjobName - the job instance's name
Job
IOException
public void setNumReduceTasks(int tasks)
throws IllegalStateException
tasks - the number of reduce tasks
IllegalStateException - if the job is submitted
public void setWorkingDirectory(Path dir)
throws IOException
dir - the new current working directory.
IllegalStateException - if the job is submitted
IOException
public void setInputFormatClass(Class<? extends InputFormat> cls)
throws IllegalStateException
InputFormat for the job.
cls - the InputFormat to use
IllegalStateException - if the job is submitted
public void setOutputFormatClass(Class<? extends OutputFormat> cls)
throws IllegalStateException
OutputFormat for the job.
cls - the OutputFormat to use
IllegalStateException - if the job is submitted
public void setMapperClass(Class<? extends Mapper> cls)
throws IllegalStateException
Mapper for the job.
cls - the Mapper to use
IllegalStateException - if the job is submittedpublic void setJarByClass(Class<?> cls)
cls - the example classpublic String getJar()
getJar in class JobContext
public void setCombinerClass(Class<? extends Reducer> cls)
throws IllegalStateException
cls - the combiner to use
IllegalStateException - if the job is submitted
public void setReducerClass(Class<? extends Reducer> cls)
throws IllegalStateException
Reducer for the job.
cls - the Reducer to use
IllegalStateException - if the job is submitted
public void setPartitionerClass(Class<? extends Partitioner> cls)
throws IllegalStateException
Partitioner for the job.
cls - the Partitioner to use
IllegalStateException - if the job is submitted
public void setMapOutputKeyClass(Class<?> theClass)
throws IllegalStateException
theClass - the map output key class.
IllegalStateException - if the job is submitted
public void setMapOutputValueClass(Class<?> theClass)
throws IllegalStateException
theClass - the map output value class.
IllegalStateException - if the job is submitted
public void setOutputKeyClass(Class<?> theClass)
throws IllegalStateException
theClass - the key class for the job output data.
IllegalStateException - if the job is submitted
public void setOutputValueClass(Class<?> theClass)
throws IllegalStateException
theClass - the value class for job outputs.
IllegalStateException - if the job is submitted
public void setSortComparatorClass(Class<? extends RawComparator> cls)
throws IllegalStateException
Reducer.
cls - the raw comparator
IllegalStateException - if the job is submitted
public void setGroupingComparatorClass(Class<? extends RawComparator> cls)
throws IllegalStateException
Reducer.reduce(Object, Iterable,
org.apache.hadoop.mapreduce.Reducer.Context)
cls - the raw comparator to use
IllegalStateException - if the job is submitted
public void setJobName(String name)
throws IllegalStateException
name - the job's new name.
IllegalStateException - if the job is submittedpublic void setSpeculativeExecution(boolean speculativeExecution)
speculativeExecution - true if speculative execution
should be turned on, else false.public void setMapSpeculativeExecution(boolean speculativeExecution)
speculativeExecution - true if speculative execution
should be turned on for map tasks,
else false.public void setReduceSpeculativeExecution(boolean speculativeExecution)
speculativeExecution - true if speculative execution
should be turned on for reduce tasks,
else false.public String getTrackingURL()
public float setupProgress()
throws IOException
IOException
public float mapProgress()
throws IOException
IOException
public float reduceProgress()
throws IOException
IOException
public boolean isComplete()
throws IOException
true if the job is complete, else false.
IOException
public boolean isSuccessful()
throws IOException
true if the job succeeded, else false.
IOException
public void killJob()
throws IOException
IOException
public TaskCompletionEvent[] getTaskCompletionEvents(int startFrom)
throws IOException
startFrom - index to start fetching events from
TaskCompletionEvents
IOException
public void killTask(TaskAttemptID taskId)
throws IOException
taskId - the id of the task to be terminated.
IOException
public void failTask(TaskAttemptID taskId)
throws IOException
taskId - the id of the task to be terminated.
IOException
public Counters getCounters()
throws IOException
IOExceptionpublic void setCancelDelegationTokenUponJobCompletion(boolean value)
public void submit()
throws IOException,
InterruptedException,
ClassNotFoundException
IOException
InterruptedException
ClassNotFoundException
public boolean waitForCompletion(boolean verbose)
throws IOException,
InterruptedException,
ClassNotFoundException
verbose - print the progress to the user
IOException - thrown if the communication with the
JobTracker is lost
InterruptedException
ClassNotFoundException
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||