|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Objectorg.apache.hadoop.mapreduce.OutputFormat<K,V>
org.apache.hadoop.mapreduce.lib.db.DBOutputFormat<K,V>
@InterfaceAudience.Public @InterfaceStability.Stable public class DBOutputFormat<K extends DBWritable,V>
A OutputFormat that sends the reduce output to a SQL table.
DBOutputFormat accepts <key,value> pairs, where
key has a type extending DBWritable. Returned DBOutputFormat.DBRecordWriter
writes only the key to the database with a batch SQL query.
| Nested Class Summary | |
|---|---|
class |
DBOutputFormat.DBRecordWriter
A RecordWriter that writes the reduce output to a SQL table |
| Constructor Summary | |
|---|---|
DBOutputFormat()
|
|
| Method Summary | |
|---|---|
void |
checkOutputSpecs(JobContext context)
Check for validity of the output-specification for the job. |
String |
constructQuery(String table,
String[] fieldNames)
Constructs the query used as the prepared statement to insert data. |
OutputCommitter |
getOutputCommitter(TaskAttemptContext context)
Get the output committer for this output format. |
RecordWriter<K,V> |
getRecordWriter(TaskAttemptContext context)
Get the RecordWriter for the given task. |
static void |
setOutput(Job job,
String tableName,
int fieldCount)
Initializes the reduce-part of the job with the appropriate output settings |
static void |
setOutput(Job job,
String tableName,
String... fieldNames)
Initializes the reduce-part of the job with the appropriate output settings |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
|---|
public DBOutputFormat()
| Method Detail |
|---|
public void checkOutputSpecs(JobContext context)
throws IOException,
InterruptedException
OutputFormatThis is to validate the output specification for the job when it is a job is submitted. Typically checks that it does not already exist, throwing an exception when it already exists, so that output is not overwritten.
checkOutputSpecs in class OutputFormat<K extends DBWritable,V>context - information about the job
IOException - when output should not be attempted
InterruptedException
public OutputCommitter getOutputCommitter(TaskAttemptContext context)
throws IOException,
InterruptedException
OutputFormat
getOutputCommitter in class OutputFormat<K extends DBWritable,V>context - the task context
IOException
InterruptedException
public String constructQuery(String table,
String[] fieldNames)
table - the table to insert intofieldNames - the fields to insert into. If field names are unknown, supply an
array of nulls.
public RecordWriter<K,V> getRecordWriter(TaskAttemptContext context)
throws IOException
RecordWriter for the given task.
getRecordWriter in class OutputFormat<K extends DBWritable,V>context - the information about the current task.
RecordWriter to write the output for the job.
IOException
public static void setOutput(Job job,
String tableName,
String... fieldNames)
throws IOException
job - The jobtableName - The table to insert data intofieldNames - The field names in the table.
IOException
public static void setOutput(Job job,
String tableName,
int fieldCount)
throws IOException
job - The jobtableName - The table to insert data intofieldCount - the number of fields in the table.
IOException
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||