|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Objectorg.apache.hadoop.mapred.lib.CombineFileRecordReader<K,V>
public class CombineFileRecordReader<K,V>
A generic RecordReader that can hand out different recordReaders
for each chunk in a CombineFileSplit.
A CombineFileSplit can combine data chunks from multiple files.
This class allows using different RecordReaders for processing
these data chunks from different files.
CombineFileSplit| Field Summary | |
|---|---|
protected RecordReader<K,V> |
curReader
|
protected FileSystem |
fs
|
protected int |
idx
|
protected JobConf |
jc
|
protected long |
progress
|
protected Reporter |
reporter
|
protected Class<RecordReader<K,V>> |
rrClass
|
protected Constructor<RecordReader<K,V>> |
rrConstructor
|
protected CombineFileSplit |
split
|
| Constructor Summary | |
|---|---|
CombineFileRecordReader(JobConf job,
CombineFileSplit split,
Reporter reporter,
Class<RecordReader<K,V>> rrClass)
A generic RecordReader that can hand out different recordReaders for each chunk in the CombineFileSplit. |
|
| Method Summary | |
|---|---|
void |
close()
Close this InputSplit to future operations. |
K |
createKey()
Create an object of the appropriate type to be used as a key. |
V |
createValue()
Create an object of the appropriate type to be used as a value. |
long |
getPos()
return the amount of data processed |
float |
getProgress()
return progress based on the amount of data processed so far. |
protected boolean |
initNextRecordReader()
Get the record reader for the next chunk in this CombineFileSplit. |
boolean |
next(K key,
V value)
Reads the next key/value pair from the input for processing. |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Field Detail |
|---|
protected CombineFileSplit split
protected JobConf jc
protected Reporter reporter
protected Class<RecordReader<K,V>> rrClass
protected Constructor<RecordReader<K,V>> rrConstructor
protected FileSystem fs
protected int idx
protected long progress
protected RecordReader<K,V> curReader
| Constructor Detail |
|---|
public CombineFileRecordReader(JobConf job,
CombineFileSplit split,
Reporter reporter,
Class<RecordReader<K,V>> rrClass)
throws IOException
IOException| Method Detail |
|---|
public boolean next(K key,
V value)
throws IOException
RecordReader
next in interface RecordReader<K,V>key - the key to read data intovalue - the value to read data into
IOExceptionpublic K createKey()
RecordReader
createKey in interface RecordReader<K,V>public V createValue()
RecordReader
createValue in interface RecordReader<K,V>
public long getPos()
throws IOException
getPos in interface RecordReader<K,V>IOException
public void close()
throws IOException
RecordReaderInputSplit to future operations.
close in interface RecordReader<K,V>IOException
public float getProgress()
throws IOException
getProgress in interface RecordReader<K,V>0.0 to 1.0.
IOException
protected boolean initNextRecordReader()
throws IOException
IOException
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||